ABS Hacking

They go by many names, P01, P59, VPW, '0411 etc . Circa 1999 to 2006. All VPW OBD2 PCMs.
User avatar
Posts: 324
Joined: Sun Jan 25, 2015 4:21 pm
Location: Sydney

Re: ABS Hacking

Postby j_ds_au » Fri Jul 31, 2020 10:45 pm

jlvaldez wrote:For some reason, when threading and trying to store datalogs, this throughput drops to a very inconsistent 5-10 hz. I'm finding it damn near impossible to get a python thread to consistently trigger with any timing accuracy in the low ms range. If it's running by itself, then it's decently consistent.

If "trying to store datalogs" means writing data to a file, that may explain your drop in performance. I believe that file I/O operations are computationally very expensive and can screw up realtime performance.

Joe.

Posts: 656
Joined: Sat Dec 15, 2018 7:38 am

Re: ABS Hacking

Postby Gampy » Sat Aug 01, 2020 12:52 am

jlvaldez,

j_ds_au is exactly right!

You need to buffer and block write.

You will likely also need to go to a lower level language, like C/C++ or even better Assembly.
A high level language (easy to write) is just to heavy for fast intensive IO ...

One of the tricks I pull is to use compiler options to output the Assembly or something like Compiler Explorer to optimize your code ... With Compiler Explorer, it really doesn't matter what compiler you select, the idea is to look at the assembly, pretty much the fewer the instructions the better.

-Enjoy
I'm not angry, I'm a brash, blatant, driven, passionate individual that is extremely intense!

Posts: 99
Joined: Mon Feb 11, 2019 12:48 pm
Location: DFW, Texas

Re: ABS Hacking

Postby jlvaldez » Sat Aug 01, 2020 2:14 am

j_ds_au wrote:
jlvaldez wrote:For some reason, when threading and trying to store datalogs, this throughput drops to a very inconsistent 5-10 hz. I'm finding it damn near impossible to get a python thread to consistently trigger with any timing accuracy in the low ms range. If it's running by itself, then it's decently consistent.

If "trying to store datalogs" means writing data to a file, that may explain your drop in performance. I believe that file I/O operations are computationally very expensive and can screw up realtime performance.

Joe.

I'm familiar with writing to file in chunks due to how slow disks are. I was storing everything in memory and write to file at the end, when I'm no longer datalogging.

I was trying to buffer in memory and write once a second but I realized that iI have enough ram to buffer everything in memory and write to disk when I stop the log.

I'm not sure which part of the program is actually taking all the time. Seems like the call to SPI is pretty slow and taking 10-15 ms to complete. Weird that by itself it takes microseconds.

I normally write this type of stuff in C on embedded devices but I'm not quite as familiar with writing close to hardware on a raspberry pi.

I also switched the library I'm using for holding data in memory, which seems to have slowed things down considerably as well. I'll at with yanking calls and seeing where the delay is.

One of my frustrations is that I cannot seem to find how to create a periodic timer that isn't affected by the function it runs. It's annoying because this is trivial to do with hardware timers in c, but I can't find a way in python lol

Posts: 99
Joined: Mon Feb 11, 2019 12:48 pm
Location: DFW, Texas

Re: ABS Hacking

Postby jlvaldez » Sat Aug 01, 2020 3:37 am

After a nights rest and coming back to it, I switch my row library to be a simple python list instead of Pandas (python library). I also moved some stuff around in the code and made a few things lighter.

I am now able to get ~150 samples/second with decent accuracy. If I renice the process to -20, timing gets more consistent. I have a flip switch input that starts and stops the datalogs.

Turns out the majority of my slow down was trying to "dynamically" put my rows together. I had a list of parameters i wanted it to log, and then it would perform an indexing function on each of them to find which index to pull the data from. This was very slow. Removing this and making it static was the single biggest speed up. I'll play with it later to have it perform indexing only once and then itll populate a list of indexes so it doesn't have to look up the index every iteration. I figured it'd be slow, but I didn't think it would affect it that much.

Python is definitely not geared for this, but it is working at lower logging rates if you put higher priority on it.

Posts: 656
Joined: Sat Dec 15, 2018 7:38 am

Re: ABS Hacking

Postby Gampy » Sat Aug 01, 2020 4:06 am

Awe shucks, stuck my nose in where it wasn't needed ... o-wells win some, loose some!

I'll just set back and watch in amazement as you meander the paths of your endeavor ...
I'm not angry, I'm a brash, blatant, driven, passionate individual that is extremely intense!

Posts: 99
Joined: Mon Feb 11, 2019 12:48 pm
Location: DFW, Texas

Re: ABS Hacking

Postby jlvaldez » Sun Aug 02, 2020 9:17 am

I like the ideas, keep em coming :) you've got a lot more experience with this type of software than I do. I deal with low level embedded stuff for work and almost never write for an OS.

Well I learned that python has a GIL which basically means multi threading doesn't work as you'd expect because only 1 thread ever runs at a time in a python interpreter. This explains why my seperate thread to handle SPI comms at a specific interval goes to complete shit once the digital inputs get loaded up.

I realized that what slows it down is the combination of the fact anytime the digital inputs switch, a call back occurs in python, where I determine frequency of wheel input and the fact that the SPI ADC calls are all blocking and I can't kick them to a separate thread like I'd expect. When these two happen together, CPU usage hits 100% (on a core) and the script gets backed up with so many call backs that it takes a while for it to do anything

This means I have 2 options:
1) write the critical code in c like I should have done from the start.
2) have multiple python threads launched separately that somehow communicate to each other.

I know doing this via C is the correct method, just sucks I'll have to rewrite all of my code :'(

Guess I'll need to look into how timers work on this thing and if I have access to them at any low level.

I shouldn't need the CPU to run at 1.5 GHz to accomplish 100 Hz sampling.

It was so easy in python... Damn :(

Previous

Return to GM LS1 512Kbyte and 1Mbyte

Who is online

Users browsing this forum: No registered users and 10 guests