Calculating checksums? Can't get OS to work

They go by many names, P01, P59, VPW, '0411 etc. Also covering E38 and newer here.
jlvaldez
Posts: 155
Joined: Mon Feb 11, 2019 12:48 pm
cars: '01 - Corvette Z06
'20 - Sierra Denali
'03 - Volvo S80 T6
'16 - Accord V6
Location: DFW, Texas

Calculating checksums? Can't get OS to work

Post by jlvaldez »

Hey all, Gampy was helpful with information calculating the checksums for the various segments. I threw together a simple python script to generate and check the checksums of the various segments in a binary. Right now, I'm trying the P01/411 binary off my corvette, and I can get all segments to work correctly, except the OS.

From this thread that Gampy pointed me to: viewtopic.php?f=42&t=6198#p91839

Code: Select all

success &= this.ValidateRange(      0, 0x7FFFD,   0x500, "Operating system");
            success &= this.ValidateRange( 0x8002, 0x13FFF,  0x8000, "Engine calibration");
            success &= this.ValidateRange(0x14002, 0x16DFF, 0x14000, "Engine diagnostics.");
            success &= this.ValidateRange(0x16E02, 0x1BDFF, 0x16E00, "Transmission calibration");
            success &= this.ValidateRange(0x1BE02, 0x1C7FF, 0x1BE00, "Transmission diagnostics");
            success &= this.ValidateRange(0x1C802, 0x1E51F, 0x1C800, "Fuel system");
            success &= this.ValidateRange(0x1E522, 0x1EE9F, 0x1E520, "System");
            success &= this.ValidateRange(0x1EEA2, 0x1EF9F, 0x1EEA0, "Speedometer");
And Gampy pointed out that the correct OS segment information is:
0x000000 -> 0x0004FF
0x000502 -> 0x003FFF
0x020000 -> 0x07FFFF
Checksum is stored at 0x500

Yet for whatever reason, whenever I go through and add up these segments, I get invalid checksums, but the rest of the addresses in the code block work correctly (checksum + sum of 16-bit words = 0).

To be clear about how I'm doing this:
1) Sum each 16-bit word within each segment together.
2) Sum each of the sums from each segment
3) Bitwise and with 0xFFFF (truncate)

This strategy works for all segments except the OS, which leads me to believe an address range is wrong?

Code: Select all

python3 checksum.py 
Sum iteration: 0x0 0x4ff Size: 1280
Sum iteration: 0x502 0x3fff Size: 15102
Sum iteration: 0x20000 0x7ffff Size: 393216
jlvaldez
Posts: 155
Joined: Mon Feb 11, 2019 12:48 pm
cars: '01 - Corvette Z06
'20 - Sierra Denali
'03 - Volvo S80 T6
'16 - Accord V6
Location: DFW, Texas

Re: Calculating checksums? Can't get OS to work

Post by jlvaldez »

Turns out the address range was wrong. The last segment of the OS section is 0x07FFFD instead of 0x7FFFF. Now it works for all segments of the 411 binary.

However, it seems now that the P59 segments are different because all sections except for the updated OS fail to get valid checksums for any of my P59 segments. But P01/411 is working now :)

Code: Select all

class p01_segments:
	# Format: Checksum location, start address, end address, ... start address, end address
	os          = [0x000500, 0x000000, 0x0004FF, 0x000502, 0x003FFF, 0x020000, 0x07FFFD]
	engine_cal  = [0x008000, 0x008002, 0x013FFF]
	engine_diag = [0x014000, 0x014002, 0x016DFF]
	trans_cal   = [0x016E00, 0x016E02, 0x01BDFF]
	trans_diag  = [0x01BE00, 0x01BE02, 0x01C7FF]
	fuel_system = [0x01C800, 0x01C802, 0x01E51F]
	system      = [0x01E520, 0x01E522, 0x01EE9F]
	speedometer = [0x01EEA0, 0x01EEA2, 0x01EF9F]

Code: Select all

class p59_segments:
	# Format: Checksum location, start address, end address, ... start address, end address
	os          = [0x000500, 0x000000, 0x0004FF, 0x000502, 0x003FFF, 0x020000, 0x0FFFFD]
	engine_cal  = [0x008000, 0x008002, 0x013FFF]
	engine_diag = [0x014000, 0x014002, 0x016DFF]
	trans_cal   = [0x016E00, 0x016E02, 0x01BDFF]
	trans_diag  = [0x01BE00, 0x01BE02, 0x01C7FF]
	fuel_system = [0x01C800, 0x01C802, 0x01E51F]
	system      = [0x01E520, 0x01E522, 0x01EE9F]
	speedometer = [0x01EEA0, 0x01EEA2, 0x01EF9F]
jlvaldez
Posts: 155
Joined: Mon Feb 11, 2019 12:48 pm
cars: '01 - Corvette Z06
'20 - Sierra Denali
'03 - Volvo S80 T6
'16 - Accord V6
Location: DFW, Texas

Re: Calculating checksums? Can't get OS to work

Post by jlvaldez »

Continuing my rubber ducky debugging:

Thanks to a post from NSFW (viewtopic.php?f=42&t=6240), I added a function to pull the segment addresses starting at address 0x514.

Dump from my Truck's custom OS (P59):

Code: Select all

>>> truck.read_segments()
 Engine Cal: 0x8000 0x15dff
Engine Diag: 0x15e00 0x18f8f
  Trans Cal: 0x18f90 0x1d26f
 Trans Diag: 0x1d270 0x1dbcf
Fuel System: 0x1dbd0 0x1f18f
     System: 0x1f190 0x1fb4f
Speedometer: 0x1fb50 0x1fd7f
Dump from a supposed bone stock 03 Silverado (P59)

Code: Select all

>>> truckGeneric.read_segments()
 Engine Cal: 0x8000 0x15dff
Engine Diag: 0x15e00 0x18f8f
  Trans Cal: 0x18f90 0x1d26f
 Trans Diag: 0x1d270 0x1dbcf
Fuel System: 0x1dbd0 0x1f18f
     System: 0x1f190 0x1fb4f
Speedometer: 0x1fb50 0x1fd7f

Dump from my Corvette, running a standard OS (but has been tuned) (P01)

Code: Select all

>>> vette.read_segments()
 Engine Cal: 0x8000 0x13fff
Engine Diag: 0x14000 0x16dff
  Trans Cal: 0x16e00 0x1bdff
 Trans Diag: 0x1be00 0x1c7ff
Fuel System: 0x1c800 0x1e51f
     System: 0x1e520 0x1ee9f
Speedometer: 0x1eea0 0x1ef9f
Looks like I've got my updated segments for P59... All my checksums work correctly now :)

Now to package this up a bit more and i'll share the python file on here for others to use.
Last edited by jlvaldez on Tue Dec 31, 2019 11:52 am, edited 1 time in total.
User avatar
antus
Site Admin
Posts: 8237
Joined: Sat Feb 28, 2009 8:34 pm
cars: TX Gemini 2L Twincam
TX Gemini SR20 18psi
Datsun 1200 Ute
Subaru Blitzen '06 EZ30 4th gen, 3.0R Spec B
Contact:

Re: Calculating checksums? Can't get OS to work

Post by antus »

Great, good work :thumbup:
Have you read the FAQ? For lots of information and links to significant threads see here: http://pcmhacking.net/forums/viewtopic.php?f=7&t=1396
jlvaldez
Posts: 155
Joined: Mon Feb 11, 2019 12:48 pm
cars: '01 - Corvette Z06
'20 - Sierra Denali
'03 - Volvo S80 T6
'16 - Accord V6
Location: DFW, Texas

Re: Calculating checksums? Can't get OS to work

Post by jlvaldez »

I am not sure if it makes sense to make a new thread, or post this into another thread.

But I'm posting up a quick n dirty python script I wrote to check the checksums of P59 and P01 binaries.

It runs in Python 3.x, and can be used by other libraries, if desired.

To use it, simply call python3 in a command line with python installed and give it your input binary, and an output path file name if you want it to copy/update checksums.

Code: Select all

python3 checksum.py inputFile.bin [outputFile.bin]
Here's a few examples of the output when ran. I ran it against the my stock Vette01.bin, which is a P01

Code: Select all

jvaldez$ python3 checksum.py ./Vette01.bin
GM Gen 3 P01 & P59 Checksum Checker/Updater
Version: 0.1, Released December 30, 2019

Loaded binary file: ./Vette01.bin
ECU type: P01
Binary size: 524288 bytes
OS ID: 12593358

Analyzing segments
           OS : [ PASS ] Bin Checksum: 0x9314 | Calculated Checksum: 0x9314
   Engine Cal : [ PASS ] Bin Checksum: 0x4721 | Calculated Checksum: 0x4721
  Engine Diag : [ PASS ] Bin Checksum: 0x24b7 | Calculated Checksum: 0x24b7
    Trans Cal : [ PASS ] Bin Checksum: 0xefde | Calculated Checksum: 0xefde
   Trans Diag : [ PASS ] Bin Checksum: 0xdfd7 | Calculated Checksum: 0xdfd7
  Fuel System : [ PASS ] Bin Checksum: 0x16ba | Calculated Checksum: 0x16ba
       System : [ PASS ] Bin Checksum: 0x7c27 | Calculated Checksum: 0x7c27
  Speedometer : [ PASS ] Bin Checksum: 0xca39 | Calculated Checksum: 0xca39

All segments have valid checksums.

Here's another example with my truck's P59, where I've modified a few instructions in a hex editor, but the checksum is invalid. I also put an output filename for it to duplicate the binary.

Code: Select all

jvaldez$ python3 checksum.py ./SilveradoTest.bin ./SilveradoTestOut.bin 
GM Gen 3 P01 & P59 Checksum Checker/Updater
Version: 0.1, Released December 30, 2019

Opening with request to output file, for corrected checksums
Loaded binary file: ./SilveradoTest.bin
ECU type: P59
Binary size: 1048576 bytes
OS ID: 1283050

Analyzing segments
           OS : [ FAIL ] Bin Checksum: 0x5384 | Calculated Checksum: 0x6b19
   Engine Cal : [ PASS ] Bin Checksum: 0xe947 | Calculated Checksum: 0xe947
  Engine Diag : [ PASS ] Bin Checksum: 0xb1da | Calculated Checksum: 0xb1da
    Trans Cal : [ PASS ] Bin Checksum: 0xe896 | Calculated Checksum: 0xe896
   Trans Diag : [ PASS ] Bin Checksum: 0x40fe | Calculated Checksum: 0x40fe
  Fuel System : [ PASS ] Bin Checksum: 0xb3c0 | Calculated Checksum: 0xb3c0
       System : [ PASS ] Bin Checksum: 0x3318 | Calculated Checksum: 0x3318
  Speedometer : [ PASS ] Bin Checksum: 0xc641 | Calculated Checksum: 0xc641

One or more segments have an invalid checksum!
Writing updated binary to: ./SilveradoTestOut.bin



Now if I run the output file that was generated back through the checker, it has the updated checksums, and should be ready for upload via PCM hammer.

Code: Select all

jvaldez$ python3 checksum.py ./SilveradoTestOut.bin 
GM Gen 3 P01 & P59 Checksum Checker/Updater
Version: 0.1, Released December 30, 2019

Loaded binary file: ./SilveradoTestOut.bin
ECU type: P59
Binary size: 1048576 bytes
OS ID: 1283050

Analyzing segments
           OS : [ PASS ] Bin Checksum: 0x6b19 | Calculated Checksum: 0x6b19
   Engine Cal : [ PASS ] Bin Checksum: 0xe947 | Calculated Checksum: 0xe947
  Engine Diag : [ PASS ] Bin Checksum: 0xb1da | Calculated Checksum: 0xb1da
    Trans Cal : [ PASS ] Bin Checksum: 0xe896 | Calculated Checksum: 0xe896
   Trans Diag : [ PASS ] Bin Checksum: 0x40fe | Calculated Checksum: 0x40fe
  Fuel System : [ PASS ] Bin Checksum: 0xb3c0 | Calculated Checksum: 0xb3c0
       System : [ PASS ] Bin Checksum: 0x3318 | Calculated Checksum: 0x3318
  Speedometer : [ PASS ] Bin Checksum: 0xc641 | Calculated Checksum: 0xc641

All segments have valid checksums.
Attachments
checksum-0p1.zip
Version 0.1
(3.29 KiB) Downloaded 293 times
User avatar
joukoy
Posts: 392
Joined: Tue Dec 17, 2019 3:27 am
cars: Pontiac Firebird 1978

Re: Calculating checksums? Can't get OS to work

Post by joukoy »

I think your script is not working correctly for all P59 binaries, because of different segment addresses.
You should always read addresses from binary. (use routine already in script)
Right?

For example, OS 12587603 have:
Engine Cal: 0x8000 0x162cf
Engine Diag: 0x162d0 0x195ff
Trans Cal: 0x19600 0x1d8af
Trans Diag: 0x1d8b0 0x1e1af
Fuel System: 0x1e1b0 0x1f6bf
System: 0x1f6c0 0x1feaf
Speedometer: 0x1feb0 0x1ffdf

Script have fixed addresses:
engine_cal = [0x008000, 0x008002, 0x015DFF]
engine_diag = [0x015E00, 0x015E02, 0x018F8F]
trans_cal = [0x018F90, 0x018F92, 0x01D26F]
trans_diag = [0x01D270, 0x01D272, 0x01DBCF]
fuel_system = [0x01DBD0, 0x01DBD2, 0x01F18F]
system = [0x01F190, 0x01F192, 0x01FB4F]
speedometer = [0x01FB50, 0x01FB52, 0x01FD7F]
jlvaldez
Posts: 155
Joined: Mon Feb 11, 2019 12:48 pm
cars: '01 - Corvette Z06
'20 - Sierra Denali
'03 - Volvo S80 T6
'16 - Accord V6
Location: DFW, Texas

Re: Calculating checksums? Can't get OS to work

Post by jlvaldez »

I think you're right. I wasn't sure if the address containing the addresses is always correct. I will update this
ColPaul
Posts: 36
Joined: Tue Dec 31, 2019 11:24 am

Re: Calculating checksums? Can't get OS to work

Post by ColPaul »

jlvaldez, did you update the script for P59? I downloaded the version 0.1 from this thread, but still see the hard-coded memory ranges. If not, I'll start working on modifying the script for P59 files.
ColPaul
Posts: 36
Joined: Tue Dec 31, 2019 11:24 am

Re: Calculating checksums? Can't get OS to work

Post by ColPaul »

jlvaldez, I made a quick hack to your script by reinitializing p59_segments in the gen3_checksum.__init__ function. Below are the lines that I added right after self.p59 = True. I also modified the header that gets printed to that everyone can tell which version is being run at runtime. The modified file is attached. Thanks for this vey useful piece of code!

p59_segments.engine_cal = [self.read_uint32(0x514),self.read_uint32(0x514)+2,self.read_uint32(0x518)]
p59_segments.engine_diag = [self.read_uint32(0x51c),self.read_uint32(0x51c)+2,self.read_uint32(0x520)]
p59_segments.trans_cal = [self.read_uint32(0x524),self.read_uint32(0x524)+2,self.read_uint32(0x528)]
p59_segments.trans_diag = [self.read_uint32(0x52c),self.read_uint32(0x52c)+2,self.read_uint32(0x530)]
p59_segments.fuel_system = [self.read_uint32(0x534),self.read_uint32(0x534)+2,self.read_uint32(0x538)]
p59_segments.system = [self.read_uint32(0x53c),self.read_uint32(0x53c)+2,self.read_uint32(0x540)]
p59_segments.speedometer = [self.read_uint32(0x544),self.read_uint32(0x544)+2,self.read_uint32(0x548)]
Attachments
checksum.py.zip
(3.46 KiB) Downloaded 206 times
jlvaldez
Posts: 155
Joined: Mon Feb 11, 2019 12:48 pm
cars: '01 - Corvette Z06
'20 - Sierra Denali
'03 - Volvo S80 T6
'16 - Accord V6
Location: DFW, Texas

Re: Calculating checksums? Can't get OS to work

Post by jlvaldez »

Awesome! Thanks for doing this. Sorry I have had some personal issues to deal with the last several months and haven't been around as much :(
Post Reply