Segment Swap utility

They go by many names, P01, P59, VPW, '0411 etc. Also covering E38 and newer here.
Post Reply
User avatar
joukoy
Posts: 392
Joined: Tue Dec 17, 2019 3:27 am
cars: Pontiac Firebird 1978

Re: Segment Swap utility

Post by joukoy »

joukoy wrote:
joukoy wrote:https://github.com/joukoy/UniversalPatcher

Quick 'n dirty copy from PCMBinBuilder, all extra removed.
Version 0.3.0 includes basic functionality for excluding blocks.
0.4.0
Support for .ini file (exclusion blocks) loading as command line parameter
Small fixes
kur4o
Posts: 948
Joined: Sun Apr 10, 2016 9:20 pm

Re: Segment Swap utility

Post by kur4o »

The universal patcher works flawless. I test a random bin and it was patched without an issue. To make it even more universal maybe some form of checksums can be implemented, the checksum can be updated after the patch is applied and can be user specified. Some config files can be generated and linked with the patch. Or a separate checksum function can be added. The checksum can be configurable like start-end address with skips added between start and end address, with numerous checksums areas available in a file.
The checksum type can be selected between sum, 1`s complement sum, 2`s complement sum. The sum type can be add byte, add word, add dword. And store location and store location width[byte,word,dword].
The other type of checksum can be crc16 with byte swapped and crc32.

It turns out that gm uses different algorithms for CVN. The ls1 types of files use crc-16 byte swapped, and the result is not stored at the bin.
Some 07-11 files use crc32 CVN, and the later file 12-17 uses cvn crc16 byte swapped but the result is stored in the segment, The 12-17 CVNs calculation skip the first 2bytes and the cvn location.

There is also one gm checksum algorithm i haven`t figure how it is done. This code seems to do it but I can`t decoded it.

Code: Select all

NewChecksum += (BinaryFile(i) * 256) + BinaryFile(i + 1)
Next i
Is it some form of crc or some sum.


roughneck427
posted some bin archive but all files have some messed up eeprom area. Maybe it is time to add some exceptions for extracting bins with bad eeprom data.

It will be nice to add the cvns to the bin file info or even to the segment name. That way any one can check for sure if the segment is stock or not and non stock duplicate segments can be easily identified. There is also gm site that can verify cvn vs p/n for the 02-03+ BINS.

The ls1 checksum algorithm for segments is used upto 2017 for almost any bins and modules I have checked. Maybe we can used that for the universal patcher and add universal checksums too.
roughneck427
Posts: 49
Joined: Thu Feb 07, 2013 1:49 pm
cars: 2002 Camaro
2008 Pontiac G8

Re: Segment Swap utility

Post by roughneck427 »

Yes the eeprom area has no vin or serial as a neutral generic bin. My conversion tool guts that area out. I have about 3000+ files that I have personally read and or modded from over the years using Tunercat OBD2
User avatar
joukoy
Posts: 392
Joined: Tue Dec 17, 2019 3:27 am
cars: Pontiac Firebird 1978

Re: Segment Swap utility

Post by joukoy »

kur4o wrote:The universal patcher works flawless. I test a random bin and it was patched without an issue. To make it even more universal maybe some form of checksums can be implemented, the checksum can be updated after the patch is applied and can be user specified. Some config files can be generated and linked with the patch. Or a separate checksum function can be added. The checksum can be configurable like start-end address with skips added between start and end address, with numerous checksums areas available in a file.
The checksum type can be selected between sum, 1`s complement sum, 2`s complement sum. The sum type can be add byte, add word, add dword. And store location and store location width[byte,word,dword].
The other type of checksum can be crc16 with byte swapped and crc32.

It turns out that gm uses different algorithms for CVN. The ls1 types of files use crc-16 byte swapped, and the result is not stored at the bin.
Some 07-11 files use crc32 CVN, and the later file 12-17 uses cvn crc16 byte swapped but the result is stored in the segment, The 12-17 CVNs calculation skip the first 2bytes and the cvn location.

There is also one gm checksum algorithm i haven`t figure how it is done. This code seems to do it but I can`t decoded it.

Code: Select all

NewChecksum += (BinaryFile(i) * 256) + BinaryFile(i + 1)
Next i
Is it some form of crc or some sum.

Uploaded UniversalPatcher-Beta.exe + example config: OS-12587603.xml
This is NOT functional yet, checksum calculations fail and so on.
But:
When progam start, click "Segments" => Load, select example file.
Then load modifier file => Segments are checked based on config.

You can also add segments and save to file, but can't modify or delete yet.

Waiting for feedback, how should I continue, or abandon this completely?

(I try to implement calculation from your code snippet: Invert SUM, don't know is it even close to real checksum calculation)

Edit: fixed checksum calculation for gen3 LS files, still not so functional.
kur4o
Posts: 948
Joined: Sun Apr 10, 2016 9:20 pm

Re: Segment Swap utility

Post by kur4o »

It is getting some nice shape for configuration. I guess the calculations are not yet implemented. Maybe you can add some options for byte sum, word sum and dword sum, and a separate settings for 1`s and 2`s compliment.

Can we rip segments` data addresses from bin. Usually the first 2 bytes of the start addresses are the checksum. That way a single config file can support all OS with the variable segment length. For some of the checksum you can hard build them than make them configurable, like ls1 checksum, e38 checksum, blackbox checksum and so on. I will get you some examples.


That inverted checksum is used on v6 pcms, It is really weird. Here is the full code
CHEKCSUMS_V6_FWD.rtf
(70 KiB) Downloaded 258 times
from a open source program called tiny tuner. I hope it does make some sense to you.
User avatar
antus
Site Admin
Posts: 8237
Joined: Sat Feb 28, 2009 8:34 pm
cars: TX Gemini 2L Twincam
TX Gemini SR20 18psi
Datsun 1200 Ute
Subaru Blitzen '06 EZ30 4th gen, 3.0R Spec B
Contact:

Re: Segment Swap utility

Post by antus »

Im not sure I understand this question

There is also one gm checksum algorithm i haven`t figure how it is done. This code seems to do it but I can`t decoded it.

Code: Select all

NewChecksum += (BinaryFile(i) * 256) + BinaryFile(i + 1)
Next i
Is it some form of crc or some sum.
Its a 16bit sum, but was that the question? Or is it because i steps by 1 byte only?
Have you read the FAQ? For lots of information and links to significant threads see here: http://pcmhacking.net/forums/viewtopic.php?f=7&t=1396
kur4o
Posts: 948
Joined: Sun Apr 10, 2016 9:20 pm

Re: Segment Swap utility

Post by kur4o »

antus wrote:Im not sure I understand this question

Its a 16bit sum, but was that the question? Or is it because i steps by 1 byte only?
Since I am not that good with C++ I am looking for someone to translate the C++ instructions to human readable math.
User avatar
antus
Site Admin
Posts: 8237
Joined: Sat Feb 28, 2009 8:34 pm
cars: TX Gemini 2L Twincam
TX Gemini SR20 18psi
Datsun 1200 Ute
Subaru Blitzen '06 EZ30 4th gen, 3.0R Spec B
Contact:

Re: Segment Swap utility

Post by antus »

OK. Read each pair of bytes, add to a 16bit some. move on.

BinaryFile(i) is the upcoming 16 bits
* 256 moves that byte to the upper 8 bytes (256 is 8 bits)

say were summing:

11 22

then i points to 11

11*256=
1100

(In C we'd use 11 << 8 for the same thing)

then i+1 points to 22

then summing them together gives us a 16 bit value of 11 22

Then I think just from the snippet that it increments i by one, and i expect another one in an outer loop that I cant see, so its then ready to sum up the next 16 bits.

Another way to do it would be to read a 16 bit value in one hit, then step by 2 bytes. many ways you could write it.

i points to 1122
Have you read the FAQ? For lots of information and links to significant threads see here: http://pcmhacking.net/forums/viewtopic.php?f=7&t=1396
User avatar
joukoy
Posts: 392
Joined: Tue Dec 17, 2019 3:27 am
cars: Pontiac Firebird 1978

Re: Segment Swap utility

Post by joukoy »

kur4o wrote:It is getting some nice shape for configuration. I guess the calculations are not yet implemented. Maybe you can add some options for byte sum, word sum and dword sum, and a separate settings for 1`s and 2`s compliment.

Can we rip segments` data addresses from bin. Usually the first 2 bytes of the start addresses are the checksum. That way a single config file can support all OS with the variable segment length. For some of the checksum you can hard build them than make them configurable, like ls1 checksum, e38 checksum, blackbox checksum and so on. I will get you some examples.


That inverted checksum is used on v6 pcms, It is really weird. Here is the full code
CHEKCSUMS_V6_FWD.rtf
from a open source program called tiny tuner. I hope it does make some sense to you.
Program can be totally universal or specific to GM.
If it try to be both, it will be complicated.

Now I am not sure what I should do:
- Continue developing universal patcher
* make it totally universal, or
* Make GM specific configurations (ls1 checksum, eeprom information etc)
- Abandon universal patcher and make PCMBinBuilder configurable:
* Config file defines:
+ where is segment addresses stored
+ where is segment info stored & how (bytes, hex/uint,text)
+ where is checksum stored & how (bytes, hex/uint,text[Display])
+ checksum method
+ Does eeprom_data need config?
+ other?

For config, I already tested this style: textbox where you can supply segment addressess (example: 8000-9000)
If you supply @514, program will read address from bin at address 514 (4 bytes for start, 4 for end address)
If you sypply #0:2:hex in checksum address, checksum is 0 bytes from segment start address, length 2 bytes, show as hex
(@ = read from bin at address, #= address is relative to segment address)

Comments, please.

If I understand V6 checksum correctly, it is word sum, read from 4 blocks, stored in 1 byte
kur4o
Posts: 948
Joined: Sun Apr 10, 2016 9:20 pm

Re: Segment Swap utility

Post by kur4o »

I worked on the newer pcm checksums. the config is a little different, but similar.

Here is a description and a bin from e38 pcm. Other pcms with 2mb flash are almost identical. I am still trying to find an early 4mb bin to look at.
e38_bin_decode.txt
(1.93 KiB) Downloaded 243 times
escalade_2007.bin
(2 MiB) Downloaded 245 times
The universal approach will be best. The map you made is good I hope it can cover all situations.

The e38 have 2 checksum, a sum and byte swapped crc-16[cvn]. The crc is done first, the sum checksum and the crc checksum are omitted from calculation. Usually the cvn is stored at start of segment address +$1e or +$20[on 4mb bins]. The sum is always at the start of segment[first 2 bytes].

Here is an example how to decode the segment info on 2mb files

Code: Select all

D3 06 checksum
81 02 [4th byte is numerical segment number 01-08, 3rd byte varies]
20 -the 5th byte is always 20[can be used for identification]
00 C0 7A 59 [segment id in hex]
41 42 [version number in ascii]
FF 00 00 31 32 36 31 34 32 33 33 00 00 00 00 00 00 00 00 
E6 1B [cvn]


eeprom data can be used for identification but I am still too far from there. It is not used for checksums at all.

On the config file, only calibration segments have varying start-end addresses. These are a must to be taken from bin.

For config, I already tested this style: textbox where you can supply segment addressess (example: 8000-9000)
If you supply @514, program will read address from bin at address 514 (4 bytes for start, 4 for end address)
If you sypply #0:2:hex in checksum address, checksum is 0 bytes from segment start address, length 2 bytes, show as hex
(@ = read from bin at address, #= address is relative to segment address)
This looks like it will work for most of the pre can pcms. Since they are configured just like the ls1 files.
I am eager to test it with some other files when done.
Post Reply