CRC's..Checksums.. Reverse Engineering!

[Tazzi]

Well.. Iv ran through every polynomial and starting value possible... using each byte. Although this turned up nothing.
I would assume.. for the sake or reliability.. GM would have used the standard polynomial and a starting of 0000 or FFFF as using other values can be "unstable" from what I have read! I do believe the answer could be reflecting the value before/after the process, or swapping the bytes before/after

Next things to test out:
-Reflect the bytes going in/out of the CRC process.
-Processing 16bit bytes instead of 8bit
-Process 32bit bytes
-Attempt to check if there is a common "add/minus" at the end.

If anyone has suggestions.. let me know!

[Tazzi]

Alright! Converted the bin to 00's.. then made the checksum recalc... and when it is all 00's, both checksums go to 00 as well.

When changing 1 byte to 01, this causes the checksums to then change to a proper value!.
So... I then changed the next byte to 01 and changed the previous byte to 00. And then checksum was different! I thought, it would be the same.. but it seems that the 00's are actually affecting the final checksum.

Soooo.. a standard CRC16 calc does not change like that... buuuuuut these CRC's do change by changing where the 01 is located:
CRC-CCITT (0xFFFF) 0x8CE2
CRC-CCITT (0x1D0F) 0xA488
CRC-CCITT (Kermit) 0x8911

[Tazzi]

Alright! Starting to see a pattern.. I think..

So I know for certain how checksum2 (CVN) is being made. It is simply (FFFF-checksum1)+TotalBin - 1.

As for checksum1. Pretty sure it is a crc16 ccitt checksum. Im starting to see patterns! Well.. I think I am.. only so long one man can stare at binary! Seen below:

*Edit
Nope.. completely off course there. I rotated and shifted so much that it was all turning constant *dough*

[Tazzi]

*Sigh*.. been looking at it all wrong... 43hours later I think Iv made some proper progress!

[yoda69]

CRC's are getting out of my league.
At least using Excel can setup For/Next loops, HEX/BIN/DEC, etc and try and automate a brute force method to try and crack them. Biggest issue is the size of late model bins being so big makes it a huge data crunching exercise.
Good luck with it, know the feeling dreaming about hex, binary and excel formulas!!

[antus]

Ive been sitting here this weekend trying to figure this one out too. I thought it should be straight forward enough but everything I have tried so far has not found the answer. Even if we could identify the code in the bin which turns on the CEL on CRC fail at bootup, but with little known about those bins to date thats not helped either.

[Tazzi]

CRC's are getting out of my league.
At least using Excel can setup For/Next loops, HEX/BIN/DEC, etc and try and automate a brute force method to try and crack them. Biggest issue is the size of late model bins being so big makes it a huge data crunching exercise.
Good luck with it, know the feeling dreaming about hex, binary and excel formulas!!

Yeah, Im currently running three applciations on three computers which are testing every polynomial, reflect-in,reflect-out,initial value and xor final value
Slowly but surely making progress... after reading up on every possible reverse engineering essay and documentation I could find.. its finally making sense!

But yeah, its the bin size that slows it down drastically, since it needs to process 20,000+ bytes each time! Bin size isnt too much of a matter, since it seems each section has its own checksum thus ends up being small"er" chunks anyways.

[Tazzi]

Ive been sitting here this weekend trying to figure this one out too. I thought it should be straight forward enough but everything I have tried so far has not found the answer. Even if we could identify the code in the bin which turns on the CEL on CRC fail at bootup, but with little known about those bins to date thats not helped either.

Same here mate. I literally didnt knock off to bed until 4am this morning staring at my excel sheet.. feels like the movie inception and Im seeing patterns inside of patterns inside of patterns! And those patterns have patterns!
Im sure this is being way overthought, but will get there.

[Tazzi]

Nope, thought I was onto something.. but just keeps escaping me. Starting to wonder if it is a CRC32 checksum.. and only the first 16bits are taken? Maaaaaybe?

[antus]

Yeah, I think you've nailed the CVN thing, and that shows it is a 16bit checksum. I dont think they'd go 32bit as its an embedded system and to calculate 32bits and throw half away would waste cpu time, which I dont think would be justified. I did have the LS1 checksum algo figured out so I just went and checked my notes (its just a regular sum). The catch there is that the OS checksum skips the saved checksum bytes, so its not a contiguous block. Perhaps the E38 checksum is skipping the CVN bytes, or something like that. Since were looking at sums i'll post that up in a new thread.

Code: Select all

for ( UINT ui = pCalcInfo->dwRegionStartAddress; ui <= pCalcInfo->dwRegionEndAddress; ui+=2 ) {
    // implement os hack jumps, real start and end are configured in xdf, this just jumps unchecked parts
    if (os_hack && ui==0x500) { ui=0x502; OutputDebugString("Skip 0x500->0x501");}
    if (os_hack && ui==0x4000) { ui=0x20000; OutputDebugString("Skip 0x4000->0x1FFFF");}
    temp=pCalcInfo->pBaseData[ui]<<8;
    temp+=pCalcInfo->pBaseData[ui+1];
    wChecksum -= temp;
}

When your calculating CRC16, are you reading the file 16 bits at at time? Have you tryed swapping the 8 bit halves?

I think it would help to map out the E38 segments as they are on disk. Do you have that info to post up?