CRC's..Checksums.. Reverse Engineering!

[Tazzi]

Yeah, I think you've nailed the CVN thing, and that shows it is a 16bit checksum. I dont think they'd go 32bit as its an embedded system and to calculate 32bits and throw half away would waste cpu time, which I dont think would be justified. I did have the LS1 checksum algo figured out so I just went and checked my notes (its just a regular sum). The catch there is that the OS checksum skips the saved checksum bytes, so its not a contiguous block. Perhaps the E38 checksum is skipping the CVN bytes, or something like that. Since were looking at sums i'll post that up in a new thread.

Code: Select all

for ( UINT ui = pCalcInfo->dwRegionStartAddress; ui <= pCalcInfo->dwRegionEndAddress; ui+=2 ) {
    // implement os hack jumps, real start and end are configured in xdf, this just jumps unchecked parts
    if (os_hack && ui==0x500) { ui=0x502; OutputDebugString("Skip 0x500->0x501");}
    if (os_hack && ui==0x4000) { ui=0x20000; OutputDebugString("Skip 0x4000->0x1FFFF");}
    temp=pCalcInfo->pBaseData[ui]<<8;
    temp+=pCalcInfo->pBaseData[ui+1];
    wChecksum -= temp;
}

When your calculating CRC16, are you reading the file 16 bits at at time? Have you tryed swapping the 8 bit halves?

I think it would help to map out the E38 segments as they are on disk. Do you have that info to post up?

Nah, dont have that info yet, all I know is the engine section and its checksum locations. I havent investigated any further than that!
Im doing 8,16, and 32bit reads at a time. I can see the logic in it being a waste of time.. just depends on how secure they decided to make it!

[antus]

For the record Ive just posted the ls1 dll up here: http://pcmhacking.net/forums/viewtopic.php?f=3&t=3845

Whats the start and end addresses of the engine segment?

[Tazzi]

Checksum 1 = 1CD974 & 1CD975
Checksum 2 = 1CD992 & 1CD993
And the data section = 1CD976 to 1FFFFF

Iv *skipped* through each part.. roughly every 0x500 and changed a byte to see if the checksum changed, and it did. There are some large 00 and FF sections which may/maynot be apart of it?
If they aren't, could be a massive deal depending on the CRC type since some are affected by even 00's! All dependings on the XOR at the end as well for each byte I guess

If the above is the case... will have to find a much faster way of checking..

[antus]

Code: Select all

0x00010000 | 04 A0 00 01 20 00 C0 73 1D 41 41 00 00 00 31 32 | .... ..s.AA...12
0x00010010 | 36 31 32 33 38 31 00 00 00 00 00 00 00 00 50 D6 | 612381........P.
0x00010020 | 00 01 02 01 00 01 00 00 00 1B FF FF 00 01 00 02 | ................
0x00010030 | 00 01 00 1D 00 01 00 20 00 1B FF FF 00 01 00 02 | ....... ........
0x00010040 | 00 1B FF FF 05 01 02 01 00 1C 00 00 00 1C 12 73 | ...............s
0x00010050 | 00 1C 00 02 00 1C 00 1D 00 1C 00 20 00 1C 12 73 | ........... ...s
0x00010060 | 00 1C 00 02 00 1C 12 73 01 02 01 00 1C 12 74 00 | .......s......t.
0x00010070 | 1C 33 DB 00 1C 12 76 00 1C 12 91 00 1C 12 94 00 | .3....v.........
0x00010080 | 1C 33 DB 00 1C 12 76 00 1C 33 DB 01 02 01 00 1C | .3....v..3......
0x00010090 | 33 DC 00 1C 37 0B 00 1C 33 DE 00 1C 33 F9 00 1C | 3...7...3...3...
0x000100A0 | 33 FC 00 1C 37 0B 00 1C 33 DE 00 1C 37 0B 01 02 | 3...7...3...7...
0x000100B0 | 01 00 1C 37 0C 00 1C D9 73 00 1C 37 0E 00 1C 37 | ...7....s..7...7
0x000100C0 | 29 00 1C 37 2C 00 1C D9 73 00 1C 37 0E 00 1C D9 | )..7,...s..7....
0x000100D0 | 73 01 02 01 00 1C D9 74 00 1F FF FF 00 1C D9 76 | s......t.......v
0x000100E0 | 00 1C D9 91 00 1C D9 94 00 1F FF FF 00 1C D9 76 | ...............v

Interesting, couple of references to "00 1F FF FF 00 1C D9 76" at the end there. Perhaps the segments are mapped from 0x10000, perhaps the skip blocks as well.

[yoda69]

The 12612381 is the operating system ID starting at 1000E.
There are also sectionsthat look like possible references to other areas. Think you might be onto something, may be entry exit points?
Might be time to go an play with the hex editor and excel again.... And probably a lot of this

[antus]

Tazzi, have you tried calculating from the end back to the start instead of forwards (eg 1FFFFFF through to 1CD976)? I would expect 00 and FF to make a difference, and the order of processing the block too. If the data above is a map, and in order of execution, it suggests maybe backwards is the way to go.

[yoda69]

Looking at the code Antus provided above, when I broke it up I got the following.
NOTE: I'm not sure about the unknown and just guessing at the Start/End headings, but it kind of makes sense.

Code: Select all

Hex addr		Unknown		Start		        End
10021		    010201				
10024				                00010000		001BFFFF
1002C				                00010002		0001001D
10034				                00010020		001BFFFF
1003C				                00010002		001BFFFF
10044		    05010201				
10048				                001C0000		001C1273
10050				                001C0002		001C001D
10058				                001C0020		001C1273
10060				                001C0002		001C1273
10068		    010201				
1006B				                001C1274		001C33DB
10073				                001C1276		001C1291
1007B				                001C1294		001C33DB
10083				                001C1276		001C33DB
1008B		    010201				
1008E				                001C33DC		001C370B
10096				                001C33DE		001C33F9
1009E				                001C33FC		001C370B
100A6				                001C33DE		001C370B
100AE		    010201				
100B1				                001C370C		001CD973
100B9				                001C370E		001C3729
100C1				                001C372C		001CD973
100C9				                001C370E		001CD973
100D1		    010201				
100D4				                001CD974		001FFFFF
100DC				                001CD976		001CD991
100E4				                001CD994		001FFFFF
100EC				                001CD976		001FFFFF

[Tazzi]

Tazzi, have you tried calculating from the end back to the start instead of forwards (eg 1FFFFFF through to 1CD976)? I would expect 00 and FF to make a difference, and the order of processing the block too. If the data above is a map, and in order of execution, it suggests maybe backwards is the way to go.

ohhhhh...... I didnt think of that one..

Backwards will have to be tomorrows job. Still running the forward iterations

[antus]

I think those tables are the ticket. Im looking at the speedometer segment because its small, and thats:

Code: Select all

1008B          010201            
1008E                            001C33DC      001C370B
10096                            001C33DE      001C33F9
1009E                            001C33FC      001C370B
100A6                            001C33DE      001C370B

So the left column yoda has called 'unknown'. They are all 01 02 01, except 05 01 02 01. I'd suggest that this is the operating system segment. The others would be the calibration segments.

The next row (i'll call it A) is the complete segment.
The second row (B) is the first block to sum (skips the 1st 2 bytes of checksum (CS), ends before the CVN).
The third row (C) skips the CVN and picks up just after and ends at segment end.
The last row (D) is the whole segment, minus the CS.

So I reckon we need to sum up areas B and C (which will start after the CS, skip the CVN and continue to segment end).

These values change from one bin to the next, so I think we need to use this index table when loading the bins. I suspect these values are actually used by the bins internal checksum routines, though Ive not identified those yet.

Its guesswork at this stage, but it seems likely. We still need a direction (forward or backwards)... probably forwards now we've layed it out more clearly... and an algo. Im hoping CRC16 or CRC16-CCITT. Once we can match the CS, we can use tazzi's algo above to calculate a CVN for it.

[Tazzi]

Iv got another lappy processing backwards now. Will probably be all done within a couple days hopefully if there are no crazy curve balls.!

What Yoda posted doesnt quite add up.. since I know the byte after the cvn does not be used in the checksum.. changing it does not seem to affect the checksums what so ever. They look like a starting point though!