CANBUS log reading

[fastboatster]

Today I was attempting to gather some debug info by logging the bus traffing while testing v0004. And, on my first attempt while logging bus traffic, it read the bin successfully!

Code: Select all

initialising Tool J2534 PassThru
Bosch - MDI...
ok.
connecting...
start diagnostic session (0x10 0x03)...ok.
disable normal communication (0x28)...ok.
security access (0x27)...
	 request seed...NOT ok, no valid seed received.
	 Wait 30s after power on or failed attempt.
connecting...
start diagnostic session (0x10 0x03)...ok.
disable normal communication (0x28)...ok.
security access (0x27)...
	 request seed...ok.
	 seed: 	 0x0000335F
	 key: 	 0x000075A3
	 sending key...ok.
request programming mode...ok.
sending enable programming mode...done.
downloading prog...
busy.
ok.
run prog...ok.
ready to upload flash or download calibration.
upload flash...
upload progress: 4% (0x010000)
upload progress: 7% (0x020000)
upload progress: 10% (0x030000)
upload progress: 13% (0x040000)
upload progress: 16% (0x050000)
upload progress: 19% (0x060000)
upload progress: 22% (0x070000)
upload progress: 25% (0x080000)
upload progress: 29% (0x090000)
upload progress: 32% (0x0A0000)
upload progress: 35% (0x0B0000)
upload progress: 38% (0x0C0000)
upload progress: 41% (0x0D0000)
upload progress: 44% (0x0E0000)
upload progress: 47% (0x0F0000)
upload progress: 50% (0x100000)
upload progress: 54% (0x110000)
upload progress: 57% (0x120000)
upload progress: 60% (0x130000)
upload progress: 63% (0x140000)
upload progress: 66% (0x150000)
upload progress: 69% (0x160000)
upload progress: 72% (0x170000)
upload progress: 75% (0x180000)
upload progress: 79% (0x190000)
upload progress: 82% (0x1A0000)
upload progress: 85% (0x1B0000)
upload progress: 88% (0x1C0000)
upload progress: 91% (0x1D0000)
upload progress: 94% (0x1E0000)
upload progress: 97% (0x1F0000)
upload progress: 100% (0x200000)
done.
validating upload...ok, valid.

pcmflasher0004-2.log
pcmflasher0004-2.bin

The log is AVT Format, so the first byte, or two bytes depending on the length of the packet are the length. If you understand the can packets you will be able to see where they start. It was logged on the bus.

So I was just looking at the log in this post. I'm somewhat unfamiliar with GMLAN protocol, so did I understand it correctly that the flashloader (0x14e0 bytes, in service id 0x34) gets loaded to 0x3fb800 RAM address (message with sid 0x36)

--> 0B 00 07 E0 05 34 00 00 14 E0 00 00
--> 0B 00 07 E8 03 7F 34 78 AA AA AA AA
--> 05 00 07 E8 01 74
--> 0B 00 07 E0 14 06 36 00 00 3F B8 00

and then the routine at that 0x3fb800 address gets executed:

--> 0B 00 07 E0 06 36 80 00 3F B8 00 00

I'm not quite clear what 0x14 in 0B 00 07 E0 14 06 36 00 00 3F B8 00 does, doesn't look like the number of bytes in the message, is that a response code?

[antus]

0B = 0x0B (11 bytes incoming from AVT - AVT protocol)
CAN packet:
00 07 E0 06 = Address information on the CAN bus
36 80 = 36 Upload 80 Execute bit set
00 3F B8 00 00 = at address 3F B8 00 00

I'll do some reading on the other one, still coming up to speed with this myself. Will post an answer back shortly unless someone else gets there first.

[fastboatster]

0B = 0x0B (11 bytes incoming from AVT - AVT protocol)
CAN packet:
00 07 E0 06 = Address information on the CAN bus
36 80 = 36 Upload 80 Execute bit set
00 3F B8 00 00 = at address 3F B8 00 00

For the other one:
07 E0 14 06 I think.. is Destination 7E0, Source 1406.
0B is AVT Protocol packet length = 11, the rest is the CAN packet.

Anyone with more or better information please correct me about the address info. I have not gone anywhere near as deep with CAN on the wire as I have with VPW. I know the payload is the same, but the addressing is different bit lengths and I am not certain how the AVT handles it - if its doing magic to pull bits off the wire and pad the data so that we can read it with 8 bit bytes.

hmm, 3F B8 00 00 or 0x3FB80000 is not a valid RAM address, is it? mpc561 manual states that

The internal memory map is organized as a single 4-Mbyte block. The user can assign this block to one of
eight locations by programming a register in the USIU (IMMR[ISB]).

and what I see at https://www.nxp.com/docs/en/data-sheet/MPC561RM.pdf on the page 1-11 (96) is that it only goes up to 0x01FFFFFF.
Anyhow, I was primarily not sure about 0x14 0x06 bytes function.

[antus]

Yes you are right. Looking more closely at your capture, the request was for address 3F B8 00, the response has an extra byte on the end which I copied, but the address must be 3FB800

[antus]

Edit: leaving this for context, but see ironduke's post below for accuracy about 1 406 NOT 14 06.

I think its to do with multi-frame and length. From earlier:

Code: Select all

0B 00 07 E8 03 7F 34 78 AA AA AA AA
            03 = Length 
               01 02 03
                        PADDING AA
End

Then

Code: Select all

0B 00 07 E0 14 06 36 00 00 3F B8 00
            14=multi frame as defined in ISO 15765-2?
               06=6 bytes
                  01 02 03 04 05 06
(further frames contain the rest of the message eg the 36/80 payload)

Still trying to understand the multi frame part...

[ironduke]

ok, just getting on here.. it's kind of weird not seeing 4 zeros in the beginning of each message.. no idea if that's normal and I just read 00 00 in the beginning with all my logs.. Edited.. Looks like it's a length of message??

as far as 07 E0 14 06 36 00 00 3F B8 00 message..
07 e0 is the address to talk to the ecm.
1 is indicative of a 10 which means it's a multiline message.
406 is the chunk, or size your about to send.. We're using the 0 from the multiline command. I'm sure there's an easier way to explain it??
Notice that 36 00 00 3F B8 00 is 6 bytes right there.. So your actually sending 400 bytes of the kernel..
36 is the data transfer command.
the 00 right after the 36 can be either 00 or 80... 80 means execute.. You'll see at the end it send a 36 with an execute command.
The next 4 bytes is the address it's sending too.. You'll see that address changes in chunks of whatever is being sent, in this case 0x400 bytes at a time. Sometimes the address increment, sometimes other software decrements it but it is stacked the same in the end.

After the 36 command you'll see the 76 ack and then the data is sent with a line number at the beginning of each line, starts at 0x21 to 0x2f, then 0x20 and it just scrolls and scrolls for the entire segment.
That enough of a starter?

btw.. Antus.. If I start a topic named something like CANbus log reading can we push this over to that.. I feel bad hijacking this entire post the way that I am..

[ironduke]

I'm just starting this topic to try and clean up another one a little bit.. Might be a bit messy in the beginning but it should work out better in the end.. I am by no means an expert but I have learned a fair amount here and I'm just trying to get others up to my speed.. Not saying I'm smart, just that this might help some of you. To everyone smarter than I am feel free to correct me as needed, some of the things I might be lying about a little bit just because I feel I can explain it easier than overloading my brain trying to get it 100% factual and I was not taught by a school, or a class, just parsed from some documents I found, a LOT of trial and error and a few people here that have been very helpful, not going to name any names for fear I would forget someone.

Not sure where to start, something basic maybe?
Not everything will be 100%, this is 99% off the cuff from memory, I am horrible at taking notes. This is about breaking down the log a little, not trying to send anything just yet.

address 07E0 is scan tool talking specifically to the ECM
address 7DF is bus wide command, all modules will reply I believe.
address 07E8 is a reply from the ECM
101FE address us sent using extended addressing.. This is a little different than 7DF address. This is usually sent with the tool present command but also with commands like a 1abo request. not all modules will respond, they'll respond if they have a reply..

The 101fe013e command is the tool present command but the modules don't respond to this one. if I were to send a 7E0 01 3E command then the ecm will reply. This can throw a hicuup writing your own code if you don't know this. I do not know if this is every module but it happened to me on most so I learned how to handle it.

Unless specified everything is CAN RAW. meaning it is as it shows on the data bus. This will only apply later if you decide to try and send anything yourself. I learned using CAN RAW instead of iso16765. That might be easier to use as it does the length byte(s) for you along with reading multiline messages a lot easier. I'm learning it and can send with it but it's harder for me to correct my code accordingly since I think I do many things the 'wrong' way round, lol.

Some simple messages. I'm just removing the first 5 zeros.. I have only ever seen zero's so I just cropped them out for ease.

101FE021AB000000000 // 1AB0 command from scan tool. basically pinging canbus for programmable modules.
7E8035AB011AAAAAAAA // 7E8 is the address that is replying, 03 is message length. 5AB0 is the reply of the 1AB0 command. 0x40 is added to the first byte as a way to validate the response 0x11 is ECM DID. Been awhile but short for DataIDentifier I think.
7E8 is the ecu address
03 is message length
5AB0 is ack of the 1AB0 request
11 is the ecu DID

ok first multiline..
7E0021A90 // This is a vin request command. 7E0 is asking ONLY the ECM, 02 is message length and 1A 90 is vin request.
1A is read data by identifier and 90 is the vin.

7E810135A9031474352 //7E8 is ECM replying, 0x10 is multiline, 13 is number of bytes in hex. 5A90 is response to 1A90.
7E03000000000000000 // this is the tool ack to say it saw the first and now send the rest.
7E8214353453058445A
7E822323139383330AA

So with the first line you see the 10 so you know it's multiline. Since it's less than 0x100 bytes being sent you see it as a 10.
This will be different when your sending multiline messages greater than 0x100 bytes.
If you start counting bytes after the 0x13 you'll end up with 19 bytes. 2 bytes are the 5A90 and the test is the vin. Just copy and paste that into a hex to ascii decoder and you'll see the vin..

This is just a single line request and response, this is an OS request.
7E0021AC1 // just 2 byte message 1A C1
7E8065AC100C1146AAA
7E8 is the ECM replying
06 is the number of bytes in the message
5AC1 is the request with 0x40 added to it for validation. that's 2 bytes so we only use the next 4 bytes
00C1146A is in hex, put that in your calculator in hex and you get 12653674 which is the OS of a 2013 E38 ecm for a truck.
Note that the last byte AA is ignored.. sometimes you'll see AA's sometimes you'll see remnants of previous messages, etc.. You read the amount of bytes specified in the length byte and not more..

What else do you guys want to see?? down the road for those who want to try and write some code to talk to a J2534 device should I work on some sort of primer or quick write up to get started? My old brute force code is up on github, I don't think the newest code is online, I could start with that and explain it out a little bit. If you can start with that and expand you'll learn simple things like a vin write utility.. I have some time in the winter and that's going to end in a couple months and then I'll be busy with life.. This is just a hobby for me, none of this is related that much to my day job except for actually working on vehicles and writing factory calibrations in, no custom work like this.
Hopefully antus can just move some posts over her from another topic, I kinda hijacked one trying to help but it went off track a bit from the original.. Once he's done that we can keep on if desired..
Thanks to everyone here who has helped get motivated and started, hopefully I can pay i forward a little..

[fastboatster]

ok, just getting on here.. it's kind of weird not seeing 4 zeros in the beginning of each message.. no idea if that's normal and I just read 00 00 in the beginning with all my logs.. Edited.. Looks like it's a length of message??

as far as 07 E0 14 06 36 00 00 3F B8 00 message..
07 e0 is the address to talk to the ecm.
1 is indicative of a 10 which means it's a multiline message.
406 is the chunk, or size your about to send.. We're using the 0 from the multiline command. I'm sure there's an easier way to explain it??
Notice that 36 00 00 3F B8 00 is 6 bytes right there.. So your actually sending 400 bytes of the kernel..
36 is the data transfer command.
the 00 right after the 36 can be either 00 or 80... 80 means execute.. You'll see at the end it send a 36 with an execute command.
The next 4 bytes is the address it's sending too.. You'll see that address changes in chunks of whatever is being sent, in this case 0x400 bytes at a time. Sometimes the address increment, sometimes other software decrements it but it is stacked the same in the end.

After the 36 command you'll see the 76 ack and then the data is sent with a line number at the beginning of each line, starts at 0x21 to 0x2f, then 0x20 and it just scrolls and scrolls for the entire segment.
That enough of a starter?

btw.. Antus.. If I start a topic named something like CANbus log reading can we push this over to that.. I feel bad hijacking this entire post the way that I am..

thank you, this is very useful information, especially the multipart 0x36 message structure. The line numbering was unusual since it didn't start from 0x00 or 0x01 but wasn't that hard to figure out.
I guess since the OP of the original thread didn't want people to "look into" his app and the flashloader, we shouldn't discuss what happens next after CAN message exchange when the flashloader gets loaded and executed.

[ironduke]

ok, just getting on here.. it's kind of weird not seeing 4 zeros in the beginning of each message.. no idea if that's normal and I just read 00 00 in the beginning with all my logs.. Edited.. Looks like it's a length of message??

as far as 07 E0 14 06 36 00 00 3F B8 00 message..
07 e0 is the address to talk to the ecm.
1 is indicative of a 10 which means it's a multiline message.
406 is the chunk, or size your about to send.. We're using the 0 from the multiline command. I'm sure there's an easier way to explain it??
Notice that 36 00 00 3F B8 00 is 6 bytes right there.. So your actually sending 400 bytes of the kernel..
36 is the data transfer command.
the 00 right after the 36 can be either 00 or 80... 80 means execute.. You'll see at the end it send a 36 with an execute command.
The next 4 bytes is the address it's sending too.. You'll see that address changes in chunks of whatever is being sent, in this case 0x400 bytes at a time. Sometimes the address increment, sometimes other software decrements it but it is stacked the same in the end.

After the 36 command you'll see the 76 ack and then the data is sent with a line number at the beginning of each line, starts at 0x21 to 0x2f, then 0x20 and it just scrolls and scrolls for the entire segment.
That enough of a starter?

btw.. Antus.. If I start a topic named something like CANbus log reading can we push this over to that.. I feel bad hijacking this entire post the way that I am..

thank you, this is very useful information, especially the multipart 0x36 message structure. The line numbering was unusual since it didn't start from 0x00 or 0x01 but wasn't that hard to figure out.
I guess since the OP of the original thread didn't want people to "look into" his app and the flashloader, we shouldn't discuss what happens next after CAN message exchange when the flashloader gets loaded and executed.

This is another topic so imo if we keep it generalized to simply loading a kernel such as an sps kernel we should be ok.
As far as what your thinking that kernel will only ever be able to write the calibration segments.. It's hardcoded into the kernel which is pushed encrypted.
The sps kernel can write the OS and calibration segments. If you look around enough you'll see that at least one major player in the aftermarket field uses the sps kernels byte for byte. They push the kernel backwards but it's the exact same sps kernel.

[fastboatster]

ok, just getting on here.. it's kind of weird not seeing 4 zeros in the beginning of each message.. no idea if that's normal and I just read 00 00 in the beginning with all my logs.. Edited.. Looks like it's a length of message??

as far as 07 E0 14 06 36 00 00 3F B8 00 message..
07 e0 is the address to talk to the ecm.
1 is indicative of a 10 which means it's a multiline message.
406 is the chunk, or size your about to send.. We're using the 0 from the multiline command. I'm sure there's an easier way to explain it??
Notice that 36 00 00 3F B8 00 is 6 bytes right there.. So your actually sending 400 bytes of the kernel..
36 is the data transfer command.
the 00 right after the 36 can be either 00 or 80... 80 means execute.. You'll see at the end it send a 36 with an execute command.
The next 4 bytes is the address it's sending too.. You'll see that address changes in chunks of whatever is being sent, in this case 0x400 bytes at a time. Sometimes the address increment, sometimes other software decrements it but it is stacked the same in the end.

After the 36 command you'll see the 76 ack and then the data is sent with a line number at the beginning of each line, starts at 0x21 to 0x2f, then 0x20 and it just scrolls and scrolls for the entire segment.
That enough of a starter?

btw.. Antus.. If I start a topic named something like CANbus log reading can we push this over to that.. I feel bad hijacking this entire post the way that I am..

thank you, this is very useful information, especially the multipart 0x36 message structure. The line numbering was unusual since it didn't start from 0x00 or 0x01 but wasn't that hard to figure out.
I guess since the OP of the original thread didn't want people to "look into" his app and the flashloader, we shouldn't discuss what happens next after CAN message exchange when the flashloader gets loaded and executed.

This is another topic so imo if we keep it generalized to simply loading a kernel such as an sps kernel we should be ok.
As far as what your thinking that kernel will only ever be able to write the calibration segments.. It's hardcoded into the kernel which is pushed encrypted.
The sps kernel can write the OS and calibration segments. If you look around enough you'll see that at least one major player in the aftermarket field uses the sps kernels byte for byte. They push the kernel backwards but it's the exact same sps kernel.

agreed. I'm not interested in PCM flasher's kernel per se, more like in any kind of E38/E67/T43 kernel as an example of best practices on how reading and writing of the flash memory is done.