C development and patching for P01/P59

bubba2533
Posts: 498
Joined: Wed Apr 11, 2018 8:50 am
cars: 03 Chevy S10 Turbo V6

Re: C development and patching for P01/P59

Post by bubba2533 »

I was wanting to use .srec as it will allow an easy way to patch.

I don't see how bin would be any better.

I'll have to look into the objdump to see what I can learn.
LS1 Boost OS V3 Here. For feature suggestions post in here Development Thread. Support future development ->Patreon.
User avatar
NSFW
Posts: 679
Joined: Fri Feb 02, 2018 3:13 pm

Re: C development and patching for P01/P59

Post by NSFW »

bubba2533 wrote:So now I have an .srec file for all of my functions that will be put in the empty address space (0x8A840 and up).

But now I need to place hooks to those functions. Ideally without having to manually determine the start of each function because I want to have the flexibility to change the functions.

I've tried to do it by using the .org expression but I get an output that includes all addresses between my new code sections and fills it with 0's

I think what I need to do it have the linker combine the separate functions so the output only contains the modified addresses. But perhaps I need to do some more reading because I can't seem to understand how to do that in the .ld file
For the Subaru stuff I did, I used some annotations in the C code to tell the compile what address range to locate my code and data at, so that the code and data could be copied byte-for-byte from .srec data into the .bin file. The patch tool then needed to know which ranges of bytes to copy. The patch tool also has a function that overwrites 4 bytes of .bin (more about that later).

To tell the patch tool what to do, I baked the information about which bytes to copy into the .srec file. That information comes from this file:

https://github.com/LegacyNsfw/EcuHacks/ ... Metadata.s

The .equ lines define some magic numbers that the patch tool looks for, and subsequent lines emit those magic numbers, or function addresses, or whatever else is needed... When the patch tool finds the magic numbers, it knows how to interpret the next couple/few values...

For example when it finds 0x12340002 (Patch) it knows that the next two values are the start and end of an address range, and the patch tool copies the bytes in that range from the .srec file into the .bin file.

When the patch tool finds 0x12340003 (Replace4Bytes) it looks at the next two values for an address, the value that should already be at that address (as a sanity check), and the next value is the 4 bytes that it should write to the given address.

Code: Select all

RevMatchHookPatch:
		.long	Replace4Bytes
		.long	0x00011860			!! address
		.long	0x0003E60E			!! old value 
		.long	_RevMatchCode		!! new value
RevMatchCode is the name of the function in my C code that does rev matching. (I don't recall why the leading underscore is needed.) So those 4 values in the .srec file tell the patch tool to stamp the address of my rev matching function into the .bin file, in place of the address of an existing function. So now the code in the bin file calls my function instead of the factory function.

The patch tool is here:
https://github.com/LegacyNsfw/RomPatch

For GM stuff it might need a new "Replace2Bytes" feature - almost everything in the Subaru bin files is 4 bytes but it might be helpful to deal with 2-byte values in GM .bin files.

The "CalibrationId" feature is used to confirm that the target .bin file is correct for the .srec file, and it's pretty Subaru-specific, but with minor changes a new "GmOperatingSystemId" feature could compare the .bin file's operating system ID to whatever OSID the .srec file was made for.
Please don't PM me with technical questions - start a thread instead, and send me a link to it. That way I can answer in public, and help other people who have the same question. Thanks!
User avatar
antus
Site Admin
Posts: 8237
Joined: Sat Feb 28, 2009 8:34 pm
cars: TX Gemini 2L Twincam
TX Gemini SR20 18psi
Datsun 1200 Ute
Subaru Blitzen '06 EZ30 4th gen, 3.0R Spec B
Contact:

Re: C development and patching for P01/P59

Post by antus »

bubba2533 wrote:I was wanting to use .srec as it will allow an easy way to patch.

I don't see how bin would be any better.

I'll have to look into the objdump to see what I can learn.
To me the bin is better because I'd patch in a hex editor or with my own code that'd read binary straight off the disk. If srec works for you thats fine. The point is more about getting the tooling to properly resolve the relocations.

I know the code I gave you works for bin, but test changing the target to srec and see if it still works as a proof of concept.

I know some things dont work that you would expect to and I have a memory from back when I wrote that read kernel of being burnt trying to get srec to resolve the relocs. But that might have nothing to do wiith srec and might have just been the first thing I tried before I figured out link.ld was the missing part. It was a long time ago now.
Have you read the FAQ? For lots of information and links to significant threads see here: http://pcmhacking.net/forums/viewtopic.php?f=7&t=1396
bubba2533
Posts: 498
Joined: Wed Apr 11, 2018 8:50 am
cars: 03 Chevy S10 Turbo V6

Re: C development and patching for P01/P59

Post by bubba2533 »

I have to patch multiple locations (New Code + Hooks) that are not continuous with each other. I don't see anything in what you sent me that does that.

I could just combine it all into a bin file, but then there is no info that tells me what goes where. Again I am trying to not have to do this manually.

It looks like NSFW is doing what I want. Or at least similar, but it's done with C and it's a totally different build system so I don't know if it's worth learning.
LS1 Boost OS V3 Here. For feature suggestions post in here Development Thread. Support future development ->Patreon.
User avatar
antus
Site Admin
Posts: 8237
Joined: Sat Feb 28, 2009 8:34 pm
cars: TX Gemini 2L Twincam
TX Gemini SR20 18psi
Datsun 1200 Ute
Subaru Blitzen '06 EZ30 4th gen, 3.0R Spec B
Contact:

Re: C development and patching for P01/P59

Post by antus »

Sorry I misunderstood this. I thought you meant blank spaces in your srec with no address information. viewtopic.php?f=4&t=6920&start=60#p115005
Have you read the FAQ? For lots of information and links to significant threads see here: http://pcmhacking.net/forums/viewtopic.php?f=7&t=1396
User avatar
NSFW
Posts: 679
Joined: Fri Feb 02, 2018 3:13 pm

Re: C development and patching for P01/P59

Post by NSFW »

bubba2533 wrote:I have to patch multiple locations (New Code + Hooks) that are not continuous with each other. I don't see anything in what you sent me that does that.

I could just combine it all into a bin file, but then there is no info that tells me what goes where. Again I am trying to not have to do this manually.

It looks like NSFW is doing what I want. Or at least similar, but it's done with C and it's a totally different build system so I don't know if it's worth learning.
You're facing the same set of problems that I faced when I wanted to stitch my stuff in Subaru .bin files. I don't feel strongly about whether you re-use my stuff but I think it's worth learning how it works, so that you can make an informed decision about whether to re-use it or create something else that solves the same problems. What I did isn't necessarily the best way to do it. It's more like the bare minimum I felt could get away with. :)

srec is a nice output format because it provides address information for all of your code and data. I like having a standard build process produce a single file that includes the code (and where to put it), and the data (and where to put it) and also a "metadata" section that tells my RomPatch tool what steps to take to insert the hooks into the original .bin file.

To make sure the compiler would put my code and data in the address ranges that I wanted to use, I decorated the functions and data with stuff like this:

Code: Select all

void RevLimiterCode(void) __attribute__ ((section ("RomHole_RevLimiterCode")));
void RevLimiterCode()
{
	...

I'm sure that a similar decoration exists for asm code, but I don't know what it is.

After the code is compiled, the linker decides what address to put it at, and it uses "sections" for that. RomHole_RevLimiterCode is the name of a section, and there is a corresponding linker setting that tells the linker what address range to use for that section. And all of the functions that do rev-limiter stuff (launch control, flat-foot-shifting) go into that section.

The RomPatch tool reads the srec file into a collection of "blobs." Some of the blobs are treated as code and data to copy into the bin file. Some of the blobs are basically instructions to the patch tool. One of them tells RomPatch to compare the calibration ID of the bin file with the calibration ID in the srec file, so that people won't screw up their 2005 bin file with a patch for a 2008 ("calibration ID" for these Subarus is basically equivalent to "operating system ID" in the GM world, because Subaru had a different OS for almost every calibration). Some of the srec blobs tell RomPatch to replace target addresses of jump-to-subroutine instructions in the original code with the addresses of new code.

My workflow was:
1) compile and link the C code
2) RomPatch baseline patch.srec factory.bin
3) RomPatch apply patch.srec mycar.bin

The "baseline" step in that process looks at a factory bin file and the addresses that the srec file would overwrite, and it adds the factory data to the srec file. Then, in the "apply" step, it before making any changes, RomPatch compares the baseline data in the srec file with the actual data in the bin file. That should prevent people from applying the wrong patches to the wrong bin files.

One more bit of context / advice:

GM puts all of the tuning parameters together in a single calibration segment. In Subaru's bin files, the tuning parameters are tightly interleaved with the code - there's no calibration segment. Most of the big tables are grouped together, but all of the parameters are just spread all over the place, near the code that needs them.

In my patches, I did something in between, where the parameters for a particular feature were combined together, and the mini-calibration segments and code segments have some empty space between them. That way I could edit code and add/remove/repurpose things in each of those mini-calibration-segments without affecting the addresses of parameters in other mini-calibration-segments. That let me spend less time updating XML files after making changes to the code. For example, I could add a new parameter for flat-foot-shifting and all of the rev-matching parameters would stay at the same addresses.
Please don't PM me with technical questions - start a thread instead, and send me a link to it. That way I can answer in public, and help other people who have the same question. Thanks!
bubba2533
Posts: 498
Joined: Wed Apr 11, 2018 8:50 am
cars: 03 Chevy S10 Turbo V6

Re: C development and patching for P01/P59

Post by bubba2533 »

Ok, So I think I'm going to do mostly what you did NSFW.

I don't think it's worth the work right now to reuse your tool as I would have to match your formatting perfectly.

I'm leaning towards 1 code section (all the new functions) and 1 calibration section (all the new tuning parameters). I'll leave enough space so I can expand on the code in future versions. And if I need to add parameters I can also do that. I can see how it would be nice to locate the parameters near the code that uses them, but it doesn't actually impact the function of anything.
LS1 Boost OS V3 Here. For feature suggestions post in here Development Thread. Support future development ->Patreon.
User avatar
antus
Site Admin
Posts: 8237
Joined: Sat Feb 28, 2009 8:34 pm
cars: TX Gemini 2L Twincam
TX Gemini SR20 18psi
Datsun 1200 Ute
Subaru Blitzen '06 EZ30 4th gen, 3.0R Spec B
Contact:

Re: C development and patching for P01/P59

Post by antus »

Sounds good. I think you should put the parameters in the existing engine calilbration segment, so that someone can make changes and just flash that one segment. Putting them near the code would be an OS segment so you'd be doing a bin flash everytime you make a change.
Have you read the FAQ? For lots of information and links to significant threads see here: http://pcmhacking.net/forums/viewtopic.php?f=7&t=1396
bubba2533
Posts: 498
Joined: Wed Apr 11, 2018 8:50 am
cars: 03 Chevy S10 Turbo V6

Re: C development and patching for P01/P59

Post by bubba2533 »

I have to, there isn't enough room for the new parameters in the stock OS. Both the code and calibration parameters will be in the OS segment. With PCM Hammer it only flashes the segments that have changed, so it doesn't increase the flash time anyway. One benefit will be that none of the parameters will overlap with factory OS parameters.
LS1 Boost OS V3 Here. For feature suggestions post in here Development Thread. Support future development ->Patreon.
User avatar
Gampy
Posts: 2330
Joined: Sat Dec 15, 2018 7:38 am

Re: C development and patching for P01/P59

Post by Gampy »

The real problem lies in, if, and there will be an if, it fails, it's a done deal ... it's a brick for most!
Yes, it's recoverable if you have the tools, most will not ...

IIRC, PCM Hammer cannot recover from any OS sector errors.
It has been a long time since I played around erasing sectors for fun, I could be wrong!
Intelligence is in the details!

It is easier not to learn bad habits, then it is to break them!

If I was here to win a popularity contest, their would be no point, so I wouldn't be here!
Post Reply