PCMTec Development Blog

[Tazzi]

Taught myself http over the weekend. I've never actually had a need to write a http service before so thought this would be a good time to learn. I previously understood http and had implemented raw TCP services (using modbus etc) but never one over http.

Wrote a small Heroku service (they host web services so you don't need to manage security on server) that will host the templates and talk to the editor program via simple JSON over HTTP GET requests. Handles the user database, encryption, licenses etc all off site so there is nothing to protect within the actual software as it is all done offsite and protected that way.

Learnt all about storing passwords using bcrypt so you store a secure password hash only (so you don't have to handle actual passwords) and use a different salt for every single password. This makes all sorts of password attack vectors like rainbow tables etc useless.

Also implemented SQL that uses parameterised queries so the server is immune to SQL injection attacks and things like that. Server uses unique session identifiers and HTTPS so theres no possibility of a replay attack.

What is even better is due to the extremely low bandwidth and storage required it fits into the Heroku free version that doesn't even cost anything.

After reading it seems a large number of people still do not know what they are doing in this regard. Eg storing actual passwords in databases, or using constant salts. Non constant time password comparison where you can use statistical analysis on how long it took for a password to be rejected to infer how many characters of the password were correct.

Stuff like writing dynamic SQL queries which are susceptible to sql injection to get a server to spit out a list of user accounts and passwords or even drop the entire database!

Seems even back in 2012 linkedin who are a massive company did dumb stuff like using no salt on their passwords!

So its going to be a user login based application? That sound about right?

[rolls]

So its going to be a user login based application? That sound about right?

Yes. So you can purchase license etc per user account. Require net access to purchase and download, then everything else can be done offline.

[MeZmeriZe]

Its fantastic to see that you've given the UI some serious thought.

I've had a play with HP tuners, and sniper in the past and I used Binary editor loads of times and I must say sometimes I find myself wondering what they were thinking. (aside from tuning software I think that a lot when looking at recent UI choices from Google and Microsoft too)

I love the way the big companies try to "simplify" their user interfaces, by hiding stuff behind specific mouse movements or whatever. If you had to be shown how to use the UI, you didn't simplify it, you made it worse. You may need to learn how to use the advanced features of a program, but the UI itself should be obvious. Recent google mapps (android) is a good example of going from an app that was intuitive and obvious to one that requires you dick around for 5 minutes to find some function that used to take a second.

Interestingly I found one of the problems with my LPG system running way too rich when I get off the throttle causing it to stall when you push the clutch in and pull up at lights.. I pulled the LPG rail pressure regulator off and apart and discovered that the needle valve used in the boost actuator (it allows extra LPG though and is triggered by a vacuum/boost hose to the intake manifold) was half siezed.. the boost has enough force to push it open but it took forever to work it's way closed again.. so after dumping off all the boost the valve was taking a while to shut and continued dumping too much LPG into the system.

Still running to rich.. but much better than it was.


One of the things I've been wondering recently.

Why hasn't someone written an autotune program for cars? I mean it would probably never be as good as an experienced tuner.. but a machine learning type app with access to a ton of varying load data logging including wideband and all OBD2 stuff, one of these 6000 var schema's that mr Rolls speaks of, and a wishlist of what you are trying to achieve with a tune (ultimate economy, lean mode cruise, max power, widest power band etc etc) would surely be able to get pretty close with the basic stuff. Hell with in memory changes and mapping you could put it in "try" mode while driving around a track somewhere and have it go though a pre determined number of realtime changes while logging the results... use that data to better work out what adjustment is needed to fine tune.

If the local laptop lacked the power to do that analysis in real time, most laptops now have a SIM slot.. get the cloud involved. (or have the laptop generate and store the prep work for upload to a cloud app) That app would make tuning something any beginner could jump in with without risking burning holes in their pistons. I suspect this will be the killer tuning app of the future.. even if it just gets you started with a starting point for continued fiddling.. it would be a fantastic learning tool too.

Think about it.. plug a tablet into your ODB2 port and stick it under the seat plugged into the cig lighter for charging and the exhaust for wideband... drive around until the app signals that it's logged a wide enough variety of conditions and loads and suggests you find somewhere on private property where it can go into live test mode.... you thrash your car for a bit somewhere. (or try freeway type driving to test lean mode or whatever based on your requirements)

It's an interesting thought. I considered trying to code such a thing but I have too much to learn and it will almost certainly exist before I'm good enough to tackle it with any authority.

[Tazzi]

Yes. So you can purchase license etc per user account. Require net access to purchase and download, then everything else can be done offline.

Will send a message about this

If you had to be shown how to use the UI, you didn't simplify it, you made it worse.

True.. to an extent.
I think they have attempted to make all common actions accessible. But theres only so much that can be fit on a screen, so no matter what they change, there will always be elements that are in undesirable locations. A popup editable 'shortcut' menu could be a solution..

Hell with in memory changes and mapping you could put it in "try" mode while driving around a track somewhere and have it go though a pre determined number of realtime changes while logging the results... use that data to better work out what adjustment is needed to fine tune.

The entire problem there, is 'real time'.
As Rolls covered a few pages back, would be a massive undertaking to develop a custom OS to allow real time.
I think the closest that could occur.. is log live data.. stop.. analyse.. upload new tune.

If the local laptop lacked the power to do that analysis in real time, most laptops now have a SIM slot.. get the cloud involved.

Cloud.. nah.. laptops will happily keep up with processing live engine data. What lags them down is refresh rate of screen for nice UI controls.

[MeZmeriZe]

If you had to be shown how to use the UI, you didn't simplify it, you made it worse.

True.. to an extent.
I think they have attempted to make all common actions accessible. But theres only so much that can be fit on a screen, so no matter what they change, there will always be elements that are in undesirable locations. A popup editable 'shortcut' menu could be a solution.

Look at the win7 UI, and then look at the Win8 UI, people didn't need to be shown how to use Win7, but Win8 was a mess that most people hated simply because it was hard to figure out how to do stuff that used to be obvious. In Google maps on Android the older app was a bit nosier with regards to stuff onscreen, the newer one is much cleaner looking but it's also no longer obvious how to get to more advanced features that used to be down the bottom of the pop up menu and are not anymore.

Hell with in memory changes and mapping you could put it in "try" mode while driving around a track somewhere and have it go though a pre determined number of realtime changes while logging the results... use that data to better work out what adjustment is needed to fine tune.

The entire problem there, is 'real time'.
As Rolls covered a few pages back, would be a massive undertaking to develop a custom OS to allow real time.
I think the closest that could occur.. is log live data.. stop.. analyse.. upload new tune.

You misunderstand.. BE and some others allow you to make "in memory" changes to the PCM while you are driving and immediately see the result... so you can change stuff while driving around in real time if you were that way inclined.. I imagine that feature is great for when you are dyno tuning. I was saying that if a connected program is running on the laptop logging and watching everything including the usual wideband, advance etc etc then it could make in memory changes on the fly the same way BE does and evaluate the results. So for example add a degree of advance under load and see if it pings. (BA-FG all have knock detection) or make small changes to a fuel table and see how much effect it has on o2 and other data being logged.

So basically, change something in memory, log the result. Put it back. Change something else, measure result.. put it back and so on. So it gets an idea of the scale of change on that particular engine. (not explaining myself well. a Storm ute smashed into the house next door at 1am landed up on it's side and I had to smash the windscreen with a brick to get the guy out. was pretty late by the time it was all over and I'm stuffed now.. just in time for another day of work.

If the local laptop lacked the power to do that analysis in real time, most laptops now have a SIM slot.. get the cloud involved.

Cloud.. nah.. laptops will happily keep up with processing live engine data. What lags them down is refresh rate of screen for nice UI controls.

Lol.. if a modern laptop couldn't do it I'd be wondering how an MPC5xx was.. no, I didn't mean controlling it, I meant the machine learning of studying the output of logs and testing and using it to create a new tune.. in a reasonable time frame. but a 6000 point tune would be trivial to a modern laptop anyway so I doubt you'd need anything cloud.. but if the software written to do the analysis was a cloud app, you'd have a lot more power (at least in terms of number of threads.. per thread the cloud is generally a lot slower)

[rolls]

You misunderstand.. BE and some others allow you to make "in memory" changes to the PCM while you are driving and immediately see the result... so you can change stuff while driving around in real time if you were that way inclined.. I imagine that feature is great for when you are dyno tuning. I was saying that if a connected program is running on the laptop logging and watching everything including the usual wideband, advance etc etc then it could make in memory changes on the fly the same way BE does and evaluate the results. So for example add a degree of advance under load and see if it pings. (BA-FG all have knock detection) or make small changes to a fuel table and see how much effect it has on o2 and other data being logged.

So basically, change something in memory, log the result. Put it back. Change something else, measure result.. put it back and so on. So it gets an idea of the scale of change on that particular engine. (not explaining myself well. a Storm ute smashed into the house next door at 1am landed up on it's side and I had to smash the windscreen with a brick to get the guy out. was pretty late by the time it was all over and I'm stuffed now.. just in time for another day of work.

You are describing live tuning. That means writing a custom OS. BE does not do that for the PPC platform.

It is not possible to edit the maps whilst the car is running as they are stored in ROM in a separate flash chip. You must edit the OS to copy the maps to RAM, then edit them in RAM and when you are happy you write the RAM back to the ROM. The PCM does not work this way currently.

People have written live tuning OSs for other vehicles, but no one has done it for the Ford PPC platform.

It is a huge amount of work. It will require writing your own recompiler (weeks of work to do that alone) as nothing exists for PPC that I'm aware of. Probably over a months work fulltime if you know what you are doing.

Afaik BE have written a recompiler for their motorola based vehicles they tune, but not PPC.'

[MeZmeriZe]

You misunderstand.. BE and some others allow you to make "in memory" changes to the PCM while you are driving and immediately see the result... so you can change stuff while driving around in real time if you were that way inclined.. I imagine that feature is great for when you are dyno tuning. I was saying that if a connected program is running on the laptop logging and watching everything including the usual wideband, advance etc etc then it could make in memory changes on the fly the same way BE does and evaluate the results. So for example add a degree of advance under load and see if it pings. (BA-FG all have knock detection) or make small changes to a fuel table and see how much effect it has on o2 and other data being logged.

So basically, change something in memory, log the result. Put it back. Change something else, measure result.. put it back and so on. So it gets an idea of the scale of change on that particular engine. (not explaining myself well. a Storm ute smashed into the house next door at 1am landed up on it's side and I had to smash the windscreen with a brick to get the guy out. was pretty late by the time it was all over and I'm stuffed now.. just in time for another day of work.

You are describing live tuning. That means writing a custom OS. BE does not do that for the PPC platform.

It is not possible to edit the maps whilst the car is running as they are stored in ROM in a separate flash chip. You must edit the OS to copy the maps to RAM, then edit them in RAM and when you are happy you write the RAM back to the ROM. The PCM does not work this way currently.

People have written live tuning OSs for other vehicles, but no one has done it for the Ford PPC platform.

It is a huge amount of work. It will require writing your own recompiler (weeks of work to do that alone) as nothing exists for PPC that I'm aware of. Probably over a months work fulltime if you know what you are doing.

Afaik BE have written a recompiler for their motorola based vehicles they tune, but not PPC.'

Lol, I've owned BE for ages now and I didn't know that.. It doesn't make that distinction in the program itself about not doing the PPC stuff for more than read/write offline and to be honest I never tried to use it as my schema was insufficient for that purpose anyway. I was wondering how that worked. So for the feature to work BE makes changes to the default RTOS to have it put the editable vars into ram where they can be tweaked on the fly. I guess I just assumed that considering the size and simplicity of RTOS and the fact that it has to be totally consistent in timing that it likely loaded the maps into ram on boot.

Out of curiosity do we know what RTOS the mcp 5xx fords use? Seems unlikely that ford/Visteon would write their own when so many options already exist. I know freeRTOS supports the tricore but I've not read anything about the 5xx motorolas (later freescale, later NXP, and now Qualcomm.)

[rolls]

Lol, I've owned BE for ages now and I didn't know that.. It doesn't make that distinction in the program itself about not doing the PPC stuff for more than read/write offline and to be honest I never tried to use it as my schema was insufficient for that purpose anyway. I was wondering how that worked. So for the feature to work BE makes changes to the default RTOS to have it put the editable vars into ram where they can be tweaked on the fly. I guess I just assumed that considering the size and simplicity of RTOS and the fact that it has to be totally consistent in timing that it likely loaded the maps into ram on boot.

Out of curiosity do we know what RTOS the mcp 5xx fords use? Seems unlikely that ford/Visteon would write their own when so many options already exist. I know freeRTOS supports the tricore but I've not read anything about the 5xx motorolas (later freescale, later NXP, and now Qualcomm.)

Creating a live tune for BA would be a terrible return on investment, if they had implemented it (like I said months of work) they would be advertising it and everyone would be using it. Not something you'd just do for lols.

RTOS would just be custom code I imagine that has been carried over from the old EEC PCMs. I know bosch sell software modules and also ECU software so it is possible they have purchased something. The operation documents don't mention anything third party about the code so I suspect it is entirely custom to ford.

The GM code that uses PPC for example looks very different to the Ford code.

[MeZmeriZe]

I guess my misunderstanding was just the wording on his site.

makes no mention of "only available for selected PCMs" or anything like that... I just re-read it and it says

With the Moates Quarterhorse the tuning parameters, such as MAF (Mass Air Flow sensor) curve can be changed on the fly. Yes this means you can finally tune while the car is running. It is no longer required to turn the car off and then download a new tune, just simple make the desired change and press the update button. It is not recommended or safe to make changes while driving. Make sure to have someone other than the driver make changes if live tuning is going to be done.

so I guess that means ECU's that take a J3 chip?

I just read something in a forum from Matt at BPR. http://ford.carstalks.com/hey-guys-need ... ing-103294
"Live tuning can be done, but need a dev PCM from Visteon"

That is an interesting thought.. wonder how you'd go about getting one of those.... probably not cheap even if they would sell them to normal folks.

[rolls]

I guess my misunderstanding was just the wording on his site.

makes no mention of "only available for selected PCMs" or anything like that... I just re-read it and it says

It must be. I bet it is only for 1 strategy in 1 PCM type as well, no one is going to write a live tuning solution that covers all PCMs, that would be hundreds of thousands in development time.

edit: If you read through that thread

"Live table mapping with qhorse in eseries and au eecv cars. Ppc like ba-fg and tricore is reflash only. Data logging support if the strategy has DMR info."