Open source GM OBD2 flash tool using a ELM327 device

[pgbpro2.0@gmail.com]

So you're using the Allpro with custom firmware to solve the hardware challenge with open source hw/sw, cool... And since this has been done before (with a source from dimented 24x7 that you have) with the AVT cable it makes sense that the earliest working versions of this software will support that.

Is the 'state of the art' Allpro firmware available on github now? Is that planned?
How about dimented's 24x7 LS1 flasher tool source?
Considering the history of these LS ECM tuning projects in the past, I think stamping the project GPL now might be a good idea

I'm trying to get a sense of, if I and other members of the public want to contribute to this what can and can we not do? Looks like there are some OBDII open source ECU tuning software, but nothing that's been updated in a while or GM friendly. The suburu / mitsu guys appear to have all the luck in this regard. I'm thinking my contribution would be in the actual tuning software as my expertise is in Data Science (machine learning) and UI vs. low level hardware interfaces - which I've only done a few projects with and they were simple async tcp/ip jobs. But I know that I'm getting ahead of myself if we can't even reliably write to just the 0411's with open source software and cheapish hardware.

Appreciate your help getting me brought up to speed.

[antus]

Allpro patches are against a prior version and are here to maintain GPL compliance. Its only a proof of concept, which is now proven. viewtopic.php?f=31&t=4632&start=20#p83464
It does 4x and it supports 180 byte packets (as far as I could push it without bigger changes). Firmware changes will be released as open source. Whether it becomes an alternate firmware fork, or can be merged back in to the orginal firmware remains to be seen.

My agreement with Dimented24x7 did not include an open source release although I do have the source. The current thought is that if its 'my' tool a binary blob should be ok, but that is out of context for what we are doing here. If/when we can get hold of Dimented24x7 we can flesh that out. If we cant, a fresh implementation of the code is still possible. We will not be releasing any of Dimented24x7s source. If we can get hold of him and he wants to join the project then he'd be welcome.

Ive provided my complete working flash tool implementation source to NSFW, who has brought his C# skills to the table and now that re-write of the code base ive been talking about for literally years can get off the ground, and will be open source. Mine is not OO and not multi-threaded and was very much an introduction to C# and OO for me. Keep in mind its still early days (and Im about to head off to a long weekend car show away from home). Any contribution you want to make would be welcome. I'll keep half an eye on this thread from my phone and im sure NSFW will be around too. I also need to mention is was 160plus who brought us together so far, thanks for that!

AVT is because a lot of them are out there in the field and I have one on my desk. My tool supports it too, and its a very capable device which I am comfortable with.

Keep in mind its still early days yet.

[NSFW]

In a nutshell:

The AVT is proven, but expensive and not readily available.

The Allpro with Antus' firmware is probably the best bet but needs more testing.

Good ELM327 devices (not halfbaked knockoffs) probably can work but will be pretty slow, but more testing is needed there too.

Thaniel (another user here) is making good progress with an Arduino-based interface as well.

The Tactrix J2534 won't work, I just happen to have one because I have been playing with Subarus for a long time. (And I completely agree about Subaru and Mitsubish being way ahead in this stuff. I just want the same stuff for my Corvette!) I put some placeholders in the code because I want to support the J2534 standard... The VCX Nano seems pretty promising - I don't have one but might try one before long.

Note that the "testing" I referred to above is mostly blocked until the app gets a little further along. The people working on them have their own tools for that of course, and you can contact them if you want to dive in right away... but I hope that after a week or three this app should be able to at least read VIN and OS info, and it will be good to have help testing that with different devices and cars. Reading will follow (no idea how long that will take) and then writing (also no idea). I think I can get the app doing VIN and OS reads on my own as soon as I get a PCM set up on my workbench, but reading and writing is pretty much up to Antus (and/or Dimented if/when he gets back).

The main things we need right now:

1) People with VPW-capable interfaces can help the app support those interfaces. (Kudos to Tazzi for jumping right in.)

2) People with experience with the internals of the LS1 Flash tool can port that logic to the new app. Apparently Dimented is going to be offline for a while, but Antus is working on this.

3) People with C# / .Net / Windows Forms can work on the GUI side of things. That's where I've been busy, and I think it's pretty close to complete. I'll add file-picker dialogs for opening and saving files, and a write button... not sure what else is left.

4) When we get more ROM files read from 0411 PCMs, I suspect we'll need help finding all of the tables and constants so we can tune them. There's been a ton of work done on a 2001 (by Dimented, if I remember right?), and I suspect we'll want to translate that info to files from other years. Or maybe we can flash the 2001 file to other 0411 PCMs? I don't know. In the Subaru world, tons of work went into reverse engineering every model/year/trim.

5) I think we need a data logging app as well. Not sure where to start on this. I think there's an opportunity for data-science stuff to be applied to tuning, but until there are data logs, there isn't much data to science with.

I already added a "COPYING" file to the github repository with the GPLv3 license, but I need to go back and add headers to all of the source files as well, to be crystal clear about the open-source-ness of this. If anyone is worried about this ever not being open-source, just fork the repository and keep your own copy, so the project will continue if we all get abducted by aliens or distracted by visions of profit. But honestly I don't think there's much risk there.

[antus]

We have free data logging, jayme released an adx for tuner pro. Its avt but can be ported to other devices. Tunerpro xdf definitions seem like a good bet to me but open to options.

viewtopic.php?f=10&t=2314

[NSFW]

Thanks, I found Jayme's thread. If we get another interface proven to work for reflashing then it would be great to port that ADX over to it. I'd also love to have an Android-based logger that works with Bluetooth elm devices, just because it would be so much more convenient than using a laptop... but that's a whole other project and I'm not even sure it's feasible.

I just updated my previous post with a note about the Arduino interface that Thaniel is working on.

[Tazzi]

The only J2534 device I own is a Tactrix OpenPort 2.0, which doesn't support VPW. I just added the J2534 classes in hopes that something would come along eventually... and look what happened!

What's your account name on GitHub? I'll add you to the project.

Price wise looks like the (genuine) nano are cheaper than the tactrix, still much cheaper than an AVT.

Tazzi3 on github

I can see what your going for with the messagefactory class. May get kinda messy with multiple different tools requiring different formats, header/filter setups ect.

I kinda bypassed it as a quick job and directly setup J2534 messages to then send/receive in the queryvin function.

[160plus]

I'd also love to have an Android-based logger that works with Bluetooth elm devices, just because it would be so much more convenient than using a laptop... but that's a whole other project and I'm not even sure it's feasible.

[youtube]https://www.youtube.com/watch?v=8S8JgMX3Upc[/youtube]
This was my initial "Proof of Concept" for an Android data logger I started on last year. It was parsing to a CSV that was saved to the phones root directory. You could rename the file to save the log or just leave it as the default name and it would write over the previous file the next time it was started. I wasn't happy with the speed using the single pid request format, even with the ObdLink Mx I wasn't able to get the pids to stream fast enough using the "Normal" pid request method. I turned to professional scan tools and started breaking down logs of "How" they were able to pull data so much faster and I found the answer. There is a multi pid format that does in fact work with VPW it just doesn't use any "Normal" multi pid format like you would find on CAN based stuff. Each data pid becomes a 2 byte request rather then a 4 byte and your able to request a several pids at once with this method. I had started sorting out what each pid request was some time ago and had to take a break due to the weather. I was actually talking to Antus about this a couple of weeks back and he was able to provide me with a number of the data pids I hadn't even begun to sort out yet.

Once I had discovered and tried out the multi pid format I figured a reasonable goal was to be able to log 20-25 pids with a refresh rate of around 15 updates per pid per second. Using less total pids the refresh rate could be bumped up a bit further for better resolution but that was pretty close to the maximum speed that ObdLinkMx could handle. I posted some information about the idea and even talked to Lance at Pantera EFI about it. He felt that even at 15 fps on each pid that was already a bit overkill and there wasn't any reason to try and bring that up any higher; we have talked a couple of times since our initial conversation about this and possibly adapting my app to work with his stand alone EFI systems log viewing software for a "Free" way to view and analyse logs but he's been pretty busy for the last month or so and I haven't heard back on any more details regarding the use of his program.

DatZap.me works decently well and would defiantly be an alternative log viewer people could use....however since it's wed browser based; larger(longer) longs can be an issue.

[NSFW]

Tazzi3 on github

I can see what your going for with the messagefactory class. May get kinda messy with multiple different tools requiring different formats, header/filter setups ect.

I kinda bypassed it as a quick job and directly setup J2534 messages to then send/receive in the queryvin function.

You have an invitation waiting at Github.

The MessageFactory class should just produce the exact sequence of bytes that finally reaches the PCM - that's going to be the same no matter what hardware sits between the PC and the PCM. Anything that's specific to particular tool should go into the Device class for that tool (ScanToolDevice, Avt852Device, etc). For now the Device class just has Initialize, Send, and Receive, but if it needs more we can add more.

Would it help to have the app call something like device.SetFilter(...) before it calls device.Receive(...) ?

[Tazzi]

You have an invitation waiting at Github.

The MessageFactory class should just produce the exact sequence of bytes that finally reaches the PCM - that's going to be the same no matter what hardware sits between the PC and the PCM. Anything that's specific to particular tool should go into the Device class for that tool (ScanToolDevice, Avt852Device, etc). For now the Device class just has Initialize, Send, and Receive, but if it needs more we can add more.

Would it help to have the app call something like device.SetFilter(...) before it calls device.Receive(...) ?

Ahhh yes ok, makes sense. Ill need to do a fresh pull and reimplement what Iv done, as Iv gone around that device class for the moment while getting the working implementation.

Definitely need to have a set mask and filter before doing anything. For J2534, its basically enforced to do that before being able to send or receive anything.

[NSFW]

This was my initial "Proof of Concept" for an Android data logger I started on last year. It was parsing to a CSV that was saved to the phones root directory. You could rename the file to save the log or just leave it as the default name and it would write over the previous file the next time it was started. I wasn't happy with the speed using the single pid request format, even with the ObdLink Mx I wasn't able to get the pids to stream fast enough using the "Normal" pid request method. I turned to professional scan tools and started breaking down logs of "How" they were able to pull data so much faster and I found the answer. There is a multi pid format that does in fact work with VPW it just doesn't use any "Normal" multi pid format like you would find on CAN based stuff. Each data pid becomes a 2 byte request rather then a 4 byte and your able to request a several pids at once with this method. I had started sorting out what each pid request was some time ago and had to take a break due to the weather. I was actually talking to Antus about this a couple of weeks back and he was able to provide me with a number of the data pids I hadn't even begun to sort out yet.

Once I had discovered and tried out the multi pid format I figured a reasonable goal was to be able to log 20-25 pids with a refresh rate of around 15 updates per pid per second. Using less total pids the refresh rate could be bumped up a bit further for better resolution but that was pretty close to the maximum speed that ObdLinkMx could handle. I posted some information about the idea and even talked to Lance at Pantera EFI about it. He felt that even at 15 fps on each pid that was already a bit overkill and there wasn't any reason to try and bring that up any higher; we have talked a couple of times since our initial conversation about this and possibly adapting my app to work with his stand alone EFI systems log viewing software for a "Free" way to view and analyse logs but he's been pretty busy for the last month or so and I haven't heard back on any more details regarding the use of his program.

DatZap.me works decently well and would defiantly be an alternative log viewer people could use....however since it's wed browser based; larger(longer) longs can be an issue.

That's really cool. Subaru has a logging scheme where the tool can send a list of parameters just once, and the the ECU sends those the values continuously, which gave us something like 20 rows per second with about a dozen parameters if I remember right. If we're lucky there will be something in the GM PCM code that works similarly, but if not maybe we can add it. But in the beginning we had half that rate (about 10 rows/second with a dozen parameters) and it was still enough... and naturally aspirated motors like most V8s could probably do just fine with a couple fewer parameters.

I made a really simple log viewer for Windows that might be handy... no charting, just a big table of numbers. But, I was getting tired waiting for Excel to open my logs, and since this has almost no features, it loads everything pretty quickly:
http://romraider.com/forum/viewtopic.php?t=5446