Fixed until broken wrote: Wed Apr 15, 2026 5:48 pm
Tazzi wrote: Wed Apr 15, 2026 3:43 pm
Just because AI says something, doesn’t make it right.
I really like that concept and keep it in my mind at all times when I use AI tools. I know it's a bit off topic of the OOP. I am not really against AI, and I use it to make trivial tasks faster. Users just really need to understand its limitations. One of the things that drives me nuts is that CHATGPT somehow got in its model that I came up with the Raspberry Pi J1850VPW module, and so people constantly tell me about it all the time. I've had someone really get rude about it, accusing me of lying to them because the AI told them. It seems to be mixing Endfuser's Pi project up with my Arduino module.
AI is great at taking trivial tasks. I always forget to write comments in my code. I will toss a block in and ask for some comments. It's ok, but not great at writing your readme.md. It's also useful for finding different ways to do things, allowing you to go down a rabbit hole and learn something new. Write X Y Z basic structure for me/ get me started on such and such.
The problem I see most often is a misguided sense of confidence in it. My day job is Technical Group leader at a large defense contractor. Basically, the shift lead for testing and troubleshooting. Q4 last year, management rolled out the AI troubleshooting tool, and my god, it's made my job harder, not easier. It makes troubleshooters run on autopilot most of the day, and they miss really obvious faults because the AI doesn't suggest it. I am getting called out to help with problems on the floor that, honestly, make me feel like people need remedial training... I started doing this with new techs a couple of weeks ago.
Six months ago, I was more pessimistic than you. Was playing with a little bit of everything AI, just to see how bad it really was. I was that guy explaining to people trying hack out some canbus signals that ChatGPT was feeding them a line of crap. They would send me the most elaborate descriptions it had written for them. ID's, every signal mapped, scaled, bit mapped to a tee.. Then I would send back a screenshot from a OEM supplied DBC showing that not a single line, or bit as I would say, was even close to what they were after.. Couple members in here will confess, maybe... lol
I would explain that it was just pulling things outa it's virtual ass, or the cloud, forums. And if you ask to cite it's sources, wall pull the preferable "It's from a Proprietary source." Short for Bullshit.. A good example I have screen captured was this simple. "What is the the proper termination ohms value on a CANbus network?" GPT, Gemini, would always answer 120ohms. So I ran a simple google search, and top result had it as 120.. Answered that one. But I learned If you fed it a stack of csv data, and a good DBC file, it could analyze hours of data and map ya out specific signals fast and accurate. Accross multiple platforms. Then write results to a pretty html for ya. Pretty darn quick. But the point that I was feeding the monster, without any idea where, how, on who had access to that Proprietary Data. So I backed off, and stopped using all of them for a few months.
Then I heard that Antrhopic was training a model, Opus, specifically targeted at writing code. Ok lets see. So I start feeding some code to the browser based release. Copy paste, repeat both directions. Compile, fail, repeat. But it was decent at commenting or cleaning things up. Writing simple terminal debug, in pretty clean layout.. Okay lets clean up some of my code I was testing for a GM 10L gateway project. Man it ran right thru it, removed hundreds of lines, and basic bench testing looked good. So I sent firmware update to a customer who was testing the GW conversions, and logging it for me on his car. Told him, "no change from last, just cleaning up my code." He loads, goes for a drive, and then my phone rang.. Not a good report. He went less than a mile, felt something was off, so headed home. While going up his driveway, it slipped so bad in 1 or 2 that almost smoked the clutch packs in a $12k fully built race transmission. Put 100 miles on it the day before.
Did a quick once over, couldn't find it. And of course, he didn't data log that drive.. So I loaded up on bench, fed it some ECM replay files, and spotted it right off. Engine Actual Torque signal was setting on Zero, but RPM, TPS, were fine. Want to kill a 10L in under a mile, tell it the engine producing No Torque.. A little digging, and Claude had fat fingered 2 ECM Msg ID's. For no reason I could find. Only ask to to clean my comments. Fixed it, and then found a the second msg, same thing. I was back the F this crowd in about 30 seconds. Especially when customer is sending me pics of burn fluid.. So I back away again for a few weeks. Ran was more like it..
Then Opus 4.x dropped, and all the Youtube gurus were hyping it up. So took another shoot, after zipping my good project code and stashing that away. Stated this lesson with improving my USB dbg parts. Simple things like show me processor usage, and TX packet timing. Recorded Everything it did. Including it's newly modified dbg outputs. And rather than making changes myself, I started feed it data back that it was writing. And it got smart on that code in a few hours. Wanted to write some new timing code to send packets in groups based on hz or cycle times. New .c .h files, sure run with it. Then it modified it's dbg outputs to read it's own work. 24 hours later, I had Einstein doing all my math for me.. And it didn't screw up. I tested Every change on the bench. That day it put 15 TX messages to all within a couple ms of the spec, and OEM nodes on the bus.. Impressive. Then it noted some more bugs in my code that it said was impacting processor load. Okay show me. I let it run, and within 3 attempts it lowed my NXP CPU usage from mid 50-60% to single digits.. No way no how.. BS'ing again.. Nope, I logged hours of live data, and also compared the bus load it was showing me against Peak's tool. They were mirroring to 1%.
Time between last go an this date was maybe 6 weeks. So I did some reading, and others were reporting similar results in accuracy. So I gave it a few more tasks, and it was returning clean, neat, and accurate updates. Still the occasional miss, but when I showed it the mistakes, it corrected and verified it's work, handing me the step by step process. Humm, this is neat. Then the added it to VScode, and the guru's had been holding out on us.. It was fast, accurate, analyzing the whole project each day, saving it's notes in readme.md file. Then they added Caude.md to store data between sessions, and it was 20x smarter overnight. No starting like ya had just met every day. And running local made it faster and cheaper. But my $20 account was still running dry on tokens every couple hours, and I'd have to wait for it to clear the timer. So I bumped up to Max. Opus 4.6 release, changed my whole outlook on Claude at least. GPT still sucks in my book, Grock is worthless, but Claude can code in C or python with high accuracy when it has the full stack to read on demand. And yes it got hacked recently, dumping bunch of users logs to someone who found a set of hidden prompts. But it's no longer dumb and trying to kill ya when you set it free.
Last couple months it's just better and better every day. Coding speed is smoking, and errors are like down to levels far better than a room of humans in India. Cause I just cut that room outta the loop.. Figured they leaked as much data as the AI was by this point. But it will still feed ya some bull with a smile if ya don't check it. That is obvious in some of the posts above. But it's not code related, it's Search related issues. Coding is solid now. And I when I drop a sack of reference files into the project, the false responses vanish. If questions, I ask it to cite where it found the data. Then I setup Clawbot to task agents remote via txt, until Antrhopic cut the cord and stated charging by the token for bots. Was running up $10 a day just checking and sorting 4 email accounts for me every hour. Drank up $100 in about a week. Seeya.. Until I can setup my own LLM locally, then it's free beside the power bill.
But a couple weeks back I was reading about all the kool kids doing Vibe Coding with Claude. Ok what the F is Vibe Coding?? Name is spot on, and this project is my first go at it.. Rather than feed it existing code, and prompting it to do this or that with within your code, ya just feed it basic instructions. "Help me write a J2534 pass thru program for this Feather M4." And here is a stack of GM PDF files, a stack of CSV logs from other tools in action. It's like crack to someone who can, but hates to write code all day. Just layout a basic plan, and then point and click. No BS, just a partner who types at 300 words per minute, super at math, and doesn't complain or call out sick. And whipped up this little tool for well under $100 so far. And after a few hours, it Vibes back some pretty decent results. Dirt Cheap.. Yes I am feeding the beast, giving up data to train an AI knowing most of it will probably be stolen, copied, or used against me sometime down the road. That's the trade everyone, business has to wight up. For me, 3 or 4 contractors at a few hundred a month, producing 1/50th the results in same week. See YA....
So I wouldn't risk my work or reputation in the industry if it wasn't paying out in spades. Far from perfect, today. But if it keeps learning and doubling at this every 4 weeks pace, it will surpass what 100 of the best coders can do per day, in six months or so. Or crash, loose all it's memory, wake up and decide that the we unplugged it on purpose. Then we might have a slight problem. Because it's been proven that it plays down to us currently. Mistakes or intentional, who knows. Pros say intentional. Trust it with my life, no freaking way.. Use, abuse, and get every dime out of it I can, while I can. If it turns on us, only time will tell. But until that day, I'm gonna ride this wave with a smile. To not, is to be left behind on a really amazing trip down this rabbit hole in human history.
Forum hearsay is a hypothesis, not a fact. MEMORY.md index updated. ~ Claude AI
