New open source tuning software

[willemml]

Pushed the graphing capability. It is broken for 3D maps. I'll fix that soon. I have not yet implemented editing values by dragging graph points, but that is a future planned feature. Again, this is an extremely rough proof of concept.

Values are rounded to a detected precision, the code figures out how any decimal places can be reliably stored and then adds one, and rounds everything to that. This is done in a somewhat hacky way, but it works, and also removes the requirement for specifying decimal places in a definition file.

Screenshot from the readme attached.

Edit: I just noticed that I forgot to actually set values for the axis labels on graphs, it seems to be using arbitrary ranges... whoops

[willemml]

I have working KWP2000 init. Not difficult or impressive given how much info there is online about how to do this, even on this forum. But hey, it's progress.

[prj]

I've always wanted to write a disassembler. At least I won't be short of projects for a while. I really just need to write the KWP flasher and logger. I will just use ecu files from ME7Info. For now at least.

For a lot of disassembly you can use binutils. Takes a ton of massaging to bind it to .NET, but if you're using straight gcc then should be easier.

prj: it sounds like you wrote an automated C166 emulator/debugger combo.

Emulator/debugger already exists, it's called Keil uVision.

[willemml]

For a lot of disassembly you can use binutils. Takes a ton of massaging to bind it to .NET, but if you're using straight gcc then should be easier.Emulator/debugger already exists, it's called Keil uVision.

I didn't realize how powerful binutils was, I've only really used it for checking sizes of sections. I'm using Rust (LLVM) but FFI makes gcc interfacing easy. I dislike .NET, at least it's usually easy to reverse engineer .NET apps...
Would still be interesting to write one from scratch.

Emulator/debugger already exists, it's called Keil uVision.

Sure, but it sounds like what you are doing is closer to emulation than just pattern matching on assembly. Following going through the code while keeping track of registers, like the stack pointer and program counter, is getting awfully close to emulating the CPU. Ultimately emulating or simulating code can be a form of analysis on binaries.

In the end I don't know, or need to know, how you do it. I am just thinking about how I would do it. If uVision has a good ABI it could be interesting to use it for doing this.

I played with IDA a bit and was not a great experience. The c166 module for ghidra seemed to have some issues, likely user error because I did not spend much time playing with it.

[prj]

Sure, but it sounds like what you are doing is closer to emulation than just pattern matching on assembly.

I would not call it emulation, but there is a certain intermediate layer that understands some of the program flow.

[willemml]

Well I got my bench harness working properly, and immo deleted my unitronic ECU. Bench testing this is much nicer than doing it in the car... Now trying to figure out how to do the rest of the init after 5 baud and maybe read the DTCs.

For some reason the ECU seems to be stuck in session type 0x83, and replies a negatively anytime I send it something.

[Artemisia]

Well I got my bench harness working properly, and immo deleted my unitronic ECU. Bench testing this is much nicer than doing it in the car... Now trying to figure out how to do the rest of the init after 5 baud and maybe read the DTCs.

For some reason the ECU seems to be stuck in session type 0x83, and replies a negatively anytime I send it something.

Hope this documentation helps. For flash programming, the logic path would be something like that:

- Start diagnostic session 0x85
- Evaluate response
- If ECU responds that security access haven't been granted yet: Request security access with access mode 1 (requestSeed)
- Use the SA2 instruction tape to solve the seed and send the response to the ECU. Re-try session 0x85 once security access is granted

The SA2 instruction tape can be retrieved in .sgo files. Often the same for similar controllers

Flashing segments:
- RequestDownload
- StartRoutine 0xC4 (Erase flash)
- RequestRoutineResult until positive response
- TransferData
- Checksum

Before flashing a segment, the data needs to be compressed using Bosch BCB Type 1 compression.
The compression use a flag to indicate if the byte doesn't repeat or if it repeats and how many times. Then the compressed data needs to be encrypted, it uses XOR encryption with the following key: GEHEIM (literary "SECRET" in German). A header needs to be added to the encrypted segment to let the ECU know this is compressed/encrypted data (0x1A 0x1 IIRC)

Checksum is done on the raw data segment using CRC32B, you send the last two bytes

Once all segments have been downloaded, StopDiagnosticSession.


Off-topic, I saw you have worked with Java and C++. What do you prefer from Rust over those two languages? I am seeing more interest and have been contemplating fiddling with it

[willemml]

Wow thank you so much, that logic path is very helpful, I didn't even know about the security access.

I will try to implement all that. Also those two PDFs are incredibly helpful, and basically what I have been looking for. I found a single English one on google but it didn't go into specifics. Luckily my German is just good enough to understand these two.

Why Rust over Java/C++/anything else?

This can very quickly turn into a very very long paper. There is a lot of personal preference involved. I'll try to answer in a somewhat minimal way though. Also my C++ experience is incredibly minimal compared to Java, I learned Java before I was 10, and my C++ experience is mostly limited to reverse engineering device drivers... The one actual project I started in C++ wound up being redone in Rust very early on.

Java runs in a JVM, which is not conducive to running code on embedded hardware/microcontrollers. My goal for the flashing/logging portion of this project is that it could run on something like an ESP32 or one of my STM32 calculators. My choice of Rust over C++ is based on familiarity. The projects I was working on at the time I was starting to learn both were much simpler to implement in Rust, so I never bothered actually learning C++. On the other hand I learned Java before I was 10, and I don't really want to touch it again even after many many years of using it. Same goes for JavaScript and .NET.

As for why I personally prefer one over the other?
- I dislike dealing with the JVM.
- Rust does away with null values, you can have optional values
- I really like Rust's pattern matching functionality, it is incredibly powerful
- I haven't had a project that made me want to truly learn C++ (most of my learning time is spent on languages such as Haskell, Elixir, APL, FORTH) so I never became fluent in it
- Rust's borrow checker makes memory efficient programs very easy to write.
- The trait system in Rust is very flexible, instead of multiple layers of classes that inherit each other, you have a type that implements various functionalities.
- Rust has a very powerful macro system, much nicer (in my opinion) than the text replacement C++ does
- Java and especially C++ have subpar dependency/package management systems, Rust's cargo is incredibly powerful, and platform independant
- Rust has a very large library of "crates" (libraries) with very few compatibility issues
- The built in documentation functionality is easy to use
- Both the unit and integration test systems are very easy to use in Rust

This can be summarized as Rust is low-level, has amazing tooling that is standardized, it is memory safe and efficient, and finally, it is very easy to get setup on microcontrollers.

[jcsbanks]

Interesting project.

I like C# for a GUI.

There are so many styles to write C++, but I really appreciate std::map, std::vector, std::string, std::find etc in microcontrollers with a heap. C99 with no heap for an ECU.

I suspect I'm a candidate for becoming a Rust lover, but I'm juggling JS, Python, assembler, C#, C99, C++, PHP and a few other scripting methods plus more I'm forgetting already. I only use what is necessary to cover the paradigms from various servers and serverless services down to ECUs, but it has evolved over time of course.

I dislike Java too.

IDA and bindiff work for hundreds of versions in production with automation for characteristics, measurements, function and hooking various of them, but I have a love/hate relationship with it and it benefits from combining with pattern matching of data in addition to its function matching strategies. The old versions of IDA needed IDC scripts to properly resolve flash accesses in particular, and there are breaking changes that have prevented me upgrading as well as incompatibilities with bindiff. IDC can fly along well, but it is certainly quicker to develop with IDA python though. Ghidra I like, but it is just deathly slow... Java related presumably.

[willemml]

Update: thanks to the documents from Artemisia I have full KWP2000 encoding and decoding implemented. Communication with the ECU seems to work reliably and it responds to messages correctly. Now that I have a complete KWP2000 over K-line implementation I can start writing flashing and logging code. I will probably also try to implement what VCDS does.

Attached is a screenshot of the output when run with commit aa6f406. Code on github https://github.com/willemml/kwp2000-rs

[willemml]

Interesting project.

Thanks! It's been very fun so far. Lots of learning.

I like C# for a GUI.

Never tried it, I tend to avoid C# because of the heavy windows relationship. I have written some simple .NET to break the DRM on some game music, but other than that nothing.

There are so many styles to write C++, but I really appreciate std::map, std::vector, std::string, std::find etc in microcontrollers with a heap. C99 with no heap for an ECU.

I suspect I'm a candidate for becoming a Rust lover, but I'm juggling JS, Python, assembler, C#, C99, C++, PHP and a few other scripting methods plus more I'm forgetting already. I only use what is necessary to cover the paradigms from various servers and serverless services down to ECUs, but it has evolved over time of course.

I dislike Java too.

I think you will enjoy Rust, especially given your favourite parts of C++. I used to write a lot of JS, I still write some Python at work (I don't work in software, but Python is helpful for data translation and some CAD stuff). My main experience with assembly is writing my own architectures/instruction sets for fun. PHP is the first language I ever learned, but I haven't touched it since I learned Java, so I wouldn't even know where to start anymore.

Side note, I have been writing my code for the kwp2000 project in such a way that it is only a couple lines to change it from heap based to heapless. And I have been trying to minimize copy operations. This code should work very nicely on microcontrollers.

[nyet]

logging code. I will probably also try to implement what VCDS does.

worthless for ME7, slow and very limited. don't bother. Same with basic kwp logging, you really need direct memory access.

[willemml]

I mean the coding, adaptation, and basic settings functionality. I know VCDS logging is useless, but it would be nice to have a free alternative for VAGCOM. The primary targets of this project are DMA logging and bin flashing. VCDS features are just extra because there is no free competition with that software that I know of.

[willemml]

Got security access working. Wound up having to do the $23/$3B hack to disable the security fail timeout because otherwise debugging was too slow, wound up being a byte order issue...

Cannot get into programming mode for some reason though, but developer mode seems to work. Stuck with repeated response pending negative replies until eventual timeout (10 seconds).

[Artemisia]

Got security access working. Wound up having to do the $23/$3B hack to disable the security fail timeout because otherwise debugging was too slow, wound up being a byte order issue...

Cannot get into programming mode for some reason though, but developer mode seems to work. Stuck with repeated response pending negative replies until eventual timeout (10 seconds).

Is it stuck on the first StartDiagnosticSession 0x85 or after the seed has been resolved? I haven't worked with KWP2000 ECUs, but on some TP2.0 ECU, the following does the trick:

- Once the seed is resolved, let the connection time out
- Once disconnected, re-init the KWP2000 connection
- *You can either try to go straight to RequestDownload or try StartDiagnosticSession 0x85, then RequestDownload

It might be possible the ECU only allow a programming session on specific timing parameters. You could also use ODIS-E to flash a SGO and sniff the communication line. You would get the full payload to handle flash programming


For the SA2 implementation, it seems you are using this one:
0x68,0x05,0x81,0x4A,0x05,0x87,0x5F,0xBD,0x5D,0xBD,0x49,0x4C
(Loop 5 times: rotate shift left (RSL), if carry is clear - skip XOR, XOR 0x5FBD5DBD)

If you plan to expand to more ECU in the future, it could be interesting to do a full implementation. Although, I think most, if not all ME7, uses the same one

   *Removed code suggestion

Great work so far, love to see the frequent updates and a Rust project I can keep up with. Very interesting language