NefMoto

Technical => Data Logging => Topic started by: setzi62 on November 05, 2010, 04:40:15 AM



Title: Logging with KWP-2000 protocol
Post by: setzi62 on November 05, 2010, 04:40:15 AM
I was wondering which method should be used for data logging using a KWP2000-protocol connection.
Can't have a look how VCDS (turbo-button) or ECUx are doing it, but maybe somebody knows more details?

Increasing the measurement rate by setting up higher communication speed is clear.
But which service is best used to read out measurement values?
You can read data using the service readDataByLocalIdentifier or read RAM contents
using readMemoryByAddress (if allowed by the ECU) or use whatever method to retrieve
memory contents.

I know that some ECUs support the readDataByLocalIdentifier with *localIdentifiers*,
which reports measurement groups in a similar form as done with KW1281. This would be relatively
easy to use but seems to be not working for most ECU's if I'm not wrong.
Most ECUs support dynamically defined identifiers by memoryAddress, allowing to read also
RAM cells, but this could be slow since it's reading only a limited number of bytes
per service.

Also could read RAM in blocks and then filter out the measurement values. This
could make sense, if a lot of measurement values are lying at adjacent addresses.
It's clear you need a mapping table to show which addresses hold the values you want
to measure, and this table will be specific for each ecu sw-version.
Here it is unclear to me how to generate such a table, where to start gathering
information about the addresses?

And finally, could it cause problems when reading without being somehow synchronized to
the engine control process?  I mean, is it possible to get wrong results when retrieving
values that are currently under update?


Title: Re: Logging with KWP-2000 protocol
Post by: spen on November 09, 2010, 01:51:14 PM
I'd go for read32BitAddress with the following absolute addresses.

(http://i272.photobucket.com/albums/jj172/spen0007/ramcells.jpg)



Title: Re: Logging with KWP-2000 protocol
Post by: iznogoud on November 09, 2010, 02:58:09 PM
The question about "race conditions" is an interesting one. I think there must be a 'trigger" somewhere in the algorithm.

Can you please state OS and hardware? Thanks.


Title: Re: Logging with KWP-2000 protocol
Post by: ArgDub on November 09, 2010, 10:13:29 PM
for speed, definitely readDataByLocalIdentifier with dynamicallyDefinedLocalIdentifier, using readDataByLocalIdentifier with 1byte header each request message is only 4 bytes long.


Title: Re: Logging with KWP-2000 protocol
Post by: spen on November 10, 2010, 01:48:28 AM

For speed, the can bus has to be the way to go?


Title: Re: Logging with KWP-2000 protocol
Post by: setzi62 on November 10, 2010, 03:03:25 AM
I'd go for read32BitAddress with the following absolute addresses.

(http://i272.photobucket.com/albums/jj172/spen0007/ramcells.jpg)


thanks a lot for the address table, that's a good starting point for me.
Will try to find some matches in other images for these variables.
I don't know a service with 32bit address on the ME7.x, so I guess you mean the
service readMemoryByAddress(sid=0x23)?

P.S. can't see the ramcells.jpg in your post here, seems to be filtered by the firewall.


Title: Re: Logging with KWP-2000 protocol
Post by: setzi62 on November 10, 2010, 03:07:44 AM
The question about "race conditions" is an interesting one. I think there must be a 'trigger" somewhere in the algorithm.

Can you please state OS and hardware? Thanks.
not shure about your question for OS and HW. In general I'm looking at ME7.x ECUs with
ERCOS on it. The logging application could run on any platform.
If "regular" mechanisms to read values are used like the official measurementGroup reading,
I believe the ECU will take care to eliminate race conditions.
But how is the situation when using inofficial mechanisms for reading engine process variables?
It could come out that this in fact is no problem, since the KWP2000 messages are handled
by the ecu in a task that's scheduled on a regular basis and thereby somehow synchronized
to the rest of the engine process. At least that's what I hope ...



Title: Re: Logging with KWP-2000 protocol
Post by: setzi62 on November 10, 2010, 03:11:43 AM
for speed, definitely readDataByLocalIdentifier with dynamicallyDefinedLocalIdentifier, using readDataByLocalIdentifier with 1byte header each request message is only 4 bytes long.
yes, this would allow the fastest reading of small ram portions.
But also processing timing in the ecu comes into place (messages handled only each 10ms)
so that the number of bytes in a request probably do not affect the number of messages
per second.
Then, only 10 dynDefLocalIdent's are available and I could associate only a single memory
location with each dynDefLocalIdent. That could limit the number of variables being logged
in parallel. But on the other side, can log up to 253 sequent bytes in one read and I could
use this service on any ecu tested so far. Many pro's and con's.


Title: Re: Logging with KWP-2000 protocol
Post by: spen on November 10, 2010, 03:53:10 AM
I don't know the service names which bosch use.  I know there is a routine to read an address though.

Just a crazy idea,  we could upload a function, and add it as a process ID for the ecu to complete and then end comms.  Our new function would then broadcast a set frame of preamble + addresss data we define (n addresses) + crc check when the k line is empty.    Then we switch our datalogger to 'sniff' mode and reap the frames?  Very low overhead, as long as we don't saturate the K line and / or drive the other controllers too hard with interrupts.

Still not as fast as the can bus though!


Title: Re: Logging with KWP-2000 protocol
Post by: Rick on November 10, 2010, 05:58:47 AM
Spen,

would the use of CAN not rule out certain ECU's?

Rick


Title: Re: Logging with KWP-2000 protocol
Post by: spen on November 10, 2010, 11:51:47 AM
I think even cars without esp still have canbus coming out of the ecu.  Have a look at your wiring diagram for the c ecu, it should be labelled can hi, can lo, or maybe Chi Clo.

CAN is better than KWP2000 for us (c and d ecu users).  Our cars won't take KWP2000 over the k line.  My ecu will talk KWP2000 on a bench :(  but not in the car.  I think you verified this with your hybrid ecu?

I dount our cars have CAN at the OBD socket.  Mine has the infotainment CANBUS but not the management can bus.  Infotainment bus!  It links the TV tuner to the RNS-D, nothing more!  My DIS is clk/data lines!





Title: Re: Logging with KWP-2000 protocol
Post by: Tony@NefMoto on November 10, 2010, 12:19:29 PM
My data logging software using ReadMemoryByAddress to do the data logging. ReadDataByLocalIdentifier and ReadDataByCommonIdentifier don't appear to work on the ME7. The message handlers are hooked up, but there are no local or common identifier defined, so you can't read anything besides the ECU identification data. You can use DynamicallyDefineLocalIdentifer by memory address, but it is not as fast as using ReadMemoryByAddress. Also, you are required to be in a Development diagnostic session to use ReadMemoryByAddress, while DynamicallyDeifineLocalIdentifier only requires a Standard diagnostic session.

Requests for ReadMemoryByAddress are fulfilled synchronously in one loop, so you don't need to worry about having half the memory being read from the previous ECU update or anything. But you do need to worry about how often and how much data you are requesting be read. My ECU crashed at 5000RPM while doing a data logging test on the street. Engine shut off and wheels locked up, then the engine restarted a second later, but I nearly lost control of the car when it happened. Turns out if you ask for too much data to be read when the engine is running at a high RPM, the watch dog timer goes off and resets the ECU. I now test data logging before driving by hitting the rev limit in the driveway, and I have also reduced how much data I read with each request.

The data logger I plan to release will require a user to supply a memory layout file specifying the location, data type, offset, and scale of the different variables to log. On the ME7 as far as I can tell, the only generic way to data log is with the slow KWP1281 protocol. With the KWP2000 protocol since common and local identifiers aren't defined, you have to do everything by memory addresses, which change with each ECU. I believe this is why APR ultimately discontinued work on ECUx.


Title: Re: Logging with KWP-2000 protocol
Post by: Rick on November 10, 2010, 02:10:55 PM
I think even cars without esp still have canbus coming out of the ecu.  Have a look at your wiring diagram for the c ecu, it should be labelled can hi, can lo, or maybe Chi Clo.

CAN is better than KWP2000 for us (c and d ecu users).  Our cars won't take KWP2000 over the k line.  My ecu will talk KWP2000 on a bench :(  but not in the car.  I think you verified this with your hybrid ecu?

I dount our cars have CAN at the OBD socket.  Mine has the infotainment CANBUS but not the management can bus.  Infotainment bus!  It links the TV tuner to the RNS-D, nothing more!  My DIS is clk/data lines!


Spen,
very interesting that you can't use KWP over  OBD.  Have you tried Tony's software in your car?  Do you know why it doesn't work in car?  I've not looked at the wiring diagram yet, but is it due to the other modules on the Kline not liking KWP2000?

If so, I will run a cable straight from the ECU plug to the OBD socket, and have an isolation switch which disconnects everything from the Kline apart from the ECU and OBD socket.  That should work as that's how it is on the bench.  Pain in the arse having to bench flash each time.

Rick


Title: Re: Logging with KWP-2000 protocol
Post by: ArgDub on November 10, 2010, 05:39:50 PM
DynamicallyDeifineLocalIdentifier allows to define a LocalIdentifier made of blocks of non consecutive memory. The document don't specify what is maximun length of a LocalIdentifier, but suppose that its 100 bytes, then you can define a LocalIdentifier of 100 consecutive bytes or a LocalIdentifier of 100 bytes at non consecutive addresses. I think this is more economic in terms of message length than read 250 bytes to use only a few


Title: Re: Logging with KWP-2000 protocol
Post by: Tony@NefMoto on November 10, 2010, 10:31:59 PM
There are 10 dynamically defined local identifiers.

Each dynamically defined local identifier can have a max of three entries from what I could determine from the assembly code. Each entry can be used to read a maximum of 255 bytes of memory it seems. But a KWP2000 message has a maximum of 253 data bytes after the service ID and local identifier in the response. So each local identifier read using a dynamically defined local identifier can essentially read a maximum of 253 bytes spread over three different memory ranges.

Read data by local identifier requests take the service ID and the local identifier as data for a total of two bytes of data in addition to the header. The response includes one byte local identifier and then the read data.

Reading memory by address, you can read a maximum of 254 bytes from one memory range.

Read memory by address requests takes a three byte memory address and a one byte memory size as data for a total of four bytes of data in addition to the message header. The response just includes the read data.

Ultimately if you have 30 different memory regions you want to read, then it may better to use dynamically defined local identifiers. If you want to read many many different memory regions, then you may be better off using read memory by address. This is because you have to reset and redine dynamic identifiers if you want to change them.

And like I said, ReadMemoryByAddress requires a Development session, and ReadDataByLocalIdentifier requires a Standard session.


Title: Re: Logging with KWP-2000 protocol
Post by: setzi62 on November 11, 2010, 05:56:34 AM
Just a crazy idea,  we could upload a function, and add it as a process ID for the ecu to complete and then end comms.  Our new function would then broadcast a set frame of preamble + addresss data we define (n addresses) + crc check when the k line is empty.   
Starting an own process/task on the ecu would be a hard job. Need to setup
a task table and code in ram and then find a way to start the task periodically.
Probably an alarm is free for this use. But conflicts with the running tasks that already
use the serial port could not be avoided.
What can be done is: you can load own code and table into the external ram and then
link it in by redirecting a pointer. So your code gets called whenever a message comes
in and works as an additional message handler for your own logging message.
The code can work on a table and collect all memory locations in one go.


Title: Re: Logging with KWP-2000 protocol
Post by: setzi62 on November 11, 2010, 06:03:55 AM
On the ecu's I inspected only one memory section could be defined for reading with dynDefLI's.
Or I am too stupid to define multiple sections? Shouldn't it be done like this:
2C F0 04                            - clear
2C F0 03 01 10 38 00 00     - define 1st section
2C F0 03 02 10 38 10 00     - define 2nd section
21 F0                                 - read
And on some ECU's the readMemoryByAddress is NOT supported, even after
securityAccess & turning on debugSession. Seems like the list of different communication
implementations on ME7 is endless ...
So a "general" data logger most probably needs to support different methods
for reading the memory.


Title: Re: Logging with KWP-2000 protocol
Post by: ArgDub on November 11, 2010, 11:17:11 AM
Tony, seems that developer of ecux found a way to bypass the three entries restriction. I've just checked that it can pull at least 67 values (8-16bit mix at non consecutive address) with a single readDataByLocalIdentifier. Also, noted it doesn't use DynamicallyDeifineLocalIdentifier, it defines LocalIdentifier writing directly to ram.

It's funny that whatever baud rate you choose, it never change timing nor baud rate, so reading 67 values it achives 12 samples per second at default 10400 bauds, and 14 sps if you read one value.

Guillermo


Title: Re: Logging with KWP-2000 protocol
Post by: ArgDub on November 11, 2010, 11:21:33 AM
Forgot to add it uses development session.


Title: Re: Logging with KWP-2000 protocol
Post by: Tony@NefMoto on November 11, 2010, 12:37:10 PM
Tony, seems that developer of ecux found a way to bypass the three entries restriction. I've just checked that it can pull at least 67 values (8-16bit mix at non consecutive address) with a single readDataByLocalIdentifier. Also, noted it doesn't use DynamicallyDeifineLocalIdentifier, it defines LocalIdentifier writing directly to ram.

It's funny that whatever baud rate you choose, it never change timing nor baud rate, so reading 67 values it achives 12 samples per second at default 10400 bauds, and 14 sps if you read one value.

Guillermo

Thanks for looking into that. The dynamically defined local identifier entries are stored in RAM, so it is possible you could use WriteMemoryByAddress and manually write them into RAM without using DynamicallyDefineLocalIdentifier. Writing the entires straight into RAM would allow you to bypass the three entry limit per local identifier. The three entry limit is only enfored when defining identifiers.

But this trick is even less portable across ECUs, because you need to find where the local identifier entries are stored and make sure to not to use RAM used by anything else. You could find the location of the local identifier entries by reading RAM, defining local identifiers, and reading RAM again. I am not sure how you can confirm how much RAM is free to use if you want to use more than the standard three entries per identifier. Also, by doing it this way you lose the ability to run in a Standard diagnostic session.

I am planning on releasing my data logger based on ReadMemoryByAddress before I work on trying any new techniques. But it's pretty cool to be hearing some new ideas on the subject. ;D


Title: Re: Logging with KWP-2000 protocol
Post by: setzi62 on November 12, 2010, 03:02:10 AM
Yeah, thanks for the information that you all gave in this thread, that was really what
I hoped to find. This presents again a good starting point for my own experiments  ;)

Guillermo, did you log the serial interface to find out what happens, and would you
share such a log?


Title: Re: Logging with KWP-2000 protocol
Post by: ArgDub on November 12, 2010, 10:16:38 PM
setzi, I would like to hear about those experiments ;D

Attached are two logs, first RPM and IAT were logged, in second 68 variables were logged. I wrote an explanatory note next to messages but there are many question marks for you guys.

Edit:
ECU 06A906032HS


Title: Re: Logging with KWP-2000 protocol
Post by: Tony@NefMoto on November 14, 2010, 01:16:35 PM
The first write of 8 bytes is manually defining a dynamically defined local identifier. The five writes of 61 bytes are all going to unused RAM regions. I believe this is manually writing the dynamically defined local identifier entries.


Title: Re: Logging with KWP-2000 protocol
Post by: setzi62 on November 15, 2010, 09:02:54 AM
It's like Tony said, the data table of the local identifier (describing ram address +
number of bytes for each variable to be read) is written into unused ram and
the data pointer in the local identifier table is redirected to the new data table.
For the 68 variables log the communication speed should have been 57600 baud.
I did some small updates in the excel file comments about this.

At all, the used logging method is a nice trick to get over some limits of the ecu functionality.

Just for curiosity: when playing with dynDefLI's, I saw two things:
- could use 3 memory regions per ddli only if the first two regions each just contain one byte.
- with a Cbox image for the definition of a ddli the memory address had to be
  given in swapped order (low byte first, the middle and finally last high byte), strange??


Title: Re: Logging with KWP-2000 protocol
Post by: prj on January 20, 2013, 05:51:51 AM
Using some specs and some data in this thread I implemented a data logger.

I wanted to try readByMemoryAddress, as that seems a bit more portable, and hoped to use it on a few other ECU's as well.
However, as I see it:
1. readMemoryByAddress only takes a single address and then a length in bytes, I can not give it an array of addresses.
2. The maximum update rate I got out of my ECU on my bench is about 17hz using this method.

So unless I am doing something wrong, it is my understanding that I have to read out whole big blocks of RAM.
If I read two different blocks I will get about 8-9 samples per second.
No idea if I ever get a reply to this, as it seems that most people who have understanding of these things have left this forum for greener pastures, but here is to trying...


Title: Re: Logging with KWP-2000 protocol
Post by: prj on January 20, 2013, 06:12:25 AM
I think I missed the accessTimingParameters service...
Will try it, and probably increase the sample rate :)


Title: Re: Logging with KWP-2000 protocol
Post by: prj on January 20, 2013, 11:18:05 AM
Yup, that allowed me to get 50hz...
So if I use the dynamic identifier hack I can get 50hz with arbitrary addresses, but this is ECU specific and we already have ME7Logger for that.
ReadMemoryByAddress should work on the diesel ECU's as well, so I need to get some of them and test.


Title: Re: Logging with KWP-2000 protocol
Post by: nyet on January 20, 2013, 10:17:05 PM
No idea if I ever get a reply to this, as it seems that most people who have understanding of these things have left this forum for greener pastures, but here is to trying...

I am definitely interested. In particular, i'd also like to try to reproduce ME7L functionality (even if it isn't general!) so it can be built as some sort of portable library for use in other platforms, as well as being able to redirect the output somewhere useful (like a domain socket to a local process, or tcp socket to a remote host).

Do you have sample code anywhere in your repo?

I might try to rewrite it in C

i goddamn hate vb with a passion.


Title: Re: Logging with KWP-2000 protocol
Post by: prj on January 21, 2013, 06:17:02 AM
Pretty sure ME7L is already written in C.
It's not hard to get the logging working, but for ME7 there is already ME7Logger and I am doing this mostly to get it to work on some Diesel ECU's, and maybe newer ECU's like MED9 and EDC16.
Not sure if they will work on K-Line or need a CAN interface though.


Title: Re: Logging with KWP-2000 protocol
Post by: nyet on January 21, 2013, 02:49:17 PM
Pretty sure ME7L is already written in C.

I'm pretty sure setzi isn't going to give me the source code :(


Title: Re: Logging with KWP-2000 protocol
Post by: prj on January 21, 2013, 03:56:20 PM
This thread contains all the relevant data, and also there are KWP2000 specs posted on this forum as well.
It literally took me 2 days from knowing nothing about KWP2000 to getting a working logger.

Of course, nothing even closely as polished as ME7Logger, and of course it does not have the binary processor that ME7Logger has.
But it's not exactly rocket science.

That said, there is nothing you will accomplish functionality wise by making your own logger.
ME7Logger already uses the best possible method to log ME7, period.


Title: Re: Logging with KWP-2000 protocol
Post by: nyet on January 21, 2013, 04:51:04 PM
ME7Logger already uses the best possible method to log ME7, period.

And its closed source, which means I can't extend it to do what I want it to do. Which means I am forced into making my own logger.


Title: Re: Logging with KWP-2000 protocol
Post by: prj on January 21, 2013, 05:05:29 PM
Well, make one then.
The complicated part is detecting where the variables are in the bin, not making the logger itself.
And well, finding a bit of free RAM you can safely overwrite, and where the pointer the dynamic table is (unless it's always the same of course, which would make sense).

There is an XLS file in this very thread that shows exactly how ecuxplorer operates.
ME7Logger does _exactly_ the same thing, the only difference is, a higher baud rate is used and the AccessTimingParameters service is used to reduce the inter-byte and inter-message minimums to 0, and achieve the 50hz rate.

setzi62 took ecuxplorer's method and improved it, but the logger it is not original at all.
What is really good about ME7Logger is ME7Info - that is a much finer and more original piece of engineering, and this is what makes the package really good.


Title: Re: Logging with KWP-2000 protocol
Post by: nyet on January 21, 2013, 05:25:51 PM
What is really good about ME7Logger is ME7Info - that is a much finer and more original piece of engineering, and this is what makes the package really good.

Agreed. Thankfully that part of it works just fine; there's really nothing i need to add to it.

Hopefully I can make a library in C that is crossplatform and extensible for logging.

I'm not sure its really possible to make it truly crossplatform, so ill initially just concentrate on cygwin/mingw and msvs

I really want to avoid using cmake etc.


Title: Re: Logging with KWP-2000 protocol
Post by: Basano on March 28, 2014, 03:27:31 AM
Hi guys,

Been poking around a bit with logging MED9.1 over CANBUS.

The information in this thread is really good, especially the traces in the spreadsheets and I’ve had a go at replicating this on MED9.1 over CANBUS. Up to now I’ve been logging using VCDS but I wanted to see how the RAM logging works. prj, I see you looked at something similar here (http://nefariousmotorsports.com/forum/index.php?topic=3110.0title=). How did you get on? This might be a bit niche, even for this forum ;D, but if anyone has played with this I’d really like to hear from you! All the services and codes I mention below are described in the KWP2000 protocol.

1)   What service are you using to read the RAM? I tried #23 ReadMemoryByAddress but it was rejected. Instead I used #2C DynamicallyDefineLocalIdentifier and read this with #21 ReadDataByLocalIdentifier.

2)   If you are also using #2C DynamicallyDefineLocalIdentifier, have you seen that only the last entry can have a size > 1 byte?

Valid examples (accepted by ECU)
One memory address:
{0x00, 0x08, 0x2C, 0xF0, 0x03, 0x01, 0x0F, 0x80, 0x00, 0x00}
Two memory address:
{0x00, 0x0E, 0x2C, 0xF0, 0x03, 0x01, 0x01, 0x80, 0x00, 0x00, 0x03, 0x02, 0x0F, 0x80, 0x00, 0x10}
Three memory address:
{0x00, 0x14, 0x2C, 0xF0, 0x03, 0x01, 0x01, 0x80, 0x00, 0x00, 0x03, 0x02, 0x01, 0x80, 0x00, 0x10, 0x03, 0x03, 0x0F, 0x80, 0x00, 0x20}

Invalid examples (rejected by ECU)
Two memory address:
{0x00, 0x0E, 0x2C, 0xF0, 0x03, 0x01, 0x0F, 0x80, 0x00, 0x00, 0x03, 0x02, 0x0F, 0x80, 0x00, 0x10}
Three memory address:
{0x00, 0x14, 0x2C, 0xF0, 0x03, 0x01, 0x0F, 0x80, 0x00, 0x00, 0x03, 0x02, 0x0F, 0x80, 0x00, 0x10, 0x03, 0x03, 0x0F, 0x80, 0x00, 0x20}

3)   Do you have the RAM address and scaling of some common variables (RPM, requested boost, actual boost etc)? I’d like to see if I can actually read them, but I don’t have the skill to disassemble myself and find them ??? I’ve attached my bin if it helps.

4)   If there is a size limit of just one byte for all except the last entry in the DynamicallyDefineLocalIdentifier, how do you work around this for 16bit variables (two bytes)? Define two consecutive memory locations (one right after the other) and read one byte from each, effectively reading two byte?

5)   I was able to define at least 17 entries in a single DynamicallyDefineLocalIdentifier. I didn’t try any more than this, but it seemed at least it’s not restricted to just three like the thread suggest on ME7.1

6)   I also found the equivalent RAM tables where this resides in MED9.1 The addresses are 0x80 4334 and 0x80 38CC. For example, if I define a DDLI like this KWP2000 {0x00, 0x08, 0x2C, 0xF0, 0x03, 0x01, 0xAF, 0x80, 0x11, 0x22} it will populate the RAM tables like this:

RAM address - Contents
0080 4334 - 00 01 00 00 00 80 38 CC <- one entry in the DDLI, points to 0x80 38CC
0080 38CC - 00 AF 00 00 00 80 11 22 00 00 <- contains the size and address. In this case, size 0xAF starting at 0x80 1122

7)   I saw 10 DDLI’s in the RAM
0080 4334 - 00 01 00 00 00 80 38 CC
0080 433C - 00 00 00 00 00 80 39 6C
0080 4344 - 00 00 00 00 00 80 39 84
0080 434C - 00 00 00 00 00 80 39 9C
0080 4354 - 00 00 00 00 00 80 39 B4
0080 435C - 00 00 00 00 00 80 39 CC
0080 4364 - 00 00 00 00 00 80 39 E4
0080 436C - 00 00 00 00 00 80 39 FC
0080 4374 - 00 00 00 00 00 80 3A 14
0080 437C - 00 00 00 00 00 80 3A 2C

8)   I can see that ecuxplorer and setzi62’s ME7Logger write to the RAM tables directly #3D WriteMemoryByAddress but unfortunately when I try this on MED9.1 it is rejected. Has anyone been able to write to the RAM on MED9.1?

9)   Also attached is a log where I define the DDLI and then dump the RAM. This helped me to find the RAM tables that the DDLI lives in.

As always, thanks!


Title: Re: Logging with KWP-2000 protocol
Post by: Aurélien on March 28, 2014, 07:51:56 AM
1) 0x2C and 0x21, just like you.

2) I don't remember this. I remember doing a log with rl_w and nmot_w. I will check.

3) I don't have a definition matching your file so I made a quick search in ida :

0x7FD830 : nmot_w
0x802554 : ldtv_w
0x7FEE0B : zwist

4) If this is the case, yes.

5) It's restricted to 20.

6) Cool. :)

8) I never tried but with 6) you can probably try !


Title: Re: Logging with KWP-2000 protocol
Post by: nyet on March 28, 2014, 09:42:44 AM

4)   If there is a size limit of just one byte for all except the last entry in the DynamicallyDefineLocalIdentifier, how do you work around this for 16bit variables (two bytes)? Define two consecutive memory locations (one right after the other) and read one byte from each, effectively reading two byte?


This seems like a bad idea, since you sample the two bytes before and after the value has changed.


Title: Re: Logging with KWP-2000 protocol
Post by: Aurélien on March 28, 2014, 10:15:45 AM
This seems like a bad idea, since you sample the two bytes before and after the value has changed.

I don't think so.
It would be the case if you do in 2 request.
But there we are talking about one single request that is asking lo byte and hi byte :)


Title: Re: Logging with KWP-2000 protocol
Post by: B234R on March 28, 2014, 11:20:46 AM
@Basono

At which security level (seed&key) did you try use service #23 ReadMemoryByAddress?


Title: Re: Logging with KWP-2000 protocol
Post by: Basano on March 29, 2014, 05:11:07 AM
1) 0x2C and 0x21, just like you.

2) I don't remember this. I remember doing a log with rl_w and nmot_w. I will check.

3) I don't have a definition matching your file so I made a quick search in ida :

0x7FD830 : nmot_w
0x802554 : ldtv_w
0x7FEE0B : zwist

4) If this is the case, yes.

5) It's restricted to 20.

6) Cool. :)

8) I never tried but with 6) you can probably try !


Auré, thank you!

If anyone can confirm the single byte size limit it would be nice to know. Maybe I've got it wrong?

Reading two consecutive bit's of memory was just a suggested workaround. I'd prefer not to do it, much better just to be able to specify size and address.

@Basono

At which security level (seed&key) did you try use service #23 ReadMemoryByAddress?

I was using diagnostic session 86 (development session) with key = seed + 0x11170. Any other suggestions? Also note that #35 RequestUpload worked and I used that to dump all the the RAM at once (32kb's worth)

Using Auré's suggestion for nmot_w, I logged this last night on the way home. It does indeed change as the rpm's change, from about 11 @ 760rpm to about 47 @ 3000rpm. Is this an 8 bit value or 16 bit value? I just logged one bytes worth but would like to know if it's bigger.

I got sample rates of about 40 samples per second. That's with just the default VW TP2.0 timings. I'm sure it could be faster if you changed the timers. Probably my coding is pretty rubbish  ;D





Title: Re: Logging with KWP-2000 protocol
Post by: prj on March 29, 2014, 03:51:16 PM
20 definitions is the limit, but if I recall it correctly, then each definition signifies a memory range!
So you can define 20 memory RANGES that you can log simultaneously.

This is more than enough considering many variables are placed sequentially after one another.

To find the addresses of the variables you have to parse the table that contains all of them. Look at the MED9 FR, it lists a lot of variables in a certain order. All you need to do is find where this list is stored, parse it, and voila, you have all the variables required.


Title: Re: Logging with KWP-2000 protocol
Post by: Basano on March 30, 2014, 12:51:03 PM
Look at the MED9 FR, it lists a lot of variables in a certain order.

Thanks prj!

That list of variables, are you refering to TKMWL ???


Title: Re: Logging with KWP-2000 protocol
Post by: Basano on March 31, 2014, 01:34:43 PM
Using Auré's suggestion for nmot_w, I logged this last night on the way home. It does indeed change as the rpm's change, from about 11 @ 760rpm to about 47 @ 3000rpm. Is this an 8 bit value or 16 bit value?

Whoops. It was staring me in the face :-[

The w in _w is for word. So nmot_w is 16 bits. Answered my own question.

Logged nmot_w again (two bytes this time!) and multiplied by 0.25 (scaling factor). I get the rpm in high resolution ;D


Title: Re: Logging with KWP-2000 protocol
Post by: Aurélien on April 01, 2014, 09:31:26 AM
20 definitions is the limit, but if I recall it correctly, then each definition signifies a memory range!
So you can define 20 memory RANGES that you can log simultaneously.

This is more than enough considering many variables are placed sequentially after one another.

To find the addresses of the variables you have to parse the table that contains all of them. Look at the MED9 FR, it lists a lot of variables in a certain order. All you need to do is find where this list is stored, parse it, and voila, you have all the variables required.

Agreed.

BUT, it's not that easy. First not all interresting variable are listed, second the list is indirect.




Title: Re: Logging with KWP-2000 protocol
Post by: Aurélien on April 01, 2014, 09:34:40 AM

I got sample rates of about 40 samples per second. That's with just the default VW TP2.0 timings. I'm sure it could be faster if you changed the timers. Probably my coding is pretty rubbish  ;D


What hardware are you using ? Lawicell ?
Also, C ? C++ ? C# ?



Title: Re: Logging with KWP-2000 protocol
Post by: Basano on April 08, 2014, 03:14:45 AM
 :D no, nothing as fancy as that.

Arduino with Sparkfun CANBUS shield and the Arduino IDE language (very similar to C).

Cheap and cheerful!


Title: Re: Logging with KWP-2000 protocol
Post by: dream3R on April 08, 2014, 10:41:59 AM
Nice, I started in Arduino, the Uno has limited RAM but worked fine.

Chipkit is nice if you want more power ;)


Title: Re: Logging with KWP-2000 protocol
Post by: MIL_on on April 08, 2014, 10:41:13 PM
did anyone else start implementing ME7Logger on Arduino?  Thats what i am just in...
the great thing about it would be that you are able to display every var you want on a display, log them via SD-Card without Laptop (maybe not so important) and connect external sensors and view everything in ECUxPlot!


Title: Re: Logging with KWP-2000 protocol
Post by: Basano on April 16, 2014, 01:37:02 PM
Hi all,

I’ve managed to modify the MED9.1 assembly code to allow me to specify ram locations larger than one byte for logging.

(The below explanation makes a lot more sense if you read the last couple of pages of this thread first)

Earlier in this thread I described a problem that I encountered when specifying DDLI’s (DefineDynamicLocalIdentifier) for logging. If I specified a block of memory locations, only the last location could have a size larger than one byte. All the other memory locations had to be one byte in size otherwise the command would fail. I tried a few combinations of the command, but never found a way around it. I don’t know if it’s my ecu or a general MED9.1 issue. If I wanted to log a 16 bit variable (two bytes) I specified the memory location (size one byte) and then the next consecutive memory location (again size one byte) and put them together to get the full 16 bits. Also, if your variables lay close together in memory, you could read a chunk of memory and parse it for what you want. Hmmm.

However I thought it would be neater when defining the DDLI to simply specify a memory location and size thereof. This is similar to how ME7Logger works and I don’t see why it couldn’t work here.  You’ve got twenty locations in the first DDLI and I didn’t want to waste them.

The mechanism of ME7Logger is really well described at the start of this thread. I began by defining a few random DDLI and then dumping the contents of the RAM (since the DDLI’s should live in RAM like ME7). Dumping the RAM was surprisingly simple – exactly the same as reading the flash, use the same KWP commands you’d use to retrieve the flash, but just specify the start location to be 0x800000 instead of 0x000000. After that I had a snapshot of the RAM. Search through that for the random DDLI’s previously defined and you can see the locations where they are written to. Then disassemble with IDA and look at the cross references that refer to those RAM locations. That gave me an entry point into the code that handles the DDLI definition.

After many dead-ends and puzzlement, I reached a section in the assembler that I’m reasonably sure checks the size of the DDLI’s. Basically it compares the size to another variable. I simply replaced the CMP operation with a NOP. Now when I specify a block of memory locations in a DDLI, I can specify the size of each location (no longer restricted to one byte for the prior locations) and the command is accepted. Once the command is accepted, the ecu happily works with the size parameter and outputs the expected data.

Attached are the before and after bins so you can see the change I’m talking about if you are curious. It’s a very small change (replace CMP with NOP).

I’d love to hear from anyone logging on MED9.1 to see if you faced a similar issue and how you’ve worked with it.


Title: Re: Logging with KWP-2000 protocol
Post by: prj on April 16, 2014, 01:48:29 PM
Most people use VCDS for MED9.
The patch I don't like. This involves modifying the binary instead of just logging the car.

You should check where that variable comes from and see if it is possible to modify it.
Another approach is, since you know the variable address, then if it is not populated often, maybe it is possible to WriteMemoryByAddress on that variable before starting to log.

Or, just do what ME7Logger does and fill the DDLI structure directly via WriteMemoryByAddress?


Title: Re: Logging with KWP-2000 protocol
Post by: Basano on April 17, 2014, 01:02:09 AM
Most people use VCDS for MED9.

I use VCDS as well. I have the genuine article and it’s helped me more times than I can count. I worked on the ram logging because a) it was interesting b) I was looking at variables that aren’t present in the measuring blocks that VCDS reads (like rl_w and the PID variables)  c) the sample rate was a bit faster.

The patch I don't like. This involves modifying the binary instead of just logging the car.

Yes. If you are a pro and working on many vehicles, patching binaries is a bit impractical. I’ve got the one car which I tinker with day in and day out. I think you can choose how to log as circumstances allow and what is appropriate . Ram logging via DDLI works very nicely without any patches at all. The patch allows me to make the most of the DDLI’s in my scenario. ymmv.

You should check where that variable comes from and see if it is possible to modify it.
Another approach is, since you know the variable address, then if it is not populated often, maybe it is possible to WriteMemoryByAddress on that variable before starting to log.

Or, just do what ME7Logger does and fill the DDLI structure directly via WriteMemoryByAddress?

Read and WriteMemoryByAddress doesn’t work on my MED9.1. I’ve tried all sorts of variations, diagnostic levels and security access. I even tried defining a DDLI and then using WriteDataByLocalIdentifier to indirectly write the memory. No joy. Attached is an extract from the FR showing what services are described. I’d loved to be proved wrong, but even in the Funktionsrahmen, WriteMemoryByAddress isn’t mentioned (having said that, #34 RequestDownload isn’t described either, but that definitely works!)



Title: Re: Logging with KWP-2000 protocol
Post by: prj on April 22, 2014, 01:24:35 PM
No easy way then besides making an automatic patcher to patch binaries.
That's pretty easy to do though.
I have so many ideas what to do, but I just do not have time when I have a carpark outside the shop needing tuning.

To make this accessible, one would need to implement this with a easy to get cable as opposed to standalone MCU setup.
For example ELM327, BMW K+DCAN or ask Ross-tech for specs and implement it with the VCDS cable.

The more interesting part is to parse the variable structure found in the ECU, so that the variable config can be auto generated, like ME7Logger does.


Title: Re: Logging with KWP-2000 protocol
Post by: Basano on April 28, 2014, 02:18:25 AM
Thanks,

I've got some ideas I will post up soon :-)


Title: Re: Logging with KWP-2000 protocol
Post by: melodic on August 12, 2014, 09:08:55 PM
I have an interesting idea.
Take the arduino due (it has 2 canbus). Read needed groups by OBD can bus and transfer they to the dashboard instead song name. Also write logs to SD card. Maybe log G-sensor and GPS or anyone else. To control we can use cruise buttons.


Title: Re: Logging with KWP-2000 protocol
Post by: dream3R on January 01, 2016, 09:39:19 PM
I'd like to reignite this if interest just as a hobby.


Title: Re: Logging with KWP-2000 protocol
Post by: dream3R on January 01, 2016, 09:41:36 PM
I use VCDS as well. I have the genuine article and it’s helped me more times than I can count. I worked on the ram logging because a) it was interesting b) I was looking at variables that aren’t present in the measuring blocks that VCDS reads (like rl_w and the PID variables)  c) the sample rate was a bit faster.

Yes. If you are a pro and working on many vehicles, patching binaries is a bit impractical. I’ve got the one car which I tinker with day in and day out. I think you can choose how to log as circumstances allow and what is appropriate . Ram logging via DDLI works very nicely without any patches at all. The patch allows me to make the most of the DDLI’s in my scenario. ymmv.

Read and WriteMemoryByAddress doesn’t work on my MED9.1. I’ve tried all sorts of variations, diagnostic levels and security access. I even tried defining a DDLI and then using WriteDataByLocalIdentifier to indirectly write the memory. No joy. Attached is an extract from the FR showing what services are described. I’d loved to be proved wrong, but even in the Funktionsrahmen, WriteMemoryByAddress isn’t mentioned (having said that, #34 RequestDownload isn’t described either, but that definitely works!)



With the Simos FR and comparing the Audi Diag DB there is $cmds missing from the FR for sure, here not so sure.


Title: Re: Logging with KWP-2000 protocol
Post by: kuebk on September 01, 2016, 01:39:47 PM
Does anyone have experience with EDC17 logging over KWP2000?
ReadMemoryByAddress is not supported by this ECU, dynamicallyDefineLocalIdentifier (reading by memory address) should be supported (as FR says) but actually its not.

> 23 40 73 50 02 01 01
< 7F 23 11
> 2C F0 03 01 02 40 73 50
< 7F 2C 11

I had success with requestUpload/transferData but are there more efficient ways to read RAM?




Title: Re: Logging with KWP-2000 protocol
Post by: ozzy_rp on September 02, 2016, 03:30:57 AM
Does anyone have experience with EDC17 logging over KWP2000?
ReadMemoryByAddress is not supported by this ECU, dynamicallyDefineLocalIdentifier (reading by memory address) should be supported (as FR says) but actually its not.

> 23 40 73 50 02 01 01
< 7F 23 11
> 2C F0 03 01 02 40 73 50
< 7F 2C 11

I had success with requestUpload/transferData but are there more efficient ways to read RAM?

Try to use 0x35 command

Here line from Diagnostic Service Config with supported command:
DiagCom_ServiceTable_t <0x35, 0xFF, 0xFFFB, 8, sub_8007E8B2, sub_8007E880, off_8002ED18> ; requestUpload


Title: Re: Logging with KWP-2000 protocol
Post by: dream3R on September 02, 2016, 12:27:29 PM
Sounds like uds that?


Title: Re: Logging with KWP-2000 protocol
Post by: kuebk on September 02, 2016, 01:04:50 PM
Try to use 0x35 command

Here line from Diagnostic Service Config with supported command:
DiagCom_ServiceTable_t <0x35, 0xFF, 0xFFFB, 8, sub_8007E8B2, sub_8007E880, off_8002ED18> ; requestUpload


I'm already using 0x35/0x36, but is there any other way?

Sounds like uds that?

It's KWP.


Title: Re: Logging with KWP-2000 protocol
Post by: elRey on May 06, 2017, 08:55:46 PM
edit: my post may violate license of me7logger. removing it for now.


Title: Re: Logging with KWP-2000 protocol
Post by: de_F on October 31, 2018, 06:42:26 AM
I'm already using 0x35/0x36, but is there any other way?

It's KWP.


Sorry for resurrecting this old thread,

Service 35 RequestUpload seems to work on MED17 and as long as I aim for an upload within 0x40000h and 0x7FFFF I get my response as requested.

The RAM tables on MED17 for example are possibly located at 0xC0000000 or 0xD0000000 ect (32bit address). according to A2L's floating around.
It looks like the standard KWP protocol can only handle 24bit addresses, so up to 0xFFFFFF.

I understand you got this working kuebk to make your logger over 0x35 RequestUpload / 0x36 TransferData. But for the moment I keep on getting answer 52 canNotUploadFromSpecifiedAddress (so my addressing is definitely wrong as I am aiming for a blocked/protected address)

Any pointers or possibly just a way to calculate address 0xC0000000 so my request passes ?


Title: Re: Logging with KWP-2000 protocol
Post by: kuebk on November 01, 2018, 01:55:36 AM
Memory regions are covered in detail in A2L.


Title: Re: Logging with KWP-2000 protocol
Post by: de_F on November 03, 2018, 12:33:46 PM
Thanks for the heads up. Should work with 0x400000 and 0x500000 then. I'll give it a shot in the next few days.