Pages: [1] 2
Author Topic: mmll: the Most Minimal Linux Logger  (Read 35797 times)
trichard3000
Full Member
***

Karma: +6/-1
Offline Offline

Posts: 57


« on: July 03, 2013, 10:53:24 AM »

mmll runs under Python 2.  It's FTDI-only and you’ll need pylibftdi (as well as libftdi) installed before running it.

This early-alpha code is for those who want to log from non-windows environments or who want to poke around with Python code that talks directly to the ECU using KWP2000.  It's not for those who expect anything other than a hobby-quality attempt. Feel free to let me know what's broken or better yet, submit your own tweaks.  However, don't expect any "support" on this as I only work on it when I can.

It's seriously buggy, poorly programmed, non-standard, rough around the edges, and cobbled together from a few dozen different examples and recipes.  I’m sure that if I knew more about python or programming in general I could’ve achieved the same results with half the code.  But hey, it’s a start!

Also, it’s a complete hack.  I started with raw captures of the serial traffic from ME7L and I tried to pick out what’s going on from a KWP2000 point of view.  The info I had was skimpy and there are a few parts where I really don’t know how it works.  So far, I’ve only coded in support for baud rates of 19200, 38400, 56000, and 57600.  Also it’s only HM0. 

It's not particularly fast, the logging maxes out at about 25 records per second (of 40 logging values each) on my Raspberry Pi at 57600.  I also tested it on an older Gentoo box I have.  I doubt it would work on Windows in its current form.

On the good side, I made it compatible with the ME7L cfg and ecu files.  Just drop the cfg and ecu files in the same directory as mmll.py and point it to the cfg.  For example:

mmll.py –c config.cfg –o logoutput.csv

Finally, the output is also “cloned” from the ME7L output so it’s already compatible with ECUxPlot.

Here’s the command line:

usage: mmll.py [-h] -c CONFIGFILE [-o OUTPUTFILE] [-d {0,1,2,3,4}]

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIGFILE, --configfile CONFIGFILE
                        The logging config file
  -o OUTPUTFILE, --outputfile OUTPUTFILE
                        The desired output log file - No entry outputs log
                        data to STDOUT
  -d {0,1,2,3,4}, --debug {0,1,2,3,4}
                        Increase the Debug Level (experimental and currently broken)


Here's the link:  https://github.com/trichard3000/mmll

Have fun!
« Last Edit: July 03, 2013, 03:10:36 PM by trichard3000 » Logged
julex
Hero Member
*****

Karma: +79/-4
Offline Offline

Posts: 923


« Reply #1 on: July 03, 2013, 12:19:58 PM »

I will never use it (windows guy!) but cool to see alternatives!
Logged
oldcarguy85
Full Member
***

Karma: +15/-1
Offline Offline

Posts: 247


« Reply #2 on: July 03, 2013, 01:12:50 PM »

Just had a really quick read through your code.  I don't think it's poorly coded at all -- Granted, i'm not a python programmer, but it seems well organized.  Good work!! Hopefully someone can explain that init sequence.  I definitely read an explanation of how to make this connection at some point but i can't remember where i found it.  Anyway, good work!  Hopefully i'll get a chance to try this out soon.
Logged
trichard3000
Full Member
***

Karma: +6/-1
Offline Offline

Posts: 57


« Reply #3 on: July 03, 2013, 02:47:12 PM »

Maybe I'm being hard on myself.  :-)   If everything is set up right, and if your ECU uses the same communications settings as mine does (S4 M-box), then this should work just fine.

However, when it doesn't work, it doesn't fail very gracefully.  There's not much error checking.  When I started, I wasn't quite sure how the protocol would work out so there's evidence of a lot of trial and error in the code.  As my python-fu improved, I started standardizing things a bit more but I still ended up doing the same kinds of things numerous different ways.  I'm sure that some of what I did could've been done much easier if I only had known about some other modules.

Anyway, the code is loosely organized into a MVC structure with the model provided by the ME7L configs and me7lconfig.py, the controller provided by pylibme7.py, and the view produced by mmll.py.

I'd like to see this extend into other ECU functions, such as code reading/resetting, measuring block views, and maybe even flashing.  In the short-term, I just wanted to get some logging done using the Raspberry Pi. 

Please let me know if you run in to any issues.
Logged
jibberjive
Hero Member
*****

Karma: +23/-2
Offline Offline

Posts: 536


« Reply #4 on: July 05, 2013, 01:36:25 AM »

Very cool.  I like the idea of possibly permanently throwing in a raspberry pi to log whenever.
Logged
CoupedUp
Full Member
***

Karma: +0/-0
Offline Offline

Posts: 59


« Reply #5 on: July 09, 2013, 05:11:33 AM »

In reply to Jibberjive: Throw in a Wifi USB dongle and you've got a remote 'server' for data collection when it's parked in the garage or driveway. Cheesy
Logged
jibberjive
Hero Member
*****

Karma: +23/-2
Offline Offline

Posts: 536


« Reply #6 on: July 09, 2013, 07:25:51 AM »

I have no idea if it is possible nor how easy/difficult it might be, but if this logger (or ME7L) could be ported to android (edit: after a quick google, it looks like it shouldn't be too hard at all to run the python program in android), it could be relatively simple (and really cool) to build an 'android stick' based 'car pc' headunit.  Maybe with something like this that comes already equipped with wifi and bluetooth and install a touchscreen in the dash http://slickdeals.net/f/6132802-CX-919-Android-PC-Stick-Quad-core-1-6Ghz-Bluetooth-WiFi-4-2-JellyBean-OS-2GB-RAM-Best-Rated-62-99-FREE-SHIPPING .  Could add a usb or wifi/bluetooth webcam as well for a cheap dashcam. Add a bluetooth OBD II dongle for cool gauges if you want them.

Or I know you can run ubuntu on some of these sticks, so that could take care of the logger aspect.
« Last Edit: July 09, 2013, 07:33:51 AM by jibberjive » Logged
rnagy86
Sr. Member
****

Karma: +6/-11
Offline Offline

Posts: 367


« Reply #7 on: July 09, 2013, 07:32:57 AM »

It is possible to run Python code on Android devices, however it would be way nicer if  this would be re-written in C  Grin

That way one could basically create a module for any language out there or port it to any device what one could imagine.
« Last Edit: July 09, 2013, 07:34:35 AM by rnagy86 » Logged
trichard3000
Full Member
***

Karma: +6/-1
Offline Offline

Posts: 57


« Reply #8 on: July 10, 2013, 12:17:23 PM »

In reply to Jibberjive: Throw in a Wifi USB dongle and you've got a remote 'server' for data collection when it's parked in the garage or driveway. Cheesy

Bingo!  As I flesh this thing out, I want it to connect via WiFi when I get home and automatically sync the log files with a Dropbox folder.  My WiFi dongle just came in the mail yesterday.  So did a GPS dev board.  Smiley

...it looks like it shouldn't be too hard at all to run the python program in android
...Add a bluetooth OBD II dongle for cool gauges if you want them.
...you can run ubuntu on some of these sticks

I don't know anything about Android.  Python certainly runs on Android but there are probably a couple of differences in file management. The only non-pure-python requirement is libftdi and it looks like there's already a port (or one in progress). 

No need for the OBDII dongle.  I'm already logging on the K-wire and I can do all kinds of stuff with the data.  I'm already working up a gauges proof-of-concept that uses HTML5 to display gauges via the browser on on Android, iPhone, laptop, whatever.  This can easily be expanded to be the control panel for the whole thing.  Smiley

Once I get this more fleshed out, I'll post the code and details of the hardware.

I'm running the Raspberry Pi build of Debian Linux which has a ton of support.   


 
...it would be way nicer if  this would be re-written in C  Grin

That way one could basically create a module for any language out there or port it to any device what one could imagine.

I chose python for a few reasons: 
  • It's the go-to language for Raspberry Pi projects
  • Very approachable for a newbie
  • Existing, open-source drivers and libraries for FTDI
  • Highly portable to multiple platforms

In addition, you could even call Python from C if you wanted to.  I think you'd find that porting this to other devices would be pretty straightforward.  On the other hand, my code could be quick and dirty reference for someone wanting to do the same thing in C. 

One thing I've learned with all of this is that the intersection of embedded computing and electronics is very, very approachable these days.  Hobbyist computers like the Raspberry Pi make it easy and inexpensive to just jump in and start trying things.  My development environment is simply a $40 Pi, an SD card, and my bench flashing rig.  Here's a (very ugly) pic:



Have fun!
Logged
Axis
Full Member
***

Karma: +4/-4
Offline Offline

Posts: 91


« Reply #9 on: July 10, 2013, 01:07:27 PM »

So did a GPS dev board.  Smiley
GPS coordinates in log file to display on googlemaps when you replay recorded data in html5 would be nice, to remember where and under which conditions things in file happened. Perhaps also a future option to record audio synchronized with log to be able to make voice notes while driving/logging, which would of course go to an external .wav file or similar.
Logged
jibberjive
Hero Member
*****

Karma: +23/-2
Offline Offline

Posts: 536


« Reply #10 on: July 10, 2013, 01:18:14 PM »

No need for the OBDII dongle.  I'm already logging on the K-wire and I can do all kinds of stuff with the data.  I'm already working up a gauges proof-of-concept that uses HTML5 to display gauges via the browser on on Android, iPhone, laptop, whatever.  This can easily be expanded to be the control panel for the whole thing.  Smiley
I'm curious to see what you come up with regarding gauges.  My reasoning behind the bluetooth dongle would be because there already are well-developed apps with nice gauges etc. If you're developing gauges though, that would be awesome to be able to put any variable we can log into a gauge.
Logged
blah
Newbie
*

Karma: +0/-0
Offline Offline

Posts: 4


« Reply #11 on: September 24, 2013, 03:34:38 PM »

No need for the OBDII dongle.  I'm already logging on the K-wire and I can do all kinds of stuff with the data. 
How are you wiring this up to the Raspberry Pi? 

Thanks!  awesome project, motivating me to learn Py
Logged
trichard3000
Full Member
***

Karma: +6/-1
Offline Offline

Posts: 57


« Reply #12 on: December 04, 2013, 11:46:07 AM »

So...  I got very busy with work for the past few months and this all went on the shelf.  Right after I put this rough code out, Setzi released the Linux port of his ME7L logger.  His is WAY more developed than this code.  This code represents only a peek behind the curtain of coaxing logging records out of ME7 and would probably crash on anything other than an m-box ECU.  I kind of figured that the need for an alternate, more limited logger had passed.   

Now I'm starting to dig back in to this.  I'm still RaspberryPi based and I still want to run everything from Python.  I was thinking about scrapping this whole approach and writing a Python wrapper around Stezi's C-based ME7L library.

Advantages:  Better support for more cars and options.  Disadvantages:  Lack of transparency into core logging process and closed-source code. 

At this point, I'm more interested in tinkering with logging control and doing stuff with the output, rather than the guts of logging, itself.  I'm kind of okay with outsourcing the low-level logging engine to ME7L. 

Anyone have any opinions on this?  I can keep working on the native Python version but I'd need a lot of help with understanding the protocols and logic behind how ME7's set up memory for logging.

Thanks.
Logged
trichard3000
Full Member
***

Karma: +6/-1
Offline Offline

Posts: 57


« Reply #13 on: December 04, 2013, 11:55:48 AM »

How are you wiring this up to the Raspberry Pi? 

Thanks!  awesome project, motivating me to learn Py

Sorry for the slow follow-up!  At the moment, I'm simply using my blue, eBay, VAGCOM-compatible cable.  With access to logged values, we can do all kinds of stuff with the data. 

My gauge demo (that I'm dusting off) is one example to pipe the data to a web-based UI.  I'll pull some stuff together to show off.   
Logged
Lauria112
Jr. Member
**

Karma: +0/-0
Offline Offline

Posts: 39


« Reply #14 on: May 23, 2016, 12:08:38 AM »

Trichard3000, did you continue to work on this or the wrapper for ME7L library?
I have similar plans to install RPi (3) on my A6. I have also build small Python program to view some data out of logged data in "real time".
It is a quick and dirty program (my first on python), so I need to clean it up before publishing.
Problem was that setzis ME7Logger fails to run on RPi or did you get that to work somehow?

I will most likely fire this up before weekend on 4B0907551G ECU and I will let you know how things go.

Regards
Lauri
Logged
Pages: [1] 2
  Print  
 
Jump to:  

Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Page created in 0.028 seconds with 17 queries. (Pretty URLs adds 0.001s, 0q)