|
Title: Open Source Checker for ME7... Post by: 360trev on December 30, 2012, 08:38:33 AM ETA (nyet): new thread here
http://nefariousmotorsports.com/forum/index.php?topic=3347.0 Q. How many C developers/lurkers on here? Well for you guys, here is my attempt to calculate checksums for my ME7.x firmwares from the info I found (mostly on Andy's site) so it was indeed inspired by good old Andy Whittakers work and done quite a long time ago now, sorry its taken so long to upload it. Decided to upload this it since some guys on here won't release their source code to ME7 stuff and I thus feel innovation dies, come on guys step up and help your fellow brothers. ... This software compiles under win32 using Mingw (executables included in the zip too) but very easy to cross compile to Linux (I did try and it worked fine in Ubuntu). Anyway, this stuff is completely freeware, do anything you want with it, its BSD licensed so free as in free beer. My tool uses config files (ini files) so will work with many ME7 dumps out there, it would be trivial to update the checksums and write them back to the file if so desired (no need to buy ANY tools to do this!!). Let me know if someone wants this feature. Anycase I will provide that feature in the next update, I just ran out of time at the time I did this. It may be useful to some. This was done to work with the Ferrari 360 dump I did (using a custom home made bench flash cable). I've uploaded the firmware dump in the usual place. Here's an example of the config file (as it stands today) # # Ferrari 360 (For Bosch Ignition Computers) # # This is the base configuration for Ferrari 360 firmware dumps # for Bosch ME7.3 firmware decoding & checksum generation # [ignition] # main rom checksumming rom_firmware_start=0x800000 rom_checksum_block_start=0x1fc20 rom_checksum_block_len=0x10 rom_checksum_offset=0x01bc88 rom_checksum_final=0x07ffe0 # boot sector validation rom_boot_Startaddr=0x00800000 rom_boot_Endaddr=0x00803fff rom_boot_Chksum=0x0fa0f5cf rom_boot_InvChksum=0x0f05fa30 # rom data segment information dumps [dumps] dump_show=8 dump_1_type=String dump_1_visible=true dump_1_label=EPK : dump_1_offset=0x1002b dump_1_len=43 dump_2_type=String dump_2_visible=true dump_2_label=SW Number : dump_2_offset=0x1015f dump_2_len=8 ... cut ... Relatively straight forward right? The dumps section allows you to print out interesting ascii string information to shell.. This means you can indeed create new version of the ini files for other ME ROM based cars easily. I plan to add more stuff soon like a command line shell environment to allow you to edit roms (from a cli environment) and re-calc the checksums and then flash back to car. Will likely create a GIT project when I get some time.. Right now its very simple indeed... Hope its useful to someone. It allowed me to go much further at the time.... If anyone's interested in the pinout diagram to dump these roms from the Ferrari Bosch ecu's let me know... I have pictures and pinout information I did at the time on my camera somewhere. This is a great starting point for the Ferrari 360 for which there is very little information posted around the web... I've also included in the zip me7stuff pointing out where one of the maps is in this dump, again easy enough to create 'templates' to find these in as future tool. (and add it as editable) so people can modify from cli environment. Anyway let me know if there is interest in this concept (i.e. fully open source) tool. Free that everyone can contribute to. Regards, Trev https://github.com/360trev/ME7Sum/ Title: Re: Open Source Checker for ME7... Post by: rnagy86 on December 30, 2012, 09:01:58 AM I thought x-mas was over :o ;D
Title: Re: Open Source Checker for ME7... Post by: 360trev on December 30, 2012, 09:08:54 AM I thought x-mas was over :o ;D :) Perhaps we can all create some config files for the various roms available? Title: Re: Open Source Checker for ME7... Post by: rnagy86 on December 30, 2012, 09:26:10 AM :) Perhaps we can all create some config files for the various roms available? First I would really upload it to github so that it's easier to contribute. Title: Re: Open Source Checker for ME7... Post by: 360trev on December 30, 2012, 10:25:37 AM As requested,
There is now an open source code repository on GitHub at: https://github.com/360trev/ME7Sum (https://github.com/360trev/ME7Sum) This is where all of the source-code is now posted. Regards, Trev Title: Re: Open Source Checker for ME7... Post by: prj on December 30, 2012, 10:34:59 AM I haven't looked at this yet, but you are off to a very good start here.
Title: Re: Open Source Checker for ME7... Post by: 360trev on December 30, 2012, 10:48:29 AM I haven't looked at this yet, but you are off to a very good start here. Thanks, hopeful its useful to others too. Would be great to have some more good open source cross platform tools that everyone can contribute to. Title: Re: Open Source Checker for ME7... Post by: rnagy86 on December 30, 2012, 11:58:38 AM I will start with replacing the ini parser because that is not portable for sure -- sometime around next week
Title: Re: Open Source Checker for ME7... Post by: nyet on December 30, 2012, 12:08:01 PM Awesome. Thanks. I will try to get a working csum corrector working asap.
Title: Re: Open Source Checker for ME7... Post by: nyet on December 30, 2012, 12:10:46 PM You should remove the .o files from the repo.
Title: Re: Open Source Checker for ME7... Post by: nyet on December 30, 2012, 12:18:56 PM Made a few changes.
Intermixing pointers and integers is not 64-bit clean. Title: Re: Open Source Checker for ME7... Post by: 360trev on December 30, 2012, 12:23:11 PM I will start with replacing the ini parser because that is not portable for sure -- sometime around next week Actually surprisingly it is fully 100% portable, ini files are just a text format. Not as popular or expressive as XML but easy to use. I've already tested the code under Linux aand even ran it under an embedded microcontroller so its 100% cross platform. Its just a text format after all. May not be the trendiest but its simple. Feel free to replace with XML if you want though. Title: Re: Open Source Checker for ME7... Post by: 360trev on December 30, 2012, 12:24:25 PM Made a few changes. Intermixing pointers and integers is not 64-bit clean. Great find. Yes, I only tried to compile so far on 32-bit toolchain so far! Title: Re: Open Source Checker for ME7... Post by: 360trev on December 30, 2012, 12:25:55 PM Awesome. Thanks. I will try to get a working csum corrector working asap. I think this would make it super useful to the average (non coder) vs us guys. ;) Title: Re: Open Source Checker for ME7... Post by: 360trev on December 30, 2012, 12:48:09 PM You should remove the .o files from the repo. whoops. These tools work so much better under Linux than Windows... been trying to commit the change but its failing for obscure reasons. Will dual boot my machine and revert to Linux to continue dev. Title: Re: Open Source Checker for ME7... Post by: nyet on December 30, 2012, 05:07:56 PM oh, also the .dll
i'll be working on the makefile as well. i'll be targeting cygwin/gcc and linux initially. I'm sure i can get cygwin/vc working, but i dont want to deal with nmake etc. so ... i hope that isn't an issue going forwards. if people really want a vc/msvs environment, i figure they're on their own and can deal with it themselves. also cloned your tree and opened a pull request for a few things. Thanks so much for making the source available, I've been waiting a long time for an excuse to get a proper checksummer. Title: Re: Open Source Checker for ME7... Post by: nyet on December 30, 2012, 08:33:43 PM Here is what i have for C-box (from andy's site)
Code: [ignition] Here is what i have for mbox Code: [ignition] https://github.com/nyetwurk/ME7Sum Title: Re: Open Source Checker for ME7... Post by: nyet on December 30, 2012, 10:01:32 PM Question: why is the rom boot checksum hardcoded in the ini? the m box location (for example) appears to be at 1fbda in the multipoint area..
is that just a sanity check? Also, is there code for the rolling XOR crc32? it is missing from your code (and obviously andy's) Code: The ME7.1 also contains a 3 stage CRC32 based checksum. This is the very well known crc32 algo. Andys' analysis is rather silly, since it can be summarized as: Code: while (size--) Also, see this thread: http://www.nefariousmotorsports.com/forum/index.php?topic=343.0 I have the CRC code written (including reading the seed table, which is pointless, since they are known constants) but i can't figure out what blocks to apply it to, and where the values are stored... can anybody help? Title: Re: Open Source Checker for ME7... Post by: 360trev on December 31, 2012, 08:05:46 AM Wow, you've been busy!
Great work. I'd not even remembered about this pesky CRC checksum. I am very familiar with CRC's having used them a lot before. The easiest way to detect their position is to take a known dump and simply modify some bytes at various points and then got some to use a working tool to re-calculate the crc's. Then all thats required is a binary 'diff', you'll easily identify the bytes which changes (and where re-calculated). These will be a very good starting point to derive the address ranges used too. Alternatively (more work) from the other threads people quote addresses of the functions in the machine code, if you provide me with exact firmware they where talking about (or point me to the location to d/l it) I'll fire up the old C16x dissassembler and reverse it. T Title: Re: Open Source Checker for ME7... Post by: 360trev on December 31, 2012, 11:10:25 AM Just added another VERY simple open source project. Bdiff to my git repositories.
https://github.com/360trev/bdiff (https://github.com/360trev/bdiff) Quite simply this is a VERY simple tool written in 5 minutes. It can be used to generate difference reports for checksum calculated files. I.e. If you have the file with a bad checksum and then get it corrected by a checksum tool you can see the byte offset in the file and the differences it made. This is a great start to understand exactly what bytes where changed when a checksum has been calculated by some other tool. Best way to use this is take 2 files from the requests for checksums folder and run it on them. You will see the exact bytes that have been changed. Also attached initial checked in version with compiled for win32 (windows cli) to this reply for people whom don't have a toolchain installed and just want to use the app from the CLI. Title: Re: Open Source Checker for ME7... Post by: nyet on December 31, 2012, 11:14:22 AM Still can't work out the purpose of the rom boot checksum in the ini... can I remove it? Or omit it for files that already have it as part of the multipoint checksum?
thanks for your work, btw! Title: Re: Open Source Checker for ME7... Post by: 360trev on December 31, 2012, 11:17:13 AM yep. It was originally just to validate the boot before proceeding further.. by all means knock yourself out..
Here's an example of the bdiff tool I just wrote against 2 files from the checksum requests folder... --- The original 2 files (chosen at random from that folder, both 1mb each) http://nefariousmotorsports.com/forum/index.php?topic=2665.0title= (http://nefariousmotorsports.com/forum/index.php?topic=2665.0title=) And the output... binary difference tool - 0.1 Last Built: Dec 31 2012, Time: 18:37:46 þ Opening 'My_Stock_B_BOX_ECU_Firmware_EMISSION_DELETE.bin' file þ Getting length of 'My_Stock_B_BOX_ECU_Firmware_EMISSION_DELETE.bin' file þ Allocating buffer of 1048576 bytes þ Reading file to buffer þ Validating size correct 1048576=1048576 þ Closing file þ Opening '8D0907551B - 0261206109 - 352738 CRC corrected' file þ Getting length of '8D0907551B - 0261206109 - 352738 CRC corrected' file þ Allocating buffer of 1048576 bytes þ Reading file to buffer þ Validating size correct 1048576=1048576 þ Closing file 0x0001fc1a ( 130074): be -> c3 0x0001fc1b ( 130075): b4 -> 81 0x0001fc1e ( 130078): 41 -> 3c 0x0001fc1f ( 130079): 4b -> 7e 0x0001fc3a ( 130106): 25 -> 1c 0x0001fc3b ( 130107): c1 -> 7c 0x0001fc3e ( 130110): da -> e3 0x0001fc3f ( 130111): 3e -> 83 0x0001fc4a ( 130122): 86 -> 2f 0x0001fc4b ( 130123): fe -> e0 0x0001fc4c ( 130124): d7 -> d9 0x0001fc4e ( 130126): 79 -> d0 0x0001fc4f ( 130127): 01 -> 1f 0x0001fc50 ( 130128): 28 -> 26 0x0001fd9a ( 130458): e1 -> d8 0x0001fd9b ( 130459): b6 -> 50 0x0001fd9c ( 130460): 86 -> 85 0x0001fd9e ( 130462): 1e -> 27 0x0001fd9f ( 130463): 49 -> af 0x0001fda0 ( 130464): 79 -> 7a 0x00073ae6 ( 473830): be -> 9c 0x00073ae7 ( 473831): 4e -> 62 0x00073ae8 ( 473832): 14 -> 8a 0x00073ae9 ( 473833): ff -> 69 0x00073af2 ( 473842): 26 -> f3 0x00073af3 ( 473843): f0 -> 7b 0x00073af4 ( 473844): 94 -> 6a 0x00073af5 ( 473845): b8 -> 48 0x000fffe0 ( 1048544): 13 -> 0a 0x000fffe1 ( 1048545): fa -> 94 0x000fffe2 ( 1048546): 13 -> 12 0x000fffe4 ( 1048548): ec -> f5 0x000fffe5 ( 1048549): 05 -> 6b 0x000fffe6 ( 1048550): ec -> ed Total byte differences: 34 (22) bytes þ All Done... Title: Re: Open Source Checker for ME7... Post by: prj on December 31, 2012, 11:20:11 AM Once you get this working decently I can make a TPro plugin out of it...
I am worried it might piss off MTX though. Title: Re: Open Source Checker for ME7... Post by: 360trev on December 31, 2012, 11:21:43 AM Once you get this working decently I can make a TPro plugin out of it... I am worried it might piss off MTX though. Lol. They've been MILKING this for far too long if I'm totally honest! Title: Re: Open Source Checker for ME7... Post by: nyet on December 31, 2012, 11:25:40 AM Once you get this working decently I can make a TPro plugin out of it... Yea, for sure. The key question is if you want to autodetect the checksum table locations by pattern recognition, or make a lookup table for the various binaries. Also, I could use your help finding the CRC tables :) Title: Re: Open Source Checker for ME7... Post by: prj on December 31, 2012, 11:33:09 AM Pattern recognition... if it was my way of doing it.
Adding every binary based on EPK's is going to drive you insane. Title: Re: Open Source Checker for ME7... Post by: jibberjive on December 31, 2012, 11:41:15 PM I can't do anything but be a cheerleader and say this is awesome. Slowly but surely I'll learn programming/disassembly and hopefully be able to contribute. Until then, I can just say that I appreciate the work.
Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 01, 2013, 05:29:39 PM Question: why is the rom boot checksum hardcoded in the ini? the m box location (for example) appears to be at 1fbda in the multipoint area.. is that just a sanity check? Also, is there code for the rolling XOR crc32? it is missing from your code (and obviously andy's) Code: The ME7.1 also contains a 3 stage CRC32 based checksum. This is the very well known crc32 algo. Andys' analysis is rather silly, since it can be summarized as: Code: while (size--) Also, see this thread: http://www.nefariousmotorsports.com/forum/index.php?topic=343.0 I have the CRC code written (including reading the seed table, which is pointless, since they are known constants) but i can't figure out what blocks to apply it to, and where the values are stored... can anybody help? This should be accurate... Check zone 1: Addresses 0x810002 - 0x813FFF 176 bytes at a time + remaining 13 Checked at 0x87A866 Check zone 2: Addresses 0x814252 - 0x817F4F 176 bytes at a time + remaining 125 Checked at 0x87A86C Check zone 3: Addresses 0x818192 - 0x81FBB1 176 bytes at a time + remaining 111 Checked at 0x87A872 Title: Re: Open Source Checker for ME7... Post by: nyet on January 01, 2013, 05:49:32 PM Wow. That is rather bizarre. Are those read from a descriptor someplace, or hardcoded?
Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 01, 2013, 06:05:15 PM Wow. That is rather bizarre. Are those read from a descriptor someplace, or hardcoded? Hardcoded. I did some coding and logging to confirm as well. Title: Re: Open Source Checker for ME7... Post by: nyet on January 01, 2013, 06:12:36 PM HRPMPH... the values aren't matching up.. here is what i have
zone 1: calc=0x412a5cc2 read=0x214cd272 zone 2: calc=0x0d75b941 read=0x17b90f53 zone 3: calc=0x8bcc3b89 read=0x7de91383 AH nvm ETA: those are non-inclusive ranges? :) zone 1: calc=0x214cd272 read=0x214cd272 zone 2: calc=0x17b90f53 read=0x17b90f53 zone 3: calc=0x7de91383 read=0x7de91383 Title: Re: Open Source Checker for ME7... Post by: nyet on January 01, 2013, 06:17:35 PM Not sure how we're going to autodetect those ranges... going to need prj's help for that.
For now i'll hard code them into my .ini Title: Re: Open Source Checker for ME7... Post by: 360trev on January 01, 2013, 06:35:36 PM Hardcoded. I did some coding and logging to confirm as well. Q. Did you check this against the stock 360 firmware I uploaded? (uploaded in the firmware section?) Here.. http://nefariousmotorsports.com/forum/index.php?topic=2992.0title= (http://nefariousmotorsports.com/forum/index.php?topic=2992.0title=) ...These boundaries quoted don't seem to map against any regions and obvious boundary ends like the end of a block of 'ff ff ff's and start of '00 00 00's. Also the check code addresses quoted are filled with ff's in my dumped image when I looked, not a CRC32 routine or any C16x assembly. What's the physical byte offset where the checksums are done? I will reverse it if need be... Title: Re: Open Source Checker for ME7... Post by: nyet on January 01, 2013, 07:00:57 PM Those are specific to S4 2.7t mbox
have you had a chance to check out my code? Title: Re: Open Source Checker for ME7... Post by: nyet on January 01, 2013, 07:30:06 PM What i dont get is that ALL the multipoint checksums are stored in one of the blocks checked by the multipoint checksum itself!
i.e. M-box multipoint is stored from 0x1fbd2-0x1ffc2, which is in block #8 (0x1c000-0x1ffff)... so how does one set up the checksums for that block, if the checksum itself is included in its own checksum area? or am i missing something? Also, i see that the main checksum is stored at fffe0, which is in the block 64 (fc000-fffff).. but thats ok, i can store that first, before doing multipoint, since the main checksum doesn't cover the multipoint area. Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 12:30:21 AM Oh well. I am going to ignore it since it seems to magically work!
I have just finished the part which actually corrects the binary image. * I ONLY HAVE CRC LOCATIONS FOR MBOX * The checked in code works under linux and cygwin. Currently trying to get a build working for windows using visual studio If you have linux or cygwin, please try it out! https://github.com/nyetwurk/ME7Sum prj, i could use your help figuring out how to autodetect stuff. I am pretty sure I can auto-detect the multipoint descriptors by pattern (they're obvious) The main checksum ones are also kind of easy (at least the block descriptors.. the checksum location might be harder to detect) The hardest part will be finding the CRC locations. ** MAKE A BACKUP OF ANY IMAGE YOU RUN THROUGH THIS! it does not make a backup of your image.. it modifies it in place! ** Title: Re: Open Source Checker for ME7... Post by: prj on January 02, 2013, 05:35:41 AM This should be accurate... Check zone 1: Addresses 0x810002 - 0x813FFF 176 bytes at a time + remaining 13 Checked at 0x87A866 Check zone 2: Addresses 0x814252 - 0x817F4F 176 bytes at a time + remaining 125 Checked at 0x87A86C Check zone 3: Addresses 0x818192 - 0x81FBB1 176 bytes at a time + remaining 111 Checked at 0x87A872 Which software revision are you referring to? M-Box? Title: Re: Open Source Checker for ME7... Post by: prj on January 02, 2013, 05:38:11 AM Never mind, I see it.
Title: Re: Open Source Checker for ME7... Post by: prj on January 02, 2013, 05:42:57 AM Ok so I checked a few different images.
The locations are mostly the same, but in some there are 3, in some there are 4 and the 29F400 ones are a different story. But I think I can come up with an algorithm to detect these fairly reliably... But really for this tool to be any good all the checksums have to be found via pattern search. Per-ECU revision ini files will make it completely useless as a real tool. Title: Re: Open Source Checker for ME7... Post by: 360trev on January 02, 2013, 06:42:26 AM Ok so I checked a few different images. The locations are mostly the same, but in some there are 3, in some there are 4 and the 29F400 ones are a different story. But I think I can come up with an algorithm to detect these fairly reliably... But really for this tool to be any good all the checksums have to be found via pattern search. Per-ECU revision ini files will make it completely useless as a real tool. prj, Could you do me a huge favour and see if you can see the checksum locations in the 360 dump I posted? Also i've re-merged the changed back in and got it re-working under Mingw again in a win32 environment. All sync'd back into github. Title: Re: Open Source Checker for ME7... Post by: 360trev on January 02, 2013, 06:46:21 AM Per-ECU revision ini files will make it completely useless as a real tool. Agreed, just too many variations on location otherwise, a different config for every rom would be very silly. Title: Re: Open Source Checker for ME7... Post by: 360trev on January 02, 2013, 08:38:41 AM nyet... I've done a few changes to your work as it no longer compiled under Win32 mingw env.
At first I managed to find an emulation/remapping function for mmap() [since this isn't part of MinGW and isn't supported natively by Windows without remapping to other api call equivalents under the nt kernel libs]. While this WORKED and is simple enough to do I didn't realize it actually caused infection with a bloody GPLv2 license so I have now completely removed it all, and replaced with simple memory loader/save functions. So its back to BSD compliant again. The happy consequence of this is it continues to works across multiple platforms again now (ones without mmap() support, i.e. non linux environments) as its using simple stdio read/writes. Also I have updated the cli arguments now to specify a destination for the crc updated file. This now means it does NOT overwrite your original rom file so you are free to compare the diffs now with my bdiff tool (or other linux equivalent tool). Trev Title: Re: Open Source Checker for ME7... Post by: prj on January 02, 2013, 09:06:20 AM prj, Could you do me a huge favour and see if you can see the checksum locations in the 360 dump I posted? The routines are completely different in this ... I am looking though. Title: Re: Open Source Checker for ME7... Post by: 360trev on January 02, 2013, 09:14:03 AM The routines are completely different in this ... I am looking though. Thx! Very much appreciated! ;D Title: Re: Open Source Checker for ME7... Post by: prj on January 02, 2013, 09:22:20 AM Thx! Very much appreciated! ;D Not familiar at all with other variants of ME7 other than VAG. I could not find any similar routines in 10 minutes. The routines for the CRC checksumming in the VAG ECU's are in the flash, not in the processor ROM. I don't know how this is solved on the Ferrari ECU. It is possible it has the CRC check, but implementation can be completely different. Or maybe I am just blind. Title: Re: Open Source Checker for ME7... Post by: prj on January 02, 2013, 10:10:08 AM I looked at a few binaries.
I think the CRC ranges are the same for almost all (if not all) 29F800BB ECU's. And also the same for 29F400BB ECU's. I looked at ME7.5, ME7.1.1, ME7.1 and so on. So I don't think a pattern is needed here :) Title: Re: Open Source Checker for ME7... Post by: 360trev on January 02, 2013, 10:17:34 AM Not familiar at all with other variants of ME7 other than VAG. I could not find any similar routines in 10 minutes. The routines for the CRC checksumming in the VAG ECU's are in the flash, not in the processor ROM. I don't know how this is solved on the Ferrari ECU. It is possible it has the CRC check, but implementation can be completely different. Or maybe I am just blind. Prj, Is there a possibility it doesn't actually have this checksum? If someone could patch a few bytes in the 360 dump and then run it through a working tool we should be able to binary diff the differences... At least we'd know then for sure if it actually has any extra checks! Title: Re: Open Source Checker for ME7... Post by: prj on January 02, 2013, 10:47:35 AM Sorry, I don't know much about the ECU in Ferraris, other than that it is a ME7.
I think the CRC checksum could be indeed vendor specific. From the VAG side, the checksum plugin should assume the ranges, and then verify these ranges using pattern search before trying to compute a checksum. Just to make sure. And if the range can not be found via pattern offset, then error out, rather than modify a wrong portion of the file. Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 10:54:58 AM 360: i changed a few things so it can compile with nmake/cl.exe as well.
Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 10:59:13 AM Also, we should maybe reorder the args so it can be
me7sum ini in.bin out.bin so we can optionally do me7sum ini in.bin for "readonly" mode (see Config.readonly)... ETA: i've changed my version to do that.. Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 01:34:06 PM Prj, Is there a possibility it doesn't actually have this checksum? If someone could patch a few bytes in the 360 dump and then run it through a working tool we should be able to binary diff the differences... At least we'd know then for sure if it actually has any extra checks! Well, fortunately, the 360 bin you gave me has all 0xffffffff in the stored CRCs, so we can easily skip the CRC check if it sees 0xffffffff.. should i make that change? Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 01:39:57 PM I looked at a few binaries. I think the CRC ranges are the same for almost all (if not all) 29F800BB ECU's. And also the same for 29F400BB ECU's. I looked at ME7.5, ME7.1.1, ME7.1 and so on. So I don't think a pattern is needed here :) My c-box is reporting Step #1: Reading main ROM CRC... Adr: 0x010002-0x013FFE @0x7a866 CRC: 0x4088F86A CalcCRC: 0x52FFEEB7 ** NOT OK ** Adr: 0x014252-0x017F4E @0x7a86c CRC: 0x4088362E CalcCRC: 0x7BBE718F ** NOT OK ** Adr: 0x018192-0x01FBB0 @0x7a872 CRC: 0xFDE63610 CalcCRC: 0x6DD1EF27 ** NOT OK ** Title: Re: Open Source Checker for ME7... Post by: prj on January 02, 2013, 01:49:47 PM 29F400BB should be... (might be wrong, just having a quick look at a ME75 file)
0x810000 - 0x814000 0x814300 - 0x817F68 0x818191 - 0x81FC00 Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 01:53:02 PM F:
Step #1: Reading main ROM CRC... Adr: 0x010002-0x013FFE @0x7a866 CRC: 0xF5E6700E CalcCRC: 0x77D806F1 ** NOT OK ** Adr: 0x014252-0x017F4E @0x7a86c CRC: 0xA696F564 CalcCRC: 0x72F852F7 ** NOT OK ** Adr: 0x018192-0x01FBB0 @0x7a872 CRC: 0x8EF7A691 CalcCRC: 0xDA302687 ** NOT OK ** G: Step #1: Reading main ROM CRC... Adr: 0x010002-0x013FFE @0x7a866 CRC: 0x25804ED4 CalcCRC: 0xFB8B3F8F ** NOT OK ** Adr: 0x014252-0x017F4E @0x7a86c CRC: 0x25A4FCE6 CalcCRC: 0x1575B32F ** NOT OK ** Adr: 0x018192-0x01FBB0 @0x7a872 CRC: 0xCE000207 CalcCRC: 0xD65C0509 ** NOT OK ** K: Step #1: Reading main ROM CRC... Adr: 0x010002-0x013FFE @0x7a866 CRC: 0x5088F800 CalcCRC: 0xA13F9F2B ** NOT OK ** Adr: 0x014252-0x017F4E @0x7a86c CRC: 0x50888A40 CalcCRC: 0xC52D9DB4 ** NOT OK ** Adr: 0x018192-0x01FBB0 @0x7a872 CRC: 0xF5E622C0 CalcCRC: 0xABABDCAD ** NOT OK ** T: Step #1: Reading main ROM CRC... Adr: 0x010002-0x013FFE @0x7a866 CRC: 0x8056F3F6 CalcCRC: 0x0B927F86 ** NOT OK ** Adr: 0x014252-0x017F4E @0x7a86c CRC: 0xF1F20207 CalcCRC: 0x9070CCBA ** NOT OK ** Adr: 0x018192-0x01FBB0 @0x7a872 CRC: 0x020750D7 CalcCRC: 0x14D9A89C ** NOT OK ** Title: Re: Open Source Checker for ME7... Post by: 360trev on January 02, 2013, 02:08:27 PM This is getting confusing ;)
Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 02:14:57 PM prj/phila: one of you mentioned that the CRC check isn't actually done most of the time? Can you elaborate? Maybe I can ditch it altogether for now, and concentrate on auto-finding the multipoint and main checksum areas.
Title: Re: Open Source Checker for ME7... Post by: prj on January 02, 2013, 06:17:17 PM Nye, the CRC locations are wrong too, you can't use the ones from 29F800.
On 29F400 they are 0x86693C, 0x866942 and 0x866948. Checksum algorithms are not exactly my strong suite, so if there is anything else you need to calculate those CRC's let me know ... Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 06:47:50 PM Well i am hoping they aren't important...
BUT alternately, if they are, i need some sort of pseudo code that describes how I might automatically find them.. Same goes for multipoint and main, if you have an opinion on either ;) My basic idea is to look for 1) block start/stop patterns (they seem to be 0x4000 aligned most of the time) 2) csum/inv-csum pairs the CRC blocks start/stop seem to be somewhat arbitrary, and the CRC value locations don't seem to be paired with inverse CRC values :( Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 06:52:57 PM Nye, the CRC locations are wrong too, you can't use the ones from 29F800. On 29F400 they are 0x86693C, 0x866942 and 0x866948. Checksum algorithms are not exactly my strong suite, so if there is anything else you need to calculate those CRC's let me know ... no joy on C-box Step #1: Reading main ROM CRC... Adr: 0x010000-0x014000 @0x6693c CRC: 0xF4F209DD CalcCRC: 0xEB231C60 ** NOT OK ** Adr: 0x014300-0x017F68 @0x66942 CRC: 0xA5A2F442 CalcCRC: 0xDDE14F6C ** NOT OK ** Adr: 0x018191-0x01FC00 @0x66948 CRC: 0xF57425E0 CalcCRC: 0xA2562CE0 ** NOT OK ** Step #1: Reading main ROM CRC... Adr: 0x010000-0x013FFF @0x6693c CRC: 0xF4F209DD CalcCRC: 0x22402F9B ** NOT OK ** Adr: 0x014300-0x017F67 @0x66942 CRC: 0xA5A2F442 CalcCRC: 0xE359D549 ** NOT OK ** Adr: 0x018191-0x01FBFF @0x66948 CRC: 0xF57425E0 CalcCRC: 0x3E37E205 ** NOT OK ** which 400 are you looking at? Title: Re: Open Source Checker for ME7... Post by: prj on January 02, 2013, 07:37:00 PM ME7.5 8N...somethingAC.
I guess that pattern searcher might come in handy after all. I will disassemble the 551C later. Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 02, 2013, 08:00:35 PM I was definitely wrong when I said that the CRC checks aren't important. They are checked constantly, one zone at a time, and are responsible for the P0601 DTC.
I will try to take a look at other files when I get the chance. Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 09:53:58 PM I have the checksum searching working.
Checked in to git. i have made ini loading optional (since eventually we shouldn't need the inis) 360trev: the autodetection doesn't work for your bin, since your main range block 1 is different from ALL of my ME7 audi images (0x800000-0x80FBFF vs 0x800000-0x80FE6D). I could loosen up the search, but i'd like to have somebody else find other binaries or soemthing so i have more datapoints. ETA: i have added masks to the search, so the 360 search works now too. All thats left is the CRC search; sounds like it is critical. Title: Re: Open Source Checker for ME7... Post by: nyet on January 02, 2013, 11:10:33 PM Hmm. Unless i'm crazy, i don't see winols fixing the CRCs..
Title: Re: Open Source Checker for ME7... Post by: prj on January 03, 2013, 03:13:20 AM 551c
0x810000 - 0x814000 @ 0x86BD8A 0x814300 - 0x817F68 @ 0x86BD90 0x818191 - 0x81FC00 @ 0x86BD96 It looks like what needs detection are the checksum locations, but this is very easy to do. Title: Re: Open Source Checker for ME7... Post by: 360trev on January 03, 2013, 04:22:24 AM 551c 0x810000 - 0x814000 @ 0x86BD8A 0x814300 - 0x817F68 @ 0x86BD90 0x818191 - 0x81FC00 @ 0x86BD96 It looks like what needs detection are the checksum locations, but this is very easy to do. Surely if we can identify the CRC32 function itself with a simple binary search once we find it with IDA or similar dissassembler. (I'm guessing that code doesn't change that much between different firmwares, probably identical or very few variants). ..We should then be able to find all function call's which make reference to it (i.e. call it) by searching for 'call/jsr' asm functions which reference it in any subsequent code blocks. e.g. calls in the code itself. Perhaps any jmp's too? and then look back a few instructions to determine their arguments (parameters passed). e.g. it must be using the function something like this.. Code: crc32 = do_crc32(start,length); Q. Do we not just need to understand the binary machine code signature of those calls (i.e. the raw hex format of the machine code?). Then once we know the file offsets which are actually calling the crc32() function (which we stored in a simple array or table) we can identify the "start address" and "end address" (or length and derive the end address from that) and the scan the C16x m/c for any variants of call it can make. Looking at the C166 instruction set guide these variants are; Quote CALLR rel Call relative subroutine PCALL reg, caddr Push direct word register onto system stack and call absolute subroutine CALLA cc, caddr Call absolute subroutine if condition is met CALLI cc, [Rw] Call indirect subroutine if condition is met CALLS seg, caddr Call absolute subroutine in any code segment So a search for all call functions signatures referencing the CRC32 code block address we found earlier in the binary signature search should tell us everything we need to know right? Or am I missing something? Title: Re: Open Source Checker for ME7... Post by: prj on January 03, 2013, 04:33:56 AM You are over thinking it slightly, but this is the general idea.
Also for the C box, if you compare to the locations I gave you will see that these ranges are correct ;) Quote Adr: 0x010000-0x013FFF @0x6693c CRC: 0xF4F209DD CalcCRC: 0x22402F9B ** NOT OK ** Adr: 0x014300-0x017F67 @0x66942 CRC: 0xA5A2F442 CalcCRC: 0xE359D549 ** NOT OK ** Adr: 0x018191-0x01FBFF @0x66948 CRC: 0xF57425E0 CalcCRC: 0x3E37E205 ** NOT OK ** Here is something useful from my good old hacking days: Code: DWORD FindMemoryClone(DWORD start, DWORD end, const UCHAR *clone, UINT size, HANDLE processHandle) Code: BOOL CompareMemory(DWORD address, const UCHAR *mask, UINT size, HANDLE processHandle) And the mask you have to find: E6 F4 FF FF E6 F5 FF FF DA 00 D8 7E FF - masked out byte. You need to find all matches of that mask, the first FF FF is the low word of the address, the second FF FF is the high word. There should be three sequential ones, where the the pointed values are 6 bytes apart, these are the checksum locations. I am sure you guys can adapt this function or the idea behind it to work on a file instead of RAM ;) Title: Re: Open Source Checker for ME7... Post by: 360trev on January 03, 2013, 04:49:24 AM And the mask you have to find: E6 F4 FF FF E6 F5 FF FF DA 00 D8 7E FF - masked out byte. You need to find all matches of that mask, the first FF FF is the low word of the address, the second FF FF is the high word. There should be three sequential ones, where the the pointed values are 6 bytes apart, these are the checksum locations. I am sure you guys can adapt this function or the idea behind it to work on a file instead of RAM ;) We've already loaded the entire firmware now into ram in the latest app. Again looking at the C166 Instruction reference manual, the hex signature you've provided us means; { E6 F4 xx xx }, MOV reg, #data16 { E6 RR xx xx } { E6 F5 yy yy }, MOV reg, #data16 { E6 RR yy yy } { DA 00 D8 7E }, CALLS seg, caddr { DA SS MM MM } (where the xx and yy are the address ranges we want) e.g. does this (roughly) translate to; { MOV r4, #xxxx // load the value of xxxx (e.g. 0x1111) into register R4 MOV r5, #yyyy // load the value of yyyy (e.g. 0x2222) into register R5 CALLS 0, 0xD87E // execute function in segment 0, offset 0xD87E } Q. So looking at your signature, your looking for 2 mov's and one CALLS. Is that really enough to make an exact match? I'm surprise it doesn't find calls to other functions too if searching over the entire rom. Is this because of the segmentation register meaning its unique for this particular region. I need to read up more on the C166x asm architecture! Also what happens if the CRC routine is in a different location than 0xD87e segment 0. It wouldn't find it right? Can someone point me to the exact dump that this works with so I can see myself. -T Title: Re: Open Source Checker for ME7... Post by: prj on January 03, 2013, 05:05:03 AM You got the endianness wrong, but other than that you are right.
However, if you note, then I gave another set of criteria which will filter out the false positives. I told to look for three consecutive addresses in the form of YYYYXXXX that are 6 bytes apart. This should work for almost any VAG ME7 dump. Title: Re: Open Source Checker for ME7... Post by: prj on January 03, 2013, 05:09:17 AM 2nd part is to make some masks that will identify the regions as well, as they do vary a bit, I found at least three different ones.
The routines between 29F400 and 29F800 are a bit different too. Title: Re: Open Source Checker for ME7... Post by: 360trev on January 03, 2013, 05:11:46 AM You got the endianness wrong, but other than that you are right. However, if you note, then I gave another set of criteria which will filter out the false positives. I told to look for three consecutive addresses in the form of YYYYXXXX that are 6 bytes apart. This should work for almost any VAG ME7 dump. Ah, I see. and thanks for the endian thing! I wrote assembly language for a decade (in a former life as a games programmer!) so I'm very familar with asm but not the C166 (yet!). I think I will modify the open source TinyC compiler as a cross compiler target to C166, that would help me in 2 way, (get VERY familar with the instructions) and also make it VERY easy to provide the community a way to automatically patch in C code into ROM's (!). I.e we could search for free space filled with FF's and then patch it in (all automatically) after compilation... Nice idea? Title: Re: Open Source Checker for ME7... Post by: prj on January 03, 2013, 05:17:56 AM I prefer to just write in ASM...
But regardless, someone needs to implement the pattern finding and I can give the rest of the patterns as well. I am not very fond of C/C++, I'd rather just help with ASM. I know it's not much help with your Ferrari, but oh well... Title: Re: Open Source Checker for ME7... Post by: 360trev on January 03, 2013, 05:20:48 AM You got the endianness wrong, but other than that you are right. However, if you note, then I gave another set of criteria which will filter out the false positives. I told to look for three consecutive addresses in the form of YYYYXXXX that are 6 bytes apart. This should work for almost any VAG ME7 dump. Thinking about this a little further... I can understand and follow the logic well.. i.e. instead of looking for just one call to a function with 2 arguments .e..g Code: result = anyfunc(address1, address2); we are looking for 3 of them in succession. i.e. Code: result = anyfunc(address1, address2); Which roughly translates to something like this... Code: crc32_number_1 = calc_crc32(start_1,end_1); I can fully appreciate this would indeed be quite unique however what in non VAG rom's? Is there ALWAYS 3 checksums? In the method I originally gave, finding the actual CRC32 routine itself and then ALL references to it should lead to perfect results every time with zero false positives... No? However its obviously a bit more work to code.... Title: Re: Open Source Checker for ME7... Post by: 360trev on January 03, 2013, 05:23:48 AM I prefer to just write in ASM... But regardless, someone needs to implement the pattern finding and I can give the rest of the patterns as well. I am not very fond of C/C++, I'd rather just help with ASM. I know it's not much help with your Ferrari, but oh well... Lol. I used to be like that... swore by assembly ;) but in my older age I've got very used to C. (laziness i guess.. having to think less... ) Title: Re: Open Source Checker for ME7... Post by: 360trev on January 03, 2013, 05:38:18 AM I suppose if I want to get the 360's final crc32's found i'm going to have to crack open the old disassembler...
I think I may actually write my own (anyone written an open source C166 disasm yet?) or cpu emulator? This is a relatively simple (by modern standards) cpu and I have an excellent reference document (the Instruction set is freely downloadable on the web). Would be quite fun... Title: Re: Open Source Checker for ME7... Post by: prj on January 03, 2013, 05:38:57 AM I can fully appreciate this would indeed be quite unique however what in non VAG rom's? Is there ALWAYS 3 checksums? You can find at least 3 checksums, and then see if there are any other matches, and if there are other matches, then see if any of them are around there 6 bytes off, if you want to do it.As for other then VAG roms, I am clueless. I just mainly tune VAG for now. However, if you donated me a Ferrari, I might be more inclined to figure things about Ferraris. ;) Quote In the method I originally gave, finding the actual CRC32 routine itself and then ALL references to it should lead to perfect results every time with zero false positives... No? However its obviously a bit more work to code.... There is no "actual CRC32 routine" it just compares the calculated sum to stored value...If you looked and understood the ECU routine, you'd see what I mean. For now you will have to trust me and implement it the way I say ;) Title: Re: Open Source Checker for ME7... Post by: 360trev on January 03, 2013, 05:51:01 AM You can find at least 3 checksums, and then see if there are any other matches, and if there are other matches, then see if any of them are around there 6 bytes off, if you want to do it. As for other then VAG roms, I am clueless. I just mainly tune VAG for now. However, if you donated me a Ferrari, I might be more inclined to figure things about Ferraris. ;) There is no "actual CRC32 routine" it just compares the calculated sum to stored value... If you looked and understood the ECU routine, you'd see what I mean. For now you will have to trust me and implement it the way I say ;) I'd love to be able to donate a Ferrari to you but then I'd be Ferrari-less so it would be a bit pointless really ;) ...It took me many years to get that toy i can tell you that! Just to clarify the "no actual crc32 routine" point. Are you saying they have 'in-lined' the calculated sum function in the code? You describe it compares the 'calculated' sum with the 'stored' value - so where exactly did it 'calculate' it? inline or in a subroutine? (or multiple subroutines?). I can see you have considerable experience with this architecture and I appreciate all the help. I aim to get up to speed quickly (due to my asm background I'm confident I will). In terms of the routines, I'd love to explore them, what rom file are we talking about and can you give me a byte offset? Also what the segmentation register values should be... -T Title: Re: Open Source Checker for ME7... Post by: prj on January 03, 2013, 06:11:25 AM Okay basically it calculates one sum at a time per pass of the routine.
1st pass, calculate 1st sum. 2nd pass, check 1st sum against stored ROM value. 3rd pass, calculate 2nd sum. 4th pass, check 2nd sum against stored ROM value. etc. This is probably done to reduce load on the ECU as it is a very computation intensive process. So while there is a CRC routine, it's not called where the calculated value is checked, the routine at 0x7ED8 (which is in CPU ROM btw), just loads stuff from ROM to RAM. Damn you and your curiosity :D I was unable to locate this in the Ferrari 360 binary btw. At least not in the 15 minutes I spent looking at it. Because I do not tune Ferraris, it is not in my interest to allocate more time to it at the moment :/ Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 03, 2013, 06:48:07 AM There are nested loops.
Outer loop starts at the beginning of it's designated region/zone and calculates current start address to pass to CRC routine. The outer loop also passes number of bytes to be processed per iteration of the CRC routine, which is 176 (B0h) in the ME7 files that I looked at. The outer loop runs until the end of the desinated region is reached. The start address, end address - 1, and number of bytes to process per inner loop iteration are hardcoded in this portion of code. The inner loop is the CRC routine itself. It runs from the start address passed from the outer for the number of bytes passed (176 except final iteration). This calculates the actual CRC. The hash table location is hardcoded in this routine. On the final iteration of the outer loop, a flag is set signalling the calculated CRC to be compared to the checksum stored in the file. After the checks are compared, the flag is changed again to signal for the next region's calculation to begin. It continues like this until all regions are complete. I hope that makes sense, it was from memory. It shouldn't be hard at all to use a pattern to find everything needed. The code is unique and everything needed is hardcoded. The code for the different regions is duplicated sequentially. I took a quick peek at the K-box and it is a little different. The third regions end address was different and there was another portion that gets processed after the final loop of the third region. I didn't get a chance to really go through it though. Also looked at the Op's file briefly and found similar checking routines. Again, didn't really get a chance to go through it. Title: Re: Open Source Checker for ME7... Post by: 360trev on January 03, 2013, 06:50:12 AM Okay basically it calculates one sum at a time per pass of the routine. 1st pass, calculate 1st sum. 2nd pass, check 1st sum against stored ROM value. 3rd pass, calculate 2nd sum. 4th pass, check 2nd sum against stored ROM value. etc. This is probably done to reduce load on the ECU as it is a very computation intensive process. So while there is a CRC routine, it's not called where the calculated value is checked, the routine at 0x7ED8 (which is in CPU ROM btw), just loads stuff from ROM to RAM. Damn you and your curiosity :D I was unable to locate this in the Ferrari 360 binary btw. At least not in the 15 minutes I spent looking at it. Because I do not tune Ferraris, it is not in my interest to allocate more time to it at the moment :/ Thanks for the explanation. It makes sense now. Ah, okay, so in addition to the 512kbyte FIRMWARE DUMP file I pulled using Bootmode there is in fact also some resident ROM functions too! (How big is this rom in kbytes, any idea?). This actual CRC function code is one of them in the ROM then! ... So, these rom functions are referenced by the upgradable Firmware but they themselves are not upgradable (sort of like a re-usable utils library). Hmmm, so if I can dump the rom functions I should be able to understand exactly what they do.... Q. Has anyone dumped the ROM regions (and reversed them?), are they read protected? If not I assume someone could write a similar tool to the dump EEPROM one for this purpose (anyone actually done it?). I can only assume the VAG rom is quite a bit different than the 360 one hence the reason why the searching didn't find anything. Its probably in a different location and may even be a different variant of the function (slightly modified or older/newer version of the routine). Last question then (sorry to have consumed so much time!) how necessary are these additional checks? could I not just 'nop' them out? :) since they seem to be crc's on top of crc's (i.e possibly a bit of overkill...) -T Title: Re: Open Source Checker for ME7... Post by: prj on January 03, 2013, 07:40:15 AM Thanks for the explanation. 32kb, you can pull it out through minimon.It makes sense now. Ah, okay, so in addition to the 512kbyte FIRMWARE DUMP file I pulled using Bootmode there is in fact also some resident ROM functions too! (How big is this rom in kbytes, any idea?). Quote This actual CRC function code is one of them in the ROM then! No, the only thing in the ROM here is the function that loads some values from a specified offset into RAM.Quote So, these rom functions are referenced by the upgradable Firmware but they themselves are not upgradable (sort of like a re-usable utils library). Correct.Quote Q. Has anyone dumped the ROM regions (and reversed them?), are they read protected? If not I assume someone could write a similar tool to the dump EEPROM one for this purpose (anyone actually done it?). Yes, there are a few different ROM versions. Some VAG ME7 flash firmware files actually have the ROM image embedded in the first 0x0000-0x7FFF, but in others 0x0000-0x7FFF already contains different user area code.Quote I can only assume the VAG rom is quite a bit different than the 360 one hence the reason why the searching didn't find anything. Its probably in a different location and may even be a different variant of the function (slightly modified or older/newer version of the routine). The CRC32 stuff is entirely in flash.Quote Last question then (sorry to have consumed so much time!) how necessary are these additional checks? could I not just 'nop' them out? :) since they seem to be crc's on top of crc's (i.e possibly a bit of overkill...) You can do whatever you want. It's just a computer and you are capable of programming.Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 10:38:32 AM And the mask you have to find: E6 F4 FF FF E6 F5 FF FF DA 00 D8 7E FF - masked out byte. You need to find all matches of that mask, the first FF FF is the low word of the address, the second FF FF is the high word. There should be three sequential ones, where the the pointed values are 6 bytes apart, these are the checksum locations. I am sure you guys can adapt this function or the idea behind it to work on a file instead of RAM ;) Yes. I have a needle/haystack/mask function, but it works a bit differently since it is based on chipselect algorithms (and i'm using this to detect checksum block descriptor areas) Code: static int memcmp_mask(const void *ptr1, const void *ptr2, const void *mask, size_t len) The idea is that it can mask out individual bits, not just whole bytes, and also search for bytes that match 0xff (if both needle and mask are 0xff) so "E6 F4 FF FF E6 F5 FF FF DA 00 D8 7E" becomes Code: needle = E6 F4 00 00 E6 F5 00 00 DA 00 D8 7E I'll work on it now. Question is, how to find block start/end? Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 11:02:51 AM Ok. Code works :)
Quote Searching for main ROM CRC offsets... Found CRC #1 at 0x86bd8a (decoded from 0x6c18e) Found CRC #2 at 0x86bd90 (decoded from 0x6c220) Found CRC #3 at 0x86bd96 (decoded from 0x6c2b0) Adr: 0x010000-0x013FFF @0x6bd8a CRC: 0x22402F9B CalcCRC: 0x22402F9B CRC OK Adr: 0x014300-0x017F67 @0x6bd90 CRC: 0xE359D549 CalcCRC: 0xE359D549 CRC OK Adr: 0x018191-0x01FBFF @0x6bd96 CRC: 0x3E37E205 CalcCRC: 0x3E37E205 CRC OK thanks prj. we just need the regions now .. Title: Re: Open Source Checker for ME7... Post by: prj on January 03, 2013, 12:07:29 PM Ok for block start/end it is a bit more tricky.
I masked out bytes with XX and marked high word and low word with HH and LL. For 29F800: Offset start positions: E6 F8 LL LL E6 F9 HH 00 F2 F4 XX XX 24 8F Offset end positions: 10 9B E6 F4 LL LL E6 F5 HH 00 26 F4 29F400 routines are a wee bit different, but I think for 29F400 just assume the same locations every time, the ones that worked for C box. Test it with a few binaries, but I think all 29F400 are the same. Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 01:05:46 PM Ok. summary time:
these worked: 4B0907551L 4Z7907551K 8D0907551A 8D0907551B 8D0907551D 8D0907551G 8D0907551H 8D0907551J 8D0907551L 8D0907551M these didn't find any: 4Z7907551S 8D0907551C 8D0907551T these found more than 3 (they found 4), and none of them match up 8D0907551F 8D0907551K 8D0907551Q obviously, ferrari's didn't work at all. Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 03, 2013, 01:12:48 PM Ok. summary time: these worked: 4B0907551L 4Z7907551K 8D0907551A 8D0907551B 8D0907551D 8D0907551G 8D0907551H 8D0907551J 8D0907551L 8D0907551M these didn't find any: 4Z7907551S 8D0907551C 8D0907551T these found more than 3 (they found 4), and none of them match up 8D0907551F 8D0907551K 8D0907551Q obviously, ferrari's didn't work at all. As I stated above, the K box had something extra at the end. I didn't get to analyze it, but it ran after the final iteration of the third region, before the check IIRC. Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 03, 2013, 04:08:04 PM On the K box, the third region runs from 0x818192 everything the same as before except the end address is 0x81FBDD. At the final iteration, the iteration counter gets cleared and the flag gets set to calculate a fourth region instead of signalling to compare the check values.
The fourth region from 0x826A00 to 0x82FFD gets calculated and then checked at 0x89000C. There is also an intialization, so if the control byte or flag as I've been referring to it isn't set to indicate calculation of any of the regions or check value comparisons, then the CRC value gets an intial calculation from start address 0x8183E9 for ten bytes. I can try to explain the "control byte" better if this is confusing. Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 04:17:56 PM Thanks for the info, phila
Here is what I have (using prj's matching pattern) Code: // LL LL HH And here are the results for K box: Code: Step #1: Reading main ROM CRC... What should i do to detect that its different, and should be handled as you describe? Even the first two regions dont work right Code: Adr: 0x810002-0x813FFE @0x90000 CRC: 0x13BFD815 CalcCRC: 0xA13F9F2B ** NOT OK ** Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 03, 2013, 04:28:25 PM Add the intial CRC for the ten bytes from 0x8183E9 and see if that fixes the first two.
E6 FC LL LL E6 FD HH HH E0 sE DA XX XX XX F6 F4 s == length in bytes Title: Re: Open Source Checker for ME7... Post by: 360trev on January 03, 2013, 04:52:21 PM Nyet,
I have been studying 8D0907551C dump and I have found the full Multipoint checksum routine and from this we should be able to derive the addresses (physical file offset which contains the actual values). In this rom the function which initializes the multipoint checksum with the START and END addresses looks like this.... // // FYI: It starts at file offset 0x6B972 in 8D0907551C dump // // // [physical memory start offset = 0x86b972, dump file byte offset = x6b972] // Code: _InitMultipoint: // I further noticed the field referenced by the code are in these locations // asm offset x3fe6 - (1bfe6 is MainRom_Checksum_START_Low) - phy addr 81bfe6, file offset 1bfe6 // asm offset x3fe8 - (1bfe8 is MainRom_Checksum_START_Low) - phy addr 81bfe8, file offset 1bfe8 // asm offset x3fea - (1bfe6 is MainRom_Checksum_END_Low) - phy addr 81bfea, file offset 1bfea // asm offset x3fec - (1bfe8 is MainRom_Checksum_END_Low) - phy addr 81bfec, file offset 1bfec The question now is how to convert the asm instruction relative offsets to byte offsets in the file? ...I believe a clue is to do with the segmented memory model of the C167, [c167 asm guys help me out here!]. I can provide the full routine I reversed if anyones interested, it does the full multi-point checksum and starts in this rom at byte offset x6b9ae. Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 04:56:30 PM Phila: that fixed the first block, but nothing else.
Adr: 0x810002-0x813FFE @0x90000 CRC: 0x13BFD815 CalcCRC: 0x13BFD815 CRC OK Adr: 0x814252-0x817F4E @0x90006 CRC: 0x8C92421A CalcCRC: 0x1382383A ** NOT OK ** Adr: 0x818192-0x81FBDC+0x826A00-0x82FFFC @0x9000c CRC: 0xC95513B5 CalcCRC: 0xFF232A87 ** NOT OK ** the other 2 two still dont work (with or without the added crc) 360: it already detects the multipoint no problem; i just loosened the matching parameters some (see latest git checkin).. what i cant' find is the CRC blocks Title: Re: Open Source Checker for ME7... Post by: 360trev on January 03, 2013, 05:00:51 PM 360: it already detects the multipoint no problem; i just loosened the matching parameters some (see latest git checkin).. what i cant' find is the CRC blocks Yes, but this way should be guarenteed for ANY variant, it derives it from the actual code init function, without using signatures which may or may not always work and could hit false positives. Here's the full checksum routine incase anyone is interested, it requires fully commenting however... Code:
Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 05:09:01 PM Success!
for 4 block, i just let the CRC accumulate through all 5 regions Code: 0: Adr: 0x8183E9-0x8183F2 CalcCRC: F9A70C59 (seed 00000000) phila: i need a way to detect the 0x8183e9 range... 360: ok, i will try to use what you gave me to do a better job looking for start. actually, if you can give me an exact needle/mask pair that would be ideal :) Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 03, 2013, 05:34:42 PM This work?
E6 FC LL LL E6 FD HH HH E0 sE DA XX XX XX F6 F4 s == length in bytes Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 05:39:09 PM Phila, trying it now.
Are you sure the other two regions aren't checked anywhere? Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 03, 2013, 05:49:50 PM Phila, trying it now. Are you sure the other two regions aren't checked anywhere? Other two as in 0 and 3 in your previous post? Not checked. Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 06:26:16 PM F K and Q all work now, which leaves ferrari's bin, and
4Z7907551S 8D0907551C 8D0907551T T is curious, there seems to be a false match 8D0907551T: Code: Found CRC #1 at 0x81b494 4Z7907551S.bin cant find offsets at all Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 06:37:43 PM err after some code tweaking:
Code: Found CRC #1 at 0x81b494 ETA: works with this search pattern, but that pattern doesn't work with the other boxes :( Code: uint8_t needle[] = {0xEA, 0xA5, 0xE6, 0xF4, 0x00, 0x00, 0xE6, 0xF5, 0x00, 0x00, 0xDA, 0x00, 0xD8, 0x7E};Code: Found CRC #1 at 0x87d94c Title: Re: Open Source Checker for ME7... Post by: jooo on January 03, 2013, 07:44:36 PM The question now is how to convert the asm instruction relative offsets to byte offsets in the file? Somewhat described here...I believe a clue is to do with the segmented memory model of the C167, [c167 asm guys help me out here!]. http://nefariousmotorsports.com/forum/index.php?topic=1735.0 and some other threads in Reverse Engineering board Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 07:49:37 PM For those of you looking for patterns, here are the hexdumps near the matched areas
M box: Code: Found possible CRC offset #1 at 0x87a866 (from 0x7ac96) K box: Code: Searching for main ROM CRC offsets... T box: Code: Found possible CRC offset #1 at 0x81b494 (from 0x37de6) maybe that f6 f5 XX XX sequence? Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 08:02:54 PM This seems to work for everything but 8D0907551C and 4Z7907551S
comments? Code: // LL LL HH HH Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 03, 2013, 08:40:50 PM This seems to work for everything but 8D0907551C and 4Z7907551S comments? Code: // LL LL HH HH Different ROM? That's a call to ROM function at the end. Hardcoding addresses isn't always a good idea. Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 03, 2013, 08:48:39 PM This works in four files I checked real quick.
E6 F4 XX XX E6 F5 XX XX DA 00 XX XX E6 00 04 02 Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 09:01:49 PM Do you mean:
ETA: err nvm gimme a sec Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 09:08:54 PM Doesn't work for t-box
Code: Found possible CRC offset #1 at 0x81b494 (from 0x37de6) Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 09:11:40 PM This does though (prepend f6 f5 xx xx)
Code: needle[] = {0xF6, 0xF5, 0x00, 0x00, 0xE6, 0xF4, 0x00, 0x00, 0xE6, 0xF5, 0x00, 0x00, 0xDA, 0x00, 0x00, 0x00, 0xe6, 0x00, 0x04, 0x02};4Z7907551S is still broken, of course ETA: with the F6 F5 XX XX i dont even seem to need the e6 00 04 02 tail... Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 10:03:46 PM Ok. So it is pretty clear that we cannot detect (for example) the CRC offsets in the C box... they have to be hard coded...
so we will need *some* database of EPKs somewhere where we can list exceptions... and possibly a whitelist of known ECUs so people don't break things? Clearly the "-i inifile" stuff is a non-starter; we can't expect an end user to always specify an ini file. At some point, we need a full database that is ALWAYS loaded by the checksum checker. also note that ferrari's bin is still unsupported, along with that weird S file that I have. Finally, i don't have anywhere NEAR all the ME7 bins, so at some point I'm going to need help doing testing and adding things to the database when they can't be autodetected. Anybody have any opinions? Also, has anybody else managed to checkout and compile the code to test it on their bins? Title: Re: Open Source Checker for ME7... Post by: phila_dot on January 03, 2013, 10:24:43 PM That allroad file is different. I don't think it has a CRC32 checksum, but I did find some checksum routines.
I found some checksum routines in the ferrari bin as well. C box still doesn't work as well? Setzi's tool is accurately detecting and calculating the checksums for most ME7 files somehow. Title: Re: Open Source Checker for ME7... Post by: nyet on January 03, 2013, 10:56:52 PM C box still doesn't work as well? Yea, i can find checksums, but not CRC blocks. However, the CRC value locations are detected. Quote Setzi's tool is accurately detecting and calculating the checksums for most ME7 files somehow. Yup. He's a lot smarter than me though. Title: Re: Open Source Checker for ME7... Post by: prj on January 04, 2013, 02:04:46 AM Yea, i can find checksums, but not CRC blocks. However, the CRC value locations are detected. Hardcode them if ECU image size is 512kb. Title: Re: Open Source Checker for ME7... Post by: nyet on January 04, 2013, 09:27:14 AM Hardcode them if ECU image size is 512kb. Done. This leaves 4Z7907551S and the ferrari bin as the only two (that I have) that do not work. I'd appreciate more bins, or other testers :) Title: Re: Open Source Checker for ME7... Post by: ddillenger on January 04, 2013, 09:48:46 AM Here are a few a6 bins.
Title: Re: Open Source Checker for ME7... Post by: Bische on January 04, 2013, 10:04:59 AM Great work guys! I wish I could help, maybe I can test some files?
After im done with the 5120 project im gonna spend my time learning ASM. Title: Re: Open Source Checker for ME7... Post by: nyet on January 04, 2013, 10:21:26 AM Here are a few a6 bins. All are ok except for 4Z7907551AA.bin Great work guys! I wish I could help, maybe I can test some files? YES PLEASE Title: Re: Open Source Checker for ME7... Post by: ddillenger on January 04, 2013, 10:24:45 AM Do any of the allroad bins work?
Title: Re: Open Source Checker for ME7... Post by: nyet on January 04, 2013, 10:27:34 AM Do any of the allroad bins work? They're all good except for AA... it can't find the CRC location offsets (it finds everything else) Title: Re: Open Source Checker for ME7... Post by: AARDQ on January 04, 2013, 10:39:20 AM I didn't see 4z7907551R as being tested. I imagine it will be the same as the S and AA (i.e. not currently supported); my box was listed as R but contained S software originally.
The R is interesting from the standpoint that it can be programmed over original S or AA software to get around the 'programming not finished' stumbling block. Title: Re: Open Source Checker for ME7... Post by: nyet on January 04, 2013, 11:13:26 AM I don't have a clean copy of R. I have (what I think) is a modified one.
Title: Re: Open Source Checker for ME7... Post by: AARDQ on January 04, 2013, 12:22:34 PM The attached should be original. It was posted by Tony way back when.
Title: Re: Open Source Checker for ME7... Post by: nyet on January 04, 2013, 01:11:14 PM Thanks. As expected same result as AA and S...
I'm working on fixing it now. ETA: i failed. I will need assistance from the smarter people here :( Title: Re: Open Source Checker for ME7... Post by: nyet on January 04, 2013, 04:49:27 PM One more thing: would it be good to try to autodetect the rom info offsets? eg. here is my mbox.ini:
Code: [dumps] Title: Re: Open Source Checker for ME7... Post by: 360trev on January 05, 2013, 06:44:57 AM nyet,
In my quest to get proper signature matching working i've started to re-write Andy's ME7 IDA Plug-in, close to having something now that could be used to match signatures without recompiling any code. See the IDA Plugin thread here; http://nefariousmotorsports.com/forum/index.php?topic=35.msg29602#new (http://nefariousmotorsports.com/forum/index.php?topic=35.msg29602#new) This got me thinking though, and the best way to do signatures is via (you've guessed it) an external config file. This way it can also identify map locations too so later we can integrate all of these functions into an ME7 'busybox' style tool which can also do map editing, reversing of functions, etc. fully self contained from the xterm. Its ambitious but it wouldn't be too much of a stretch to get this working brilliantly for multiple purposes. T Title: Re: Open Source Checker for ME7... Post by: nyet on January 05, 2013, 12:27:28 PM Yea i've actually already started to put the matching stuff into a separate module but it isn't really ready for prime time.
I was hoping to get a few more matching signatures first so i can make sure the module can handle a variety of different things... in particular, it needs to be able to handle extracting a bunch of different kinds of information from the patterns, eg xxx LL LL HH HH and xxx LL LL xx xx HH HH and xxx LL LL xx xx Sx etc. Title: Re: Open Source Checker for ME7... Post by: 360trev on January 05, 2013, 03:37:58 PM nyet,
Watch this space on masking ;D I have been experimenting and you can make 'automatic' masks based on a big instruction table being defined of all valid instructions with all relative addresses being masked in the table so every time it identifies a menomic it knows what mask to apply by looking it up in the table! this means any byte sig will work with it unmodified as it knows where to apply masks automatically! Title: Re: Open Source Checker for ME7... Post by: prj on January 05, 2013, 05:33:39 PM I have been experimenting and you can make 'automatic' masks based on a big instruction table being defined of all valid instructions with all relative addresses being masked in the table so every time it identifies a menomic it knows what mask to apply by looking it up in the table! this means any byte sig will work with it unmodified as it knows where to apply masks automatically! I wanted to do this a long time ago. Keep going ;) Title: Re: Open Source Checker for ME7... Post by: nyet on January 05, 2013, 09:38:24 PM So two things are holding me back right now: detecting CRC offsets in those allroad files (AA, R, S) and the ferrari bin
Also, any opinions on detecting the position of the EPK, software/hardware revisions, etc? should i just search for known strings? eg: EPK - sesarch for ME7.x PartNUmber - search for 551 (wont work for ferrari) EngineId - search for 2.7l (wont work for ferrari) SWversion - ?? just offset from EngineID? HWnumber/SWnumber - search for 1037 obviously a more generalizable way is preferable... or is it just not important? Also, at this point it would be really great to get a few testers on board, preferably people who can use git and know how to compile... I dont think im ready to post .exes... or should i? Title: Re: Open Source Checker for ME7... Post by: ddillenger on January 05, 2013, 09:48:36 PM I'm all for testing, but my knowledge on compiling is next to nothing right now. If you post an .exe, I will put every bin I have, or can find through it and post the ones that don't work, along with a summary of those that do.
I wish I could do more, you guys are really moving on this. Title: Re: Open Source Checker for ME7... Post by: nyet on January 05, 2013, 11:40:52 PM v0.0.4
WARNING! BACK UP ALL BINARIES BEFORE USING THIS :) http://nyet.org/cars/files/me7sum-v0.0.4.zip ETA: v0.0.3 had a bad bug. please dload v0.0.4 .. Title: Re: Open Source Checker for ME7... Post by: prj on January 06, 2013, 07:27:18 AM Why are SW version, Engine ID and so on important for calculating a checksum?
Title: Re: Open Source Checker for ME7... Post by: nyet on January 06, 2013, 12:35:11 PM If i am going to make a database of exception patterns, and i can more reliably and generally find the ids/versions than I can find, say CRC offsets, then I can use the result of that pattern match to look up the id in a database to tell me either what pattern to use for that id, or what hardcoded offsets to use.
Also, if i need to make a whitelist or a blacklist. Title: Re: Open Source Checker for ME7... Post by: nyet on January 10, 2013, 10:01:21 AM And while we're working on ME7.1.1, anybody want to help take a stab at me7.5? It is pretty similar to 7.1.1 .. i can find the crc blocks but not the crc value offsets.
Title: Re: Open Source Checker for ME7... Post by: nyet on January 21, 2013, 08:16:50 PM based on the files posted here:
http://nefariousmotorsports.com/forum/index.php?topic=1002.0 There appear to be two differences that are new to me there is a large block at 16a8a-16b09 (128 bytes). i guess thats the RSA stuff he talks about there is a LL HH pair at aa46e and aa472 that is changed... must be a crc location, but its different from ME7.1 because its split into two short words rather than contiguous LLHH Unfortunately, i'd need a file that is changed in more places to see if there are patterns (i.e more than one CRC is "wrong") PLEASE, i'd love to get more help :) either way its a dead end until somebody has access to a version of winols (or checksum fixer) that can do BEL and can help me .. basically, change a bunch more stuff and see what the checksum routines try to fix. Title: Re: Open Source Checker for ME7... Post by: ddillenger on January 21, 2013, 08:18:10 PM Post up whatever BEL file you want, I'll change the sh@# out of it and post both versions before and after checksum correction.
Title: Re: Open Source Checker for ME7... Post by: AARDQ on January 21, 2013, 08:34:33 PM based on the files posted here: http://nefariousmotorsports.com/forum/index.php?topic=1002.0 There appear to be two differences that are new to me there is a large block at 16a8a-16b09 (128 bytes). i guess thats the RSA stuff he talks about there is a LL HH pair at aa46e and aa472 that is changed... must be a crc location, but its different from ME7.1 because its split into two short words rather than contiguous LLHH Unfortunately, i'd need a file that is changed in more places to see if there are patterns (i.e more than one CRC is "wrong") PLEASE, i'd love to get more help :) either way its a dead end until somebody has access to a version of winols (or checksum fixer) that can do BEL and can help me .. basically, change a bunch more stuff and see what the checksum routines try to fix. Have a go at this. It's fairly significantly changed from the CS corrected file I run. Let me know if you need still more changes. Title: Re: Open Source Checker for ME7... Post by: nyet on January 21, 2013, 08:48:44 PM These are the blocks that suspect are checked by crc (S box though, not sure its same R box, ill have to check)
0) Adr: 0x018446-0x018455 1) Adr: 0x010002-0x013FFE 2) Adr: 0x014382-0x017F4E 3) Adr: 0x018192-0x01FBDC 4) Adr: 0x026A00-0x02FFFC changing one byte in each one independently should be sufficient (i hope) to figure out where the resulting crc changes are... Title: Re: Open Source Checker for ME7... Post by: AARDQ on January 21, 2013, 08:56:57 PM Here's range No. 4. I hope, anyway.
Title: Re: Open Source Checker for ME7... Post by: AARDQ on January 21, 2013, 08:59:47 PM Range No. 3
Title: Re: Open Source Checker for ME7... Post by: nyet on January 21, 2013, 09:06:34 PM r3 and r4 seem to be identical...
Title: Re: Open Source Checker for ME7... Post by: ddillenger on January 21, 2013, 09:09:28 PM I hope this is what you were asking for.
There's a readme detailing the changes in the archive. DD Title: Re: Open Source Checker for ME7... Post by: AARDQ on January 21, 2013, 09:11:51 PM r3 and r4 seem to be identical... Hmmm. Definitely changed just one byte in each of the ranges listed. I'll try again. In the meantime, here's Range No. 2 Title: Re: Open Source Checker for ME7... Post by: nyet on January 21, 2013, 09:14:50 PM AARDQ: ddillinger got me the whole set, thanks!
btw if any of you can take a look at what he posted and see if you can see patterns that would be great :) Title: Re: Open Source Checker for ME7... Post by: AARDQ on January 21, 2013, 09:16:22 PM AARDQ: ddillinger got me the whole set, thanks! btw if any of you can take a look at what he posted and see if you can see patterns that would be great :) Oh, good! And will do. Title: Re: Open Source Checker for ME7... Post by: nyet on February 28, 2013, 03:51:00 PM Split to new thread here
http://nefariousmotorsports.com/forum/index.php?topic=3347.0title= Title: Re: Open Source Checker for ME7... Post by: vgeorgiev on September 28, 2014, 09:16:26 AM Hi guys,
Is there similar project for EDC15C5 ? |