|
Title: Damos address corrector (Python script) Post by: Polo35580 on March 05, 2019, 07:34:40 AM Hello
I'd like to share a python script I wrote to convert a damos (A2L) to fit a new dump. I wrote it in python to allow everybody to improve it. (and check there is no worm in it) ;) You can imagine that the damos need to be as close as possible of the new dump. I mean same hardware but different software is ok but it's not applicable with different hardware. The script use a valid damos/dump couple in entry and a new dump to produce a new damos file. Supported ecus: ME7.5, MED91, EDC16U34, MED17 and SID208. The format of the dumps can be Intel Hex, Motorola S19 or Binary. The size must be 1024kb, 1504kb, 2048kb or 4096kb. It's possible to add support to new ecus by adding a new ecu definition in the script. For that the script contains a supportedeculist variable which hold ecu definition dictionaries. Ex: # MED17 - 0x80045320 "ecu" : "MED17", "cpu" : "TriCore", "addresslen" : 10, "addressstart" : '0x80', "addressformat": '0x80%06X\n' Meanings: # "ecu" : Ecu string found in A2L MOD_PAR entry (Ex: ECU "MED17") # "cpu" : Cpu type string found in A2L MOD_PAR entry (Ex: CPU_TYPE "TriCore") # "addresslen" : Length of address string in A2L CHARACTERISTIC records (Ex: 0x800574D8 -> 10) # "addressstart" : Sub string to remove to obtain an hex address where data can be found in dump file (Ex: 0x800574D8 -> '0x80') # "addressformat": Format use to rebuild an address line where 6 is the len of hex address without addressstart (Ex: 0x800574D8 -> '0x80%06X\n') The script will ask for an ecu and cpu type if there is no MOD_PAR entry in the A2L. Here is how the script process: - Read the damos file and create an address table - Read the original dump - Read the new dump - Correct the address table by searching best match pattern from original dump in new dump using dycothomic search method from original address - Linearize the address table - Create a new damos using the address table And here the command line to execute it: Code: python correct-damos-address.py DamosFile.A2L OriginalDump NewDump It take less than 10 minutes to correct most damos on my computer. There may be some mistakes due to linearization but it work pretty well in most case. Feel free to test it and report if it fit your needs. :D Edit: Update script to version 1.1. # ChangeLog : v1.0 - Original release # - Add MED17 Ecus support # - Add Intel Hex dump support Edit2: Update script to version 1.2. # ChangeLog : v1.2 - Add ME7.5 Ecus support Edit3: # ChangeLog : v1.3 - Add MED9.1 Ecus support # - Add EDC16U34 Ecus support # - Add Motorola S19 dump support # - Add supported ecu list to allow adding ecu type (See "Supported Ecu List") Edit4: # ChangeLog : v1.4 - Rewrite Intel Hex and Motorola S19 support to handle address bases # - Add SID208 Ecus support Best regards Polo Title: Re: Damos address corrector (Python script) Post by: Blazius on March 05, 2019, 11:19:04 AM Did you actually test this with 2 different files ( use script to match damos, change something , flash file without issues ) on a car yet? Anyway nice job.
Title: Re: Damos address corrector (Python script) Post by: Polo35580 on March 05, 2019, 01:35:44 PM I created damos with this script for my 2 cars.
A Seat Altea 1.9TDI and a Scirocco 1.4TSI 118kw both modded and working perfectly. :) Though the Scirocco was a pain to mod but that's not related to the damos. (MED17.5.5...) I double checked all "important" address of the new damos by comparison in winols when I wrote the script. I'm currently working on converting the Scirocco to E85 and I can confirm that singles like KRKATE are at good address. Remember to use a damos as close as possible of your hardware/software to minimize mistakes. Best regards Polo Title: Re: Damos address corrector (Python script) Post by: Blazius on March 05, 2019, 01:52:28 PM I created damos with this script for my 2 cars. A Seat Altea 1.9TDI and a Scirocco 1.4TSI 118kw both modded and working perfectly. :) Though the Scirocco was a pain to mod but that's not related to the damos. (MED17.5.5...) I double checked all "important" address of the new damos by comparison in winols when I wrote the script. I'm currently working on converting the Scirocco to E85 and I can confirm that singles like KRKATE are at good address. Remember to use a damos as close as possible of your hardware/software to minimize mistakes. Best regards Polo Nice good job , maybe I'll give this a try for my N/A software see how well it works with slight mismatch, as the closest thing I can use is Me7.5 dbw narrowband turbo files :P Title: Re: Damos address corrector (Python script) Post by: Polo35580 on March 06, 2019, 10:12:18 AM Hello
I updated the script to v1.1 with binary dump file of 1504ko support. So you can now mix Intel hex and binary files. Ex: Code: python correct-damos-address.py DamosFile.A2L OriginalDump.HEX NewDump.BIN Best regards Polo Title: Re: Damos address corrector (Python script) Post by: Blazius on March 07, 2019, 02:51:46 PM I just tried this with the AWM damos and it says a2l doesnt contain any adress, while infact it does in WinOls. Any idea whats wrong?
Title: Re: Damos address corrector (Python script) Post by: Polo35580 on March 10, 2019, 10:03:59 AM Hello
I just updated the first post with a new version with ME7 support. I'll have to find a better way to identify the ecu type... I first created the script for my Seat Altea 1.9TDI but I mess the script when I convert it to MED17 and I can't find my old backup... I'm no more able to create a correct damos for my Seat Altea 1.9TDI. I'll try to find my backup and restore support for EDC16 also. Best regards Polo Title: Re: Damos address corrector (Python script) Post by: Polo35580 on March 15, 2019, 03:29:18 AM Hello
An other update with MED9 and EDC16U34 support. I also added support of Motorola S19 dump file. The Ecu is now identified by parsing the A2L. This identification allow to select an ecu definition in a list of supported ecu. It's possible to add new definitions in this list in the script. (See first post and script) Best regards Polo Title: Re: Damos address corrector (Python script) Post by: Bitshifter on March 15, 2019, 08:23:16 PM Tested it with MED 9.1 file. Script don`t work for me, it shows error "Input is not a file !!!"
Could somebody upload a working example, please. Title: Re: Damos address corrector (Python script) Post by: nyet on March 16, 2019, 10:38:49 AM Tested it with MED 9.1 file. Script don`t work for me, it shows error "Input is not a file !!!" Could somebody upload a working example, please. copy/paste what you tried to do at the command line. Title: Re: Damos address corrector (Python script) Post by: Polo35580 on March 19, 2019, 01:30:32 AM Hello
Here my test folder for MED9 and MED17. https://mega.nz/#!iFcQxArS!BNg0BZWY8XtKNFdt8f4uegHMUZ9K4gThWKBk-y8akgU (https://mega.nz/#!iFcQxArS!BNg0BZWY8XtKNFdt8f4uegHMUZ9K4gThWKBk-y8akgU) There is 2 damos + dump of Golf 5 2.0T MED9.1 and 2 damos + dump of Golf 6 1.4TSI MED17.5.5. So it's easy to compare created damos with real ones. Here the commands MED9: Code: python correct-damos-address-v1.3.py "1K0907115G_0010-41WE20BKGC1G.A2L" "1K0907115G_0010.s19" "1K0907115S_0030.s19" debug Code: python correct-damos-address-v1.3.py "1K0907115S_0030-44D100BKIA3G.A2L" "1K0907115S_0030.s19" "1K0907115G_0010.s19" debug Code: python correct-damos-address-v1.3.py "V4N350FM10KJ2g.A2L" "Ser_V4N350FM10KJ2g_Complete.HEX" "SER_V5O360F10KM01g_Complete.HEX" debug Code: python correct-damos-address-v1.3.py "V5O360F10KM01g.A2L" "SER_V5O360F10KM01g_Complete.HEX" "SER_V4N350FM10KJ2g_Complete.HEX" debug Best regards Polo Title: Re: Damos address corrector (Python script) Post by: Polo35580 on April 05, 2019, 01:54:27 AM Hello
I just updated the first post with support for SID208. I also rewrote the Intel Hex and Motorola SRec parsing to handle address bases. The script will ask for an ecu and cpu type if there is no MOD_PAR entry in the A2L so it's possible to add new ecu support even if the A2L don't contains ECU and CPU type. Best regards Polo Title: Re: Damos address corrector (Python script) Post by: jcsbanks on April 05, 2019, 02:52:24 AM Line 411 and 644 need * in:
dumplinebin[2] * 0x10000 Title: Re: Damos address corrector (Python script) Post by: jcsbanks on April 05, 2019, 04:04:41 AM I tried an MED17 and at a quick glance the KFs I looked at were well formed. Nice work.
Title: Re: Damos address corrector (Python script) Post by: Polo35580 on April 05, 2019, 06:50:02 AM Line 411 and 644 need * in: Thanks for the report. ;)dumplinebin[2] * 0x10000 I just updated the file in first post. Best regards Polo Title: Re: Damos address corrector (Python script) Post by: Bitshifter on April 06, 2019, 01:34:36 AM copy/paste what you tried to do at the command line. That was not the Problem. The Problem are missing "" ;) @ Polo35580 Great tool. Tested with 3 versions (MED9) and all looks fine, save a lot of time. Thank you for engineering, release and update. I vote: hero member :) Title: Re: Damos address corrector (Python script) Post by: IamwhoIam on April 06, 2019, 04:56:35 AM If only it worked as decently with continental as it does with Bosch stuff... there's definitely LOADS of room for improvement there, just searching for byte strings isn't enough to make it a proper "corrector"... I've tested it on 2 similar Bosch MED17 files, in one of them LAMFA doesn't exist anymore, in the original one it exists. Guess what? the corrector found LAMFA in my file that doesn't have it :D
Title: Re: Damos address corrector (Python script) Post by: Gilgamesh on June 02, 2019, 04:10:56 AM Feel free to test it and report if it fit your needs. :D Hello Polo! Very good idea and solution. Thanks a lot for your work! I've test it and it's working good, but sometimes there is strange errors: Quote Address 0x04A170 not corrected in 16384 search !!! No matching pattern !!! Correcting address 0x04A172 Address 0x04A172 not corrected in 16384 search !!! No matching pattern !!! Correcting address 0x04A174 Bestmatch for address: 0x04A174, searchaddress: 0x04B7D2, offset: 165E, matchbefore: 0, matchafter: 20 Address 0x04A174 not corrected in 16384 search !!! Keeped Best match: 0x04B7D2 - Offset 165E matchbefore: 0 matchafter: 20 !!! Correcting address 0x04A176 Bestmatch for address: 0x04A176, searchaddress: 0x04B7D4, offset: 165E, matchbefore: 4, matchafter: 16 Address 0x04A176 not corrected in 16384 search !!! Keeped Best match: 0x04B7D4 - Offset 165E matchbefore: 4 matchafter: 16 !!! Correcting address 0x04A178 Bestmatch for address: 0x04A178, searchaddress: 0x04B7D6, offset: 165E, matchbefore: 8, matchafter: 12 Address 0x04A178 not corrected in 16384 search !!! Keeped Best match: 0x04B7D6 - Offset 165E matchbefore: 8 matchafter: 12 !!! Correcting address 0x04A17A Bestmatch for address: 0x04A17A, searchaddress: 0x04B7D8, offset: 165E, matchbefore: 12, matchafter: 8 Address 0x04A17A not corrected in 16384 search !!! Keeped Best match: 0x04B7D8 - Offset 165E matchbefore: 12 matchafter: 8 !!! Correcting address 0x04A17C Address 0x04A17C not corrected in 16384 search !!! No matching pattern !!! Correcting address 0x04A180 Bestmatch for address: 0x04A180, searchaddress: 0x04B7DC, offset: 165C, matchbefore: 2, matchafter: 16 Bestmatch for address: 0x04A180, searchaddress: 0x04A224, offset: A4, matchbefore: 4, matchafter: 8 Bestmatch for address: 0x04A180, searchaddress: 0x049D34, offset: -44C, matchbefore: 2, matchafter: 8 Address 0x04A180 not corrected in 16384 search !!! Rejected Best match: 0x04B7DC - Offset 165C matchbefore: 2 matchafter: 16 !!! Correcting address 0x04A184 Bestmatch for address: 0x04A184, searchaddress: 0x04B7E0, offset: 165C, matchbefore: 10, matchafter: 8 Address corrected from 0x04A170 to 0x04B7CE - 36% - 105420/286002 - offset 165E Address corrected from 0x04A171 to 0x04B7CF - 36% - 105421/286002 - offset 165E Address corrected from 0x04A172 to 0x04B7D0 - 36% - 105422/286002 - offset 165E Address corrected from 0x04A173 to 0x04B7D1 - 36% - 105423/286002 - offset 165E Address corrected from 0x04A174 to 0x04B7D2 - 36% - 105424/286002 - offset 165E Address corrected from 0x04A175 to 0x04B7D3 - 36% - 105425/286002 - offset 165E Address corrected from 0x04A176 to 0x04B7D4 - 36% - 105426/286002 - offset 165E Address corrected from 0x04A177 to 0x04B7D5 - 36% - 105427/286002 - offset 165E Address corrected from 0x04A178 to 0x04B7D6 - 36% - 105428/286002 - offset 165E Address corrected from 0x04A179 to 0x04B7D7 - 36% - 105429/286002 - offset 165E Address corrected from 0x04A17A to 0x04B7D8 - 36% - 105430/286002 - offset 165E Address corrected from 0x04A17B to 0x04B7D7 - 36% - 105431/286002 - offset 165C Address corrected from 0x04A17C to 0x04B7D8 - 36% - 105432/286002 - offset 165C Address corrected from 0x04A17D to 0x04B7D9 - 36% - 105433/286002 - offset 165C Address corrected from 0x04A17E to 0x04B7DA - 36% - 105434/286002 - offset 165C Address corrected from 0x04A17F to 0x04B7DB - 36% - 105435/286002 - offset 165C Address corrected from 0x04A180 to 0x04B7DC - 36% - 105436/286002 - offset 165C Address corrected from 0x04A181 to 0x04B7DD - 36% - 105437/286002 - offset 165C Title: Re: Damos address corrector (Python script) Post by: Polo35580 on July 15, 2019, 02:59:13 AM If only it worked as decently with continental as it does with Bosch stuff... there's definitely LOADS of room for improvement there, just searching for byte strings isn't enough to make it a proper "corrector"... I've tested it on 2 similar Bosch MED17 files, in one of them LAMFA doesn't exist anymore, in the original one it exists. Guess what? the corrector found LAMFA in my file that doesn't have it :D HelloI can take a look to Simos support if you provide me some hex + damos couples :) Yes there is lot of things to improve as reorder the whole code with class and functions or rewrote the address linearization part which cause your LAMFA and Gilgamesh issues or add MEASUREMENT correction support ... It's a hard work for each improvement and I don't have enough time for now... Feel free to give help on any part Best regards Polo Title: Re: Damos address corrector (Python script) Post by: nabsabs on October 16, 2019, 02:40:47 AM Hello
I am trying this for the first time on an MED9.1 file I am getting the following errors and it is not being executed c:\Users\nabih\Desktop\Test>python correct-damos-address.py DamosFile.A2L OriginalDump NewDump File "correct-damos-address.py", line 734 print (('Address 0x%06X skipped because of unreconizable pattern - %d%% - %d/%d ' % (damosaddress, (100/len(damosaddresslist)*damosaddresscount), damosaddressco unt, len(damosaddresslist))), end="\r") ^ SyntaxError: invalid syntax Image attached below (http://) Can anyone help Title: Re: Damos address corrector (Python script) Post by: pukacinio on January 17, 2021, 08:53:18 AM Hello I am trying this for the first time on an MED9.1 file I am getting the following errors and it is not being executed c:\Users\nabih\Desktop\Test>python correct-damos-address.py DamosFile.A2L OriginalDump NewDump File "correct-damos-address.py", line 734 print (('Address 0x%06X skipped because of unreconizable pattern - %d%% - %d/%d ' % (damosaddress, (100/len(damosaddresslist)*damosaddresscount), damosaddressco unt, len(damosaddresslist))), end="\r") ^ SyntaxError: invalid syntax Image attached below (http://) Can anyone help Installing a newer version of python solved this problem for me. ;) Polo35580 it works for me, you're my hero ;D Title: Re: Damos address corrector (Python script) Post by: gt-innovation on February 25, 2021, 09:24:54 AM There is a bug for binaries that are loading the module table in a different offset in ram.That means the reference point will be shifted so the parsing of the a2l will not match the actual binary file.
for example in Med17.1.62 you will need 0x809 instead of 0x803. For those who do not know how to edit the code a quick workaround will be to replace all 0x809 occurrences with 0x803 and then feed the a2l to the script. Remember to change back the offset in the generated a2l before usage in ols. Other editors with direct address reference will work just fine without changing the offset. Great work polo35580. Title: Re: Damos address corrector (Python script) Post by: nyet on March 01, 2021, 01:12:39 AM for example in Med17.1.62 you will need 0x809 instead of 0x803. For those who do not know how to edit the code a quick workaround will be to replace all 0x809 occurrences with 0x803 and then feed the a2l to the script. If only there was a way for multiple people to work on a common set of source code Title: Re: Damos address corrector (Python script) Post by: OzzydaveO on July 02, 2021, 06:09:23 PM Installing a newer version of python solved this problem for me. ;) Polo35580 it works for me, you're my hero ;D Perhaps a check to make sure the running version of python is ver 3.x (preferably 3.7 or later) Some thing like this: import sys if sys.version_info < (3, 7): print('Please upgrade your Python version to 3.7.0 or higher') sys.exit() Title: Re: Damos address corrector (Python script) Post by: kirukisu on December 10, 2021, 05:46:18 PM What size should have .hex file? It doesnt like any of my ME7.5 .hex & .a2l pair . Will .dam be supported?
Code: PS C:\Users\lukas\Desktop\test> python correct-damos-address-v1.4.py a24c203g.a2l a24c203g.hex NewDump.bin Title: Re: Damos address corrector (Python script) Post by: xmifeng6 on March 19, 2022, 06:47:58 PM NICE WORK bro ！ thanks for share ！
Title: Re: Damos address corrector (Python script) Post by: Fang on February 09, 2024, 01:45:38 PM I wonder where is the script? Sorry I found them . ;D
|