NefMoto

Technical => Flashing and Chipping => Topic started by: n0ble on August 10, 2014, 03:10:22 AM



Title: FRF and SGO - Differences?
Post by: n0ble on August 10, 2014, 03:10:22 AM
For Vas-PC - Why are their 2 types of file?

As a guess, SGO is full flash and FRF is partial?

I have the stock files:
MED17.5.5 - MED1755_03C906027AD_1037518220
DQ200 - HW: 0AM927769D - v069E5110AM___getriebe_DSG_tbE5

Would i be right in thinking that the attached is the relevant update files for stock?
MED17.5.5 - FL_03C906027AD_9971.frf
DQ200 - v069E5610AM___getriebe_DSG_tbE5_sw.sgo

Also, can someone point me in the right direction of converting the DSG .SGO to a .BIN
AND
The MED17 .FRF to a .ODX?

I understand that i will not be able to update with the .FRF file using VAS-PC until it has been converted to .ODX?

Thanks again, and sorry for not be very clear i'm just a little lost with this.

Thanks



Title: Re: FRF and SGO - Differences?
Post by: TCSTigersClaw on August 18, 2014, 08:10:24 AM
this is an Ibiza FR file right ? the 9971 is almost the latest update (now it is 9972) , it is update only and VAS can write in OTP areas also.


Title: Re: FRF and SGO - Differences?
Post by: TCSTigersClaw on August 18, 2014, 08:14:27 AM
sorry forgot about the question.

frf and sgo are encrypted for VAS , I dont think they can be converted to bin. The best chance is to flash them via VAS and then read them via obd or tricore boot to get the BIN


Title: Re: FRF and SGO - Differences?
Post by: IamwhoIam on August 19, 2014, 07:42:33 AM
this is an Ibiza FR file right ? the 9971 is almost the latest update (now it is 9972) , it is update only and VAS can write in OTP areas also.

NO TOOL can write to OTP areas!!!! EVER!


Title: Re: FRF and SGO - Differences?
Post by: k0mpresd on August 19, 2014, 08:31:46 AM
sgo can be converted to bin.


Title: Re: FRF and SGO - Differences?
Post by: coreyj03 on August 19, 2014, 09:06:52 PM
i posted up the factory mk5 DSG launch control .sgo a couple years if u need it.  search its somewhere on here


Title: Re: FRF and SGO - Differences?
Post by: ddillenger on August 19, 2014, 09:29:21 PM
NO TOOL can write to OTP areas!!!! EVER!

Bullshit.

The first time they're easy to write.

It's the subsequent writes that aren't so easy.

:P



Title: Re:
Post by: n0ble on August 20, 2014, 08:26:00 PM
TCStigersclaw yes it is the Ibiza FR :-) where can I get the 9972 file? Latest I could find was 9971. I successfully flashed the 9971 file with ODIS engineering as vas-pc does not support the protocol the ECU is running on.

Vas-pc did work ok for flashing the DSG though.

ddillenger that is very true :-) although I'm lead to believe that the cks are in the OTP areas? So if the map data changes in a update file how do they deal with the CKS, as the previous CKS in the OTP will now be incorrect?


Title: Re: FRF and SGO - Differences?
Post by: k0mpresd on August 21, 2014, 11:01:19 AM
here is sgo file description, along with otp area for same ecu.

Quote
Reading ECU data, please wait...
PROCESSOR TYPE: TC1796
IROM FLASH sector configuration:
Address   Size    Access
A0000000h   4000h   read and write
A0004000h   4000h   read and write
A0008000h   4000h   read and write
A000C000h   4000h   read and write
A0010000h   4000h   read only (OTP)
A0014000h   4000h   read only (OTP)
A0018000h   4000h   read and write
A001C000h   4000h   read and write
A0020000h   20000h   read and write
A0040000h   40000h   read and write
A0080000h   80000h   read and write
A0100000h   80000h   read and write
A0180000h   80000h   read and write


Title: Re:
Post by: n0ble on August 22, 2014, 01:28:20 AM
Thanks k0mpresd.


Title: Re: FRF and SGO - Differences?
Post by: gremlin on August 24, 2014, 12:19:02 PM
sgo can be converted to bin.

No problem convert .FRF to bin also.
Just convert FRF to ODX (the same as XML-format file) and than extract BIN content (coded as BCB) from ODX.
As example below is ODX converted from frf-file given in topic start message.


Title: Re: FRF and SGO - Differences?
Post by: chli1976 on August 24, 2014, 09:39:23 PM
Just convert FRF to ODX (the same as XML-format file) and than extract BIN content (coded as BCB) from ODX.

Can you give more infos
thanks


Title: Re: FRF and SGO - Differences?
Post by: tabster on December 26, 2014, 05:51:24 PM
For Vas-PC - Why are their 2 types of file?

As a guess, SGO is full flash and FRF is partial?


SGO files are for ECUs using KWP2000 protocol.
FRF, ODX and SOX files are for ECUs using UDS protocol.

In theory all types can be converted to bin, however many different methods of compression and encryption exist, so it can take some time and effort to do it.



Title: Re: FRF and SGO - Differences?
Post by: chli1976 on December 27, 2014, 03:14:17 AM
coded as BCB

Is this the right method

http://timogruss.de/2014/01/bcd-in-dezimalwerte-umrechnen/#BCD_zu_Dezimal_umrechnen


Title: Re: FRF and SGO - Differences?
Post by: technic on December 27, 2014, 06:29:53 AM
BCB, not BCD :) BCB is a compressed format delevoped by Bosch


Title: Re: FRF and SGO - Differences?
Post by: Aurélien on December 27, 2014, 03:29:22 PM
Bosch ODX content is COMPRESSED and ENCRYPTED ( 11 )

Encryption is very easy.
Decompression is easy also... :)
Compression, the proper way ( not just tellng " following block is uncompressed " ) is a lot more work though.


Title: Re: FRF and SGO - Differences?
Post by: KmosK04 on January 16, 2015, 07:48:54 AM
Can somebody know how to convert .frf files to .bin? I have an app that converts them to .odx. Now I have to convert that to .bin? If yes how?? Thanks


Title: Re: FRF and SGO - Differences?
Post by: KmosK04 on January 19, 2015, 02:07:04 PM
Anyone please?


Title: Re: FRF and SGO - Differences?
Post by: dream3R on October 09, 2015, 03:29:27 PM
So the resulting bin from FRF hat does it all contain?


Title: Re: FRF and SGO - Differences?
Post by: nyet on October 09, 2015, 07:02:47 PM
Bosch ODX content is COMPRESSED and ENCRYPTED ( 11 )

Encryption is very easy.
Decompression is easy also... :)
Compression, the proper way ( not just tellng " following block is uncompressed " ) is a lot more work though.


Too bad nobody has the balls to release source code.

It is easy to say something is "easy".

It isn't easy to document and publish.

All balless wonders who talk a lot but not much else.


Title: Re: FRF and SGO - Differences?
Post by: n0ble on October 10, 2015, 01:17:41 PM
I'm almost there with it....

Now at the final step of trying to work out the compression, I have half worked out the compression but unfortunately my knowledge lacks here.

However i'll keep at it, I'll get there in the end.


Title: Re: FRF and SGO - Differences?
Post by: dream3R on October 11, 2015, 10:56:45 AM
Anyone know if VAG use FRF for UDS definitions i.e.  ReadDataByIdentifier?



Title: Re: FRF and SGO - Differences?
Post by: dream3R on October 11, 2015, 12:10:41 PM
I'm almost there with it....

Now at the final step of trying to work out the compression, I have half worked out the compression but unfortunately my knowledge lacks here.

However i'll keep at it, I'll get there in the end.

Care to post your progress?   I was looking before but didn't look like something i've seen before.

edit:  keeping wih it, assuiming it's within you, it's the best thing.  I nearly wen't mad doing 5 bar on my Volvo but got there.  BTW the one on here is incomplete...mods know this etc, prj knows it, pisses me off!



Title: Re: FRF and SGO - Differences?
Post by: dream3R on October 11, 2015, 12:12:50 PM
Bosch ODX content is COMPRESSED and ENCRYPTED ( 11 )

Encryption is very easy.
Decompression is easy also... :)
Compression, the proper way ( not just tellng " following block is uncompressed " ) is a lot more work though.


Come-on then friend, give me some clues, you were happy enough for my FREE MED9 help!


Title: Re: FRF and SGO - Differences?
Post by: Geremia on October 17, 2015, 03:59:52 AM
frf-to-odx is done inside SoxUtil.dll (odis or DTS7)
Code:
text:10001CC5                 call    edi ; MString::operator char const *(void) ; MString::operator char const *(void)
.text:10001CC7                 push    ebx             ; dest_zip_filename
.text:10001CC8                 push    eax             ; frf_filename
.text:10001CC9                 lea     ecx, [ebp+var_170]
.text:10001CCF                 call    MY_getKey_and_goto_descramble
.text:10001CD4                 lea     ecx, [ebp+var_170]
.text:10001CDA                 mov     byte ptr [ebp+var_4], 2
.text:10001CDE                 call    MY_unzipper_stuff
.text:10001CE3                 test    al, al
.text:10001CE5                 jnz     short loc_10001CFC
.text:10001CE7                 lea     ecx, [ebp+var_28]
.text:10001CEA                 push    offset unk_100046E0
.text:10001CEF                 push    ecx
.text:10001CF0                 mov     [ebp+var_28], offset aNotOne_odxInAr ; "Not one .odx in archive"
.text:10001CF7                 call    _CxxThrowException

key.bin is inside the resource area

Then, as told, the odx contains flash data in encrypted/compressed form.
I dont' know for ECUs, you need to RE bootarea to know the decryption/decompression algo and i did only for dq200 0CW, and yes they are simple once ported to C code, but takes some days to RE them, so i'm not surprised if they don't go opensource quickly.
Flashdata first need to be descrambled and, at least for dsg, it's the same scrambling algo found in previous sgo files, just the byte subst table is per ecu type.
About compression algo, don't know, probably dsg uses diff algo than bosch, anyway comparing compressed and uncompressed data makes the task very easy.


Title: Re: FRF and SGO - Differences?
Post by: tmbinc on November 09, 2015, 04:00:22 PM
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

const unsigned char key[4095] = {
#include "key.h"
};

int main(void)
{
   int kidx = 0;
   int seed0 = 0;
   int seed1 = 1;
   
   while (1)
   {
      unsigned char buf[1024];
      int i;
      int n = read(0, buf, sizeof(buf));
      if (!n) {
         break;
      }
      
      for (i = 0; i < n; ++i) {
         unsigned char kb = key[kidx];
         kidx += 1;
         kidx %= sizeof(key);

         seed0 = ((seed0 + kb) * 3) & 0xFF;

         buf ^= seed0 ^ 0xFF ^ seed1 ^ kb;

         seed1 = ((seed1 + 1) * seed0) & 0xFF;
      }
      
      write(1, buf, n);
   }
   return 0;
}


Title: Re: FRF and SGO - Differences?
Post by: Geremia on November 10, 2015, 03:45:36 PM
welcome! ;)
I do like this, but it's ok anyway, since keysize is 0xFFF
buf ^= seed0 ^ sizeof(key) ^ seed1 ^ kb;


Title: Re: FRF and SGO - Differences?
Post by: tmbinc on November 13, 2015, 05:16:18 PM
Also, to unpack "BCB Type1" compressed data:

import sys, struct

key = "BiWbBuD101"

img = sys.stdin.read()
img = img[img.index("\x1A\x01") + 2:]
img = ''.join(chr(ord(j)^ord(key[i%len(key)])) for i, j in enumerate(img))

p = 0
res = ""

while p < len(img):
  l = struct.unpack(">H", img[p:p+2])[0]
  p += 2

  fl = l >> 14
  l &= 0x3FFF

  if fl == 0: # literal
    res += img[p:p+l]
    p += l
  elif fl == 1: # RLE
    res += img[p] * l
    p += 1
  else:
    sys.stderr.write("remaining bytes: " + img[p:].encode('hex') + "\n")
    break

sys.stdout.write(res)



Title: Re: FRF and SGO - Differences?
Post by: Geremia on November 15, 2015, 05:11:44 PM
else //fl==3 checksum
      {
         p++;
         unsigned int chk=(inbuf[p]<<24)|(inbuf[p+1]<<16)|(inbuf[p+2]<<8)|inbuf[p+3];
         unsigned int chk2=0;
         for(unsigned int i=0;i<outsize;i++) chk2+=outbuf;
         if(chk!=chk2)
         {
            printf("Checksum mismatch, file=0x%X calc=0x%X at inbuf offset 0x%X\n", chk, chk2, p);
            error=true;
         }
         if((p+4)!=size)
         {
            printf("Checksum at offset 0x%X not EOF\n",p);
            error=true;
         }
         break;
      }


Title: Re: FRF and SGO - Differences?
Post by: H2Deetoo on November 16, 2015, 12:35:16 AM
This looks very interesting guys!

Is somebody able to post a complete example of input/output data to verify the posted routines?
I am interested in writing a (Delphi) application for this...


Thanks,
H2Deetoo


Title: Re: FRF and SGO - Differences?
Post by: DrDelphi on September 27, 2016, 11:42:38 PM
Hello guys

Too bad this thread has died.
If the members that posted the above code pieces are still around, I would have some questions if they are kind enough to clear them with me.
So, first of all, where do I get the "key" from ?
Normally, having a SGO and the BIN file, should be enough to find the key, but the seed algo doesn't check. Are you sure it is seed0 = (seed0 + key[kIdx]) * 3 ? Isn't it *2 or *4 or anything else?
The password encrypted SGOs are easier to decrypt. Some passwords I found are BiWbBuD101, GEHEIM, CodeRobert and MILKYWAY.

This should be all for now.
Looking forward for you replies.
Best regards.


Title: Re: FRF and SGO - Differences?
Post by: Geremia on September 30, 2016, 11:00:34 AM
The seed0 = (seed0 + key[kIdx]) * 3 you refer, is about decrypting frf to a zip file, inside zip there is odx file, inside odx you find DATA, which is (mont time) encrypted/compressed data you send (as is) to ecu when flashing.
In sgo, if i remember well, is a simply xor FF (or sort of) to get an sgm file, which is a container where you find DATA (again, most time compressed and/or encrypted) to send (as is) to ecu.

If you are referring to seed in the mean of seed/key auth against ecu, that's another story, SA2 data (you can find it inside sgm and odx) is what you need for passing seedkey in programming session (diag session is another story).

I came accross the milkyway too  ;)


Title: Re: FRF and SGO - Differences?
Post by: cherry on September 30, 2016, 12:58:11 PM
So far i know DATAblocks in sgm should only be "encrypted" as base64.


Title: Re: FRF and SGO - Differences?
Post by: cherry on September 30, 2016, 01:13:52 PM
Attached sgm example.


Title: Re: FRF and SGO - Differences?
Post by: DrDelphi on October 01, 2016, 01:17:50 AM
Thanks for the reply, Geremia.

I thought that the algo you posted was for decoding SGO blocks. Meanwhile I realized it was for FRF as I succeeded to turn them into ODX and then into BINs.

The problem I am dealing with now is with the SGOs.

The SGO files that are compressed, are also password encrypted and they are easy to decompress / decode. If you don't know the password, but you have the BIN from another source, I can find out the password by compressing the BIN and then XORing the result with the SGO.

The problem is with the uncompressed SGOs that don't use passwords and I couldn't yet figure out how they are encrypted.
If I take a SGO, extract its blocks, then take the corresponding BIN from another source and XOR them, I get some scrambled data. Then if I take the scrambled data and XOR it with another SGO's blocks, I get the correct data result, but it's not a solution I can rely on. It's definitely another XOR algo, but not the one used for FRFs and maybe another key too.

Any hint ?

Thanks in advance and best regards.


Title: Re: FRF and SGO - Differences?
Post by: Geremia on October 01, 2016, 03:11:15 AM
Well, sgo/odx are only containers, how DATA is compressed/encrypted is ecu and/or car brand specific, you have to reverse the service36 function of the bootloader of the specific ecu to know how it's done, or if you have some encrypted and decrypted pairs, maybe guess it.


Title: Re: FRF and SGO - Differences?
Post by: prj on October 05, 2016, 03:51:29 PM
The problem is with the uncompressed SGOs that don't use passwords and I couldn't yet figure out how they are encrypted.

If it's not encrypted then it's XOR FF and result is the file obviously, as has been posted here.
If it's BCB compressed XOR encrypted - then breaking the key takes <1 second with a single thread for any file, you don't need to know what's in there nor do you need the key, as it can be computed at runtime.
If it's some different type of encryption and/or compression, then you will have to obviously reverse the bootloader of the ECU to find the algorithm.

EVC has introduced a SGO/FRF import plugin. Upon asking them which ECU's it actually works with I did not receive a reply... and I am a customer.


Title: Re: FRF and SGO - Differences?
Post by: ladan on November 09, 2016, 11:27:11 AM
Hi everyone...  so where can I find "key" array?  Geremia ?

const unsigned char key[4095] = {
#include "key.h"
};


Title: Re: FRF and SGO - Differences?
Post by: nyet on November 09, 2016, 12:11:27 PM
If somebody wants to give me a set of sgo/bin pairs and decent documentation i'd be more than happy to write a decoder and provide source code.


Title: Re: FRF and SGO - Differences?
Post by: prj on November 10, 2016, 07:59:14 AM
If somebody wants to give me a set of sgo/bin pairs and decent documentation i'd be more than happy to write a decoder and provide source code.
Did you ignore what I wrote in the other thread?

I am bumping thic topic.
What's needed to write such converter? I would donate, others would do the same I guess.
1. Reverse SGO format enough to parse it.
2. Reverse the bootloader on every single ECU type you want to convert.

EDC15/ME7/ZF6HP/MED9/ME17 are BCB/XOR. Some have fixed key, others like EDC15 have a different key for every ECU file and version.
Other ECU's such as EDC16 and SIMOS have a completely different algorithm that can be only obtained by reversing the the bootloader.

I would happily write a converter if somebody can post the specs to the sgo file.
With all due respect, I don't think you are going to break the crypto on EDC16 for example. You need to pull the bootloader apart one by one, it is a very time consuming process. The SGO is easy to reverse even without any specs.

I don't think you understand how this works.
The code to decode ME7 is already in this forum thread, but it is different for every ecu type.


Title: Re: FRF and SGO - Differences?
Post by: ladan on November 11, 2016, 03:48:25 AM
hi Prj,
So what about my question? Don't you know? 
As far as I see - all "converters"(FRF 2 ODX/SOX) use original .dll (from DTS7) to convert FRF to ODX, but I searching algo(my solutions are running under Linux).... One posted above looks good but where to find a "key"?   


Title: Re: FRF and SGO - Differences?
Post by: prj on November 11, 2016, 05:06:57 AM
hi Prj,
So what about my question? Don't you know? 
As far as I see - all "converters"(FRF 2 ODX/SOX) use original .dll (from DTS7) to convert FRF to ODX, but I searching algo(my solutions are running under Linux).... One posted above looks good but where to find a "key"?   

Why do you feel entitled to an answer? It is not my job to answer you.
I have not even looked at the FRF format, I only wrote an SGO decoder and I had help with reversing the algorithms in some of the ECU's.


Title: Re: FRF and SGO - Differences?
Post by: ladan on November 11, 2016, 05:21:20 AM
Hm...  I just hoped you can help, since you are online....


Title: Re: FRF and SGO - Differences?
Post by: ladan on November 12, 2016, 06:57:35 AM
turned on my brain, read once again, found all I need, wrote unpacker script .... thanks Geremia, tmbinc


Title: Re: FRF and SGO - Differences?
Post by: learning1 on November 19, 2016, 11:00:44 AM
NO TOOL can write to OTP areas!!!! EVER!

That is incorrect.
OTP is write once area that starts with all binary bits unset value 00
When all binary bits are burned you have FF
You can only burn more bits so you can only move towards FF,

Tools tend to avoid OTP but the statement that no TOOL can write to this EVER is not TRUE


Title: Re: FRF and SGO - Differences?
Post by: IamwhoIam on November 22, 2016, 03:46:42 AM
That is incorrect.
OTP is write once area that starts with all binary bits unset value 00
When all binary bits are burned you have FF
You can only burn more bits so you can only move towards FF,

Tools tend to avoid OTP but the statement that no TOOL can write to this EVER is not TRUE

My bad, let me rephrase that: no OBD flash tool can EVER write OTP areas that have been programmed and set as OTP. EVAR.


Title: Re: FRF and SGO - Differences?
Post by: dera on March 16, 2017, 10:12:32 PM
My bad, let me rephrase that: no OBD flash tool can EVER write OTP areas that have been programmed and set as OTP. EVAR.

Yes they can, just that they can only go one way with it.
Let me rephrase your comment :) No tool can 0 OTP bits that have been set.