Part 1: Basic definitions

Definitions are stored in a XML format in the me7xmls directory.
There is also a file called "mapdef.xsd" which defines the schema according to which the XML files are created.
When creating definitions or editing existing ones, I strongly recommend you use an editor which supports schema and auto completing, as it will make your life a lot easier.

Let's take a look at locating the 16 bit KRKTE.
The first thing, is to find the KRKTE access in IDA Pro. This is simple - pick a binary you have a DAMOS for, and then go from there.
I will use the 551K RS4 binary for this example.

In this binary KRKTE is at 0x1C9DC. There are three places which access this value in the binary.
So we must choose one of them and concentrate on it. It is best to look for a spot that has unique code around it.
Like this one:


Time to create the initial XML:

<?xml version="1.0" encoding="UTF-8"?>
<map xmlns="http://prj-tuning.com/mapdef" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://prj-tuning.com/mapdef mapdef.xsd">
</map>

The first thing to add is the "ID". The "ID" is a unique identifier, which identifies the map.
There can be multiple XML files/definitions for the same ID. It is a good idea to name the file the same as the ID, so in this case KRKTE.xml.

Let's add the ID:
<?xml version="1.0" encoding="UTF-8"?>
<map xmlns="http://prj-tuning.com/mapdef" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://prj-tuning.com/mapdef mapdef.xsd">
   <id>KRKTE</id>
</map>

Now comes the most powerful feature of the map locator tool - the pattern.
The pattern is a way to locate a matching area in a binary. So we will have to create a pattern.

The pattern consists of the following building blocks:
HEX - bytes that should be matched exactly
XX - a byte that should be skipped.
XX<number> - means that zero to <number> bytes should be skipped.
Every time the pattern is matched, the offset that is returned is the address where the first character in the pattern matched.
If you would like to move the offset that is returned to an arbitrary place in the pattern, you can prefix any of the members of the pattern with MM.
The prefixed member will become the new reported offset.

For example F2 XX MMF2 XX.

Let's build a pattern for our location. It is a simple pattern, and comes out as follows:
F2 F4 XX XX 7C 44 E0 05 70 55.

We mask out the actual address from the pattern, because it will be different between binaries.
Let's add this to the xml:

<?xml version="1.0" encoding="UTF-8"?>
<map xmlns="http://prj-tuning.com/mapdef" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://prj-tuning.com/mapdef mapdef.xsd">
   <id>KRKTE</id>
   <pattern>F2 F4 XX XX 7C 44 E0 05 70 55</pattern>
</map>

Now the main building blocks have been added. It will find the pattern at an offset, however this is not terribly useful yet.
The way the C167 addresses memory is via a DPP and offset. So we have to specify where from the pattern location the DPP is located and where the offset is located.
In this case our DPP - 0x207h is located two bytes before the pattern start. If we omit this, the default DPP of 0x204h would be used.
Our offset - 0x09DC is located two bytes after the pattern start.

The total address is calculated as: dpp * 0x4000 - 0x800000 + offset. The subtraction is because of where the EEPROM is loaded.
Let's add this information to the XML:
<?xml version="1.0" encoding="UTF-8"?>
<map xmlns="http://prj-tuning.com/mapdef" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://prj-tuning.com/mapdef mapdef.xsd">
   <id>KRKTE</id>
   <pattern>F2 F4 XX XX 7C 44 E0 05 70 55</pattern>
   <address>
      <offset>2</offset>
      <dppOffset>-2</dppOffset>
   </address>
</map>

Alternatively a marker could be specified at F2 F4 MMXX XX... and the offset omitted, and DPP offset set to -4.

Now we just need to add the data to convert the located value to a legible form. This is done by the conversion element.

<?xml version="1.0" encoding="UTF-8"?>
<map xmlns="http://prj-tuning.com/mapdef" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://prj-tuning.com/mapdef mapdef.xsd">
   <id>KRKTE</id>
   <pattern>F2 F4 XX XX 7C 44 E0 05 70 55</pattern>
   <address>
      <offset>2</offset>
      <dppOffset>-2</dppOffset>
   </address>
   <conversion>
      <factor>0.000167</factor>
      <width>2</width>
      <endianness>LoHi</endianness>
   </conversion>
</map>

The factor is specified - the default factor is 1.0.
The offset is not needed - the default offset is 0.0, which suits us in this case.
The default width is 1 byte and the default endianness is HiLo. In this case the value is 2 bytes wide and bigendian, so we specify the endianness as LoHi.

And that's it - this XML is enough to detect KRKTE in almost any ME7 file, where it is a 16 bit value.
For 8 bit KRKTE's a slight modification is required, an example can be found in KRKTE_8.xml.

Part 3: XML reference

To create a new XML definition, simply add an XML to the me7xmls folder.
You can keep the program running and re-parse a binary as often as you like. Changes to your XML's will be picked up on the fly.

The mapdef.xsd file outlays the format. Here is a list of all the elements and details on what they do.


Map:
<map> - the root map element.
   <id> - the map ID
   <pattern> - the lookup pattern to match the map. You can specify multiple pattern elements, they get checked from top to bottom.
   <afterRowAxis> - can be used instead of pattern, if the given map is located right after the row axis.
   <afterColAxis> - can be used instead of pattern, if the given map is located right after the column axis.
   <address> - address element, describes the map location
   <conversion> - conversion element, describes the map data.
   <length> - length element. Describes the length of the map if it needs to be loaded.
   <rowAxis> - Lists references to the row axis
   <colAxis> - Lists references to the column axis

Address:
<address> - the address element describes the address offset from the pattern location.
   <offset> - the offset from the matched pattern location/marker. Defaults to 0.
   <dpp> - hard coded dpp. Defaults to 0x204h, if dpp nor dppOfset is specified.
   <dppOffset> - the offset from the matched pattern for the dpp value.

Conversion:
<conversion> - the conversion element describes the map data.
   <factor> - the factor for map data, defaults to 1.0.
   <offset> - the offset for map data, defaults to 0.0.
   <width> - the map width. 1 or 2, defaults to 1.
   <endianness> - the map endianness. LoHi or HiLo, defaults to HiLo.
   <signed> - whether the value is a signed data type, defaults to false.
   <alt> - another Conversion element. This is used if an axis is specified as 16 bit, but is actually 8 bit. Look at SNM16ZUUB for an example.

Length:
<length> - the length element is used to specify the length of a map. For maps with axes this is calculated from axis lengths.
   <hardcoded> - hardcoded length inside the XML, defaults to 1.
   <address> - address element to locate the length in the binary.
   <width> - width element, only valid with address element, defines whether length is 1 or 2 bytes long.

RowAxis/ColAxis:
<rowAxis> - the rowAxis and colAxis elements specify a hierarchical list of axis candidate ID's.
   <id> - one or more ID's of the axis maps.



Part 4: Plugin reference

If you would to define a plugin, then you will have to implement the LocatorPlugin interface, make a jar, and put the jar into the "lib" folder.
The folder is scanned for plugins on startup.

The LocatorPlugin interface defines a single method:
public interface LocatorPlugin {
  Collection<? extends LocatedMap> locateMaps(byte[] binary);
}

LocatedMap is a bean, which your plugin should fill out as completely as possible.
Each plugin is run in a separate background thread.

It is up to the community to create an extensive XML for this?

P.S. You are one of the people I was targeting with this. Give me some feedback, what do you think?

awesome. git cloning now.

I have a few java plugins in mind as well.
Using the Pattern matcher a Java plugin should be written that will find and define all DTC's by code using the algorithm Tony posted a while back.

This will allow to disable DTC's very quickly.
Maybe nyet feels up to the task?

Applied.

Odd that it built fine here, I guess I had some Eclipse leftovers in there.