NefMoto > Technical > Data Logging > trim heatmap tool for tuning KFKHFM
Pages: [1]
« previous next »
Author Topic: trim heatmap tool for tuning KFKHFM  (Read 10697 times)
nyet
Administrator
Hero Member
*****

Karma: +609/-169
Offline Offline

Posts: 12295


WWW
« on: October 24, 2024, 01:20:02 AM »
Similar to the FKVVS tuner, this is specifically made to tune KFKHFM based on fr/frm logs.

https://github.com/nyetwurk/trim-heatmap/releases

It is command line based, so you have to give it one or more log files on the command line. If you do not know what that means, this tool is not for you



Code:
usage: heatmap.py [-h] [-w WINDOW] [-l LOAD_FILTER] [-r RPM_FILTER] [-m MIN_SAMPLES] [-f] [-u] [-v] [filename ...]

Create trim heatmap for KFKHFM based on fr/frm datalog

positional arguments:
  filename              csv files(s) to parse (log.csv)

options:
  -h, --help            show this help message and exit
  -w WINDOW, --window WINDOW
                        number of sequential rows to detect constant rpm/load (5)
  -l LOAD_FILTER, --load-filter LOAD_FILTER
                        change in load which is still "constant" load (10)
  -r RPM_FILTER, --rpm-filter RPM_FILTER
                        change in RPM which is still "constant" RPM (100)
  -m MIN_SAMPLES, --min-samples MIN_SAMPLES
                        minimum number of samples required to generate a cell (10)
  -f, --use-fr          use "fr" instead of "frm" (the default)
  -u, --use-unweighted-mean
                        use unweighted mean instead of weighted average (the default)
  -v, --verbose
« Last Edit: October 24, 2024, 01:22:52 AM by nyet » Logged
ME7.1 tuning guide
ECUx Plot
ME7Sum checksum
Trim heatmap tool

Please do not ask me for tunes. I'm here to help people make their own.

Do not PM me technical questions! Please, ask all questions on the forums! Doing so will ensure the next person with the same issue gets the opportunity to learn from your ex
ratosluaf
Jr. Member
**

Karma: +1/-3
Offline Offline

Posts: 39
« Reply #1 on: January 06, 2025, 03:31:43 AM »
I've done something similar in the past for myself.
Now I had some free time so I've rewritten it to streamlit and deployed it on the community cloud.

Here's the link: https://fixhfm.streamlit.app/

It does depend on nmot_w, rl_w and fr_w from me7logger, so idk if it will work with all logs. It does stick your data to nearest nmot_w and rl_w in kfkhfm table and calculate it depending on weighted mean.

You can copy your table from tunerpro or import your table from csv file - example of log file and map csv file in github repo.

github repo: https://github.com/rwalkuski/hfm_fixer

Any feedback is welcome since now i have no possibility to test output myself, but had success with my own car in the past.


Edit: for now, when you want to copy from site to tunerpro, please buffer the data in notepad or any other text editor Copying directly into tunerpro causes destroying of data table, but i don't know why for now.
edit2: copying directly from site lacks of carriage return which tunerpro interprets as new line. At the moment, i do not know how to fix that, so please buffer your data.
« Last Edit: January 06, 2025, 05:09:25 AM by ratosluaf » Logged
Crazy18T
Newbie
*

Karma: +0/-0
Offline Offline

Posts: 13
« Reply #2 on: April 29, 2025, 11:20:31 PM »
Finally have my car back together (in a slightly temporary fashion), so I was eager to try this. Smiley After struggling to get the output to not cause issues with the tune (tried every setting and many iterations), I decided to take a closer look at the code. It seems the bucket creation code is quite broken.

Applying this patch, we can see both RPM and load bucket calculation are way off:
Code:
diff -ur trim-heatmap-0.0.5/src/heatmap.py trim-heatmap-0.0.5-testing/src/heatmap.py
--- trim-heatmap-0.0.5/src/heatmap.py	2024-11-06 02:39:02.000000000 -0700
+++ trim-heatmap-0.0.5-testing/src/heatmap.py	2025-04-29 23:49:57.097326680 -0600
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # internal modules
 import os
@@ -78,6 +78,7 @@
 		frdata[lk]={}
 		for rk,rv in rpms.items():
 			query = f"{rv[0]} <= @lc.rpm <= {rv[1]} & {lv[0]} <= @lc.load <= {lv[1]}"
+			print(query)
 			res = lc.query(query).copy()
 			if len(res.index) >= (args.min_samples, 1)[args.no_filter]: # ternary is (false, true)
 				# size of cell, center to corner

A few rounds of output with patch applied (example log.csv):
Quote
$ ./heatmap.py
0 <= @lc.rpm <= 859.0 & 0 <= @lc.load <= 14.0
860.0 <= @lc.rpm <= 1119.0 & 0 <= @lc.load <= 14.0
1120.0 <= @lc.rpm <= 1379.0 & 0 <= @lc.load <= 14.0
1380.0 <= @lc.rpm <= 1759.0 & 0 <= @lc.load <= 14.0
1760.0 <= @lc.rpm <= 2259.0 & 0 <= @lc.load <= 14.0
2260.0 <= @lc.rpm <= 2759.0 & 0 <= @lc.load <= 14.0
2760.0 <= @lc.rpm <= 3259.0 & 0 <= @lc.load <= 14.0
3260.0 <= @lc.rpm <= 3759.0 & 0 <= @lc.load <= 14.0
3760.0 <= @lc.rpm <= 4259.0 & 0 <= @lc.load <= 14.0
4260.0 <= @lc.rpm <= 4759.0 & 0 <= @lc.load <= 14.0
4760.0 <= @lc.rpm <= 5259.0 & 0 <= @lc.load <= 14.0
5260.0 <= @lc.rpm <= 5759.0 & 0 <= @lc.load <= 14.0
5760.0 <= @lc.rpm <= 6259.0 & 0 <= @lc.load <= 14.0
6260.0 <= @lc.rpm <= 10000 & 0 <= @lc.load <= 14.0
0 <= @lc.rpm <= 859.0 & 15.0 <= @lc.load <= 24.125
860.0 <= @lc.rpm <= 1119.0 & 15.0 <= @lc.load <= 24.125
1120.0 <= @lc.rpm <= 1379.0 & 15.0 <= @lc.load <= 24.125
1380.0 <= @lc.rpm <= 1759.0 & 15.0 <= @lc.load <= 24.125
1760.0 <= @lc.rpm <= 2259.0 & 15.0 <= @lc.load <= 24.125
2260.0 <= @lc.rpm <= 2759.0 & 15.0 <= @lc.load <= 24.125
2760.0 <= @lc.rpm <= 3259.0 & 15.0 <= @lc.load <= 24.125
3260.0 <= @lc.rpm <= 3759.0 & 15.0 <= @lc.load <= 24.125
3760.0 <= @lc.rpm <= 4259.0 & 15.0 <= @lc.load <= 24.125
4260.0 <= @lc.rpm <= 4759.0 & 15.0 <= @lc.load <= 24.125
4760.0 <= @lc.rpm <= 5259.0 & 15.0 <= @lc.load <= 24.125
5260.0 <= @lc.rpm <= 5759.0 & 15.0 <= @lc.load <= 24.125
5760.0 <= @lc.rpm <= 6259.0 & 15.0 <= @lc.load <= 24.125
6260.0 <= @lc.rpm <= 10000 & 15.0 <= @lc.load <= 24.125
0 <= @lc.rpm <= 859.0 & 25.125 <= @lc.load <= 33.875
860.0 <= @lc.rpm <= 1119.0 & 25.125 <= @lc.load <= 33.875
1120.0 <= @lc.rpm <= 1379.0 & 25.125 <= @lc.load <= 33.875
1380.0 <= @lc.rpm <= 1759.0 & 25.125 <= @lc.load <= 33.875
1760.0 <= @lc.rpm <= 2259.0 & 25.125 <= @lc.load <= 33.875
2260.0 <= @lc.rpm <= 2759.0 & 25.125 <= @lc.load <= 33.875
2760.0 <= @lc.rpm <= 3259.0 & 25.125 <= @lc.load <= 33.875
3260.0 <= @lc.rpm <= 3759.0 & 25.125 <= @lc.load <= 33.875
3760.0 <= @lc.rpm <= 4259.0 & 25.125 <= @lc.load <= 33.875
4260.0 <= @lc.rpm <= 4759.0 & 25.125 <= @lc.load <= 33.875
4760.0 <= @lc.rpm <= 5259.0 & 25.125 <= @lc.load <= 33.875
5260.0 <= @lc.rpm <= 5759.0 & 25.125 <= @lc.load <= 33.875
5760.0 <= @lc.rpm <= 6259.0 & 25.125 <= @lc.load <= 33.875
6260.0 <= @lc.rpm <= 10000 & 25.125 <= @lc.load <= 33.875
0 <= @lc.rpm <= 859.0 & 34.875 <= @lc.load <= 44.0
860.0 <= @lc.rpm <= 1119.0 & 34.875 <= @lc.load <= 44.0
1120.0 <= @lc.rpm <= 1379.0 & 34.875 <= @lc.load <= 44.0
1380.0 <= @lc.rpm <= 1759.0 & 34.875 <= @lc.load <= 44.0
1760.0 <= @lc.rpm <= 2259.0 & 34.875 <= @lc.load <= 44.0
2260.0 <= @lc.rpm <= 2759.0 & 34.875 <= @lc.load <= 44.0
2760.0 <= @lc.rpm <= 3259.0 & 34.875 <= @lc.load <= 44.0
3260.0 <= @lc.rpm <= 3759.0 & 34.875 <= @lc.load <= 44.0
3760.0 <= @lc.rpm <= 4259.0 & 34.875 <= @lc.load <= 44.0
4260.0 <= @lc.rpm <= 4759.0 & 34.875 <= @lc.load <= 44.0
4760.0 <= @lc.rpm <= 5259.0 & 34.875 <= @lc.load <= 44.0
5260.0 <= @lc.rpm <= 5759.0 & 34.875 <= @lc.load <= 44.0
5760.0 <= @lc.rpm <= 6259.0 & 34.875 <= @lc.load <= 44.0
6260.0 <= @lc.rpm <= 10000 & 34.875 <= @lc.load <= 44.0
Logged
nyet
Administrator
Hero Member
*****

Karma: +609/-169
Offline Offline

Posts: 12295


WWW
« Reply #3 on: April 29, 2025, 11:57:17 PM »
post the log ill take a look

what are you expecting to see?
« Last Edit: April 29, 2025, 11:59:11 PM by nyet » Logged
ME7.1 tuning guide
ECUx Plot
ME7Sum checksum
Trim heatmap tool

Please do not ask me for tunes. I'm here to help people make their own.

Do not PM me technical questions! Please, ask all questions on the forums! Doing so will ensure the next person with the same issue gets the opportunity to learn from your ex
Crazy18T
Newbie
*

Karma: +0/-0
Offline Offline

Posts: 13
« Reply #4 on: April 30, 2025, 12:03:42 AM »
This is the example log.csv as stated. I assume the bucket code is AI generated? Why do you not see the issue here?
Logged
Crazy18T
Newbie
*

Karma: +0/-0
Offline Offline

Posts: 13
« Reply #5 on: April 30, 2025, 12:34:06 AM »
I expect to see the RPM and load buckets populated as you provided (rpms loads) for the 2.7T. In my case they are adjusted for my setup, but that's beyond the scope of this thread. Sorry! I didn't mean to come across as rude.
Logged
nyet
Administrator
Hero Member
*****

Karma: +609/-169
Offline Offline

Posts: 12295


WWW
« Reply #6 on: April 30, 2025, 10:39:27 AM »
Quote from: Crazy18T on April 30, 2025, 12:34:06 AM
I expect to see the RPM and load buckets populated as you provided (rpms loads) for the 2.7T. In my case they are adjusted for my setup, but that's beyond the scope of this thread. Sorry! I didn't mean to come across as rude.

You're not printing the output. You're printing a query that happens in the core code, before anything else happens, including all processing. What is the input you are giving it? What command line parameters?
« Last Edit: April 30, 2025, 10:41:11 AM by nyet » Logged
ME7.1 tuning guide
ECUx Plot
ME7Sum checksum
Trim heatmap tool

Please do not ask me for tunes. I'm here to help people make their own.

Do not PM me technical questions! Please, ask all questions on the forums! Doing so will ensure the next person with the same issue gets the opportunity to learn from your ex
Crazy18T
Newbie
*

Karma: +0/-0
Offline Offline

Posts: 13
« Reply #7 on: April 30, 2025, 03:31:55 PM »
I'm aware I'm not printing the output, but the output can't be correct if the buckets/cells aren't populated using the proper values. The command line is included with my post. It's just showing what buckets are created using your example log.csv.
Logged
Crazy18T
Newbie
*

Karma: +0/-0
Offline Offline

Posts: 13
« Reply #8 on: April 30, 2025, 03:58:17 PM »
calc_ranges() seems to be completely broken. I don't understand how you aren't seeing this? It creates rpms as 14 buckets/cells as expected, but for example the first bucket shouldn't be 0-859. It should be 720-999. And it continues on with both bucket ranges in this confusing way.
Logged
nyet
Administrator
Hero Member
*****

Karma: +609/-169
Offline Offline

Posts: 12295


WWW
« Reply #9 on: April 30, 2025, 10:08:57 PM »
the map numbers are the *centers* of the cells. We either do a linear average or a weighted average by distance from the center of the cell (which is the map axis data)
« Last Edit: April 30, 2025, 10:11:22 PM by nyet » Logged
ME7.1 tuning guide
ECUx Plot
ME7Sum checksum
Trim heatmap tool

Please do not ask me for tunes. I'm here to help people make their own.

Do not PM me technical questions! Please, ask all questions on the forums! Doing so will ensure the next person with the same issue gets the opportunity to learn from your ex
Pages: [1]
  Print  
« previous next »
 
Jump to:  

Navigation
Personal Tools
  • May 22, 2025, 04:57:19 PM
  • Welcome, Guest. Please login or register.
    Did you miss your activation email?

    Login with username, password and session length
Search
 
Advanced Search
About Disclaimer
All Rights Reserved 2010 Nefarious Motosports
Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Page created in 0.021 seconds with 14 queries. (Pretty URLs adds 0.001s, 0q)