Pages: [1]
Author Topic: value change mystery in WinOls  (Read 6933 times)
userpike
Hero Member
*****

Karma: +22/-1
Offline Offline

Posts: 763


« on: May 21, 2013, 12:02:28 PM »

I have been moving some maps over to a newer software revision and in TVUB particularly the values don't stay what I type in! They are close but not exactly what I typed in. For instance: I typed 1.5015 and go to the next value and it saves as 1.4912. WTF? Does this have something to do with the checksums being corrected or something? I'm confused.
Logged
ddillenger
Hero Member
*****

Karma: +641/-21
Offline Offline

Posts: 5640


« Reply #1 on: May 21, 2013, 12:17:30 PM »

Normal. There is no exact hex match for the value you entered, it rounds it to the nearest possibility.
Logged

Please, ask all questions on the forums! Doing so will ensure the next person with the same issue gets the opportunity to learn from your experience!

Email/Google chat:
DDillenger84(at)gmail(dot)com

Email>PM
k0mpresd
Hero Member
*****

Karma: +146/-54
Offline Offline

Posts: 1655


« Reply #2 on: May 21, 2013, 01:36:36 PM »

copy the actual hex and not the map value in decimal. see if that helps.
sometimes what i enter will round down to FE for example, instead of FF. have to actually physically change the map to FF.
Logged
nyet
Administrator
Hero Member
*****

Karma: +608/-168
Online Online

Posts: 12270


WWW
« Reply #3 on: May 21, 2013, 02:37:02 PM »

This is a HUGE problem for 8-bit values with obnoxious scalars...

if i see a scalar <.01 for an 8-bit value, occasionally i will multiply it by 100 or 1000 (depending on what it is), or even different units entirely, and make a duplicate table definition for it.
Logged

ME7.1 tuning guide
ECUx Plot
ME7Sum checksum
Trim heatmap tool

Please do not ask me for tunes. I'm here to help people make their own.

Do not PM me technical questions! Please, ask all questions on the forums! Doing so will ensure the next person with the same issue gets the opportunity to learn from your ex
userpike
Hero Member
*****

Karma: +22/-1
Offline Offline

Posts: 763


« Reply #4 on: May 22, 2013, 12:34:28 PM »

This is a HUGE problem for 8-bit values with obnoxious scalars...

if i see a scalar <.01 for an 8-bit value, occasionally i will multiply it by 100 or 1000 (depending on what it is), or even different units entirely, and make a duplicate table definition for it.

Will you go into further detail please? What is the advantage of multiplying the scalar? To be honest, I don't even know what the scalars are there for... I've seen them, but haven't messed with them. I assume they will change what is seen on the maps but don't know why you would want to do that.



Logged
nyet
Administrator
Hero Member
*****

Karma: +608/-168
Online Online

Posts: 12270


WWW
« Reply #5 on: May 22, 2013, 12:57:43 PM »

Because for some reason WinOLS does precision by "digits right of the decimal point" rather than significant figures, and for all 8 bit values, the precision should always be 2-3 sigfigs.

However, there is no way to specify "digits right of the decimal point" such that it is correct for all scalars.

I don't really feel like going into an explanation of sig figs vs. decimal points.. science 101 and all..
Logged

ME7.1 tuning guide
ECUx Plot
ME7Sum checksum
Trim heatmap tool

Please do not ask me for tunes. I'm here to help people make their own.

Do not PM me technical questions! Please, ask all questions on the forums! Doing so will ensure the next person with the same issue gets the opportunity to learn from your ex
Axis
Full Member
***

Karma: +4/-4
Offline Offline

Posts: 91


« Reply #6 on: May 22, 2013, 02:04:39 PM »

Userpike: Read this
http://mathforum.org/library/drmath/view/58335.html
Logged
Acki
Full Member
***

Karma: +1/-1
Offline Offline

Posts: 75


« Reply #7 on: May 22, 2013, 03:19:46 PM »

Sounds like the very old excel bug.
Incorrect calc of floating point.
Without using x.xx you have integer and not the problem.
Logged
userpike
Hero Member
*****

Karma: +22/-1
Offline Offline

Posts: 763


« Reply #8 on: May 22, 2013, 10:25:02 PM »

Because for some reason WinOLS does precision by "digits right of the decimal point" rather than significant figures, and for all 8 bit values, the precision should always be 2-3 sigfigs.

However, there is no way to specify "digits right of the decimal point" such that it is correct for all scalars.

I don't really feel like going into an explanation of sig figs vs. decimal points.. science 101 and all..
 

I expect they did it like that because it's more accurate, but I think I get the problem in doing this with the 8 bit values. Anything with higher than 2-3 sig figs would take up too much space being that 8 bits is the max for each value so the value gets rounded to something that will fit?

 
Thanks for the link Axis, it was a good refresher.
Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Page created in 0.019 seconds with 16 queries. (Pretty URLs adds 0s, 0q)