Another thread about summing in scope

[garyb]

[Warp69]

.........32bit float has more resolution than 32bit integer, and that conversion in the working range of float (-1 to 1?) to integer will yield no loss of data?
how is that logical?

First of all - Im usinf the "." as 1000 seperator and the "," as fractional point - that's standard in Denmark - sorry - that could be confusing.

If we use the following range [-1;1] (since that's the normal audio range, as you pointed out) the conversions between float and int will be like this :

If you convert 32bit float to 32bit int (audio range [-1;1]) you'll get :

(+/-) 0,00390625 - (+/-) 1 : no rounding errors
0 - (+/-) 0,00390625 : possible rounding errors = loss of precision

If you convert 32bit int to 32bit float (audio range [-1;1]) you'll get :

(+/-) 0,0078125 - (+/-) 1 : possible rounding errors = loss of precision
0 - (+/-) 0,0078125 : no rounding errors

Remeber my use of "," as a fractional point

As we can see of the above - you can NOT convert 32bit float -> 32bit int or 32bit int -> 32bit float without loss of precision in some ranges of the data.

The reason for the above is demonstrated in my earlier post.

.........
this means that each "bit" reprisents a number:
23rd bit is 1.0
22nd bit is .5
21st bit is .25
and so on
so 1.75
is the 23rd, 22nt and 21st bits turned on.

Let us use 1,75 as an example - this would translate to the following :

1,75 (Decimal) = 1,11 exponent 0 (We haven't move the fractional point therefore 0)

2,75 would be :

10,11 - exponent 0 / as you can see we need to move the fractional point so the result is :

2,75 (Decimal) = 1,011 exponent 1 (since we moved the fractional point 1 step to left)

As you can see - the first bit is "1" and it will always be "1" no matter which number I choose. So why should we encode it into the mantissa when we know it will always be "1"? We don't! Therefore :

1,75 (Decimal) = 1,11 exponent 0 becomes : 11 exponent 0 - so our value 1,75 have only the 23rd and 22nd bits on in our mantissa. And our value 2,75 becomes 011 exponent 1 which have the 22nd and 21st bits on.

As you can see the value of the mantissa bits change according to the exponent, so :

At exponent 0 : 23rd = 0,5 / 22nd = 0,25 / 21st = 0,125 etc.
At exponent 1 : 23rd = 1 / 22nd = 0,5 / 21st = 0,25 etc.
..
..
..

What about the value 0,75 then?

0,75 (Decimal) = 0,11 exponent 0 - We can now move the fractional point to the right since we want a "1" in the first bit :

0,75 (Decimal) = 1,1 exponent -1 (we moved the fractional point 1 step to the right - therefore : -1)

We dont encode the first bit, since it's always "1", we'll then get 0,75 (decimal) = 1 exponent -1, so only the 23rd bit is on.

At exponent -1 : 23rd = 0,25 / 22nd = 0,125 / 21st = 0,0625 etc.
At exponent -2 : 23rd = 0,125 / 22nd = 0,0625 / 21st = 0,03125 etc
..
..
..

.........which means
that the lower the number, the less bits you have for it.
since its not a linear format but an exponential one, you have more range - which means less resolution for the same range.

As you can see from the above : Everytime we decrease the exponent, the range gets smaller, but we preserve all 23bit - so the precision gets better each time we decrease the exponent.

[Warp69]

Warp thank you for this very detailed post! I cant say that I completely understand the math mind you. Would you be willing to put this into a non mathematically sentence or two for me?

Im not trying to ignore you, I just dont know what to write

You'll lose precision with conversion between 32bit float and 32bit int. Which format is the best? It depends

The human ear is more sensitive to changes in volume at low levels compared to high levels and that's exactly what floats represents - higher precision at low numbers compared to higher numbers. But math is not particually sensitive to any range, so when we use floats for coefficients in filters etc we have pretty good precision near zero, but much worse precision near 1 or above. And second, rounding errors and cancellations are inherent in float computations - so it's better to calculate (x+y)(x-y) instead of x^2 - y^2 in float.

[bill3107]

and vice versa ...

Just kidding that thread is obviously not for me

[ReD_MuZe]

yes, but float is not from -1 to 1. you have a given headroom.

take scope for instance - you are getting the slight rounding error you are talking about, not with 32bit float, but with 40bit float.
the mixer atoms in scope for instance have extra headroom that gets clipped (lost) when converted back to 32bit integer.

to test this do the following: take a mix3 and tune all the gains to max. and connect the following:
in1 : maximized drumloop
in2 : maximized drumloop
in3 : -maximized drumloop

you will get the loop without any clipping. if you want to make sure there is no folding taking place, you can make a filter out of it

what i have seen is that you proove that if float 32float can have rougly the same resolution as 32int, providing they have the same range used for audio. and thats obvious they have the same amount of 0 and 1 combinations.
but they don't use the same range for audio.
data is getting lost. about 8 bits.
last paragraph here:
http://www.dspguide.com/ch4/4.htm
and in simple forums english:
http://www.gearslutz.com/board/masterin ... float.html

where does the extra headroom go in 32bit integer? this is the question to answer.

[hifiboom]

most 90% of DSP math is 4th grade math (+-/*) 8% is 8th grade math (trigonometry, exponents , logs) and the rest of the 2% you can learn as you go along.

true, its simple at all.
but for the rest 10% of DSP math you need at least a good amount of logical thinking.

and the more you are expecting from the results the more difficult it gets.

What I mean is:
a single line of code is also understandable by most usual guys, but producing 10000s of codelines and still being able to find through it make it more difficult.

no offense, just an added side note for the fun of the discussion.

by the way i stand the point that the fixed point 32bits in scope are superior to floating point in most audio areas.
I think its up to the coder in fixed point when he degrades soundquality. for example you mix two full 32 bit signals which are filled up to -0dbs. than you have to thing about headroom and gain both signals with 0,5.

The float version is the more easy to handle but with less control on the developer side.

[Warp69]

yes, but float is not from -1 to 1. you have a given headroom.
..
..
what i have seen is that you proove that if float 32float can have rougly the same resolution as 32int, providing they have the same range used for audio. and thats obvious they have the same amount of 0 and 1 combinations.
but they don't use the same range for audio.
data is getting lost. about 8 bits.

Floats have 1bit sign, 8bit exponent and 23bit mantissa - the 8bit exponent represents -126 -> 127. I would imagine that by headroom, you mean all numbers above (+/-)1, right? The "headroom" is handle by a POSITIVE exponent. The range 0 - (+/-)1 is handle by a NEGATIVE exponent.

You're forgetting that the exponent also represents -126 -> 0.

When converting (audio data [-1;1]) - you don't need the POSITIVE part of exponent, only the NEGATIVE.

Try this one - http://babbage.cs.qc.edu/IEEE-754/Decimal.html

last paragraph here:
http://www.dspguide.com/ch4/4.htm

Yes?!? Not sure what you mean, since that proves nothing regarding negative exponents.

and in simple forums english:
http://www.gearslutz.com/board/masterin ... float.html

Ehm, they're completely................WRONG!

[ReD_MuZe]

yes we are getting closer finally
of course exponent is signed (bi polar) otherwise you wouldn't have ANY way to represent numbers smaller than 1 thats not the issue. the question is how much precision do you retain by converting 32bit single precision float to 32int signed.

yes we have lost anything above 0 in the exp value which is exactly like loosing an entire bit (which kills your earlier argument just buy itself, since now we can only use 31 bits of info, and we have half the resolution just because of that. (+ - 2^30)), but thats not all because now you need to compute also the small numbers, too see they don't crunch into integer. since integer cannot reach the ultra low numbers float can produce. so my questions what happens to this range :

floating point range:

(1 / (2^23)) * (2^(-127)) = 7.00649232 * (10^-46)
to
to(1 / (2^23)) * (2^(-8)) = 4.65661287 * (10^-10)

this is the lowest number in fractional 32bit integer:
1 / (2^31) = 4.65661287 * (10^-10)

everything under that is lost as-well.
edit:
thats 5 bits, leaving us with only 2 bits of exponent for 23bits of data.
considering rounding errors, reduces about 1 more bit of precision (half)

http://en.wikipedia.org/wiki/Floating_point
here the second paragraph reads:

The advantage of floating-point representation over fixed-point (and integer) representation is that it can support a much wider range of values. For example, a fixed-point representation that has seven decimal digits, with the decimal point assumed to be positioned after the fifth digit, can represent the numbers 12345.67, 8765.43, 123.00, and so on, whereas a floating-point representation (such as the IEEE 754 decimal32 format) with seven decimal digits could in addition represent 1.234567, 123456.7, 0.00001234567, 1234567000000000, and so on. The floating-point format needs slightly more storage (to encode the position of the radix point), so when stored in the same space, floating-point numbers achieve their greater range at the expense of slightly less precision.

and ofcourse of you look here:
http://en.wikipedia.org/wiki/Single_precision

Single precision binary floating-point format

* Sign bit: 1
* Exponent width: 8
* Significant precision: 23 (24 implicit)

afaik there is no way on earth to get the same resolution from two different ranges of the same bit depths.
pure logic - no math.

[Warp69]

...........which kills your earlier argument just buy itself, since now we can only use 31 bits of info, and we have half the resolution just because of that. (+ - 2^30).

And which argument is that?

...........but thats not all because now you need to compute also the small numbers, too see they don't crunch into integer. since integer cannot reach the ultra low numbers float can produce.

Look here :

If you convert 32bit float to 32bit int (audio range [-1;1]) you'll get :

(+/-) 0,00390625 - (+/-) 1 : no rounding errors
0 - (+/-) 0,00390625 : possible rounding errors = loss of precision

If you convert 32bit int to 32bit float (audio range [-1;1]) you'll get :

(+/-) 0,0078125 - (+/-) 1 : possible rounding errors = loss of precision
0 - (+/-) 0,0078125 : no rounding errors

[ReD_MuZe]

ok let me rephrase that:
"besides what is 32bit float? it has the usable audio resolution of 24bit integer :>"
better?
this was said regarding recording files in 24bits or 32bit float.

there is no sound quality gain. just some headroom, and noise floor. the audible range is 24bit.

[Warp69]

ok let me rephrase that:
"besides what is 32bit float? it has the usable audio resolution of 24bit integer :>"
better?
this was said regarding recording files in 24bits or 32bit float.

there is no sound quality gain. just some headroom, and noise floor. the audible range is 24bit.

And we're right back to square one

32bit float has far better resolution than 24bit int.

If your original data is 24bit audio and you convert that to 32bit float - then you'll have the same presicion, since 32bit float can represent the 24bit audio range perfectly and the source is only 24bit. The same thing can be said about 32bit int, 64bit int, 128bit int etc.

That doesn't mean that 24bit and 32bit float have the same resolution as a standard and you can convert from 32bit float to 24bit int without losing precision.

If you make any calculation in the 32bit float environment with the original 24bit resolution - then you'll lose precision if you convert back to 24bit int, 32bit int or 64bit int.

Maybe this topic should continue through PM.

[valis]

You guys can feel free to dangle your floating points and integers here all you want, we're all watching

Seriously though, it might help to define things a bit more before jumping into the math.

For instance: what 'usable audio resolution' means. Usable in processing? This seems to be where Warp69 is at with representing values in 32FLT. Usable at the converter? This seems to be Assaf's point in previous posts, that no converter will handle greater than 24bit output as a datastream (and if Bob Katz & others are to be listened to, thermal noisefloor on the converters renders them incapable of even achieving full 24bits of headroom).

And beyond that, define the scope of the argument or discussion being put forth, we've pretty much taken "summing" as a "scope" of discussion to the moon & beyond so far...

[voidar]

Just did an experiment here using REAPER and Flexors Absolute Val Monitor.

Trying both ASIO2 32 and FLT drivers, I pass an absolute value from Scope through REAPER and back to Scope into a different monitor.
I discovered that the FLT driver holds greater precision at lower values (follows same value up to a specific point) than the 32 driver does, which is basically a value or two off the whole range. Closest to VDAT.

This seems to be support what Warp69 is telling us.

According to the designer, REAPER takes any ASIO driver format, but internally converts it to a 64-bit float value.

Even VDAT, though the most accurate, has a slight offset it seems. Probably some gain-staging.

[ReD_MuZe]

try the same, but mix 0.5*max with vdat

[voidar]

try the same, but mix 0.5*max with vdat

I am not sure how you mean? And why?

In SDK I see there are some rounding errors. This you wanted to show me?

[Mike Goodwin]

Just did an experiment here using REAPER and Flexors Absolute Val Monitor.

Trying both ASIO2 32 and FLT drivers, I pass an absolute value from Scope through REAPER and back to Scope into a different monitor.
I discovered that the FLT driver holds greater precision at lower values (follows same value up to a specific point) than the 32 driver does, which is basically a value or two off the whole range. Closest to VDAT.

This seems to be support what Warp69 is telling us.

According to the designer, REAPER takes any ASIO driver format, but internally converts it to a 64-bit float value.

Even VDAT, though the most accurate, has a slight offset it seems. Probably some gain-staging.

Thanks for sharing that. That is helpful information. I will switch to the FLT ASIO driver for going back and forth to my Host, Ableton Live. Just for information sakes Live works with "32-bit floating point".

[ReD_MuZe]

simple.

becouse 24bits has slightly better error correction at the top end.

look at this pic:

32bits float are better for silence (the stuff you don't hear) and teh difference between 24bits and 32bits is just by one point.

when you go to the high range (the stuff you actualy hear) 24 bits is on the upper hand with a nicer rounding algorithm that makes rounding errors of + - 64 points while 32bit float reaches errors of 128points.

both have the same precision and different rounding algos. this is the experiment i did on reaper. comparing 32bit float drivers and 24bit integer drivers.

[ReD_MuZe]

and here is a graph roughly explaining why this happens:
not that the zoom and selection size on the float side is not correct but in order to it being correct i would have to select sub pixels. so it just gives the idea across.

[Mike Goodwin]

Warp thank you for this very detailed post! I cant say that I completely understand the math mind you. Would you be willing to put this into a non mathematically sentence or two for me?

Im not trying to ignore you, I just dont know what to write

You'll lose precision with conversion between 32bit float and 32bit int. Which format is the best? It depends

The human ear is more sensitive to changes in volume at low levels compared to high levels and that's exactly what floats represents - higher precision at low numbers compared to higher numbers. But math is not particually sensitive to any range, so when we use floats for coefficients in filters etc we have pretty good precision near zero, but much worse precision near 1 or above. And second, rounding errors and cancellations are inherent in float computations - so it's better to calculate (x+y)(x-y) instead of x^2 - y^2 in float.

Thanks for getting back to me on that. Just for the record I did not feel that you where ignoring me. It seemed that there was much more debate to come! I saw that this thread got called "thankless" in another place here in the forum and I have to say that I am glad for that. I am not trying to say that summing, or tiny losses due to conversion from floating point to integer maths is going to make or break a good song. This is not spirit of this thread when I started it. I just wanted to find out what the "correct" or "best" way to go about getting the most from my high end "in the box" studio. I love the fact that there is some serious math going on here. It is very informative even though I understand about 10% of it. Knowing weather or not it is best to use FLT ASIO drivers is important to me and has seemingly gone completely uncovered. It seems to me to be a very fundamental question that almost all users of the scope environment can benefit from, if even just by a few rounding errors. It is the point that there is a "more correct-more accurate" result that we can all get just by knowing more about what we own.

Thanks to all that have contributed so far!

Don't mean to sound like I own this thread, just trying to share some appreciation to everyones time.

[ReD_MuZe]

Knowing weather or not it is best to use FLT ASIO drivers is important to me and has seemingly gone completely uncovered. It seems to me to be a very fundamental question that almost all users of the scope environment can benefit from, if even just by a few rounding errors.

my experiment shows that both 24bits and 32bit float have the same resolution after conversion, but the rounding is different.
while you get more accuracy on the quiet parts of 32bit float, you get less accuracy on the loud parts. its a give and take situation.

i think you cant tell the difference anyhow, and it all gets converted to 64bit float inside the daw anyhow.

so to answer your question - if you are recording through converters - record at 24bits integer to keep exactly the same audio that got into the card. if you are recording a synth - record in 32bits float - to keep your sound as much in tact as you can (well it gets quantized from 64bits anyhow.) all the rest is irrelevant.