headers
Paul Davis | 9 Dec 2009 20:16

symmetric vs. asymmetric float<->int conversion

at some point about 18 months ago, we switched to symmetrical
conversion for float->int in the ALSA backend of JACK. now comes the
2nd of 2 blogs by a guy who can at least make it sound as though he
knows what he is talking about.

http://blog.bjornroche.com/2009/12/linearity-and-dynamic-range-in-int.html

(link to the first post is in there)

your comments are welcome and sought after.

--p
Permalink | Reply |
headers
fons | 9 Dec 2009 22:45

Re: symmetric vs. asymmetric float<->int conversion

On Wed, Dec 09, 2009 at 02:16:33PM -0500, Paul Davis wrote:

> at some point about 18 months ago, we switched to symmetrical
> conversion for float->int in the ALSA backend of JACK. now comes the
> 2nd of 2 blogs by a guy who can at least make it sound as though he
> knows what he is talking about.
> 
> http://blog.bjornroche.com/2009/12/linearity-and-dynamic-range-in-int.html
> 
> (link to the first post is in there)
> 
> your comments are welcome and sought after.

If the author of this blog would care to measure any real
ADC or DAC - including the most expensive and esoteric ones
he can find - he would discover that its performance in
terms of noise and accuracy will swamp anything shown in
his graphs by a factor of around 30 dB. Which means that
these results are completely irrelevant in practice. This
is not going to change any time soon, as the limits of
performance of today's best converters are dictated by the
laws of physics rather than by technology.

The question he seems to ask himself seem to boil down
to the choice of the constant used when convertion between
int and float representations. There are basically two 
choices:

1. 2^N, e.g. 0x8000 for 16-bit
2. 2^N - 1, e.g. 0x7FFF for 16-bit
(Continue reading)

Permalink | Reply |
headers
Peter Nelson | 9 Dec 2009 23:38
Gravatar

Re: symmetric vs. asymmetric float<->int conversion

On Wed, 2009-12-09 at 22:45 +0100, fons <at> kokkinizita.net wrote:
> 1. 2^N, e.g. 0x8000 for 16-bit
> 2. 2^N - 1, e.g. 0x7FFF for 16-bit
> 
> (1) has the advantage that the representation is
> not really changed - for the int->float case it
> does not introduce any error at all.
> Its only 'drawback' is that the float value +1 can't
> be converted exactly to an int, and has to be clipped
> to e.g. 0x7FFF. So what ? Normal signals don't clip,
> and when they do the whole issue of accuracy becomes
> irrelevant anyway.
> 
> (2) covers the full range of -1...+1 in float to
> int conversion, at the expense of introducing 
> a mapping that is not exact. This will show up in
> the 24-bit cases, as floats have a resolution that
> is close to 24 bits.I see no reason why it should
> ever be used. 

(2) is what JACK (at least in the ALSA backend) does already.

As far as I can see there is no inexact mapping in int->float->int
conversion except that an input value of -2^N will be artificially
clipped to -1.0, and -2^N will never be output.
Permalink | Reply |
headers
fons | 10 Dec 2009 00:20

Re: symmetric vs. asymmetric float<->int conversion

On Wed, Dec 09, 2009 at 10:38:26PM +0000, Peter Nelson wrote:
> On Wed, 2009-12-09 at 22:45 +0100, fons <at> kokkinizita.net wrote:
> > 1. 2^N, e.g. 0x8000 for 16-bit
> > 2. 2^N - 1, e.g. 0x7FFF for 16-bit
> > 
> > (1) has the advantage that the representation is
> > not really changed - for the int->float case it
> > does not introduce any error at all.
> > Its only 'drawback' is that the float value +1 can't
> > be converted exactly to an int, and has to be clipped
> > to e.g. 0x7FFF. So what ? Normal signals don't clip,
> > and when they do the whole issue of accuracy becomes
> > irrelevant anyway.
> > 
> > (2) covers the full range of -1...+1 in float to
> > int conversion, at the expense of introducing 
> > a mapping that is not exact. This will show up in
> > the 24-bit cases, as floats have a resolution that
> > is close to 24 bits.I see no reason why it should
> > ever be used. 
> 
> (2) is what JACK (at least in the ALSA backend) does already.
> 
> As far as I can see there is no inexact mapping in int->float->int
> conversion except that an input value of -2^N will be artificially
> clipped to -1.0, and -2^N will never be output.

The full cycle int->float->int will be exact. The error
introduced by (2) is that for the int->float conversion
the float value can't be represented exactly, and when
(Continue reading)

Permalink | Reply |
headers
salsaman | 10 Dec 2009 01:47
Picon
Picon
Favicon

Re: symmetric vs. asymmetric float<->int conversion

On Thu, December 10, 2009 00:20, fons <at> kokkinizita.net wrote:
> On Wed, Dec 09, 2009 at 10:38:26PM +0000, Peter Nelson wrote:
>> On Wed, 2009-12-09 at 22:45 +0100, fons <at> kokkinizita.net wrote:
>> > 1. 2^N, e.g. 0x8000 for 16-bit
>> > 2. 2^N - 1, e.g. 0x7FFF for 16-bit
>> >
>> > (1) has the advantage that the representation is
>> > not really changed - for the int->float case it
>> > does not introduce any error at all.
>> > Its only 'drawback' is that the float value +1 can't
>> > be converted exactly to an int, and has to be clipped
>> > to e.g. 0x7FFF. So what ? Normal signals don't clip,
>> > and when they do the whole issue of accuracy becomes
>> > irrelevant anyway.
>> >
>> > (2) covers the full range of -1...+1 in float to
>> > int conversion, at the expense of introducing
>> > a mapping that is not exact. This will show up in
>> > the 24-bit cases, as floats have a resolution that
>> > is close to 24 bits.I see no reason why it should
>> > ever be used.
>>
>> (2) is what JACK (at least in the ALSA backend) does already.
>>
>> As far as I can see there is no inexact mapping in int->float->int
>> conversion except that an input value of -2^N will be artificially
>> clipped to -1.0, and -2^N will never be output.
>
> The full cycle int->float->int will be exact. The error
> introduced by (2) is that for the int->float conversion
(Continue reading)

Permalink | Reply |
headers
Peter Nelson | 10 Dec 2009 12:58
Gravatar

Re: symmetric vs. asymmetric float<->int conversion

On Thu, 2009-12-10 at 00:20 +0100, fons <at> kokkinizita.net wrote:
> The full cycle int->float->int will be exact. The error
> introduced by (2) is that for the int->float conversion
> the float value can't be represented exactly, and when
> the rounded value is multiplied by the conversion factor
> (e.g. 0x7FFF) it will not be equal to the original integer.

Ah, yes indeed. The difference is visible when dividing using double
instead of float.
Permalink | Reply |
headers
Jussi Laako | 9 Dec 2009 23:22

Re: symmetric vs. asymmetric float<->int conversion

On 12/09/2009 09:16 PM, Paul Davis wrote:
> at some point about 18 months ago, we switched to symmetrical
> conversion for float->int in the ALSA backend of JACK. now comes the
> 2nd of 2 blogs by a guy who can at least make it sound as though he
> knows what he is talking about.

I just checked once again results of my own resampler code, and with my
implementation of method(2) from double with TPDF dither there's nothing
else except fundamental frequency of a logarithmic sweep visible above
dither noise which lies around -180 dBFS (peak, 16k FFT) on the
spectrum. From 20 Hz to 20 kHz when converted 44.1 kHz -> 88.2 kHz.

In any case, practically anything below -150 dBFS on the spectrum
doesn't matter since that's the performance of the best DAC I've seen.
On more practical DAC hardware, anything below -130 dBFS doesn't matter.
Generally DAC chips tend to spoil these things way before...
Permalink | Reply |
headers
Jussi Laako | 9 Dec 2009 23:23

Re: symmetric vs. asymmetric float<->int conversion

On 12/09/2009 09:16 PM, Paul Davis wrote:
> http://blog.bjornroche.com/2009/12/linearity-and-dynamic-range-in-int.html

And there are three conversion possibilities (that I know of) instead of
two claimed...
Permalink | Reply |

Gmane