Re: symmetric vs. asymmetric float<->int conversion
<salsaman <at> xs4all.nl>
2009-12-10 00:47:22 GMT
On Thu, December 10, 2009 00:20, fons <at> kokkinizita.net wrote:
> On Wed, Dec 09, 2009 at 10:38:26PM +0000, Peter Nelson wrote:
>> On Wed, 2009-12-09 at 22:45 +0100, fons <at> kokkinizita.net wrote:
>> > 1. 2^N, e.g. 0x8000 for 16-bit
>> > 2. 2^N - 1, e.g. 0x7FFF for 16-bit
>> > (1) has the advantage that the representation is
>> > not really changed - for the int->float case it
>> > does not introduce any error at all.
>> > Its only 'drawback' is that the float value +1 can't
>> > be converted exactly to an int, and has to be clipped
>> > to e.g. 0x7FFF. So what ? Normal signals don't clip,
>> > and when they do the whole issue of accuracy becomes
>> > irrelevant anyway.
>> > (2) covers the full range of -1...+1 in float to
>> > int conversion, at the expense of introducing
>> > a mapping that is not exact. This will show up in
>> > the 24-bit cases, as floats have a resolution that
>> > is close to 24 bits.I see no reason why it should
>> > ever be used.
>> (2) is what JACK (at least in the ALSA backend) does already.
>> As far as I can see there is no inexact mapping in int->float->int
>> conversion except that an input value of -2^N will be artificially
>> clipped to -1.0, and -2^N will never be output.
> The full cycle int->float->int will be exact. The error
> introduced by (2) is that for the int->float conversion