The Noise That Makes It Accurate

Round a quiet sound to too few levels and the error is not noise but a staircase that rings as harmonics. Add the right noise before rounding and the average output becomes exactly the input: rectangular noise one step wide straightens the staircase, and triangular noise two steps wide also holds the hiss steady, at a cost of 4.77 dB. The proof assumes a quantiser that never clips, which is exactly what a 1-bit loop cannot promise.

Every digital audio system puts each sample on one of a fixed set of levels: once when a converter first measures the sound, and again whenever software shortens the numbers. Call the gap between two levels one step (engineers write it Δ and call it the least significant bit, LSB). Below, the steps are made enormous on purpose, so that a tone only a step or so high shows you what rounding does to it.

What the curves say

The lines are exact: for each input the page adds up every value the noise can take, weighted by how likely it is. The dots are measured: 400 random draws of noise at each input, drawn again when you press the button. Where they agree, the arithmetic is not hiding anything.

No noise. The average output is the staircase itself, up to half a step away from the input, and the error is a fixed function of the signal. A fixed error that comes back with every cycle of the tone is distortion. At a height of 1.4 steps the rounded tone's third harmonic is only 15.7 dB below the tone. Below half a step the tone is not distorted, it is gone: every sample rounds to zero.

Rectangular noise, one step wide. Each sample now lands on the level above or the level below, with odds set by where the input sits between them, and on average it lands exactly on the input. Stanley Lipshitz, Robert Wannamaker and John Vanderkooy, in their 1992 survey of the theory, put it this way: The first moment, or mean error, is zero for all inputs, indicating that the quantizer has been linearized by the use of this dither. The first chart is straight. The second is not: The error variance, on the other hand, is clearly signal dependent, so that the noise power in the signal varies with the system input. This is sometimes referred to as noise modulation and is audibly undesirable. The error power runs from 0 when the input sits on a level to Δ²/4 halfway between levels; averaged over every position between two levels it is Δ²/6, which is 3.01 dB above Δ²/12.

Triangular noise, two steps wide. Add two independent rectangular sources and the second chart goes flat: The second moment of the total error is a constant Δ²/4 for all inputs. That is three times Δ²/12, the error power of the textbook model of rounding, so the price is 10·log10(3) = 4.77 dB more noise; in the survey's own simulated spectrum the noise floor is up by 4.8 dB relative to that of Fig. 6(d) due to the tripling of the noise spectral density. The general rule is their Theorem 8, and the triangular width is the cheapest that works: triangular-pdf dither of 2-LSB peak-to-peak amplitude is unique and optimal in the sense that it renders the first and second moments of the total error input independent, while minimizing the second moment.

Theorem 8 A nonsubtractive dither signal generated by the summation of n statistically independent zero-mean rectangular-pdf random processes, each of 1-LSB peak-to-peak amplitude, renders the first n moments of the total error independent of the system input, and results, for n ≥ 2, in a total error power of (n + 1)Δ²/12.Lipshitz, Wannamaker and Vanderkooy, J. Audio Eng. Soc. 40 (1992), p. 369

Not independent. What the noise does not do is make the error independent of the music: Nonsubtractive dithering, unlike subtractive dithering, cannot render the total error independent of the system input. Switch the second chart to the third moment. With triangular noise it still waves with the input, by up to 0.048 Δ³ (with rectangular noise, up to 0.096 Δ³). The survey's authors wrote that Variations in higher moments are believed to be inaudible, and this has been corroborated by a large number of psychoacoustic tests conducted by the authors and others; of those listening tests they added, Admittedly, these tests were informal.

The widths are not a matter of taste. The survey again: using rectangular-pdf noises of peak-to-peak amplitude not equal to 1 LSB (or, rather, not equal to an integral number of LSB) will not render error moments independent of the input. Choose rectangular noise and set the width control to 0.80: the straight line bends again, by up to 0.10 step (with triangular noise built from two 0.80-step sources, by up to 0.020).

Where it matters. The first rounding, in a converter measuring a real sound, often needs no noise added on purpose, because many analog signals and digital-to-analog conversion systems exhibit a Gaussian noise component which is of large enough amplitude to act as a satisfactory inherent dither. For shortening numbers inside the digital chain, the survey notes that its recommended triangular dither is easy to generate for digital requantization purposes by simply summing two independent rectangular-pdf pseudo-random processes; its listening tests used signals, music among them, requantized very coarsely (to 8 bits from 16).

The condition. Every result above is proved for quantisers the survey calls infinite, and its first footnote says what that means: For practical purposes, this simply means that the signal is never clipped by saturation of the quantizer. Keep that sentence. The rest of this page is about a quantiser that cannot keep it.

Deeper

The loop that cannot take the whole cure

The objection: this is a fix from the age of the compact disc. Modern converters run far faster than they need to and push their rounding error up out of hearing, and the streams that really are new, in a phone's microphone or on a Super Audio CD, are 1 bit anyway.

Pushing the error out of hearing is real, and it is called noise shaping. Feed the rounding error E back through a filter H and subtract it from the next input, and the output is Y = X + (1 − H)E: the input passes untouched and the error is reshaped by 1 − H (Lipshitz and Vanderkooy, 2001). What shaping cannot do is remove error. A theorem of Gerzon and Craven, as Lipshitz and Vanderkooy state it, says that the areas of any optimal noise shaper curve above and below the unshaped PSD must be equal on a decibel scale against linear frequency: the sampled-signal cousin of the conservation law in The Error You Can Only Move. For the published shaper below, the average of 20·log10|1 − H| over the whole band, from 0 to 1.4112 MHz, computes to 0.00 dB.

Computing…

Shaping leaves the dither theorems' condition in place: the quantiser inside the loop must never clip. A quantiser with many levels keeps that condition easily: in a straight 16-bit system, the dither occupies only two out of the 65,536 levels available. A 1-bit quantiser has two output levels, +Δ/2 and −Δ/2, and behaves like a many-level one only while its input W stays within −Δ ≤ W < Δ. Triangular dither on its own already reaches almost ±Δ. Lipshitz and Vanderkooy:

We claim that a 1-bit sigma-delta converter must overload when properly dithered. This follows at once, since the TPDF dither N, which is needed to fully linearize the quantizer, itself swings the quantizer’s input W over its full no-overload range of ±∆.Lipshitz and Vanderkooy, AES Convention Paper 5395 (2001), p. 3

The argument fits in a line. W is the input, minus the fed-back error, plus the dither; the dither alone can come as close to ±Δ as you like; so the moment anything else in W is not zero, some sample lands outside. Their conclusion: full TPDF dither is never allowable in a 1-bit noise shaper or sigma-delta modulator, and hence full linearity is never achievable either in principle or, of course, in practice.

The safe region is a guarantee, not a cliff edge. At exactly zero input the loop holds out for any dither smaller than ±Δ/2, because its error can then only be −Δ/2, 0 or +Δ/2, which keeps W inside ±Δ. Line (d) of the appendix says that above Δ/4 overload is guaranteed to occur (even with no input) at some step n; at zero input this page cannot make that line true (press the zero-input button: µ = 0.30, no overload, and none is possible). Their summary restates it in general, that such a loop can operate without overload only if µ < ∆/4, but names case (d) only at µ = Δ, full triangular dither, where it does hold (below), and the argument on p. 3 does not need it. Outside the region overloads become possible, not certain: the region holds for every dither sequence, and a random one is often kinder (input 0.10 with µ = 0.25, for one, runs 200,000 samples clean). With full triangular dither even zero input fails: 93,470 of a million samples overload (seed 1), and given as many levels as it likes, the loop then uses 4.

The readout also shows the other fact about a 1-bit stream. Every sample is +Δ/2 or −Δ/2, so the stream's power is Δ²/4 whatever it carries. Since the output power is constant, the signal component of the output must come at the expense of the remainder. Raise the input x and the power left for everything that is not the signal falls by exactly x². So the noise must move with the signal; the best hope is that it moves only above the audio band, and of that hope they wrote, This cannot, however, be guaranteed.

What the published loop does

No dither. In the absence of dither, and with zero initial conditions, Lip7ZP displays a low-level limit-cycle oscillation of period 328 samples (and a very audible fundamental frequency of 8.605 kHz). Recomputed: after its first 13,904 samples the output repeats exactly every 328 samples (they wrote that the cycle takes almost 16,000 samples before it is firmly established), and 2,822,400/328 = 8604.878 Hz. Averaged over whole periods, that fundamental sits 151.3 dB below a DSD full-scale sine, whose peak the paper puts at Δ/4. Its second harmonic, at 17,209.756 Hz, is 130.2 dB below; these are the only two lines of the cycle in the audio band. The strongest line, at 1.0584 MHz, is 1.7 dB above full scale. So this page shows the cycle and does not play it.

A little dither. They found that 0.0013 step, peak to peak, of rectangular dither was the minimum necessary to break up the limit cycle, an amount of dither 60.7 dB less than full TPDF dither; recomputed, 10·log10((Δ²/6)/(0.0013²Δ²/12)) = 60.73 dB. In all ten of this page's runs, with that much dither on from the first sample, the loop never settled into the 328-sample cycle. Switched on at sample 20,000, once the cycle has settled, the same dither never changes a bit of it: the settled cycle keeps the signal reaching the quantiser at least 0.00087 step from its decision point, and this dither reaches only 0.00065 step either way. Nor is it a cure. In 5 of 10 runs (seeds 1 to 10, a million samples each, the bench's length when it averages 1,600 periods or fewer) the loop fell into a different exact cycle, one the dither was too small ever to disturb: 168, 120, 56, 48 and 56 samples long (seeds 3, 4, 5, 7 and 10), which is 16,800, 23,520, 50,400, 58,800 and 50,400 Hz. In each, the signal reaching the quantiser before the dither is added stays at least 0.0043 step from its decision point, more than six times the dither's reach of 0.00065 step, so no draw of the dither can change a single output bit again: a seed that locks stays locked, and the longer the run, the more seeds can lock. Seed 3 locks after 24,868 samples and was still locked when we stopped, three million samples in; seeds 4 and 10 lock only after 600,000 samples. Within a million samples, with 0.005 step 5 of 20 seeds locked; with 0.01, 5 of 20; with 0.02, 0.05, 0.1 and 0.17, none of 20. Which cycle a seed falls into depends on the arithmetic as well as the dither: with the filter's whole chain reversed (the gain and the pole first, then the three sections from last to first), seed 10 locks into a 48-sample cycle instead, from sample 756,736.

Full dither, as many levels as needed. Now the dither theorems apply, and the loop needs room: it occasionally needs up to ±4 levels; that is, a 3-bit quantizer is needed to avoid overload. In 40 runs of three million samples (seeds 1 to 40), six levels were enough in 39; seed 22 needed eight, once, at sample 215,551.

Full dither, forced into 1 bit. Their verdict was that the system is unstable. In seeds 1 to 8 the loop's internal signal passed a million steps between samples 453 and 650.

Find the edge. Scale the triangular dither down until the 1-bit loop copes (silence in, a million samples, seeds 1 to 8). At 0.25 of full amplitude no seed overloads; at 0.3, 0 to 3 overloads; at 0.35, 76 to 96; at 0.4 every run runs away. A quarter of the amplitude is 12.04 dB less dither power, and partial dither leaves the theorems' promises unkept. With two half-full-scale tones in, they found we can inject only 0.17 LSBs peak-to-peak of RPDF dither before additional levels become necessary, which is 1/69th of the power of full TPDF dither (0.17²/12 against 1/6 is 1 to 69.2). This page's runs of the same two tones, at 5512.5 Hz and 8268.75 Hz (seeds 1 to 8, a million samples each), agree: at 0.17 step no seed overloads; at 0.2, one seed overloads, twice; at 0.25, four seeds overload and a fifth runs away.

A cycle belongs to a realisation. The filter here is built as a chain of the printed factors, and all 144 orderings of those factors that we tried give the same 328-sample cycle from the same sample. The same H written as one seventh-order difference equation rounds differently, and in 300,000 samples it shows no exact cycle shorter than 20,000 samples. An undithered loop's cycles are a fact about the arithmetic, not only about H.

Recomputed from the printed coefficients, the noise transfer function 1 − H has its zeros where the paper says, one real zero at dc, and three complex conjugate pairs of zeros just inside the unit circle, corresponding to frequencies of 4, 12, and 20 kHz: at 4.000, 12.000 and 20.000 kHz, with radii 0.999, 0.998 and 0.995, and the real zero within 10⁻¹³ of z = 1 (checked in exact rational arithmetic, because ordinary floating point cannot place a zero so close to its neighbours that finely). All seven poles lie inside the unit circle, the largest at radius 0.967.

Where the 1-bit stream actually lives

None of this says the converter in your phone or laptop is 1-bit. The converter chips in the data sheets checked for this page are delta-sigma designs with more than two levels. Cirrus Logic's CS4344 digital-to-analog converter (data sheet dated 2013) is based on a fourth-order multibit delta-sigma modulator. Texas Instruments' PCM4222 (October 2006, revised March 2007) is a Multi-Bit Delta-Sigma Analog-to-Digital Converter offering 6-bit modulator data among its outputs. In TI's PCM510xA converters (May 2012, revised May 2015) the data are modulated all the way to 128 × f_S before arriving at the current segments. These are examples, not a census. In 2001 Lipshitz and Vanderkooy, reading the data sheets of the major high-quality converter makers, found they had almost universally given up on the 1-bit sigma-delta topology in favor of oversampling converters using more than two levels.

The genuine 1-bit stream is closer to you than the converter. Knowles' SPH0641LU4H-1 is a silicon digital microphone with a single bit PDM output, built around a sigma-delta modulator and offered for cellphones, smart phones (data sheet revision B, 2015). InvenSense's ICS-41350 (product brief, 2016) follows its MEMS element and amplifier with a fourth‐order Σ‐Δ modulator and a pulse density modulated (PDM) interface, and lists smartphones first among its applications. Wikipedia's article on the format says: A PDM bitstream is encoded from an analog signal through the process of a 1-bit delta-sigma modulation. Which phones use which microphone this page has not sourced. The other 1-bit stream is DSD itself, the format of the Super Audio CD: the same PCM4222 offers a 1-bit Direct Stream Digital (DSD) output, at 2.8224 MHz in its 64x mode.

Can anyone hear it? That is a different question, and an open one. The overload argument is arithmetic, and the instruments above check it. Whether the leftovers of a 1-bit stream are audible is disputed. Lipshitz and Vanderkooy themselves wrote that the distortion products they had shown inside the audio band might be perceptible, and added: Further research is needed here. Wikipedia's article on DSD records that Comparisons of DSD and PCM recordings with the same origin, number of channels and similar bandwidth and noise have yielded contradictory results. On Lipshitz and Vanderkooy's wider verdict, that one-bit converters are unsuitable for high-end applications, the article adds: In 2002, Philips published a paper arguing the contrary. Its next two sentences read: Lipshitz and Vanderkooy's paper was further criticized by Angus. Lipshitz and Vanderkooy responded to the criticisms. The Audio Engineering Society's library returned 403 (forbidden) to our requests for those papers, so this page does not summarise them. And the dither theory itself does not reach inside such loops: Wannamaker's 1997 thesis closes by noting that a complete statistical description of non-linear systems with feedback, such as sigma-delta converters, awaits development.

Related layers: The Grey That Isn't There does the same job for pictures with deterministic patterns of dots instead of random noise, and The Half of the Noise It Can Erase is another audio page about a limit set by arithmetic and physics: anti-noise that erases only the low, steady part of a sound.

curl -sO https://artwaste.land/checks/verify-the-noise-that-makes-it-accurate.mjs
node verify-the-noise-that-makes-it-accurate.mjs