Improved Winamp Visualizer by @x2nie (#1260)

* Improved Winamp Visualizer by @x2nie

This contains work started by @x2nie and continued by me.

There's a lot that's either broken and/or unfinished.

The main goal of this PR is to improve the visualizer of Webamp, bringing improvements to the Oscilloscope and Spectrum analyzer rendering.

Improving the Oscilloscope was achieved by taking the rendering code from the Equalizer and implanting it into the small visualizer box done by x2nie, improving the Spectrum Analyzer involved ditching the Web Audio API in favor of FFTNullsoft, providing a much more accurate representation of the audio.
The spectrum analyzer code (at least for the thicker bands) was redone that makes use of reverse engineered code from Winamp 2.65, more specifically how the bar chunking is achieved as well as how the peaks are calculated (this is broken).

* Fixed analyzer responding to data immediately
Various code comments explaining the purpose behind the new implementation
Peaks are no longer visible at the bottom

* Added support for windowshade mode (colors are broken)
Replaced old Visualizer in the Playlist Editor in favor of the new Visualizer

* Fixed analyzer exceeding bounds in unwindowshaded mode

* Removed paintFrameThin() since it's now no longer needed (processing happens in paintFrameWide())
Also removed paintWavSolid() since that too is now processed in paintWavLine
Removed variables no longer in use
Adjusted how the FFT data is first processed in Vis.tsx and how that affects the thin and wide modes (they should now be consistent in volume)

* Add proper windowshade mode visualizer support (adapts correctly to doublesize too!)
Proper color handling for the Spectrum Analyzer if in windowshade and double size mode
Removed comemnts/functions no longer in use

* Visualizer is now pushed down by 2 pixels if not in double size mode
Fixed "doubled" not being able to be used outside of Vis.tsx
Set up base for eventual additional parameters that can be passed to the visualizer
Consolidate paintWavDot into paintWavLine
Remove dispose()

* Fixed accidentally setting oscStyle to "dots" for testing (oops)

* New (non-working) parameter: "sa", dictates vis mode
Allowed "mode" to be modifiable
Adjusted frequency scaling of the FFT

* Maybe fix deploy issues?

* Replace rangeByAmplitude with a colorIndex function
Attempt at addressing a few comments from the PR review

* Missed a few variables that weren't in camelCase
Finetuned the data going into timeToFrequencyDomain

* Move FFT stuff into VisPainter.ts
Attempt at addressing more of the PR review

* Moved the FFT to be part of BarPaintHandler, instead of being global
Added checking for the state of the Main Window so that the Playlist Editor visualizer no longer bugs out like it did (incorrectly showing the small visualizer, or showing the full capacity of it)
Changed the global variable `i` to be `chunk` to avoid potential issues
Ensure `y` is scaled down (in the y axis) correctly when in windowshade mode
Skip rendering of the Oscilloscope in the windowshade mode to avoid it drawing out of bounds
Missed a few variables that werent in camelCase (again)

* Missed implementing the solid mode drawing a pixel instead of a filled line if x is 0 and if visualizer is small

* Readded drawing the visualizer background
Prevent saPeaks from going "out of bounds" when the main window is in windowshade mode

* Missed accounting for the Playlist Editor Visualizer w.r.t to the background if Main Window was not visible, in windowshade mode and not in double size

* Addressing comments of the recent review
Fixes FFT being corrupted when multiple instances of Webamp exist and are playing at the same time
Fixes multiple Webamp instances fighting over what the current state of the Main Window really is
Moved a lot of global mutable variables to instead be owned by BarPaintHandler and PaintWavHandler
Renamed visualizer functions since they now handle a lot of things

* Make canvas required

* Ensure bars are split only in "wide" bandwidth

* Some small FFTNullsoft cleanup

* Call ``painter.prepare()`` only when doublesize is engaged/disengaged

* Confirmed order of Visualization modes
VisPaintHandler is an abstract class (is that how you do it?)
Instead of logging to console about paintBarLine reusing code from "Fire mode", it's been replaced with a code comment

* move ``processFFT()`` out of ``Painter.prepare()``
It never had a place there anyway

packages/webamp/js/components/FFTNullsoft.ts

@@ -0,0 +1,203 @@
+// The Web Audio API's FFT is bad, so this exists now!
+// Taken from https://github.com/WACUP/vis_classic/tree/master/FFTNullsoft
+
+export class FFT {
+  private bitrevtable: number[];
+  private envelope: Float32Array;
+  private equalize: Float32Array;
+  private temp1: Float32Array;
+  private temp2: Float32Array;
+  private cossintable: Float32Array[];
+
+  // Constants
+  private static readonly TWO_PI = 6.2831853; // 2 * Math.PI
+  private static readonly HALF_PI = 1.5707963268; // Math.PI / 2
+
+  constructor() {
+    // Assuming these are your hardcoded values:
+    const samplesIn = 1024; // hardcoded value
+    const samplesOut = 512; // hardcoded value
+    const bEqualize = true; // hardcoded value
+    const envelopePower = 1.0; // hardcoded value
+    const mode = false; // hardcoded value
+
+    const NFREQ = samplesOut * 2;
+
+    // Initialize the tables and arrays with hardcoded values
+    this.bitrevtable = this.initBitRevTable(NFREQ);
+    this.cossintable = this.initCosSinTable(NFREQ);
+
+    this.envelope = this.initEnvelopeTable(samplesIn, envelopePower);
+    this.equalize = this.initEqualizeTable(NFREQ, mode);
+
+    this.temp1 = new Float32Array(NFREQ);
+    this.temp2 = new Float32Array(NFREQ);
+  }
+
+  private initEqualizeTable(NFREQ: number, mode: boolean): Float32Array {
+    const equalize = new Float32Array(NFREQ / 2);
+    let bias = 0.04; // FFT.INITIAL_BIAS
+
+    for (let i = 0; i < NFREQ / 2; i++) {
+      const inv_half_nfreq = (9.0 - bias) / (NFREQ / 2);
+      equalize[i] = Math.log10(1.0 + bias + (i + 1) * inv_half_nfreq);
+      bias /= 1.0025; // FFT.BIAS_DECAY_RATE
+    }
+
+    return equalize;
+  }
+
+  private initEnvelopeTable(samplesIn: number, power: number): Float32Array {
+    const mult = (1.0 / samplesIn) * FFT.TWO_PI;
+    const envelope = new Float32Array(samplesIn);
+
+    for (let i = 0; i < samplesIn; i++) {
+      envelope[i] = Math.pow(
+        0.5 + 0.5 * Math.sin(i * mult - FFT.HALF_PI),
+        power
+      );
+    }
+
+    return envelope;
+  }
+
+  private initBitRevTable(NFREQ: number): number[] {
+    const bitrevtable = new Array(NFREQ);
+
+    for (let i = 0; i < NFREQ; i++) {
+      bitrevtable[i] = i;
+    }
+
+    for (let i = 0, j = 0; i < NFREQ; i++) {
+      if (j > i) {
+        const temp = bitrevtable[i];
+        bitrevtable[i] = bitrevtable[j];
+        bitrevtable[j] = temp;
+      }
+
+      let m = NFREQ >> 1;
+      while (m >= 1 && j >= m) {
+        j -= m;
+        m >>= 1;
+      }
+
+      j += m;
+    }
+
+    return bitrevtable;
+  }
+
+  private initCosSinTable(NFREQ: number): Float32Array[] {
+    const cossintable: Float32Array[] = [];
+    let dftsize = 2;
+
+    while (dftsize <= NFREQ) {
+      const theta = (-2.0 * Math.PI) / dftsize;
+      cossintable.push(new Float32Array([Math.cos(theta), Math.sin(theta)]));
+      dftsize <<= 1;
+    }
+
+    return cossintable;
+  }
+
+  public timeToFrequencyDomain(
+    inWavedata: Float32Array,
+    outSpectraldata: Float32Array
+  ): void {
+    if (!this.temp1 || !this.temp2 || !this.cossintable) return;
+    // Converts time-domain samples from inWavedata[]
+    //   into frequency-domain samples in outSpectraldata[].
+    // The array lengths are the two parameters to Init().
+
+    // The last sample of the output data will represent the frequency
+    //   that is 1/4th of the input sampling rate.  For example,
+    //   if the input wave data is sampled at 44,100 Hz, then the last
+    //   sample of the spectral data output will represent the frequency
+    //   11,025 Hz.  The first sample will be 0 Hz; the frequencies of
+    //   the rest of the samples vary linearly in between.
+    // Note that since human hearing is limited to the range 200 - 20,000
+    //   Hz.  200 is a low bass hum; 20,000 is an ear-piercing high shriek.
+    // Each time the frequency doubles, that sounds like going up an octave.
+    //   That means that the difference between 200 and 300 Hz is FAR more
+    //   than the difference between 5000 and 5100, for example!
+    // So, when trying to analyze bass, you'll want to look at (probably)
+    //   the 200-800 Hz range; whereas for treble, you'll want the 1,400 -
+    //   11,025 Hz range.
+    // If you want to get 3 bands, try it this way:
+    //   a) 11,025 / 200 = 55.125
+    //   b) to get the number of octaves between 200 and 11,025 Hz, solve for n:
+    //          2^n = 55.125
+    //          n = log 55.125 / log 2
+    //          n = 5.785
+    //   c) so each band should represent 5.785/3 = 1.928 octaves; the ranges are:
+    //          1) 200 - 200*2^1.928                    or  200  - 761   Hz
+    //          2) 200*2^1.928 - 200*2^(1.928*2)        or  761  - 2897  Hz
+    //          3) 200*2^(1.928*2) - 200*2^(1.928*3)    or  2897 - 11025 Hz
+
+    // A simple sine-wave-based envelope is convolved with the waveform
+    //   data before doing the FFT, to emeliorate the bad frequency response
+    //   of a square (i.e. nonexistent) filter.
+
+    // You might want to slightly damp (blur) the input if your signal isn't
+    //   of a very high quality, to reduce high-frequency noise that would
+    //   otherwise show up in the output.
+
+    // code should be smart enough to call Init before this function
+    //if (!bitrevtable) return;
+    //if (!temp1) return;
+    //if (!temp2) return;
+    //if (!cossintable) return;
+
+    // 1. set up input to the fft
+    for (let i = 0; i < this.temp1.length; i++) {
+      const idx = this.bitrevtable[i];
+      if (idx < inWavedata.length) {
+        this.temp1[i] =
+          inWavedata[idx] * (this.envelope ? this.envelope[idx] : 1);
+      } else {
+        this.temp1[i] = 0;
+      }
+    }
+    this.temp2.fill(0);
+
+    // 2. Perform FFT
+    let real = this.temp1;
+    let imag = this.temp2;
+    let dftsize = 2;
+    let t = 0;
+
+    while (dftsize <= this.temp1.length) {
+      const wpr = this.cossintable[t][0];
+      const wpi = this.cossintable[t][1];
+      let wr = 1.0;
+      let wi = 0.0;
+      const hdftsize = dftsize >> 1;
+
+      for (let m = 0; m < hdftsize; m += 1) {
+        for (let i = m; i < this.temp1.length; i += dftsize) {
+          const j = i + hdftsize;
+          const tempr = wr * real[j] - wi * imag[j];
+          const tempi = wr * imag[j] + wi * real[j];
+          real[j] = real[i] - tempr;
+          imag[j] = imag[i] - tempi;
+          real[i] += tempr;
+          imag[i] += tempi;
+        }
+
+        const wtemp = wr;
+        wr = wr * wpr - wi * wpi;
+        wi = wi * wpr + wtemp * wpi;
+      }
+
+      dftsize <<= 1;
+      ++t;
+    }
+
+    // 3. take the magnitude & equalize it (on a log10 scale) for output
+    for (let i = 0; i < outSpectraldata.length; i++) {
+      outSpectraldata[i] =
+        Math.sqrt(real[i] * real[i] + imag[i] * imag[i]) *
+        (this.equalize ? this.equalize[i] : 1);
+    }
+  }
+}

packages/webamp/js/components/MainWindow/index.tsx

@@ -8,7 +8,7 @@ import MiniTime from "../MiniTime";
 
 import ClickedDiv from "../ClickedDiv";
 import ContextMenuTarget from "../ContextMenuTarget";
-import Visualizer from "../Visualizer";
+import Vis from "../Vis";
 import ActionButtons from "./ActionButtons";
 import MainBalance from "./MainBalance";
 import Close from "./Close";
@@ -106,10 +106,7 @@ const MainWindow = React.memo(({ analyser, filePickers }: Props) => {
           />
           <Time />
         </div>
-        <Visualizer
-          // @ts-ignore Visualizer is not typed yet
-          analyser={analyser}
-        />
+        <Vis analyser={analyser} />
         <div className="media-info">
           <Marquee />
           <Kbps />

packages/webamp/js/components/PlaylistWindow/index.tsx

@@ -7,7 +7,7 @@ import * as Selectors from "../../selectors";
 
 import { clamp } from "../../utils";
 import DropTarget from "../DropTarget";
-import Visualizer from "../Visualizer";
+import Vis from "../Vis";
 import PlaylistShade from "./PlaylistShade";
 import AddMenu from "./AddMenu";
 import RemoveMenu from "./RemoveMenu";
@@ -140,10 +140,7 @@ function PlaylistWindow({ analyser }: Props) {
               <div className="playlist-visualizer">
                 {activateVisualizer && (
                   <div className="visualizer-wrapper">
-                    <Visualizer
-                      // @ts-ignore Visualizer is not yet typed
-                      analyser={analyser}
-                    />
+                    <Vis analyser={analyser} />
                   </div>
                 )}
               </div>