How do I know the tests are being done properly?

[hovege]

Looking at the machine-generated-javascript code, it seems that it does invoke Math.random() and handles those numbers to the TestU01's interface unif01_CreateExternGen01. Chrome 87.0.4280.88 and Chromium 81.0.4044.138 both fail the RandomWalk test. Chrome 87.0.4280.88 runs Javascript V8 8.7.220.29 and the PRNG is apparently defined in

  https://chromium.googlesource.com/v8/v8.git/+/refs/tags/8.7.220.29/src/numbers/math-random.h
  https://chromium.googlesource.com/v8/v8.git/+/refs/tags/8.7.220.29/src/numbers/math-random.cc

The procedure RefillCache seems to generate the numbers using xorshift128:

// Create random numbers.
for (int i = 0; i < kCacheSize; i++) {
  // Generate random numbers using xorshift128+.
  base::RandomNumberGenerator::XorShift128(&state.s0, &state.s1);
  cache.set(i, base::RandomNumberGenerator::ToDouble(state.s0));
}

Here's Wikipedia's implementation of xorshift128 in C.

%cat xorshift128.c
#include <stdint.h>

struct xorshift128_state {
  uint32_t a, b, c, d;
};

/* The state array must be initialized to not be all zero */
uint32_t xorshift128(struct xorshift128_state state)
{
        / Algorithm "xor128" from p. 5 of Marsaglia, "Xorshift RNGs" */
        uint32_t t = state->d;

        uint32_t const s = state->a;
        state->d = state->c;
        state->c = state->b;
        state->b = s;

        t ^= t << 11;
        t ^= t >> 8;
        return state->a = t ^ s ^ (s >> 19);
}

#include <stdio.h>
int main() {
  struct xorshift128_state s;
  s.a = 0xdead; s.b = 0xbeef; s.c = 0x1a2b; s.d = 0x3c4d;
  uint32_t r;
  for (;;) {
    r = xorshift128(&s);
    fwrite(&r, sizeof r, 1, stdout);
  }
}

Running it against SmallCrush, it consistently fails MaxOft and not RandomWalk:

========= Summary results of SmallCrush =========
 Version:          TestU01 1.2.3
 Generator:        xorshift128
 Number of statistics:  15
 Total CPU time:   00:00:20.73
 The following tests gave p-values outside [0.001, 0.9990]:
 (eps  means a value < 1.0e-300):
 (eps1 means a value < 1.0e-15):
       Test                          p-value
 ----------------------------------------------
  6  MaxOft                           eps
 ----------------------------------------------
 All other tests were passed

I can't be sure the two generators are the same. I can't be sure the Javascript library is making sense here. I'd be surprised if this is a matter of seeds or something like that. What is your advice on this? I would like to make sure that your nice work does make sense. Thank you!