Adjust degree of freedom for each permutation?

[behinger]

"As the (approximate) degrees of freedom are based on the samples: the same statistics with the same design may produce different p-values (using methods like Satterthwaite). So maybe the scale of the statistics is not the appropriate scale 1) through the permutation and also 2) thought the DV. So we try to transform each statistics into the "parametric p-values scales", using degrees of freedom computed at each permutation to compute the permuted p-values."

Related: We likely want to compare power of brain-permutation to FDR or similar, thus we need to discuss what we want to use there. I am guessing, Phillip will argue that if you have enough data, DF will be high enough to use z-approximation.

[OlivierRenaud]

In case we decide to implement modified degrees of freedom, the reference I was mentionning is

Kuznetsova A, Brockhoff PB, Christensen RH (2017) lmerTest Package: tests in linear mixed effects models. Journal of Statistical Software 82:1–26. https://doi.org/10.18637/jss.v082.i13

which show that Satterthwaite is way faster than KR (see Figure 3) and which gives the ingredients for the computation of the df in appendix A. I did not check the details of the implementations, but Tom Maullin-Sapey mentionned to me that some steps are not that self-explanatory and his explanations in his very recent manuscript:

Maullin-Sapey T, Nichols TE (2021) Fisher Scoring for crossed factor Linear Mixed Models. http://arxiv.org/abs/2102.05103

section 2.6, might contain more specific details. I can ask him more details if needed.

[palday]

There's a derivative computation in there. I need to look at exactly where it's happening, but this might be when we bring in some of the packages for automatic differentiation in in Julia.