Strange results PCMCI better than PCMCI+, SVARRFCI , meaning of knn smaller is better results

[dynamite9999]

Hello,

Sorry for posting this also on issues, no one responded, so hoping this discussion group is more appropriate ?

I am getting surprisingly better accuracy results with PCMCI than all others. Which is surprising, since the new algorithms are supposed to be better. I define accuracy as the number of correct estimated directed edges/ true correct directed edges per node.

SETUP:
tigramite - 4.2.2.1
I am varying knn, which makes a huge difference[.05, .2, .4] and cannot figure out why having a smaller hypercube would yield better results. I am only using CMIknn[varying knn as noted above] and keeping tau_max=[4,6], pc_alpha:0.2 alpha_level:0.05 fixed_thresh: 0.8.

QUESTIONS:

why does a smaller knn for CMIknn at .05 yield better results than larger values ? I must be missing something. If anything, making the hypercube larger, with more data would yield higher power due to larger number of samples ? Please help me clarify?
I have tried fixed_thresh of 10, 20 and it ran much faster, which no real changes, however the recommended values are are at .2. I am getting better accuracy at .8. I am not really understanding why ? Please help me understand what is this parameter does and how such large values are faster ? Is there caching effect perhaps ?
I have tested with all kinds of data [non, ]distributions stationary, gaussian, linearand thought to get better results with pcmci+ and svarrfci as per paper, but I am seeing opposite. Any fixes that I am missing perhaps since 4.2.2.1 ?
Any clarifications would be very much appreciated.

Cheers,

[dynamite9999]

Hello Sagar Nagaraj Simha and Jakob,

Tigramite 4.2.2

I am getting consistently better results with PCMCI than others, even with different params and data distributions. Is there some additional processing you added in PCMCI ? That is the only explanation I can think of. My results do not match your papers results where at least PCMCI+ should be better. I am measuring accuracy by number of correct directed edges/ ( true + incorrect predicted) so this should capture both missing and incorrectly predicted edges.

Kindly advise.

[jakobrunge]

These algorithms make different assumptions about the underlying data, this is why you cannot compare them as such.