Back to the main page.

Bug 3094 - point for discussion: warning about p-value confidence intervals in ft_statistics_montecarlo

Reported 2016-03-10 14:27:00 +0100
Modified 2016-05-05 20:28:42 +0200
Product: FieldTrip
Component: core
Version: unspecified
Hardware: PC
Operating System: Mac OS
Importance: P5 normal
Assigned to: Jan-Mathijs Schoffelen
Depends on:
See also:

Jan-Mathijs Schoffelen - 2016-03-10 14:27:12 +0100

running some statistics I came across the warning: "the p-value confidence interval of positive cluster #13 includes 0.050 - consider increasing the number of permutations" Considering the fact that we should base our inferential decision on the position of the largest observed cluster within the randomization distribution, it is a bit silly to make this recommendation for any cluster that is not the largest one. It relates to the fact that people still tend to make intepretations of their data, based on the p-values associated with individual clusters, which is strictly speaking a bit silly. Should we consider only returning this warning for the clusters designated as "#1"?

Jörn M. Horschig - 2016-03-10 14:55:22 +0100

yes, good idea. That prevents people from having stupid ideas :)

Robert Oostenveld - 2016-03-15 13:40:37 +0100

(In reply to Jörn M. Horschig from comment #1) People will have stupid ideas anyway, regardless of all our attempts. But I also agree.

Jörn M. Horschig - 2016-03-15 13:55:37 +0100

(In reply to Robert Oostenveld from comment #2) hehe, let me rephrase: yes, good idea. That might reduce the likelihood of people getting stupid ideas :)

Jan-Mathijs Schoffelen - 2016-05-05 19:22:06 +0200

addressed with pull request 140