Back to the main page.
Bug 2746 - loadcnt sometimes behaves weird by trying to jump to a non-integer valued file pointer
| Status | CLOSED FIXED |
| Reported | 2014-10-23 21:19:00 +0200 |
| Modified | 2018-08-09 19:42:49 +0200 |
| Product: | FieldTrip |
| Component: | fileio |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Mac OS |
| Importance: | P5 normal |
| Assigned to: | Jan-Mathijs Schoffelen |
| URL: | |
| Tags: | |
| Depends on: | |
| Blocks: | |
| See also: | http://bugzilla.fieldtriptoolbox.org/show_bug.cgi?id=3441 |
Jan-Mathijs Schoffelen - 2014-10-23 21:19:41 +0200
this happens in line 364 where for one strange reason the startpos is non-integer, causing fseek to fail, and the function to return the incorrect data. FieldTrip specifies r.sample1 and on lines 333 and 338/339 this integer value is divided and multiplied with h.rate, which somehow occasionally results in slightly non-integer values. The user that reported a problem with reading the data correctly in FieldTrip, which is most likely caused by this issue, also reported that segmenting the data with eeglab works fine. Therefore I thought that the problem should be fixed on FieldTrip's end, but I don't know an easy way. To me, the easiest fix would be an explicit round() on l.338/339. I don't think that this would hurt in general. Arno, do you think this would be an OK change to the code? Does this need to be changed in eeglab, too?
Jan-Mathijs Schoffelen - 2014-10-24 09:18:26 +0200
created test script: test_bug2476
Jan-Mathijs Schoffelen - 2014-10-24 09:18:48 +0200
(In reply to Jan-Mathijs Schoffelen from comment #1) I mean test_bug2746
Jan-Mathijs Schoffelen - 2014-10-29 09:51:29 +0100
@Arno: we have some local test data and the test-script resides in fieldtrip/test/ (it is part of the svn-repository, but not of the FieldTrip release package). The user that reported the problem had some data uploaded on Dropbox. I post the original e-mail below (her data however is quite large, and the issue seems reproducible on our example data (and thus by extension: on any data) too: ================================================================================= ================================================================================= Hi all, When I am using ft_preprocessing to segment the data from the continuous eeg .cnt file I get segments that are exactly the same. That is, pieces of the data that correspond to different trials and that are exactly the same! I don’t really know if I am doing something wrong or if there is a problem in the interaction of the ft_preprocessing and the loadcnt function!!!! I am not new using fieldtrip, but I am new with .cnt files. I have been checking and trying different things: 1) My trl matrix seems to be properly defined; 2) When I use curry or eeglab to segment the data I don’t get this weird repeated segments; 3) Extracting the original data using ft_read_data and looking into the values that correspond to the samples of the repeated segments show that the original data of those segments is different! 4) Importantly, while I am extracting the segments from the data using ft_preprocessing I get a warning message “events imported with a time shift might be innacurate”… (this do not happen with eeglab and the function for loading .cnt files is apparently the same); 5) This is not a problem of only one participant and one of my colleagues is experiencing exactly the same problem. Here is the configuration that I am using to run ft_preprocessing cfg = dataset: 'D:\Subj01\1-OriginalData\Subj01.cnt' dataformat: 'ns_cnt32' headerformat: 'ns_cnt32' eventformat: 'ns_cnt32' demean: 'yes' baselinewindow: [0 0.2000] reref: 'yes' refchannel: {'A1' 'A2'} implicitref: {'A1'} channel: {'all' '-A1' '-TP9' '-TP10' '-FT10' '-FT9' '-Trigger'} trl: [312x5 double] Using this code on the output of the preprocessing shows me that trials 12, 85, 86, 87 and 89 are exactly the same!!!!! repeatedtrials =[]; for i = 1:length(PreProcArtifact_cue.trial) for j = 1:length(PreProcArtifact_cue.trial) if PreProcArtifact_cue.trial{i} == PreProcArtifact_cue.trial{j} if i ~= j repeatedtrials(1+end,:) = [i j]; end end end end I uploaded my original data file, my trl matrix and the output of the preprocessing here: https://www.dropbox.com/sh/k2nmg50le34v8a0/AAB6odjOx8wnRz10FdG4YlyXa?dl=0 Please can someone check if I am doing something wrong in order to get exactly the same segment in a different sample interval after using ft_preprocessing? It has been really frustrating to try to figure out what can be wrong and I am starting to believe that there is a problem in ft_preprocessing while segmenting .cnt files? Can this be the case???? Thank you very very much! Inês Bramão
Jan-Mathijs Schoffelen - 2014-10-29 15:51:39 +0100
The following code shows that the current issue seems to be a finite arithmetic issue: sample = 1:20000; rate = 500; newsample = sample./rate.*rate; % this is what is essentially done in loadcnt figure; plot(sample-newsample); % sometimes the difference is not equal to 0
Robert Oostenveld - 2014-10-29 16:35:50 +0100
(In reply to Jan-Mathijs Schoffelen from comment #4) if it is a numerical accuracy issue, then that should be addressed. Fseek indeed seems to be picky about this. I had expected it to have some eps-tolerance. >> fid = fopen('matlab_crash_dump.5277-1') fid = 3 >> fseek(fid, eps, 'bof') ans = -1 >> fseek(fid, 0, 'bof') ans = 0
Jan-Mathijs Schoffelen - 2014-10-30 13:10:36 +0100
bash-4.1$ svn commit -m "bugfix - 2746: deal with numerical inaccuracy of slightly non-integer samples requested for reading" test/test_bug2746.m external/eeglab/loadcnt.m Sending external/eeglab/loadcnt.m Sending test/test_bug2746.m Transmitting file data .. Committed revision 9932. OK, I officially think it's a rounding issue. I updated our local version of loadcnt. If Arno and Ramon agree with the suggested patch, they may want to use this in the official EEGLAB version as well