Back to the main page.

Bug 3295 - ft_appenddata does not only select the channels present in all data structures anymore

Reported 2017-05-11 14:44:00 +0200
Modified 2019-08-10 12:40:58 +0200
Product: FieldTrip
Component: core
Version: unspecified
Hardware: All
Operating System: All
Importance: P5 critical
Assigned to: Robert Oostenveld
Depends on:
See also:

Thomas Hartmann - 2017-05-11 14:44:44 +0200

Created attachment 839 demo script for the bug hi, the previous behavior of ft_appenddata was that if you would append two or more data structures over the rpt dimension and these data structures have different channels, only those channels present in all the data structures and their data would end up in the result. this behavior has changed dramatically leading to inconsistent data structures. in the attached script, i simulate some data with 10 channels. i then create a version of that data with 7 and 6 channels, both taken out of the original data structure. the two new structures only share 3 of their channels. in the next step, ft_appenddata is used to combine the three data structures. i would expect the result to have all trials containing only the info of those 3 channels, common to all the structures. but instead, the label field of the result contains all 10 original channels while the 150 trials contain the full set of channels present in the data structure they originate from. best, thomas

Arjen Stolk - 2017-05-11 17:57:05 +0200

Hi Thomas, working on a new version of ft_appendsens as we speak.

Arjen Stolk - 2017-05-11 20:33:22 +0200

Hi Thomas, Which version of ft are you using? When I used the latest: K>> size(data_few_1.trial{1}) ans = 7 200 K>> size(data_few_1.label) ans = 7 1 and K>> size(data_few_2.trial{1}) ans = 6 200 K>> size(data_few_2.label) ans = 6 1

Thomas Hartmann - 2017-05-12 09:42:28 +0200

(In reply to Arjen Stolk from comment #2) hi arjen, thanks for taking care of this. i am using the latest ft version (commit: c2f875cf6059bf19f202762d6a52248db29f0628). and concerning your comment: the two data_few_X structures are fine. they result from two calls to ft_selectdata which works perfectly. the problem becomes apparent when you look at data_final, created in line 51 using ft_appenddata: >> data_final data_final = struct with fields: label: {10×1 cell} trial: {1×150 cell} time: {1×150 cell} cfg: [1×1 struct] >> size(data_final.trial{1}) ans = 10.0000e+000 200.0000e+000 >> size(data_final.trial{51}) ans = 7.0000e+000 200.0000e+000 >> size(data_final.trial{101}) ans = 6.0000e+000 200.0000e+000 best, thomas

Arjen Stolk - 2017-05-13 02:37:47 +0200

Hi Thomas, I was able to replicate the issue. It seems append_common is doing its work properly, finding the 3 channels that overlap across your 3 datasets: line 123 in ft_appenddata: data = append_common(cfg, dummy{:}); data = label: {3x1 cell} data.label ans = 'chan_04' 'chan_05' 'chan_06' However, at lines 155 to 169, this information is not taken into account. As a result, data_final has 10 channel labels but unequally-sized trial matrices depending on the appended data (e.g. 10 rows, 7 rows, and 6 rows). I'm cc'ing Robert, to see where he thinks the fix should be implemented. That is, either in ft_appenddata or in append_common, in case the issue replicates with timelock and freq data.

Thomas Hartmann - 2017-05-19 09:26:13 +0200

(In reply to Arjen Stolk from comment #4) any updates?

Robert Oostenveld - 2017-05-19 10:36:43 +0200

I thought that it only pertained to the default behavior of the function, but actually this is a blatant bug in ft_appenddata. with ft_appenddata(cfg, data_all, data_few_1, data_few_2); it returns a data structure that is internally not consistent.

Robert Oostenveld - 2017-05-19 11:13:16 +0200

[bug3295-appenddata b7d0183] fixed bug in ft_appenddata when appending data with non-identical but overlapping channels. See and thanks to Thomas for reporting. 2 files changed, 117 insertions(+), 26 deletions(-) create mode 100644 test/test_bug3295.m

Robert Oostenveld - 2017-05-19 11:16:45 +0200

I made changes to the code, see

Arjen Stolk - 2017-05-30 01:46:05 +0200


Robert Oostenveld - 2019-08-10 12:34:47 +0200

This closes a whole series of bugs that have been resolved (either FIXED/WONTFIX/INVALID) for quite some time. If you disagree, please file a new issue on

Robert Oostenveld - 2019-08-10 12:40:58 +0200

This closes a whole series of bugs that have been resolved (either FIXED/WONTFIX/INVALID) for quite some time. If you disagree, please file a new issue on