Back to the main page.

Bug 1318 - improve the filter padding options for the ft_preproc filter functions

Reported 2012-02-08 13:25:00 +0100
Modified 2014-03-12 12:21:41 +0100
Product: FieldTrip
Component: preproc
Version: unspecified
Hardware: PC
Operating System: Mac OS
Importance: P3 normal
Assigned to: Jörn M. Horschig
Depends on: 1730
See also:

Robert Oostenveld - 2012-02-08 13:25:12 +0100

Vladimir mentioned to me that it is desired for ft_specest_hilbert to have better filter padding. Since the filtering is done with ft_preproc functions, the improvement has to be implemented there. The idea is that mirroring the data at the edges might help. TODO - create a test case (named test_bugXXX.m with XXX the right number) that demonstrates a case with hand-crafted "random" data which would benefit from improved edge padding.

Jörn M. Horschig - 2012-06-27 16:24:06 +0200

Hey Vladimir, do you mind if I take this one up? It would be in line with bug 1561 on which Robert and we just worked a few minutes ago, so this one would be a quick fix now that I just started digging into the filtering stuff ;) Best, Jörn

Vladimir Litvak - 2012-06-27 16:29:39 +0200

Hi Jorn, I would be very happy if you take this. Actually I haven't realized till now that I was supposed to work on this at all. Good luck, Vladimir

Jörn M. Horschig - 2012-06-27 16:54:53 +0200

thanks ;)

Jörn M. Horschig - 2012-08-21 14:49:01 +0200

hm, two possibilities: either the high-level function (ft_preprocessing) will pad the mirrored data and remove it (my favourite) or the low-level function will do that (not sure how, some filthy trick like negative cfg.padding, or new option like cfg.mirrorpad). since I prefer the first solution, I thought about it such that mirror padding can only be done for in-memory data and will automatically be done if cfg.padding is used for in-memory data. anyone who wants to do mirror padding without going through the high-level function has to do that manually. any thoughts on this?

Jörn M. Horschig - 2012-08-21 14:50:42 +0200

uh, just saw that the unpadding is already done by the low-level function - so only the question whether the high- or low-level function should do the padding remains

Jörn M. Horschig - 2012-08-22 14:25:30 +0200

agreed upon in Ft meeting. Vladimir, is it okay for you if the padding is done in ft_preprocessing and not in the low-level functions? I just had a quick look at specest_hilbert and saw that the function is actually doing some cranky stuff which has absolutely nothing to do ft_preprocessing, so this solution does not help you. Is mirrorpadding always more favourable than zeropadding for specest_hilbert? I could then replace the zeropadding there

Vladimir Litvak - 2012-08-22 14:32:35 +0200

Hi Jorn, ft_preprocessing is not good as we don't use it. If it's in ft_preproc_... functions then it's OK. I took the mirror padding idea from the Cardiff group who actually use Hilbert for spectral estimation. I don't have much experience with it myself but my feeling is it's quite a good idea also for filtering in general. Vladimir

Robert Oostenveld - 2012-08-22 15:59:28 +0200

in the FT meeting one hour ago we discussed it, but there it was not clear that it was motivated by ft_spectest_hilbert. This changes the situation and necessitates that it is in either ft_preproc_bandpassfilter or in ft_spectest_hilbert itself. I suggest to change it like function [filt] = ft_preproc_bandpassfilter(dat, Fs, Fbp, N, type, dir) function [filt] = ft_preproc_bandpassfilter(dat, Fs, Fbp, N, type, dir, pad) where pad is a string, which can be zero, mirror, edge. The default (current) implementation is zero. mirror would be tmpdat = [fliplr(dat) dat dat]; and edge tmpdat = [repmat(dat(:,1),1,n) dat repmat(dat(:,end),1,n)]; I would keep the mean subtraction as it is, that will always be beneficial (or neutral). The same can be implemented in low/high and bandstop. The option should be passed somehow to the user in a cfg as cfg.bandpassfiltpad and to ft_specest_hilbert as filtpad (because that one only has one type of filter, whereas ft_preprocessing has multiple).

Roemer van der Meij - 2012-08-22 16:29:27 +0200

(In reply to comment #8) I that case we would also need two padding parameters I think, i.e. padtype and padlength: padtype = 'zero', 'edge', 'mirror' padlength = number, samples needed to be padded out to To keep things consistent, it might better to create a small function like: ft_preproc_padding(dat, padtype, padlength, mode) (mode being 'add', or 'remove') This can then be called from all filter functions, with the default being padtype = []; Trivial, but for mirroring it should be: tmpdat = [fliplr(dat) dat fliplr(dat)]; Or, for longer sizes ;), tmpdat = [fliplr(dat) dat fliplr(dat) dat fliplr(dat) dat fliplr(dat)];

Jörn M. Horschig - 2012-10-24 16:45:49 +0200

function [dat] = ft_preproc_padding(dat, padtype, padlength) % FT_PREPROC_PADDING performs padding on the data, i.e. adds or removes % samples to/from the data matrix. % % Use as % [dat] = ft_preproc_padding(dat, padtype, padlength) % where % dat data matrix (Nchan1 X Ntime) % padtype 'zero', 'edge', 'mirror' or 'remove' % padlength scalar, number of samples that will be padded 513 $ svn add ft_preproc_padding.m A ft_preproc_padding.m jorhor@mentat284:~/FieldTrip/trunk/preproc 514 $ svn ci ft_preproc_padding.m -m "enhancement-#1318- added new preproc_padding function that should (later) take over all padding apart from datapadding" Adding ft_preproc_padding.m Transmitting file data . Committed revision 6786. Neither did I test the function nor did is it used anywhere, yet. The mirror padding, however, turned out to be more complex if done properly: 1) also with mirror padding, only a certain number of samples to be padded is desired (imagine a 5minute piece of data would be mirrored... hello mr. memory problem!) 2) as Roemer pointed out, if the to-be-padded data is more than the # samples available, padding has to be reversed 3) we do not want the first (and last) sample each to be duplicated (i.e. we do not want [1 2 3 3 2 1] but rather [1 2 3 2 1]) Gonna replace all padding procedures by calls to this function next week. In the meanwhile, Vladimir, is is alright for you to have it this way?

Jörn M. Horschig - 2012-10-24 16:51:00 +0200

add mean-padding in line with ft_preproc_smooth

Jörn M. Horschig - 2012-11-16 14:31:21 +0100

Vladimir, ft_specest_hilbert could get new key-value pairs 'padtype' and 'padlength' and then call ft_preproc_padding. Default would be no padding - agree? The type of filtering can be set manual, different methods are implemented. On a more general note: the function 'preproc' currently supports different length for padding in the beginning and in the end of a data matrix. Should this be supported by the new function? Currently it's not.

Jörn M. Horschig - 2012-11-16 14:31:51 +0100

(In reply to comment #12) *the type of padding not filtering, of course...

Jörn M. Horschig - 2012-11-16 14:34:19 +0100

ahjaja, and I see default behaviour is zeropadding, that would be kept as it is of course...

Jörn M. Horschig - 2012-11-23 11:11:27 +0100

padding is more flexible now. Let me know if anything more should be changed or if an error occurs. See also bug 1730.