Back to the main page.

Bug 2420 - beamformer tutorial test scripts need excessive resources on torque

Reported 2013-12-12 08:55:00 +0100
Modified 2014-03-12 12:19:57 +0100
Product: FieldTrip
Component: core
Version: unspecified
Hardware: PC
Operating System: Mac OS
Importance: P3 normal
Assigned to: Jörn M. Horschig
Depends on:
See also:

Robert Oostenveld - 2013-12-12 08:55:27 +0100

I frequently get PBS Job Id: Job Name: test_tutorial_beamformer20131122 Exec host: dccn-c016/8 Aborted by PBS Server Job exceeded some resource limit (walltime, mem, etc.). Job was aborted See Administrator for help or PBS Job Id: Job Name: test_tutorial_beamformer Exec host: dccn-c018/13 job deleted Job deleted at request of job violates resource utilization policies although I have repeatedly been increasing time and memory estimates. A tutorial should not be taking more than 2 hours, and should be able to run on a "normal" machine. My feeling is that these test scripts take much more resources than they used to do. Perhaps something changed in the code that makes specifically these test scripts inefficient. to do: run the scripts (or one of them) and profile the code. Where is most time spent, and which part requires the most memory? Subsequently the FT code and tutorial on the wiki has to be fixed to ensure that the suggested pipeline is actually feasible under normal conditions. If needed the test scripts (also the historical ones) might need a small update to reduce the resource requirements. I can imagine that downsampling the MRI prior to segmentation would resolve a lot.

Jörn M. Horschig - 2013-12-12 12:15:35 +0100

I am running the profiler since 10 in the morning. The testscript is still at computing the leadfields. Computing the sourcemodel took about 30 minutes, resulting in 1.5 million dipoles inside and 9 million dipoles outside the brain. This is a grid of 200 x 200 x 200. After tightening, 1.5 million are still inside. The gridsize is 1mm. I guess we can safe a lot of down by making it 10mm? is this truly what is suggested in the tutorial? I'm gonna check.

Jörn M. Horschig - 2013-12-12 13:16:58 +0100

yap, prepare_headmodel does know how to handle cfg.grid.resolution. Since no unit-field is present, it assumes the same units as the vol does have. In one FT meeting without you, Robert, we discussed and proposed that the sensor-description takes precedence, see bug 1832, last comment. Robert, do you agree with that? As a fix for this bug, I would propose that the tutorial should explicitly specify cfg.grid.unit = 'cm'

Jörn M. Horschig - 2013-12-12 14:20:23 +0100

yep, that was it, now it takes 600seconds, of which 80% are spent in ft_volumesegment. I added cfg.grid.unit to various locations on the wiki, because imho this is good practice and makes things more explicit. I am a bit lost in the MNE tutorial, because the grid there is defined by sourcespace.pnt. I guess cfg.grid.unit should then be sourcespace.unit, right? Also note my question in the last post ;)