Back to the main page.

Bug 1041 - ft_freqdescriptives needs NaN amount of RAM

Reported 2011-10-14 11:32:00 +0200
Modified 2012-08-23 10:35:14 +0200
Product: FieldTrip
Component: core
Version: unspecified
Hardware: PC
Operating System: Windows
Importance: P1 normal
Assigned to: Eelke Spaak
Depends on: 1537
See also:

Jörn M. Horschig - 2011-10-14 11:32:25 +0200

Output, independent of Linux or Windows: the call to "ft_freqdescriptives" took 5 seconds and an estimated NaN MB

Robert Oostenveld - 2011-10-17 22:32:32 +0200

I created a test script, but am not able to reproduce the problem. See fieldtrip/test/test_bug1041 Can you reproduce it? If so, can you update the test script and reassign to me?

Jörn M. Horschig - 2011-10-18 13:55:28 +0200

Could you check whether you comitted that test script? jorhor@mentat312:~/FieldTrip/trunk/test 505 $ svn up At revision 4499. jorhor@mentat312:~/FieldTrip/trunk/test 506 $ ls *1041* ls: *1041*: No such file or directory btw: I get it for every call to freqdescriptives. Maybe it's platform related? >> load('H:\common\matlab\fieldtrip\data\test\freq\meg\freq_mtmfft_ctf275.mat') >> ft_freqdescriptives([], freq) the input is freq data with 307 channels, 101 frequencybins and no timebins undoing the G3BR balancing the call to "ft_freqdescriptives" took 0 seconds and an estimated NaN MB

Robert Oostenveld - 2011-10-18 15:25:38 +0200

(In reply to comment #2) > Could you check whether you comitted that test script? Sorry, I indeed had forgotten. It is now. It could well be a windows specific problem, I don't recall that I ever spent serious time on figuring out the memory function on windows. So probably there is something like ifdef __WIN32__ return NAN else do something smarter end

Jörn M. Horschig - 2011-10-19 12:24:24 +0200

I guess it is line 49: if isempty(strfind(which('memprofile'), mexext)) ... output = nan;

Robert Oostenveld - 2011-10-19 12:43:46 +0200

(In reply to comment #4) so it is a feature, not a bug ;-)

Jörn M. Horschig - 2011-10-19 14:43:35 +0200

true, but can we remove that feature? :) we could use the windows-specific function memory in Matlab: >> [uV sV] = memory uV = MaxPossibleArrayBytes: 1.3207e+010 MemAvailableAllArrays: 1.3207e+010 MemUsedMATLAB: 1.0025e+009 sV = VirtualAddressSpace: [1x1 struct] SystemMemory: [1x1 struct] PhysicalMemory: [1x1 struct] so by comparing uV.MemUsedMATLAB before and after we can figure out how much memory the function used. Not quite sure how precise this will be though, because it relies on the windows' memory management.

Robert Oostenveld - 2011-10-19 16:53:30 +0200

nice idea, but does not work. The memory need could be the highest in the middle, i.e. neither at the begin or end. That is why the linux/osx mex file runs a thread that "measures" the memory every second. in peer/src/memprofile.c there is void memprofile_sample(void) { uint64_t rss, vs; memlist_t *memitem; if (getmem(&rss, &vs)!=0) { rss = 0; vs = 0; } ... and the corresponding "getmem" function is in peer/src/util.c, which contains #if defined (PLATFORM_OSX) int getmem (uint64_t *rss, uint64_t *vs) { task_t task = MACH_PORT_NULL; struct task_basic_info t_info; mach_msg_type_number_t t_info_count = TASK_BASIC_INFO_COUNT; if (KERN_SUCCESS != task_info(mach_task_self(), TASK_BASIC_INFO, (task_info_t)&t_info, &t_info_count)) { return -1; } *rss = t_info.resident_size; *vs = t_info.virtual_size; return 0; } #elif defined(PLATFORM_WIN32) || defined(PLATFORM_WIN64) int getmem (uint64_t *rss, uint64_t *vs) { /* no idea how to get the memory information on a windows computer */ *rss = 0; *vs = 0; return -1; } #elif defined(PLATFORM_LINUX) int getmem (uint64_t *rss, uint64_t *vs) { FILE *fp; uint64_t val1 = 0, val2 = 0; if ((fp = fopen("/proc/self/statm", "r")) == NULL) { DEBUG(LOG_ERR, "could not open /proc/self/statm"); return -1; } /* read the information from /proc/self/statm size total program size resident resident set size share shared pages text text (code) lib library data data/stack dt dirty pages (unused in Linux 2.6) */ if (fscanf(fp, "%llu%llu", &val1, &val2 )!=2) { DEBUG(LOG_WARNING, "could not read all elements from /proc/self/statm"); val1 = 0; val2 = 0; } /* these seem to be in 4096 byte blocks */ *vs = val1 * 4096; *rss = val2 * 4096; fclose(fp); return 0; } #endif The following might be a starting point for windows

Eelke Spaak - 2012-07-26 11:46:09 +0200

Fixed for now by removing the memory statement under Windows; Windows implementation of getmem() is for the future.

Eelke Spaak - 2012-08-23 10:35:14 +0200

closing my bugs