Back to the main page.
Bug 1041 - ft_freqdescriptives needs NaN amount of RAM
| Status | CLOSED FIXED |
| Reported | 2011-10-14 11:32:00 +0200 |
| Modified | 2012-08-23 10:35:14 +0200 |
| Product: | FieldTrip |
| Component: | core |
| Version: | unspecified |
| Hardware: | PC |
| Operating System: | Windows |
| Importance: | P1 normal |
| Assigned to: | Eelke Spaak |
| URL: | |
| Tags: | |
| Depends on: | 1537 |
| Blocks: | |
| See also: |
Jörn M. Horschig - 2011-10-14 11:32:25 +0200
Output, independent of Linux or Windows: the call to "ft_freqdescriptives" took 5 seconds and an estimated NaN MB
Robert Oostenveld - 2011-10-17 22:32:32 +0200
I created a test script, but am not able to reproduce the problem. See fieldtrip/test/test_bug1041 Can you reproduce it? If so, can you update the test script and reassign to me?
Jörn M. Horschig - 2011-10-18 13:55:28 +0200
Could you check whether you comitted that test script? jorhor@mentat312:~/FieldTrip/trunk/test 505 $ svn up At revision 4499. jorhor@mentat312:~/FieldTrip/trunk/test 506 $ ls *1041* ls: *1041*: No such file or directory btw: I get it for every call to freqdescriptives. Maybe it's platform related? >> load('H:\common\matlab\fieldtrip\data\test\freq\meg\freq_mtmfft_ctf275.mat') >> ft_freqdescriptives([], freq) the input is freq data with 307 channels, 101 frequencybins and no timebins undoing the G3BR balancing the call to "ft_freqdescriptives" took 0 seconds and an estimated NaN MB
Robert Oostenveld - 2011-10-18 15:25:38 +0200
(In reply to comment #2) > Could you check whether you comitted that test script? Sorry, I indeed had forgotten. It is now. It could well be a windows specific problem, I don't recall that I ever spent serious time on figuring out the memory function on windows. So probably there is something like ifdef __WIN32__ return NAN else do something smarter end
Jörn M. Horschig - 2011-10-19 12:24:24 +0200
I guess it is line 49: if isempty(strfind(which('memprofile'), mexext)) ... output = nan;
Robert Oostenveld - 2011-10-19 12:43:46 +0200
(In reply to comment #4) so it is a feature, not a bug ;-)
Jörn M. Horschig - 2011-10-19 14:43:35 +0200
true, but can we remove that feature? :) we could use the windows-specific function memory in Matlab: >> [uV sV] = memory uV = MaxPossibleArrayBytes: 1.3207e+010 MemAvailableAllArrays: 1.3207e+010 MemUsedMATLAB: 1.0025e+009 sV = VirtualAddressSpace: [1x1 struct] SystemMemory: [1x1 struct] PhysicalMemory: [1x1 struct] so by comparing uV.MemUsedMATLAB before and after we can figure out how much memory the function used. Not quite sure how precise this will be though, because it relies on the windows' memory management.
Robert Oostenveld - 2011-10-19 16:53:30 +0200
nice idea, but does not work. The memory need could be the highest in the middle, i.e. neither at the begin or end. That is why the linux/osx mex file runs a thread that "measures" the memory every second. in peer/src/memprofile.c there is void memprofile_sample(void) { uint64_t rss, vs; memlist_t *memitem; if (getmem(&rss, &vs)!=0) { rss = 0; vs = 0; } ... and the corresponding "getmem" function is in peer/src/util.c, which contains #if defined (PLATFORM_OSX) int getmem (uint64_t *rss, uint64_t *vs) { task_t task = MACH_PORT_NULL; struct task_basic_info t_info; mach_msg_type_number_t t_info_count = TASK_BASIC_INFO_COUNT; if (KERN_SUCCESS != task_info(mach_task_self(), TASK_BASIC_INFO, (task_info_t)&t_info, &t_info_count)) { return -1; } *rss = t_info.resident_size; *vs = t_info.virtual_size; return 0; } #elif defined(PLATFORM_WIN32) || defined(PLATFORM_WIN64) int getmem (uint64_t *rss, uint64_t *vs) { /* no idea how to get the memory information on a windows computer */ *rss = 0; *vs = 0; return -1; } #elif defined(PLATFORM_LINUX) int getmem (uint64_t *rss, uint64_t *vs) { FILE *fp; uint64_t val1 = 0, val2 = 0; if ((fp = fopen("/proc/self/statm", "r")) == NULL) { DEBUG(LOG_ERR, "could not open /proc/self/statm"); return -1; } /* read the information from /proc/self/statm size total program size resident resident set size share shared pages text text (code) lib library data data/stack dt dirty pages (unused in Linux 2.6) */ if (fscanf(fp, "%llu%llu", &val1, &val2 )!=2) { DEBUG(LOG_WARNING, "could not read all elements from /proc/self/statm"); val1 = 0; val2 = 0; } /* these seem to be in 4096 byte blocks */ *vs = val1 * 4096; *rss = val2 * 4096; fclose(fp); return 0; } #endif The following might be a starting point for windows http://msdn.microsoft.com/en-us/library/ms683219.aspx
Eelke Spaak - 2012-07-26 11:46:09 +0200
Fixed for now by removing the memory statement under Windows; Windows implementation of getmem() is for the future.