Memory issue when calculating a huge matrix in VMD's TCL - malloc: can't allocate region - matrix

Question

I wrote a *.tcl script for calculation of the pairwise RMSD between trajectory's frames (n=5000) using VMD in text mode (command line). However, when I run the script (vmd -dispdev text -e distmatrix.tcl) on the terminal, after processing couple of frames the script stops with a memory issue:
$vmd -dispdev text -e distmatrix.tcl
Info) VMD for MACOSXX86, version 1.9.3 (November 30, 2016)
Info) http://www.ks.uiuc.edu/Research/vmd/
Info) Email questions and bug reports to vmd#ks.uiuc.edu
Info) Please include this reference in published work using VMD:
Info) Humphrey, W., Dalke, A. and Schulten, K., `VMD - Visual
Info) Molecular Dynamics', J. Molec. Graphics 1996, 14.1, 33-38.
Info) -------------------------------------------------------------
Info) Multithreading available, 8 CPUs detected.
Info) Dynamically loaded 2 plugins in directory:
Info) /Applications/VMD/VMD 1.9.3.app/Contents/vmd/plugins/MACOSXX86/molfile
{/Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/tcl8.5} {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts} {/Applications/VMD/VMD 1.9.3.app/Contents/lib} /Applications/VMD/VMD 1.9.3.app/Contents/Frameworks/Tcl.framework/Versions/8.5/Resources/Scripts ~/Library/Tcl /Library/Tcl /Network/Library/Tcl /System/Library/Tcl ~/Library/Frameworks /Library/Frameworks /Network/Library/Frameworks /System/Library/Frameworks {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/vmd} {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/plugins/MACOSXX86/tcl} {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/plugins/noarch/tcl} /Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/la1.0
{/Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/tcl8.5} {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts} {/Applications/VMD/VMD 1.9.3.app/Contents/lib} /Applications/VMD/VMD 1.9.3.app/Contents/Frameworks/Tcl.framework/Versions/8.5/Resources/Scripts ~/Library/Tcl /Library/Tcl /Network/Library/Tcl /System/Library/Tcl ~/Library/Frameworks /Library/Frameworks /Network/Library/Frameworks /System/Library/Frameworks {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/vmd} {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/plugins/MACOSXX86/tcl} {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/plugins/noarch/tcl} /Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/la1.0 /Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/orient
{/Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/tcl8.5} {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts} {/Applications/VMD/VMD 1.9.3.app/Contents/lib} /Applications/VMD/VMD 1.9.3.app/Contents/Frameworks/Tcl.framework/Versions/8.5/Resources/Scripts ~/Library/Tcl /Library/Tcl /Network/Library/Tcl /System/Library/Tcl ~/Library/Frameworks /Library/Frameworks /Network/Library/Frameworks /System/Library/Frameworks {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/vmd} {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/plugins/MACOSXX86/tcl} {/Applications/VMD/VMD 1.9.3.app/Contents/vmd/plugins/noarch/tcl} /Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/la1.0 /Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/orient /Applications/VMD/VMD 1.9.3.app/Contents/vmd/scripts/vmd/get_total_charge.tcl
2.0.2
file6
5000
::m
Progress: ▏▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋ ▏ 65% (3295/5000)vmd(96350,0x5ae080) malloc: can't allocate region
*** mach_vm_map(size=8388608) failed (error code=3)
vmd(96350,0x5ae080) malloc: *** set a breakpoint in malloc_error_break to debug
vmd(96350,0x5ae080) malloc: can't allocate region
*** mach_vm_map(size=8388608) failed (error code=3)
vmd(96350,0x5ae080) malloc: *** set a breakpoint in malloc_error_break to debug
unable to realloc 1149 bytes
Abort trap: 6
It is interesting to note that when I run the same script in tclsh (./distmatrix.tcl), the program is running smoothly and finishes successfully with writing the final 5000x5000 matrix in a the output file.
$./distmatrix.tcl
Progress: ▏▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋▋ ▏ 100% (5000/5000)
Done.
For the sake of simplicity I tested a similar simple script (not the real trajectory calculation) of producing a 5000x5000 matrix (code below):
#!/usr/bin/tclsh
package require struct::matrix
################################################################
proc progress_bar {totalsize current} {
font color light green for the bar
set fgreenl [exec tput setaf 40]
no color
set nb [exec tput sgr 0]
puts -nonewline "\rProgress: \u258F"
if {$totalsize == 0} {
set totalsize $current
}
set portion [expr 1.0 * $current/$totalsize * 40]
for {set x 0} {$x <= $portion} {incr x} {
puts -nonewline "${fgreenl}\u258B${nb}"
}
for {} {$x <= 40} {incr x} {
puts -nonewline " "
}
puts -nonewline " \u258F [format "%3d" [expr int(100.0 * $current/$totalsize)]]% ($current/$totalsize)"
flush stdout
if {$totalsize == $current} {
puts "\nDone."
}
}
Output file matrix
set outfile [open mat2.txt w]
################################################################
set n 5000
Create a new matrix that will store all data (pairwise RMSD values)
struct::matrix m;
Define number of rows that is equal to the number of frames in the MD trajectory
m add columns [expr $n + 1];
for {set i 0} {$i <= $n} {incr i} {
set vecl {}
for {set j 0} {$j <= $n} {incr j} {
set vec [expr sqrt($j)]
lappend vecl $vec
m add row $vecl;
progress_bar $n $i
after 1
}
set vec_vals [m format 2string;]
puts $outfile $vec_vals
close $outfile
exit
Is there a possibility to overcome this memory issue (e.g., somehow by chunking the operations executed inside the second loop)?