Re: [Texmacs-dev] number of open files

[Joris van der Hoeven]

On Fri, Sep 16, 2011 at 11:35:25PM +0200, Josef Weidendorfer wrote:
> On Friday 16 September 2011, Joris van der Hoeven wrote:
> > > Just a quick guess: After mmap'ing a file, you can close the
> > > descriptor, but perhaps it still falls under the OS X limit until
> > > a munmap is done? This would explain the connection to freetype,
> > > which seems to map fonts into memory.
> > 
> > Thanks for this possible explanation. Is there some way to know
> > how many closed FDs are mapped into memory and still counted by the OS?
> > This would allow me to check whether your hypothesis is right.
> 
> In Linux, you can see the mappings of a process in /proc/<PID>/maps.
> Not sure about OS-X.

This does not exist on MacOS, but I now suspect that lsof actually
shows the FDs which are mapped into memory. More precisely,
this seems to correspond to the lines with REG in the listing below.

So I still do not understand why I am exceeding the 256 open files limit.
I really would like to understand this: the resulting bug is VERY annoying
(I lost several hours of work because I thought things were saved while
they were not). We really need to understand what is going on and
design a clean fix. Anyone can help me further?

Best, --Joris


Castafiore:MaGiXaLiX vdhoeven$ lsof -p 54934
COMMAND     PID     USER   FD     TYPE     DEVICE  SIZE/OFF      NODE NAME
texmacs.b 54934 vdhoeven  cwd      DIR       14,2       272 107792971 
/Users/vdhoeven/magix/vdhoeven/seminar/MaGiXaLiX
texmacs.b 54934 vdhoeven  txt      REG       14,2  10361176 108100663 
/Users/vdhoeven/texmacs/src/TeXmacs/bin/texmacs.bin
texmacs.b 54934 vdhoeven  txt      REG       14,2  25095600  84504684 
/Library/Frameworks/QtGui.framework/Versions/4/QtGui
texmacs.b 54934 vdhoeven  txt      REG       14,2   6220576  84504650 
/Library/Frameworks/QtCore.framework/Versions/4/QtCore
texmacs.b 54934 vdhoeven  txt      REG       14,2    978976  99068468 
/opt/local/lib/libguile.17.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2     40912  98654985 
/opt/local/lib/libltdl.7.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2    449632  98226399 
/opt/local/lib/libgmp.10.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2    621496  98432487 
/opt/local/lib/libfreetype.6.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2     96280   6198388 
/opt/local/lib/libz.1.2.5.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2     74456  97036117 
/opt/local/lib/libbz2.1.0.6.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2   1094920  98203618 
/opt/local/lib/libiconv.2.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2     54288   6162279 
/opt/local/lib/libintl.8.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2  13997424   7218861 
/usr/share/icu/icudt40l.dat
texmacs.b 54934 vdhoeven  txt      REG       14,2       150     13440 
/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Ink.framework/Versions/A/Resources/inkpaper.png
texmacs.b 54934 vdhoeven  txt      REG       14,2      3144      6596 
/System/Library/ColorSync/Profiles/sRGB Profile.icc
texmacs.b 54934 vdhoeven  txt      REG       14,2     93884  84521435 
/Developer/Applications/Qt/plugins/imageformats/libqgif.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2     99048  84521436 
/Developer/Applications/Qt/plugins/imageformats/libqico.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2     77568  84521439 
/Developer/Applications/Qt/plugins/imageformats/libqsvg.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2   2572288  95505749 
/private/var/folders/81/810ln2PCEHytPEy+NV2Mp++++TI/-Caches-/com.apple.LaunchServices-025501.csstore
texmacs.b 54934 vdhoeven  txt      REG       14,2   5425538     11446 
/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/Resources/Extras2.rsrc
texmacs.b 54934 vdhoeven  txt      REG       14,2     28564     10514 
/System/Library/Fonts/Keyboard.ttf
texmacs.b 54934 vdhoeven  txt      REG       14,2   1757684     10516 
/System/Library/Fonts/LucidaGrande.ttc
texmacs.b 54934 vdhoeven  txt      REG       14,2     91799   1478488 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/ecrm10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2    534528  84521437 
/Developer/Applications/Qt/plugins/imageformats/libqjpeg.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2     86972    522333 
/System/Library/Caches/com.apple.IntlDataCache.le.kbdx
texmacs.b 54934 vdhoeven  txt      REG       14,2     35752  98729219 
/opt/local/share/texmf-texlive-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     32722  98729221 
/opt/local/share/texmf-texlive-dist/fonts/type1/public/amsfonts/cm/cmr12.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     32726  98729231 
/opt/local/share/texmf-texlive-dist/fonts/type1/public/amsfonts/cm/cmr8.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2    783788  84521438 
/Developer/Applications/Qt/plugins/imageformats/libqmng.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2    858500  84504768 
/Library/Frameworks/QtSvg.framework/Versions/4/QtSvg
texmacs.b 54934 vdhoeven  txt      REG       14,2    803500  84504799 
/Library/Frameworks/QtXml.framework/Versions/4/QtXml
texmacs.b 54934 vdhoeven  txt      REG       14,2    773376  84521440 
/Developer/Applications/Qt/plugins/imageformats/libqtiff.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2    811436      7125 
/System/Library/PrivateFrameworks/CoreUI.framework/Versions/A/Resources/SArtFile.bin
texmacs.b 54934 vdhoeven  txt      REG       14,2    594556  84521411 
/Developer/Applications/Qt/plugins/accessible/libqtaccessiblewidgets.dylib
texmacs.b 54934 vdhoeven  txt      REG       14,2    144580     11448 
/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/Resources/HIToolbox.rsrc
texmacs.b 54934 vdhoeven  txt      REG       14,2     32569  98729273 
/opt/local/share/texmf-texlive-dist/fonts/type1/public/amsfonts/cm/cmsy10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2   5946480      7120 
/System/Library/PrivateFrameworks/CoreUI.framework/Versions/A/Resources/ArtFile.bin
texmacs.b 54934 vdhoeven  txt      REG       14,2    802460     15340 
/System/Library/Keyboard 
Layouts/AppleKeyboardLayouts.bundle/Contents/Resources/AppleKeyboardLayouts-L.dat
texmacs.b 54934 vdhoeven  txt      REG       14,2     86464   1478486 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/ecbx10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     78601   1478496 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/ecbx12.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     86815   1478508 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/ecrm7.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     60313   1478490 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/ecss10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     88667   1478522 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/ectt7.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     32762  98729229 
/opt/local/share/texmf-texlive-dist/fonts/type1/public/amsfonts/cm/cmr7.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2    490410     11435 
/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/Resources/English.lproj/Localized.rsrc
texmacs.b 54934 vdhoeven  txt      REG       14,2    116528   1478491 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/ecti10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     97607   1478492 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/ectt10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     93549   1478487 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/eccc10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     96899   1478565 
/Users/vdhoeven/.TeXmacs/fonts/type1/math/grmn10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     36299  98729203 
/opt/local/share/texmf-texlive-dist/fonts/type1/public/amsfonts/cm/cmmi10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     88294   1478516 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/ecst10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     82659   1478498 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/ecbx7.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     31764  98729029 
/opt/local/share/texmf-texlive-dist/fonts/type1/public/amsfonts/symbols/msam10.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     36281  98729211 
/opt/local/share/texmf-texlive-dist/fonts/type1/public/amsfonts/cm/cmmi7.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2     88854   1478500 
/Users/vdhoeven/.TeXmacs/fonts/type1/ec/eccc7.pfb
texmacs.b 54934 vdhoeven  txt      REG       14,2   1054960   7249441 
/usr/lib/dyld
texmacs.b 54934 vdhoeven  txt      REG       14,2 204906496  92270387 
/private/var/db/dyld/dyld_shared_cache_x86_64
texmacs.b 54934 vdhoeven    0r     CHR        3,2       0t0       297 /dev/null
texmacs.b 54934 vdhoeven    1u     CHR       16,4  0t480705       615 
/dev/ttys004
texmacs.b 54934 vdhoeven    2u     CHR       16,4  0t480705       615 
/dev/ttys004
texmacs.b 54934 vdhoeven    3u  KQUEUE                                count=1, 
state=0x2
texmacs.b 54934 vdhoeven    4     PIPE 0x11072a94     16384           
->0x1d78c134
texmacs.b 54934 vdhoeven    5     PIPE 0x1d78c134     16384           
->0x11072a94
texmacs.b 54934 vdhoeven    6r     REG       14,2   5425538     11446 
/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/Resources/Extras2.rsrc
texmacs.b 54934 vdhoeven    7     PIPE 0x110713ec     16384           
->0x11070898
texmacs.b 54934 vdhoeven    8     PIPE 0x11070898     16384           
->0x110713ec
texmacs.b 54934 vdhoeven    9r     REG       14,2    144580     11448 
/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/Resources/HIToolbox.rsrc
texmacs.b 54934 vdhoeven   11     PIPE 0x110704b0     16384           
->0x1d78b770
texmacs.b 54934 vdhoeven   12     PIPE 0x1d78b770     16384           
->0x110704b0
texmacs.b 54934 vdhoeven   13     PIPE 0x11071388     16384           
->0x1d78c7d8
texmacs.b 54934 vdhoeven   14     PIPE 0x1d78c7d8     16384           
->0x11071388
texmacs.b 54934 vdhoeven   15r     REG       14,2    490410     11435 
/System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/Resources/English.lproj/Localized.rsrc
Castafiore:MaGiXaLiX vdhoeven$