PROPOSED FIX: Too many open files from TiffImagePlugin.py

[lucatirel]

Working in my enterprise context where we process large folders of ".tiff" files (>100k images), i found that usage of src/PIL/TiffImagePlugin.py still cause the well known error "Too many open files" due to memory storage i suppose is related to file handlers. (from stackoverflow posts)

I found a fast-fix for that by adding the following snippet of code to TiffImagePlugin.py :

if _fp:
    try:
         os.close(_fp)
    except OSError:
         pass 

I propose to add this block after the if-else block in TiffImagePlugin.py like that:

if "_debug_multipage" in encoderinfo: 
     ...

if _fp:
    try:
         os.close(_fp)
    except OSError:
         pass

I found this solution to correctly solve my issue empirically, avoiding "too many open files" error when working with large images directories.

PS:
I am a neophyte to github pull/branch/merge/commit system, so excuse me if i did not proposed the fix in the correct way. I just would like to help you, since Pillow is a very useful and lovely library.

[radarhere]

Hi. You've created a issue, where people typically report problems they'd like fixed or ask questions they'd like answered. You have a problem that you'd like solved, so this is fine.

To formally suggest a code change, people create pull requests, but I have some comments.

1. Although you say "too many open files" is a "well known error", I would find it easier to deal with a specific problem. If you could post some example code, and what operating system and Pillow version you're using, that would be helpful.

2. Are you opening TIFFs, saving TIFFs, or both? If you're opening TIFFs, I wonder if PR Do not close provided file handles with libtiff #7199 would solve your problem. If you're unable to compile Pillow from source to try it out, let us know what operating system and Python version you're using, and we should be able to put together a wheel you can install to test it.

3. Would you agree that we've already run your proposed addition a few lines earlier?
   

   Pillow/src/PIL/TiffImagePlugin.py

   Lines 1859 to 1862 in 6ebdc52

   	try:
   	os.close(_fp)
   	except OSError:
   	pass

   
   I think you're attempting to close the same file twice with your idea. That may seem to solve your problem, but not too long ago "close" syscall run twice on same file descriptor when loading binary LZW-compressed TIFF #7042 was opened to try and stop us from attempting to close the same file twice.

[radarhere]

#7199 has been merged and released as part of Pillow 10.0.0.

If you upgrade, is your problem resolved?

[github-actions]

Closing this issue as no feedback has been received.