Add API for file handle

[ericpre]

What if we put the original file into the metadata or added it to the signal dictionary. Would that be slightly better as we could just track the initial object.

Originally posted by @CSSFrancis in #317 (comment)

[ericpre]

Should add the file handle as an item of the dictionary returned by file_reader functions? Then the consumer will do what they want with it! ;) For example, hyperspy could keep it as private attribute?

[sem-geologist]

I am scratching my head on this. What is benefits of not closing file? especially when using lazy signals (datasets larger than RAM)? Tracking through the rabbit hole I see that it has something to do with Tiffs. How compute know about closing or opening files at all? (and if it should at all?). The questions are poping to my head as I am looking at improvement of bcf reading so pixel-spectra could be retrieved lazily. My code notes internally filepath, and opens file everytime it needs to do parsing with

with open (filepath, "rb") as fhandle:
      do stuff

which closes file automatically after that block of code.
Is it that some other formats opens file, and keeps files open indefinitely unless it is ordered to close?
Is this some windows file opening performance overhead workaround?
But most importantly why compute would have only close implemented and not open?

[ericpre]

What is benefits of not closing file?

For reading data non-lazily, the file is closed when the file is finished to read. For loading lazily, the file is kept open to be able to access the data - if the file is closed, dask will complain.

How compute know about closing or opening files at all?

This is what this discussion is about. At the moment, compute knows about closing hdf5 and tif files only and this is clearly suboptimal. It would be better to have an API for the file handle so that hyperspy compute can easily close the file when needed, without having to figure out the file handle in the dask keys, etc...

Is it that some other formats opens file, and keeps files open indefinitely unless it is ordered to close?

Yes, only hdf5 and tif files can be closed explicitly with hyperspy.