🚀 Feature: Add ability to write spectrum file to a buffer instead of to disk

[suzil]

🔖 Feature description

We're working on a feature for RADIS App to allow the user to download the spectrum file (arunavabasucom/radis-app#619). To do this, we do spectrum.store(file_path), load the file from disk to serve to the user, and then delete the file. If we could instead pass a buffer object to write the file to memory, we could simplify our implementation of this feature and improve performance by skipping the write to disk. The buffer would have the same interface as someone doing fp = open(file_path) and passing fp to spectrum.store(fp), so RADIS would just write the data there and the client can take care of whether it goes to disk or to an in-memory buffer.

I propose adjusting spectrum.store(file_path) to accept either a string file path (as it is now) or a file handle-like object. If passed a file handle-like object, we would skip the open(...) lines like here:

radis/radis/tools/database.py

Line 211 in 91bfcd0

with open(fout, "wb") as f:

👉 Why you want this feature!!

Simplify RADIS App implementation of serving spectrum download files to users and improve performance.

[erwanp]

It seems we can simply dump into a StringIO object and Json works fine (json_tricks probably too).

So we'd replace

radis/radis/tools/database.py

Line 211 in 91bfcd0

with open(fout, "wb") as f:

        with open(fout, "wb") as f:
            json_tricks.dump(
                sjson,
                f,
                compression=True,  # calls gzip compression
                properties={
                    "ndarray_compact": True
                },  # use compact numpy format in json-tricks 3.15+
            )

with

        import io
        response_file = io.StringIO()
        json_tricks.dump(
              sjson,
              response_file,
              compression=True,  # calls gzip compression
              properties={
                  "ndarray_compact": True
              }, 
          )

Is that it ?

[suzil]

Yup, that looks right.

[suzil]

If you're writing binary (mode is "wb"), we'd use BytesIO() instead of StringIO().

[arunavabasucom]

@erwanp, pls assign me, gonna wrap this up !!

[erwanp]

Done