How to write file to memory filepath and read from memory filepath in Python?

Question

An existing Python package requires a filepath as input parameter for a method to be able to parse the file from the filepath. I want to use this very specific Python package in a cloud environment, where I can't write files to the harddrive. I don't have direct control over the code in the existing Python package, and it's not easy to switch to another environment, where I would be able to write files to the harddrive. So I'm looking for a solution that is able to write a file to a memory filepath, and let the parser read directly from this memory filepath. Is this possible in Python? Or are there any other solutions?

Example Python code that works by using harddrive, which should be changed so that no harddrive is used:

temp_filepath = "./temp.txt"
with open(temp_filepath, "wb") as file:
  file.write("some binary data")
model = Model()
model.parse(temp_filepath)

Example Python code that uses memory filesystem to store file, but which does not let parser read file from memory filesystem:

from fs import open_fs
temp_filepath = "./temp.txt"
with open_fs('osfs://~/') as home_fs:
  home_fs.writetext(temp_filepath, "some binary data")
model = Model()
model.parse(temp_filepath)

Can this package also accept an open file object? In that case you could use io.BytesIO. — tdelaney
– tdelaney, Commented Dec 13, 2020 at 23:47
The package can't accept an open file object, and it requires the filepath. It even validates if the filepath has the correct extension — Davma
– Davma, Commented Dec 14, 2020 at 17:58
Found a solution by Microsoft to be able to store data on the temporary directory: learn.microsoft.com/en-us/answers/questions/42218/… — Davma
– Davma, Commented Dec 16, 2020 at 19:41

ti7 · Accepted Answer · 2020-12-13 23:46:11Z

4

You're probably looking for StringIO or BytesIO from io

import io

with io.BytesIO() as tmp:
    tmp.write(content)
    # to continue working, rewind file pointer
    tmp.seek(0)
    # work with tmp

pathlib may also be an advantage

answered Dec 13, 2020 at 23:46

ti7

19.8k8 gold badges50 silver badges87 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Davma Over a year ago

Thanks for the answer ti7. Unfortunately, the package can't accept an open file object such as io.BytesIO or io.StringIO, because the Python package validates the suffix based on the filename and the type of file based on the filepath.

Collectives™ on Stack Overflow

How to write file to memory filepath and read from memory filepath in Python?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related