8

Let's say I want to open a text file for reading using the following syntax:

with open(fname,'r') as f:
    # do something
    pass

But if I detect that it ends with .gz, I would call gzip.open(). 

if fname.endswith('.gz'):
    with gzip.open(fname,'rt') as f:
            # do something
            pass
else:
    with open(fname,'r') as f:
            # do something
            pass

If "do something" part is long and not convenient to write in a function (e.g. it would create a nested function, which cannot be serialized), what is the shortest way to call with either gzip.open or open based on the return of fname.endswith('.gz')?

Improve this question
1

3 Answers 3

Reset to default
11

The context manager helps close the object.

You don't have to create the object used as a context manager, at the same time you use with to enter the context, though. The open() and gzip.open() calls return a new object that happens to be a context manager, and you can create them before you enter the context:

if fname.endswith('.gz'):
    f = gzip.open(fname,'rt')
else:
    f = open(fname, 'r')

with f:
    # do something

In both cases, the object returns self on entering the context, so there is no need to use as f here.

Also, functions are first-class citizens, so you can also use a variable to store the function and then call that in the with statement to create the context manager and file object:

if fname.endswith('.gz'):
    opener = gzip.open
else:
    opener = open

with opener(fname, 'rt') as f:  # yes, both open and gzip.open support mode='rt'
    # do something

This doesn't really buy you anything over the other method here, but you could use a dictionary to map extensions to callables if you so desire.

The bottom line is that with calls context-manager hook methods, nothing less, nothing more. The expression after with is supposed to supply such a manager, but creating that object is not subject to the context management protocol.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Thanks! Can you just explain why "opener, mode = open"? How is it possible that "opener = open" wouldn't work?
@CindyAlmighty: that was an editing error, sorry. I originally assigned both an opener and a mode (so opener, mode = gzip.open, 'rt'), but then realised that open() accepts rt to mean the same thing as r for the mode.
9

You can bind either context manager to the same name and choose early:

if fname.endswith('.gz'):
    context = gzip.open(fname,'rt')
else:
    context = open(fname,'r')

with context as f:
    # do the same thing in either case

This allows for some nice patterns, for instance if the input is possibly an opened file handle then you could use contextlib.nullcontext to get a no-op in the with block for the given case.

Improve this answer

5 Comments

It's one of the beautiful things about Python because everything is an object you can get away with lots of neat tricks like this. Another favorite of mine is using dicts as a switch.
1. can you explain the last sentence please? :) 2. @Idlehands What is this use of dict as a switch?
Here's an answer that utilizes the dict object as a switch: stackoverflow.com/questions/60208/…
@CindyAlmighty it isn't directly applicable to your use case. But imagine the scenario when you have "My fname is either a file path that needs to be opened or it has already been opened elsewhere by open(the_actual_file_name)". In this case you could do, say context = gzip.open(fname, 'rt') if isinstance(fname, str) else contextlib.nullcontext(fname) and use a single with context as f: block which will work in both cases just fine. This is exactly the second example given in the docs, actually.
@Idlehands. very true, and Python is usually pretty clear in its intent. but context managers are a bit unusual in their implicit enter/exit triggering. It’s not all that obvious that the context creation can syntactically be separated from as, so this post is extra informative - despite using them for a while this is a new trick for me.
2 
with gzip.open(fname, 'rt') if fname.endswith('.gz') else open(fname, 'r') as f:
    # do something
    pass
Improve this answer

6 Comments

Why the need to put everything on one line?
No real need of course, which is your (valid) point. But arguably it answers the actual question better, as the question mentions: "[...] what is the shortest way [...]"
I did not say that. It was the OP who asked that.
I didn't say you said that.
Although this is not as useful, it truly is what I asked. And also shows another possibility! A great answer IMHO.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.