Python with Usage

What is the Python with statement

There are some tasks that may need to be set up beforehand and cleanup work done afterwards. For such scenarios, Python’s with statement provides a very convenient way to handle them. A good example is file processing, where you need to get a file handle, read data from the file, and then close the file handle.

Without the with statement, the code would look like this:

file = open("/tmp/foo.txt")
data = file.read()
file.close()

There are two problems here:

1. One is the possibility of forgetting to close the file handle.
2. Secondly, the file reads the data with an exception and no processing is done.

The following is an enhanced version of handling exceptions.

try:
    f = open('xxx')
except:
    print 'fail to open'
    exit(-1)
try:
    do something
except:
    do something
finally:
     f.close()

Although this code works well, it is too lengthy.

This is where with comes into its own. In addition to having a more elegant syntax, with is also great for handling exceptions generated by contextual environments.

The following is the with version of the code.

with open("/tmp/foo.txt") as file:
    data = file.read()

How Python with works

• After the statement immediately following with is evaluated, the __enter__() method of the returned object is called, and the return value of this method is assigned to the variable following as.
• The __exit__() method of the preceding returned object will be called when all the code blocks following the with have been executed.

The following example can specify how with works:

#!/usr/bin/env python
# with_example01.py
class Sample:
    def __enter__(self):
        print "In __enter__()"
        return "Foo"
    def __exit__(self, type, value, trace):
        print "In __exit__()"
def get_sample():
    return Sample()
with get_sample() as sample:
    print "sample:", sample

Run the code and the output is as follows:

bash-3.2$ ./with_example01.py
In __enter__()
sample: Foo
In __exit__()

As you can see:

1. the __enter__() method is executed
2. the value returned by the __enter__() method – in this case Foo – is assigned to the variable sample
3. the code block is executed, printing the value of the variable sample as Foo 4.
4. the __exit__() method is called and the real power of with is that it can handle exceptions.

You may have noticed that the __exit__ method of the Sample class has three parameters val, type and trace. These parameters are quite useful in exception handling. Let’s change the code to see exactly how it works.

#!/usr/bin/env python
# with_example02.py
class Sample:
    def __enter__(self):
        return self
    def __exit__(self, type, value, trace):
        print "type:", type
        print "value:", value
        print "trace:", trace
    def do_something(self):
        bar = 1/0
        return bar + 10
with Sample() as sample:
    sample.do_something()

In this example, get_sample() followed by with becomes Sample(). It doesn’t matter, as long as the object returned by the statement immediately following with has __enter__() and __exit__() methods. In this case, the __enter__() method of Sample() returns the newly created Sample object and assigns it to the variable sample.

After code execution:

bash-3.2$ ./with_example02.py
type: <type 'exceptions.ZeroDivisionError'>
value: integer division or modulo by zero
trace: <traceback object at 0x1004a8128>
Traceback (most recent call last):
  File "./with_example02.py", line 19, in <module>
    sample.do_something()
  File "./with_example02.py", line 15, in do_something
    bar = 1/0
ZeroDivisionError: integer division or modulo by zero

In fact, the __exit__() method is executed when any exception is thrown by the block following the with. As the example shows, when an exception is thrown, the type, value, and stack trace associated with it are passed to the __exit__() method, so the ZeroDivisionError exception thrown is printed out. When developing libraries, clean up resources, close files, etc. can be placed in the __exit__ method.

Also, __exit__ can be used for exception monitoring and handling in addition to tear things down, note the last few arguments. To skip an exception, just return True for that function.

The following sample code skips all TypeErrors and lets other exceptions be thrown normally.

def __exit__(self, type, value, traceback):
    return isinstance(value, TypeError)

As mentioned above, the __exit__ function can do some of the exception handling, if we don’t handle the exception in this function, it will be thrown normally, at this point we can write it like this (python 2.7 and above, previous versions refer to using the library function contextlib.nested)

try:
    with open( "a.txt" ) as f :
        do something
except xxxError:
    do something about exception

In short, the with-as expression greatly simplifies the work of writing a finally each time, which is a great help in keeping the code elegant.

If there is more than one item, we can write it like this.

with open("x.txt") as f1, open('xxx.txt') as f2:
    do something with f1,f2

Thus, Python’s with statement is an effective mechanism to provide a more concise code, while making cleanup simpler when exceptions are thrown.