Python with Statement

1The with Statement: overview

The with statement runs setup code before a block and guaranteed cleanup code after it, even when the block fails, by using an object called a context manager.

Before this lesson: Python Exceptions: try, except, else and finally, Python File Handling

2The with Statement: diagram

Quick reference: The with Statement

• Python File Handling Cheat Sheet: open, read, write, CSV, JSON — Open files with `with open(path, mode, encoding="utf-8") as f:` so they close automatically, read with `read(…
• How to Read a File Line by Line in Python — Open the file with `with open("file.txt") as f:` and loop over it: `for line in f:`. Each `line` is one line …
• How to Write to a File in Python — Use `with open("notes.txt", "w", encoding="utf-8") as f:` then `f.write("text\n")`. Mode `"w"` replaces the f…
• Python open() Function — `open(file, mode='r', encoding=None)` opens a file and returns a file object you can read from or write to; u…

4The with Statement: syntax

with open("data.txt") as f:        # __enter__ runs; f is what it returned
    text = f.read()                # do all the work inside the block
                                   # __exit__ runs here, always

with open("in.txt") as src, open("out.txt", "w") as dst:   # two managers
    dst.write(src.read())

class Managed:                     # class-based context manager
    def __enter__(self):
        ...                        # setup
        return self                # bound to the name after "as"
    def __exit__(self, exc_type, exc_value, traceback):
        ...                        # cleanup (all three are None if no error)
        return False               # True would swallow the exception

from contextlib import contextmanager, suppress, redirect_stdout

@contextmanager
def managed():                     # generator-based version
    ...                            # setup
    try:
        yield value                # value is bound by "as"
    finally:
        ...                        # cleanup, even after an error

with suppress(FileNotFoundError):  # ignore only this error
    os.remove("old.tmp")
with redirect_stdout(buffer):      # print() writes into buffer
    print("captured")
buffer = io.StringIO()             # in-memory text file: write(), getvalue()

5The with Statement: run the example

Press Run (or Ctrl + Enter). The output (step 6) appears directly below the code. Change anything and run it again.

with open("scores.txt", "w") as f:
    f.write("Asha,82\nBen,oops\nChen,91\n")

total = 0
try:
    with open("scores.txt") as f:
        for line in f:
            name, score = line.strip().split(",")
            total += int(score)
except ValueError as err:
    print("Bad data:", err)

print("Total before the error:", total)
print("File closed?", f.closed)

Expected outputBad data: invalid literal for int() with base 10: 'oops'
Total before the error: 82
File closed? True

7The with Statement: how the example works

Hover or tap a step to highlight its lines in the example above.

1. Line 1-2A file is written inside a with block and closed automatically when the block ends. The middle line, Ben,oops, has a score that is not a number.
2. Line 4total collects the scores that are read successfully.
3. Line 5-6The try block opens the file for reading. __enter__ returns the file object, which is stored in f.
4. Line 7-8The loop reads one line at a time. strip() removes the newline and split(",") separates the name from the score.
5. Line 9int(score) converts the text and adds it to total. Asha's 82 works, but on the second line int("oops") raises a ValueError. The error leaves the loop and the with block, and on its way out __exit__ closes the file.
6. Line 10-11The except block catches the error and prints its message. Chen's score was never reached.
7. Line 13Only Asha's score made it into total: 82.
8. Line 14f.closed is True: the file was closed even though an error escaped from inside the block. That guarantee is the whole point of with.

8The with Statement: common mistakes

with open("scores.txt", "w") as f:
    f.write("Asha,90\n")

with open("scores.txt") as f:
    print("Reading the file...")
text = f.read()
print(text)

with open("scores.txt", "w") as f:
    f.write("Asha,90\n")

with open("scores.txt") as f:
    print("Reading the file...")
    text = f.read()
print(text)

from contextlib import contextmanager

locked = []

@contextmanager
def lock(name):
    locked.append(name)
    yield
    locked.remove(name)

try:
    with lock("report"):
        raise RuntimeError("crash")
except RuntimeError:
    pass
print(locked)

from contextlib import contextmanager

locked = []

@contextmanager
def lock(name):
    locked.append(name)
    try:
        yield
    finally:
        locked.remove(name)

try:
    with lock("report"):
        raise RuntimeError("crash")
except RuntimeError:
    pass
print(locked)

class Timer:
    def __enter__(self):
        print("start")
        return self

    def __exit__(self):
        print("stop")

with Timer():
    print("working")

class Timer:
    def __enter__(self):
        print("start")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("stop")

with Timer():
    print("working")

class Quiet:
    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("cleaning up")
        return True

with Quiet():
    number = int("abc")
print("finished")

class Quiet:
    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("cleaning up")
        return False

try:
    with Quiet():
        number = int("abc")
except ValueError as err:
    print("Handled:", err)
print("finished")

with "notes.txt" as f:
    print(f.read())

with open("notes.txt", "w") as f:
    f.write("remember the milk\n")

with open("notes.txt") as f:
    print(f.read())

9The with Statement: exercises

Write your answer, then press Check. The checks run in your browser.

def save_names(path, names):
    # write one name per line
    pass

def load_names(path):
    # return a list of names without the newlines
    return []

save_names("team.txt", ["Asha", "Ben", "Chen"])
print(load_names("team.txt"))

def save_names(path, names):
    with open(path, "w") as f:
        for name in names:
            f.write(name + "\n")

def load_names(path):
    with open(path) as f:
        return [line.strip() for line in f]

save_names("team.txt", ["Asha", "Ben", "Chen"])
print(load_names("team.txt"))

class Guard:
    def __enter__(self):
        print("enter")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("exit:", exc_type.__name__ if exc_type else "clean")
        return exc_type is KeyError

with Guard():
    print("body one")

with Guard():
    {}["missing"]
    print("body two")

print("after")

enter
body one
exit: clean
enter
exit: KeyError
after

2 more exercises → 6 more examples

Further reading

• Input and output
• itertools
• Functional programming HOWTO
• pathlib

10Next lesson

Related:
Python Exceptions: try, except, else and finallyPython File HandlingPython pathlibPython Class Methods and Static Methods

← Python File Handling · All Iteration and Files lessons · Practise in the IDE