Pyfied
Log in
Python Generators
Core Python Iteration and Files

Python Generators, Iterators and yield

By (LinkedIn) · Every code example is run and checked on Python 3.13 (how we test) · Last updated

1Python Generators: overview

A generator is a function that uses yield to hand out values one at a time, pausing after each one and resuming where it left off when the next value is requested.

GoalBy the end you can explain iterables and iterators, use iter() and next(), write a generator function with yield, use generator expressions, and say when a generator saves memory.

Before this lesson: Python Functions, Python List Comprehensions

2Python Generators: diagram

Python Generators, Iterators and yieldgen = countdown(3)nothing runs yetnext(gen)resume the bodyyield nhand out n, pausedonebody endsStopIterationpaused until the next next() call
Calling a generator function only creates the generator. Each next() runs the body up to the next yield, hands out that value and pauses there. When the body finishes, the generator raises StopIteration and a for loop stops quietly.

3Python Generators: explained step by step

Every for loop you've written has quietly used an iterator. Once you understand iterators, generators follow naturally: they are functions that produce values one at a time instead of building a whole list in memory first.

Iterables, iterators, iter() and next()

An iterable is anything you can loop over: a list, a string, a dict, a range, a file. Calling iter() on it gives you an iterator, an object that hands out the items one by one each time you call next() on it. When nothing is left, next() raises StopIteration.

Python 
colors = ["red", "green", "blue"]
it = iter(colors)
print(next(it))
print(next(it))
print(next(it))
try:
    next(it)
except StopIteration:
    print("StopIteration: nothing left")





That's exactly what for color in colors: does behind the scenes. It calls iter() once, then calls next() again and again, and stops quietly when StopIteration arrives. An iterator only moves forward, and once it's used up it stays empty.


Generator functions and yield


A function that contains yield is a generator function. Calling it does not run its body. Instead it returns a generator object, which is an iterator. Each next() runs the body until it reaches a yield. The yielded value is handed out and the function pauses right there, keeping all its local variables. The following next() resumes on the line after the yield.



  

    Python
    

      
      
      
      
    

  

  
def countdown(start):
    print("Starting")
    n = start
    while n > 0:
        yield n
        n -= 1
    print("Done")

gen = countdown(3)
print(next(gen))
print(next(gen))
for value in gen:
    print("loop got", value)

  
  
  
  

    
      
      
    
    
  

  





Notice that Starting only appears when the first value is requested, and that the for loop carries on from where the two next() calls left off. When the function body finishes, the generator raises StopIteration and the loop ends.


Generator expressions


A generator expression looks like a list comprehension with round brackets: (n * n for n in numbers). It works out each value only when it's asked for. When it's the only argument to a function, you can drop the extra brackets: sum(n * n for n in numbers).


Why generators save memory


A list keeps every item in memory at the same time. A generator only remembers where it is, so it stays tiny however many values it will produce:



  

    Python
    

      
      
      
      
    

  

  
import sys

squares_list = [n * n for n in range(1_000_000)]
squares_gen = (n * n for n in range(1_000_000))

print(sys.getsizeof(squares_list), "bytes for the list")
print(sys.getsizeof(squares_gen), "bytes for the generator")
print(sum(squares_gen))

  
  
  
  

    
      
      
    
    
  

  





Both give the same total, but the generator never stores a million numbers. The trade-off is that a generator can be looped over only once, and it has no len() and no indexing. If you need the values more than once, make a list.

      


      
      

        
Quick reference: Python Generators

        
      

      

      
      

        
4Python Generators: syntax

        
it = iter(iterable)            # get an iterator
next(it)                       # next value, or StopIteration

def numbers(limit):            # generator function
    n = 1
    while n <= limit:
        yield n                # hand out n and pause here
        n += 1

gen = numbers(3)               # creates the generator, runs nothing
for n in gen: ...              # the loop calls next() for you
(x * 2 for x in data)          # generator expression

      

      

      
      

        
5Python Generators: run the example

        
Press Run (or Ctrl + Enter). The output (step 6) appears directly below the code. Change anything and run it again.

        

  

    Interactive example
    

      
      
      
      
    

  

  
def read_lines(text):
    for number, line in enumerate(text.splitlines(), start=1):
        print(f"  (reading line {number})")
        yield line

log = "INFO start\nERROR disk full\nINFO retry\nERROR timeout"

lines = read_lines(log)
print("Generator created:", type(lines).__name__)

first = next(lines)
print("First line:", first)

errors = (line for line in lines if line.startswith("ERROR"))
for err in errors:
    print("Found:", err)

print("Lines left:", sum(1 for _ in lines))

  
  
  
  

    
      
      
    
    
  

  
Expected outputGenerator created: generator
  (reading line 1)
First line: INFO start
  (reading line 2)
Found: ERROR disk full
  (reading line 3)
  (reading line 4)
Found: ERROR timeout
Lines left: 0




      

      

      
      

        
7Python Generators: how the example works

        
Hover or tap a step to highlight its lines in the example above.

        
    
          
          
  1. Line 1-4yield makes read_lines a generator function. The print inside shows exactly when each line is actually read.
    2. 
          
          
  2. Line 6A small log as one string. In a real program this could be a huge file that you don't want to load all at once.
    3. 
          
          
  3. Line 8-9Calling read_lines(log) runs none of its body: no "reading line" message appears. It just returns a generator object.
    4. 
          
          
  4. Line 11-12next(lines) runs the body until the first yield. Line 1 is read and handed out, and the function pauses.
    5. 
          
          
  5. Line 14A generator expression that will filter the remaining lines. Creating it reads nothing yet.
    6. 
          
          
  6. Line 15-16The for loop pulls values through both generators one at a time. Each line is read just before it's checked, and only ERROR lines are printed.
    7. 
          
          
  7. Line 18The generator is now used up, so counting the lines that are left gives 0. A generator can only be looped over once.
    8. 
          
        

      

      

      
      

        
8Python Generators: common mistakes

        
        

          
Looping over the same generator twice

          

            
✗ Wrong
squares = (n * n for n in range(4))
print(sum(squares))
print(sum(squares))
Prints 14 and then 0: the second sum() finds the generator already used up

            
✓ Right
squares = [n * n for n in range(4)]
print(sum(squares))
print(sum(squares))

          

          
A generator hands out each value only once. If you need the values more than once, store them in a list, or create a fresh generator each time.

        

        
        

          
Using return instead of yield

          

            
✗ Wrong
def evens(limit):
    for n in range(0, limit, 2):
        return n

print(list(evens(10)))
TypeError: 'int' object is not iterable

            
✓ Right
def evens(limit):
    for n in range(0, limit, 2):
        yield n

print(list(evens(10)))

          

          
return ends the function on the very first value, so evens() returns the plain number 0. yield hands out a value and keeps the function going.

        

        
        

          
Asking a generator for its length or an index

          

            
✗ Wrong
shouts = (word.upper() for word in ["hi", "hey"])
print(len(shouts))
print(shouts[0])
TypeError: object of type 'generator' has no len()

            
✓ Right
shouts = list(word.upper() for word in ["hi", "hey"])
print(len(shouts))
print(shouts[0])

          

          
A generator doesn't know how many values it will produce, and it can't jump to a position. Convert it with list() when you need len() or indexing.

        

        
      

      

      


      

        
9Python Generators: exercises

        
          
Write your answer, then press Check. The checks run in your browser.

          

  

    
1. Your first generator

    Write code
    Easy
    
  

  
Write a generator function count_up(limit) that yields the numbers 1, 2, 3, … up to and including limit. For example list(count_up(3)) is [1, 2, 3]. Use yield, not a list.

  
    

  

    Your code
    

      
      
      
      
    

  

  
def count_up(limit):
    # yield 1, 2, ..., limit
    pass

for n in count_up(3):
    print(n)

  
  
  
  

    
      
      
      
      
    
    
  

  




  
  

  
  
  




  

    
2. Predict the pauses

    Predict the output
    Medium
    
  

  
Read the code and type exactly what it will print. Think about when the body of letters() actually runs. Then press Check to compare.

  
    

  

    Read this code
    

      
      
    

  

  
def letters():
    print("start")
    yield "a"
    print("middle")
    yield "b"
    print("end")

gen = letters()
print("created")
print(next(gen))
print(next(gen))
print(list(gen))

  
  
  

  

    
      
      
      
      
    
    
  

  




  
  

  
  
  



          
2 more exercises → 4 more examples

        
        

          
Finished this lesson?
Progress is saved in this browser — create a free account to keep it across devices.

          
        

      


      
      

        
Questions learners ask

        
What is the difference between yield and return?
return sends back one value and ends the function for good. yield sends back a value but only pauses the function, so the next request continues from the line after the yield with all local variables intact. A function with yield in it becomes a generator function.
When should I use a generator instead of a list?
Use a generator when you only need to go through the values once, especially when there are a lot of them or they come from a big file or an endless source. Use a list when you need len(), indexing, sorting or several passes over the same data.
Is a generator an iterator?
Yes. Every generator is an iterator: it supports next() and raises StopIteration at the end. Not every iterable is an iterator, though. Lists and range objects are iterables that give you a fresh iterator each time you loop, which is why you can loop over a list many times but a generator only once.

      

      

      
      
      

        
Further reading

        
      

      

      

        
10Next lesson

        
        Up nextPython IteratorsAn iterator is an object that hands out the items of a sequence one at a time through next(), remembers where it is, an…
        
        
Related:
Python List ComprehensionsPython IteratorsPython itertoolsPython File Handling

        
← Python List Comprehensions · All Iteration and Files lessons · Practise in the IDE