Python requests: Call Web APIs
1Python Requests: overview
The requests library lets your Python code send HTTP requests to web servers and APIs and read their responses — most often JSON that turns straight into a Python dict or list.
Before this lesson: Python Dictionaries, Python Exceptions: try, except, else and finally, Working with JSON in Python
2Python Requests: diagram
3Python Requests: explained step by step
An API (application programming interface) is a web address that answers programs instead of people: instead of a web page you get data, usually as JSON. Weather services, GitHub and online shops all offer them. requests is the most popular Python library for talking to them.
Your first GET request
A GET request asks for data. requests.get(url) sends it and returns a Response object:
import requests
response = requests.get("https://jsonplaceholder.typicode.com/todos/1", timeout=10)
print(response.status_code)
todo = response.json()
print(todo["title"], "| done:", todo["completed"])response.json() parses the JSON body into a dict or list — the same as json.loads(response.text).
Query parameters and headers
Many APIs take options after a ? in the URL, such as ?userId=1&completed=false. Rather than gluing that string together yourself, pass a dict as params=; requests builds the URL and encodes special characters safely. Headers carry extra information about the request — which format you accept, or a token — and go in a dict passed as headers=.
Sending data with POST
A POST request sends data to the server, usually to create something. Pass a dict as json=: requests converts it to JSON text and sets the Content-Type header for you. Practice APIs such as JSONPlaceholder pretend to save the data and send it back.
Status codes and raise_for_status()
Every response has a status code: 200 OK, 201 created, 404 not found, 401/403 not allowed, 429 too many requests, and 500 or above for a server problem. requests does not raise an error for a 404 — it simply hands you the response. Call response.raise_for_status() to turn any 4xx or 5xx code into an HTTPError, or check response.ok yourself.
Timeouts and errors
Always pass timeout= (in seconds) so your program can't wait forever. Then wrap the call in try/except requests.exceptions.RequestException — the parent class of every requests error, including connection failures, timeouts and HTTPError:
try:
response = requests.get(url, timeout=10)
response.raise_for_status()
data = response.json()
except requests.exceptions.RequestException as err:
print("Request failed:", err)
Requests from the browser and CORS
On this site your Python runs inside your web browser, so the browser sends the request for it. For safety, browsers only let a page read a response from another website if that website allows it with a CORS (cross-origin resource sharing) header. Many public APIs do; most ordinary websites don't. When they don't, the request fails with a network error even though the same URL opens fine in your address bar. The APIs on the Public APIs page are known to work.
This also means anything in front-end code — including Python running in a browser — can be read by whoever opens the page. Never put secret API keys or passwords in it. Code that needs a secret belongs on a server.
On your own computer, install the library with pip install requests; the same code works there without the CORS limits.
4Python Requests: syntax
import requests
r = requests.get(url, params={"key": "value"}, headers={...}, timeout=10)
r = requests.post(url, json={"title": "Hi"}, timeout=10)
r.status_code # 200, 404, ...
r.ok # True when the status code is below 400
r.raise_for_status() # raise HTTPError for 4xx / 5xx
r.json() # JSON body -> dict / list
r.text # body as a string
r.url # the final URL, including params
except requests.exceptions.RequestException as err: # any requests error
5Python Requests: run the example
Press Run (or Ctrl + Enter). The output (step 6) appears directly below the code. Change anything and run it again.
import requests
url = "https://jsonplaceholder.typicode.com/todos"
params = {"userId": 1, "completed": "false"}
try:
response = requests.get(url, params=params, timeout=10)
response.raise_for_status()
todos = response.json()
except requests.exceptions.RequestException as err:
print("Request failed:", err)
else:
print("Status:", response.status_code)
print("URL:", response.url)
print("Open tasks:", len(todos))
for todo in todos[:3]:
print("-", todo["title"])Expected outputStatus: 200
URL: https://jsonplaceholder.typicode.com/todos?userId=1&completed=false
Open tasks: 9
- delectus aut autem
- quis ut nam facilis et officia qui
- fugiat veniam minus
7Python Requests: how the example works
Hover or tap a step to highlight its lines in the example above.
- Line 1Import the library. On this site it is loaded automatically the first time you import it.
- Line 3-4The endpoint URL and a dict of query parameters: we want user 1's todos that are not completed yet.
- Line 6Anything can go wrong on a network, so the request goes inside
try. - Line 7
requests.get()adds?userId=1&completed=falseto the URL, sends the request and waits at most 10 seconds for an answer. - Line 8
raise_for_status()does nothing for a200response but raisesHTTPErrorfor codes such as 404 or 500. - Line 9
.json()parses the response body into a Python list of dicts. - Line 10-11
RequestExceptioncatches every requests error: no connection, a CORS block, a timeout or a bad status code. - Line 12The
elseblock only runs when thetryblock raised no exception. - Line 13-14The status code and the final URL show exactly what was requested and how the server answered.
- Line 15-17The data is now an ordinary list:
len()counts the todos and a slice prints the first three titles.
8Python Requests: common mistakes
Reading the data without checking the status
✗ Wrong
import requests
r = requests.get("https://jsonplaceholder.typicode.com/todos/99999", timeout=10)
todo = r.json()
print(todo["title"])✓ Right
import requests
try:
r = requests.get("https://jsonplaceholder.typicode.com/todos/99999", timeout=10)
r.raise_for_status()
print(r.json()["title"])
except requests.exceptions.HTTPError as err:
print("API error:", err.response.status_code)A 404 is not an exception in requests, so the code carried on with an empty body. Call raise_for_status() (or check r.ok) before using the data.
Sending JSON with data= instead of json=
✗ Wrong
import requests
r = requests.post("https://httpbin.org/post", data={"name": "Asha", "age": 21}, timeout=10)
print(r.json()["json"])✓ Right
import requests
r = requests.post("https://httpbin.org/post", json={"name": "Asha", "age": 21}, timeout=10)
print(r.json()["json"])data= sends the dict as form fields. Use json= to send a JSON body with the right Content-Type header.
Forgetting the parentheses on .json()
✗ Wrong
import requests
r = requests.get("https://jsonplaceholder.typicode.com/todos/1", timeout=10)
todo = r.json
print(todo["title"])✓ Right
import requests
r = requests.get("https://jsonplaceholder.typicode.com/todos/1", timeout=10)
todo = r.json()
print(todo["title"])json is a method. Without () you get the method itself instead of the parsed data.
9Python Requests: exercises
Write your answer, then press Check. The checks run in your browser.
1. Explain a status code
Write code Easy ✓ CompletedWrite describe_status(code) that returns:
"success"for codes 200–299"redirect"for 300–399"client error"for 400–499"server error"for 500–599"unknown"for anything else
def describe_status(code):
# return "success", "redirect", "client error", "server error" or "unknown"
pass
for code in [200, 404, 503]:
print(code, describe_status(code))
200 <= code <= 299. Check each range in turn and return as soon as one matches.def describe_status(code):
if 200 <= code <= 299:
return "success"
if 300 <= code <= 399:
return "redirect"
if 400 <= code <= 499:
return "client error"
if 500 <= code <= 599:
return "server error"
return "unknown"
for code in [200, 404, 503]:
print(code, describe_status(code))
2. Read a weather response
Write code Medium ✓ Completedbody is the JSON text an Open-Meteo request returns (so you can practise offline). Write current_temperature(body) that parses it and returns the current temperature followed by a space and its unit, for example "31.4 °C". The value is in current → temperature_2m and the unit in current_units → temperature_2m.
import json
body = '{"latitude": 28.625, "longitude": 77.25, "current_units": {"time": "iso8601", "temperature_2m": "°C"}, "current": {"time": "2024-05-17T10:00", "temperature_2m": 31.4}}'
def current_temperature(body):
# parse the JSON text (what response.json() would do), then dig into it
pass
print(current_temperature(body))
json.loads(body) gives a dict. Then use two keys in a row, like data["current"]["temperature_2m"].import json
body = '{"latitude": 28.625, "longitude": 77.25, "current_units": {"time": "iso8601", "temperature_2m": "°C"}, "current": {"time": "2024-05-17T10:00", "temperature_2m": 31.4}}'
def current_temperature(body):
data = json.loads(body)
value = data["current"]["temperature_2m"]
unit = data["current_units"]["temperature_2m"]
return f"{value} {unit}"
print(current_temperature(body))
2 more exercises → 4 more examples
Questions learners ask
Why does a URL open in my address bar but fail from Python on this page?
Your address bar visits the site directly, but code on a page is bound by the browser's CORS rules: the browser only hands the response to the page if the API sends a header that allows cross-origin requests. If it doesn't, you get a network error. Use an API that allows CORS, or run the code in desktop Python, where CORS doesn't apply.
Do I need to install requests?
Not on this site: it loads automatically the first time you import it. On your own computer run pip install requests once (python -m pip install requests if pip isn't on your PATH).
What is the difference between params=, data= and json=?
params= adds query parameters to the URL, which is typical for GET. json= sends a JSON body, which most modern APIs expect for POST and PUT. data= sends form fields, like an HTML form does.
Further reading
10Next lesson
Up nextPython API Example: Build an API ClientAn API client is a small program that sends HTTP requests to a web service, checks the reply, turns the JSON into Pytho…Related:
Python Exceptions: try, except, else and finallyWorking with JSON in PythonPython API Example: Build an API Client
← Python Regular Expressions (regex) · All Practical Python lessons · Practise in the IDE