Requests and Httpx
Both
requests
and httpx
are powerful HTTP clients in Python. requests
is widely used, while httpx
is an advanced alternative that supports asynchronous requests and HTTP/2.
1. Installing requests
and httpx
pip install requests httpx
2. Basic Usage: GET and POST Requests
Using requests
import requests
# GET Request
= requests.get("https://jsonplaceholder.typicode.com/todos/1")
response print(response.json()) # Convert response to JSON
# POST Request
= {"title": "New Task", "completed": False}
data = requests.post("https://jsonplaceholder.typicode.com/todos", json=data)
response print(response.status_code, response.json())
Using httpx
(Sync)
import httpx
# GET Request
= httpx.get("https://jsonplaceholder.typicode.com/todos/1")
response print(response.json())
# POST Request
= httpx.post("https://jsonplaceholder.typicode.com/todos", json=data)
response print(response.status_code, response.json())
Using httpx
(Async)
import httpx
import asyncio
async def fetch():
async with httpx.AsyncClient() as client:
= await client.get("https://jsonplaceholder.typicode.com/todos/1")
response print(response.json())
asyncio.run(fetch())
3. Key Differences Between requests
and httpx
Feature | requests |
httpx |
---|---|---|
Sync Support | ✅ Yes | ✅ Yes |
Async Support | ❌ No | ✅ Yes (AsyncClient ) |
HTTP/2 Support | ❌ No | ✅ Yes |
Connection Pooling | ❌ No | ✅ Yes |
Timeouts | ✅ Yes | ✅ Yes |
Streaming Support | ✅ Yes | ✅ Yes |
Cookies & Sessions | ✅ Yes | ✅ Yes |
5. File Uploads
= {"file": open("example.txt", "rb")}
files
# Using requests
= requests.post("https://api.example.com/upload", files=files)
response print(response.json())
# Using httpx
= httpx.post("https://api.example.com/upload", files=files)
response print(response.json())
6. Handling Timeouts and Retries
import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
= requests.Session()
session = Retry(total=3, backoff_factor=1, status_forcelist=[500, 502, 503, 504])
retries "https://", HTTPAdapter(max_retries=retries))
session.mount(
try:
= session.get("https://api.example.com/data", timeout=5)
response print(response.json())
except requests.exceptions.Timeout:
print("Request timed out!")
import httpx
= httpx.Client(timeout=httpx.Timeout(10.0))
client
try:
= client.get("https://api.example.com/data")
response print(response.json())
except httpx.TimeoutException:
print("Request timed out!")
7. Authentication (Basic, Token, OAuth)
Basic Authentication
from requests.auth import HTTPBasicAuth
import requests
= requests.get("https://api.example.com/protected", auth=HTTPBasicAuth("user", "pass"))
response print(response.status_code)
import httpx
= httpx.get("https://api.example.com/protected", auth=("user", "pass"))
response print(response.status_code)
Bearer Token Authentication
= {"Authorization": "Bearer my_token"}
headers = requests.get("https://api.example.com/data", headers=headers)
response print(response.json())
= httpx.get("https://api.example.com/data", headers=headers)
response print(response.json())
8. Handling JSON Responses & Errors
try:
= requests.get("https://api.example.com/data")
response # Raises error if status code is 4xx or 5xx
response.raise_for_status() = response.json()
data except requests.exceptions.HTTPError as err:
print(f"HTTP Error: {err}")
try:
= httpx.get("https://api.example.com/data")
response
response.raise_for_status()= response.json()
data except httpx.HTTPStatusError as err:
print(f"HTTP Error: {err}")
9. Streaming Large Responses
# With requests
with requests.get("https://api.example.com/largefile", stream=True) as r:
for chunk in r.iter_content(chunk_size=1024):
print(chunk)
# With httpx
with httpx.stream("GET", "https://api.example.com/largefile") as response:
for chunk in response.iter_bytes():
print(chunk)
10. Using Proxy Servers
= {
proxies "http": "http://proxy.example.com:8080",
"https": "https://proxy.example.com:8080"
}
# Using requests
= requests.get("https://api.example.com/data", proxies=proxies)
response
# Using httpx
= httpx.get("https://api.example.com/data", proxies=proxies) response
11. HTTP/2 and Connection Pooling (httpx
Only)
import httpx
# Enable HTTP/2
= httpx.Client(http2=True)
client = client.get("https://api.example.com/data")
response print(response.http_version) # Should return 'HTTP/2'
12. Benchmarking: Speed Comparison
```python import time import requests import httpx
url = “https://jsonplaceholder.typicode.com/todos/1”
Using requests
start = time.time() requests.get(url) print(“requests:”, time.time() - start)
Using httpx (sync)
start = time.time() httpx.get(url