HTTP Basics & urllib

HTTP (HyperText Transfer Protocol) is the foundation of data communication on the web. Python's standard library includes urllib for making HTTP requests without any third-party dependencies.

HTTP Methods

Each HTTP request uses a method (also called a verb) that describes the intended action:

REST APIs follow these conventions:

• GET /users → list all users
• POST /users → create a new user
• GET /users/42 → get user 42
• PUT /users/42 → replace user 42
• PATCH /users/42 → update specific fields of user 42
• DELETE /users/42 → delete user 42

HTTP Status Codes

The server always responds with a status code indicating success or failure:

2xx — Success

• 200 OK — request succeeded, body contains the result
• 201 Created — resource was successfully created (often after POST)
• 204 No Content — succeeded but no body (often after DELETE)

3xx — Redirection

• 301 Moved Permanently — resource has a new permanent URL
• 302 Found — temporary redirect

4xx — Client Errors (your fault)

• 400 Bad Request — malformed request syntax or invalid parameters
• 401 Unauthorized — authentication required or failed
• 403 Forbidden — authenticated but not permitted
• 404 Not Found — resource doesn't exist
• 422 Unprocessable Entity — valid syntax but semantic errors in body
• 429 Too Many Requests — rate limit exceeded

5xx — Server Errors (server's fault)

• 500 Internal Server Error — unexpected server-side crash
• 502 Bad Gateway — upstream server failure
• 503 Service Unavailable — server overloaded or in maintenance

Making a Simple GET Request

urllib.request.urlopen opens a URL and returns a response object:

The response object implements a context manager — always use with to ensure the connection is closed.

Reading JSON Responses

Most modern APIs return JSON. Use the json module to parse it:

Building Query Parameters with urllib.parse

Never manually concatenate query strings — use urllib.parse.urlencode:

Use urllib.parse.quote to percent-encode a single string value (e.g., a search term with spaces or special characters):

Custom Headers with urllib.request.Request

Use urllib.request.Request to add custom headers, change the method, or add a request body:

POST Requests with a JSON Body

Error Handling

urllib.error.HTTPError is raised for 4xx and 5xx responses. urllib.error.URLError is raised for network-level failures (DNS, connection refused, timeout):

Setting Timeouts

Prevent your program from hanging indefinitely by always specifying a timeout:

A Complete Example