JSON Parsing with Python [Practical Guide]

Use the json module. It is part of Python's standard library, so there is nothing to install. json.loads() turns a JSON string into Python objects, and json.load() does the same for a file (Python json documentation).

In this guide, we’ll explore the syntax and data types of JSON, as well as the Python libraries and methods used for parsing JSON data, including more advanced options like JMESPath and ChompJS, which are very useful for web scraping data.

1import json
2
3text = '{"title": "A Light in the Attic", "price": 51.77, "in_stock": true, "category": "Poetry"}'
4book = json.loads(text)
5
6print(book["title"])     # A Light in the Attic
7print(book["price"])     # 51.77
8print(book["in_stock"])  # True
9print(type(book))        # <class 'dict'>

A JSON object becomes a Python dict and a JSON array becomes a list. From there it is ordinary dictionary and list work.

Which json function do you need?

Four functions cover reading and writing (Python json documentation).

The keyword arguments used further down, indent, ensure_ascii, default, object_hook and parse_float, are documented on the same page.

What does each JSON value become in Python?

That is the decoder's conversion table in the Python json documentation. Going the other way, a Python tuple is written as a JSON array, the same as a list, so a tuple comes back from a round trip as a list.

However, it is essential to remember that JSON is not the same as a Python dictionary. When ready to save the data, you will need to convert it back to JSON format using the json dumps function.

Four behaviours of the module go beyond that table, and the Python json documentation states each one.

• Keys come back as strings. "Keys in key/value pairs of JSON are always of the type str. When a dictionary is converted into JSON, all the keys of the dictionary are converted to strings." (Python json documentation)
• A repeated key keeps its last value. By default the module "ignores all but the last name-value pair for a given name" (Python json documentation). Pass object_pairs_hook to see every pair.
• NaN and Infinity are accepted. The decoder "understands NaN, Infinity, and -Infinity as their corresponding float values, which is outside the JSON spec", and the encoder writes them unless you pass allow_nan=False (Python json documentation). The JSON standard does not permit them (RFC 8259, section 6).
• A number with a fraction becomes a binary float. For prices, pass parse_float=decimal.Decimal and the value arrives exact. The decimal documentation states that "numbers like 1.1 and 2.2 do not have exact representations in binary floating point" (Python decimal documentation).

1import json
2from decimal import Decimal
3
4print(json.loads(json.dumps({1: "a", "t": (1, 2)})))       # {'1': 'a', 't': [1, 2]}
5print(json.loads('{"x": 1, "x": 2}'))                      # {'x': 2}
6print(json.loads('{"price": 51.77}'))                      # {'price': 51.77}
7print(json.loads('{"price": 51.77}', parse_float=Decimal))  # {'price': Decimal('51.77')}

How do you read a JSON file?

Open it with an explicit encoding and pass the file object to json.load(). Following, we display an example using the load method.

1import json
2
3with open("book.json", encoding="utf-8") as f:
4    book = json.load(f)
5
6print(book["title"])  # A Light in the Attic

We load the data using the with open() context manager and json.load() to load the contents of the JSON file into a Python dictionary.

Name the encoding. The JSON standard says that JSON "exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8" (RFC 8259, section 8.1). Without an encoding argument, open() uses an encoding that the Python documentation calls "platform-dependent" (Python open() documentation).

If the file starts with a byte order mark, json.load() stops with Unexpected UTF-8 BOM (decode using utf-8-sig). The Python json documentation states that the module's deserializer "raises a ValueError when an initial BOM is present". Open the file with encoding="utf-8-sig" instead: on decoding, that codec skips "an optional UTF-8 encoded BOM at the start of the data" (Python codecs documentation).

How do you get a value out of nested JSON?

Chain the keys and indexes, one level at a time. This is a shortened copy of the sample document that httpbin.org, a public HTTP request and response testing service, returns from /json. Accessing elements within nested JSON structures requires using multiple keys or indices to traverse through the structure.

1import json
2
3text = """
4{
5  "slideshow": {
6    "author": "Yours Truly",
7    "title": "Sample Slide Show",
8    "slides": [
9      {"title": "Wake up to WonderWidgets!", "type": "all"},
10      {
11        "title": "Overview",
12        "type": "all",
13        "items": ["Why <em>WonderWidgets</em> are great", "Who <em>buys</em> WonderWidgets"]
14      }
15    ]
16  }
17}
18"""
19data = json.loads(text)
20
21print(data["slideshow"]["title"])               # Sample Slide Show
22print(data["slideshow"]["slides"][1]["title"])  # Overview
23
24for slide in data["slideshow"]["slides"]:
25    print(slide["title"], len(slide.get("items", [])))
26# Wake up to WonderWidgets! 0
27# Overview 2

The second slide has an items list and the first does not. slide["items"] on the first slide raises KeyError. slide.get("items", []) returns the default instead, "so that this method never raises a KeyError" (Python dict documentation).

Python's glossary names the two ways to handle a key that may be missing. EAFP "assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false". LBYL "explicitly tests for pre-conditions before making calls or lookups" (Python glossary). Both give the same result here:

1first = data["slideshow"]["slides"][0]
2
3try:
4    items = first["items"]
5except KeyError:
6    items = []
7
8items = first.get("items", [])  # the same result in one line
9print(items)                    # []

Your JSON will not parse. What does the error mean?

Every parsing failure raises json.JSONDecodeError, a subclass of ValueError. It carries msg, lineno, colno and pos, the index in the document where parsing failed (Python json documentation). Print them before you change anything.

1import json
2
3bad = "{'title': 'A Light in the Attic'}"
4
5try:
6    json.loads(bad)
7except json.JSONDecodeError as err:
8    print(f"{err.msg} at line {err.lineno}, column {err.colno}")
9    # Expecting property name enclosed in double quotes at line 1, column 2
10    print(err.doc[err.pos:err.pos + 20])
11    # 'title': 'A Light in

These are the messages Python 3.13.15 prints for the inputs in the second column. The message strings are in CPython's JSON decoder (CPython 3.13 source, Lib/json/decoder.py).

How do you parse JSON from an API response?

Check the status first, then call .json().

1import requests
2
3response = requests.get("https://httpbin.org/json", timeout=30)
4response.raise_for_status()
5
6try:
7    data = response.json()
8except requests.exceptions.JSONDecodeError:
9    print("Not JSON. The body starts with:", response.text[:200])
10    raise
11
12print(data["slideshow"]["title"])  # Sample Slide Show

raise_for_status() comes first because the Requests documentation says that "the success of the call to r.json() does not indicate the success of the response" and that "Some servers may return a JSON object in a failed response (e.g. error details with HTTP 500)." When the body is not valid JSON, .json() raises requests.exceptions.JSONDecodeError (Requests API reference).

If the body is an HTML page and the status is 403, why Python requests gets a 403 covers the causes. If the API answers in XML, XML parsing with Python covers the standard library's parser.

In a Scrapy spider, response.json() does the same job inside a callback, and the result is cached after the first call (Scrapy request and response documentation).

How do you change the data and write it back to a file?

Edit the dictionary, then write it with json.dump().

Working with JSON in Python also involves modifying the data by adding, updating or deleting elements. For those three jobs, we will be using the json built-in package, as it provides all basic functions we require to accomplish these tasks.

To add an element, you can modify the corresponding mapping in the JSON object using standard dictionary syntax. Another approach to either adding and/or updating values into a python dictionary is using the update() method. It will add or update elements in the dictionary using the values from another dictionary, or with an iterable containing key-value pairs.

To remove an element from a JSON object, you can use the del keyword to delete the corresponding value. Another approach to removing an element from a dictionary with JSON data is to use the pop method, which allows you to retrieve the value and use it at the same time it is removed.

After tweaking with a previous JSON file or JSON string, you may want to save your modified data back to a JSON file or export it as a JSON string to store data. The json.dump() method allows you to save a JSON object to a file, while json.dumps() returns a JSON string representation of an object.

1import json
2
3book = {"title": "A Light in the Attic", "price": 51.77, "in_stock": True}
4
5book["currency"] = "GBP"             # add a key
6book.update({"category": "Poetry"})  # add or replace several keys at once
7del book["in_stock"]                 # remove a key; KeyError if it is missing
8rating = book.pop("rating", None)    # remove if present; returns None if not
9
10with open("book.json", "w", encoding="utf-8") as f:
11    json.dump(book, f, indent=2, ensure_ascii=False)
12
13print(json.dumps({"symbol": "£"}))                      # {"symbol": "\\u00a3"}
14print(json.dumps({"symbol": "£"}, ensure_ascii=False))  # {"symbol": "£"}

Beware, trying to remove an element using del when the element is not present will raise a KeyError exception. pop() with a default does not. The documentation's rule for pop() is that "If default is not given and key is not in the dictionary, a KeyError is raised" (Python dict documentation).

indent=2 pretty-prints with two spaces per level. ensure_ascii is true by default, and then "the output is guaranteed to have all incoming non-ASCII and non-printable characters escaped" (Python json documentation). Pass ensure_ascii=False to keep characters such as £ as they are, and open the file with encoding="utf-8" when you do.

Do not add a second document to the end of a JSON file. The documentation puts it this way: "Unlike pickle and marshal, JSON is not a framed protocol, so trying to serialize multiple objects with repeated calls to dump() using the same fp will result in an invalid JSON file" (Python json documentation). To add records over time, write JSON Lines, covered further down.

Why does json.dumps refuse a datetime, a Decimal or your own class?

Because JSON has no type for them. The standard says JSON "can represent four primitive types (strings, numbers, booleans, and null) and two structured types (objects and arrays)" (RFC 8259, section 1). By default, the json module in Python can only serialize certain types of objects, like strings, numbers, and lists/dictionaries. On Python 3.13.15, json.dumps() on a datetime raised TypeError: Object of type datetime is not JSON serializable, and a Decimal and a set failed the same way.

Pass a function as default. The encoder calls it "for objects that can't otherwise be serialized", and it "should return a JSON encodable version of the object or raise a TypeError". To rebuild your objects on the way back in, pass object_hook: it is called with every decoded object, and its return value "will be used instead of the dict" (Python json documentation).

We can create a custom encoder by inheriting from json.JSONEncoder and overriding the default method. This allows us to convert python objects into JSON strings. The default method is called by the JSON encoder for objects that are not serializable by default. By using a custom encoding class, we can customize how our objects are serialized to JSON and handle any special cases that may not be covered by the default encoding behavior.

Just as we can use custom encoding classes to serialize custom objects to JSON, we can also use custom decoding classes to decode JSON strings back into our custom objects.

1import json
2from dataclasses import dataclass, asdict
3from datetime import datetime, timezone
4from decimal import Decimal
5
6
7@dataclass
8class Book:
9    title: str
10    price: Decimal
11    scraped_at: datetime
12
13
14def to_json(obj):
15    if isinstance(obj, Book):
16        return {"__type__": "Book", **asdict(obj)}
17    if isinstance(obj, Decimal):
18        return str(obj)
19    if isinstance(obj, datetime):
20        return obj.isoformat()
21    raise TypeError(f"Cannot serialize {type(obj).__name__}")
22
23
24def from_json(d):
25    if d.get("__type__") == "Book":
26        return Book(d["title"], Decimal(d["price"]), datetime.fromisoformat(d["scraped_at"]))
27    return d
28
29
30book = Book("A Light in the Attic", Decimal("51.77"), datetime(2026, 9, 24, 3, 0, tzinfo=timezone.utc))
31
32text = json.dumps(book, default=to_json)
33print(text)
34# {"__type__": "Book", "title": "A Light in the Attic", "price": "51.77", "scraped_at": "2026-09-24T03:00:00+00:00"}
35
36again = json.loads(text, object_hook=from_json)
37print(again == book)  # True

Three choices keep that round trip exact.

• The price is written as the string "51.77", so no float rounding enters it.
• The timestamp is written by datetime.isoformat() in ISO 8601 with its UTC offset and read back by datetime.fromisoformat() (Python datetime documentation), so every stored time has the same format and carries its offset.
• One way of making the decoder able to know the object type that it should cast is by adding metadata bound to the object type when encoding it. The __type__ key tells from_json which dictionaries to rebuild. The Python documentation's own example marks complex numbers the same way, with a __complex__ key. dataclasses.asdict() turns the dataclass into a dictionary of its fields (Python dataclasses documentation).

If a date arrives as free text, such as "two weeks ago", dateparser turns relative dates like that one into a datetime before you store it (dateparser documentation).

How do you query nested JSON without a chain of brackets?

Use JMESPath, which describes itself as "a query language for JSON" (jmespath.org). When traversing JSON data in Python, depending on the complexity of the object, there are more advanced libraries to help you get to the data with less code. JMESPath is well-suited for advanced JSON parsing tasks because it can handle complex, nested JSON structures with ease. At the same time, it is easy to use at beginner level, making it an accessible tool for anyone working with JSON data.

Install the Python library with pip install jmespath. With data from the slideshow example:

1import jmespath
2
3print(jmespath.search("slideshow.slides[*].title", data))
4# ['Wake up to WonderWidgets!', 'Overview']
5
6print(jmespath.search("slideshow.slides[?items].title", data))
7# ['Overview']
8
9print(jmespath.search("slideshow.slides[0].items", data))
10# None, where data["slideshow"]["slides"][0]["items"] raises KeyError
11
12print(jmespath.search("slideshow.slides[*].{title: title, count: length(items || `[]`)}", data))
13# [{'title': 'Wake up to WonderWidgets!', 'count': 0}, {'title': 'Overview', 'count': 2}]
14
15titles = jmespath.compile("slideshow.slides[*].title")
16print(titles.search(data))
17# ['Wake up to WonderWidgets!', 'Overview']

Those examples only display the basics of what JMESPath can do. JMESPath queries can also filter and transform JSON data. For example, you can use JMESPath to filter a list of objects based on a specific value or to extract specific parts of an object and transform them into a new structure.

A missing key gives None instead of an exception. The JMESPath tutorial states that "if you refer to a key that does not exist, a value of null (or the language equivalent of null) is returned." [?items] is a filter that keeps only the slides with items, and filters also take the comparators ==, !=, <, <=, > and >=. The {title: title, count: ...} part builds a new object for each slide. jmespath.compile() parses an expression once so you can reuse it across documents (jmespath.py repository).

The same expressions run outside Python. The JMESPath organisation lists implementations in Python, Go, Lua, JavaScript, PHP, Ruby and Rust, each marked fully compliant (JMESPath libraries).

parsel has JMESPath built in. Version 1.8.0, released on 18 April 2023, added Selector.jmespath() (parsel release notes), and Selector detects JSON input on its own (parsel usage documentation). In Scrapy, response.jmespath() is a shortcut to the same method (Scrapy request and response documentation).

1from parsel import Selector
2
3sel = Selector(text=text)
4print(sel.jmespath("slideshow.slides[*].title").getall())
5# ['Wake up to WonderWidgets!', 'Overview']

For a longer walk-through of JMESPath and ChompJS together, see extracting JSON with ChompJS and JMESPath.

The JSON is inside a web page. How do you get it out?

When you are scraping a page, check its <script> tags for two shapes of data before you write selectors against the visible markup.

The first shape is a JSON-LD block. The JSON-LD 1.1 specification embeds JSON-LD in HTML inside a script element whose type attribute is application/ld+json (JSON-LD 1.1, W3C Recommendation, 16 July 2020). The content is JSON, so json.loads() reads it once a selector has found it. In the schema.org vocabulary, an Offer carries price and priceCurrency (schema.org Offer).

1import json
2from parsel import Selector
3
4html = """
5<html>
6<head>
7<script type="application/ld+json">
8{
9  "@context": "https://schema.org",
10  "@type": "Product",
11  "name": "A Light in the Attic",
12  "offers": {"@type": "Offer", "price": "51.77", "priceCurrency": "GBP"}
13}
14</script>
15</head>
16<body><h1>A Light in the Attic</h1></body>
17</html>
18"""
19
20sel = Selector(text=html)
21for block in sel.css('script[type="application/ld+json"]::text').getall():
22    item = json.loads(block)
23    print(item["name"], item["offers"]["price"], item["offers"]["priceCurrency"])
24# A Light in the Attic 51.77 GBP

Some pages mark up the same vocabulary as microdata in the HTML instead; extracting schema.org microdata covers that case.

The second shape is a JavaScript variable. The /js/ page of quotes.toscrape.com, part of a scraping sandbox (toscrape.com), holds its quotes as var data = [...] inside a script tag. That array parses as JSON with JavaScript before and after it. Find the first bracket and let raw_decode() read one document from there. The documentation says raw_decode() "can be used to decode a JSON document from a string that may have extraneous data at the end" (Python json documentation).

1import json
2
3import requests
4from parsel import Selector
5
6response = requests.get("https://quotes.toscrape.com/js/", timeout=30)
7response.raise_for_status()
8
9script = Selector(text=response.text).xpath('//script[contains(., "var data")]/text()').get()
10start = script.index("[")
11quotes, end = json.JSONDecoder().raw_decode(script[start:])
12
13print(len(quotes))                  # 10
14print(quotes[0]["author"]["name"])  # Albert Einstein

Web scraping involves collecting data from websites, which may be embedded in JavaScript objects that initialize the page. While the standard library function json.loads() extracts data from JSON objects, it is limited to valid JSON objects. The problem is that not all valid JavaScript objects are also valid JSONs.

When the object is JavaScript that JSON rejects, with single quotes, unquoted names or trailing commas, use chompjs. Its parse_js_object() reads the "first encountered JSON-like object" and raises ValueError on failure, and by default it starts at the first { or [ it finds (chompjs repository).

1import chompjs
2
3quotes = chompjs.parse_js_object(script)
4print(len(quotes))  # 10
5
6print(chompjs.parse_js_object("{title: 'A Light in the Attic', tags: ['poetry',],}"))
7# {'title': 'A Light in the Attic', 'tags': ['poetry']}

chompjs works by parsing the JavaScript object and converting it into a valid Python dictionary. One of the benefits of using chompjs over json.loads is that it can handle a wider range of JavaScript objects. For example, chompjs can handle objects that use single quotes instead of double quotes for property names and values. It can also handle objects that have extra commas at the end of arrays or objects.

What data parsing is covers the script-tag case in more depth, with the selector choices around it. The selector libraries used here are compared in Python web scraping tools.

The file is too big to load in one go. What now?

json.load() reads the whole file before it parses anything: it calls fp.read() and passes the result to loads() (CPython 3.13 source, Lib/json/init.py). The Scrapy documentation describes the limit for the format as a whole: JSON "doesn’t scale well for large amounts of data since incremental (aka. stream-mode) parsing is not well supported (if at all) among JSON parsers (on any language), and most of them just parse the entire object in memory" (Scrapy item exporters documentation).

If you control the format, write JSON Lines: one JSON value per line, UTF-8, and \n as the line terminator (JSON Lines). Each line is parsed on its own. Scrapy's feed exports write this format when you set the format to jsonlines (Scrapy feed exports documentation).

1import json
2
3books = [
4    {"title": "A Light in the Attic", "price": 51.77},
5    {"title": "Tipping the Velvet", "price": 53.74},
6]
7
8with open("books.jsonl", "w", encoding="utf-8") as f:
9    for book in books:
10        f.write(json.dumps(book, ensure_ascii=False) + "\n")
11
12with open("books.jsonl", encoding="utf-8") as f:
13    for line in f:
14        if line.strip():
15            book = json.loads(line)
16            print(book["title"])
17# A Light in the Attic
18# Tipping the Velvet

If the file is one large array that you did not write, stream it with ijson, "an iterative JSON parser with standard Python iterator interfaces" (ijson repository). ijson.items() yields each object under a prefix, and the prefix item selects each element of a top-level array. Open the file in binary mode.

1import json
2
3import ijson
4
5with open("books.json", "w", encoding="utf-8") as f:
6    json.dump(books, f)
7
8with open("books.json", "rb") as f:
9    for book in ijson.items(f, "item"):
10        print(book["title"], repr(book["price"]))
11# A Light in the Attic Decimal('51.77')
12# Tipping the Velvet Decimal('53.74')

ijson returns numbers with a fraction as decimal.Decimal unless you pass use_float=True, which is why the prices print that way (ijson repository).

How do you check or pretty-print JSON from the command line?

Python ships a command-line tool.

1python -m json.tool book.json
2python -m json.tool --sort-keys --no-ensure-ascii book.json
3python -m json.tool --json-lines books.jsonl

It prints the document indented. On invalid input it prints the error with its line and column instead, and on Python 3.13.15 it exited with status 1, so a script can use it as a check. On Python 3.14 and later the tool also runs as python -m json, and the documentation adds that "For backwards compatibility, invoking the CLI as python -m json.tool remains supported" (Python json documentation, command-line interface).

Is it safe to parse JSON from a source you do not control?

json.loads() builds only dictionaries, lists, strings, numbers, booleans and None, unless you pass hooks that build something else. Two limits still apply, and the Python json documentation states both.

• Size. "Be cautious when parsing JSON data from untrusted sources. A malicious JSON string may cause the decoder to consume considerable CPU and memory resources. Limiting the size of data to be parsed is recommended."
• Long integers. Since Python 3.11, the default integer parsing "limits the maximum length of the integer string via the interpreter’s integer string conversion length limitation to help avoid denial of service attacks".

Check len(response.content) against a limit you set before you parse. Never pass JSON-like text to eval(), which executes the source it is given (Python eval() documentation).

What does the whole thing look like as one script?

This is every step above in one file. It runs on Python 3.13.15 with requests 2.34.2, jmespath 1.1.0, parsel 1.11.0, chompjs 1.4.1 and ijson 3.5.1. It calls two public test services, httpbin.org and quotes.toscrape.com. For fetching real pages at the start of a pipeline like this, see web scraping with Python.

1"""Every step on this page in one script.
2
3Install the five third-party packages first:
4    pip install requests jmespath parsel chompjs ijson
5"""
6import json
7from dataclasses import dataclass, asdict
8from datetime import datetime, timezone
9from decimal import Decimal
10
11import chompjs
12import ijson
13import jmespath
14import requests
15from parsel import Selector
16
17
18@dataclass
19class Book:
20    title: str
21    price: Decimal
22    scraped_at: datetime
23
24
25def to_json(obj):
26    if isinstance(obj, Book):
27        return {"__type__": "Book", **asdict(obj)}
28    if isinstance(obj, Decimal):
29        return str(obj)
30    if isinstance(obj, datetime):
31        return obj.isoformat()
32    raise TypeError(f"Cannot serialize {type(obj).__name__}")
33
34
35def from_json(d):
36    if d.get("__type__") == "Book":
37        return Book(d["title"], Decimal(d["price"]), datetime.fromisoformat(d["scraped_at"]))
38    return d
39
40
41# 1. A string, then a file
42book = json.loads('{"title": "A Light in the Attic", "price": 51.77, "in_stock": true}')
43with open("book.json", "w", encoding="utf-8") as f:
44    json.dump(book, f, indent=2, ensure_ascii=False)
45with open("book.json", encoding="utf-8") as f:
46    print(json.load(f)["title"])
47
48# 2. An API response, nested values and a JMESPath query
49response = requests.get("https://httpbin.org/json", timeout=30)
50response.raise_for_status()
51data = response.json()
52for slide in data["slideshow"]["slides"]:
53    print(slide["title"], len(slide.get("items", [])))
54print(jmespath.search("slideshow.slides[?items].title", data))
55
56# 3. Invalid JSON
57try:
58    json.loads("{'title': 'A Light in the Attic'}")
59except json.JSONDecodeError as err:
60    print(f"{err.msg} at line {err.lineno}, column {err.colno}")
61
62# 4. Types the json module cannot handle on its own
63typed = Book("A Light in the Attic", Decimal("51.77"), datetime(2026, 9, 24, 3, 0, tzinfo=timezone.utc))
64text = json.dumps(typed, default=to_json)
65print(json.loads(text, object_hook=from_json) == typed)
66
67# 5. JSON inside a web page: a JSON-LD block, then a JavaScript variable
68html = """<script type="application/ld+json">
69{"@context": "https://schema.org", "@type": "Product", "name": "A Light in the Attic",
70 "offers": {"@type": "Offer", "price": "51.77", "priceCurrency": "GBP"}}
71</script>"""
72for block in Selector(text=html).css('script[type="application/ld+json"]::text').getall():
73    print(json.loads(block)["offers"]["price"])
74
75page = requests.get("https://quotes.toscrape.com/js/", timeout=30)
76page.raise_for_status()
77script = Selector(text=page.text).xpath('//script[contains(., "var data")]/text()').get()
78quotes, end = json.JSONDecoder().raw_decode(script[script.index("["):])
79print(len(quotes), quotes[0]["author"]["name"])
80print(chompjs.parse_js_object(script) == quotes)
81
82# 6. JSON Lines, then one large array read an item at a time
83with open("quotes.jsonl", "w", encoding="utf-8") as f:
84    for quote in quotes:
85        f.write(json.dumps(quote, ensure_ascii=False) + "\n")
86with open("quotes.jsonl", encoding="utf-8") as f:
87    authors = {json.loads(line)["author"]["name"] for line in f if line.strip()}
88print(len(authors))
89
90with open("quotes.json", "w", encoding="utf-8") as f:
91    json.dump(quotes, f, ensure_ascii=False)
92with open("quotes.json", "rb") as f:
93    first = next(ijson.items(f, "item"))
94print(first["author"]["name"])

It prints, in order: A Light in the Attic, the two slide lines, ['Overview'], the error message, True, 51.77, 10 Albert Einstein, True, 8 and Albert Einstein.

When do you stop writing JSON parsing code yourself?

When the data you need is not in JSON anywhere on the page. Everything above assumes an API or the page already holds the data as JSON. When neither does, somebody writes and maintains selectors for every source.

Zyte API can return that data as JSON instead, for the data types its documentation lists, which include product, article and job posting data (Zyte API automatic extraction documentation). AI extraction describes how it works. Zyte sells it. The documentation's own example asks for the same book used on this page:

1import requests
2
3api_response = requests.post(
4    "https://api.zyte.com/v1/extract",
5    auth=("YOUR_ZYTE_API_KEY", ""),
6    json={
7        "url": (
8            "https://books.toscrape.com/catalogue"
9            "/a-light-in-the-attic_1000/index.html"
10        ),
11        "product": True,
12    },
13)
14product = api_response.json()["product"]
15print(product)

The documented output starts with "name": "A Light in the Attic", "price": "51.77" and "currency": "GBP". The price arrives as a string, so Decimal(product["price"]) keeps it exact. For a source you already parse with the code above, there is nothing to change.

In this guide, we've covered the basics of reading and parsing JSON data with Python, as well as how to access and modify JSON data using Python's built-in json package. We've also discussed more advanced JSON parsing options, such as JMESPath and ChompJS, which are useful for web scraping data. With the knowledge gained from this guide, you should be able to efficiently work with JSON data in Python and integrate it into your developer workflow.

FAQ

Q: What is the difference between json.load and json.loads?
A: json.loads() parses a JSON document held in a string, bytes or bytearray. json.load() parses one from a file object you opened. Both return the same Python objects: a JSON object becomes a dict and an array becomes a list.

Q: How do I fix Expecting value: line 1 column 1 (char 0)?
A: Look at what you passed in. Python prints this for an empty string and for an HTML page, because parsing fails at the very first character. Print the first 200 characters of the input. With Requests, call raise_for_status() before .json(), so an error page sent with an error status fails as an HTTP error instead.

Q: How do I pretty-print JSON in Python?
A: In code, use json.dumps(data, indent=2), and add ensure_ascii=False to keep non-ASCII characters readable. From a terminal, run python -m json.tool file.json, which also reports the line and column of the first error.

Q: Why does json.dumps turn characters like é and £ into escape sequences?
A: Because ensure_ascii is on by default, and with it on, every non-ASCII character in the output is escaped. Pass ensure_ascii=False, and open the output file with encoding="utf-8".

Q: Can Python's json module read JSON with single quotes?
A: No. JSON strings and names need double quotes, and the module fails with Expecting property name enclosed in double quotes. If the text is a JavaScript object from a web page, chompjs reads it. If it is a Python dictionary that was printed with print(), change the code that produced it to call json.dumps().

Q: What happens to duplicate keys in a JSON object?
A: Python keeps the last value and raises no error, so {"x": 1, "x": 2} loads as {'x': 2}. The JSON standard says names within an object should be unique and that software receiving repeated names behaves unpredictably. Pass object_pairs_hook to receive every pair and decide yourself.

Q: How do I stop prices turning into floats when I parse JSON?
A: Pass parse_float=decimal.Decimal to json.loads() or json.load(). Every JSON number with a fraction then arrives as an exact Decimal. When you write the data back out, convert each Decimal to a string in a default function, because the module cannot serialize it.

Q: How do I read a JSON Lines file in Python?
A: Open it as text with encoding="utf-8", loop over the lines, skip blank ones, and call json.loads() on each line. Calling json.loads() on the whole file fails with Extra data, because the file holds one document per line.

Q: Why did my integer keys come back as strings?
A: JSON object names are always strings, so json.dumps() converts every key to a string and json.loads() gives strings back. {1: "a"} comes back as {'1': 'a'}. Convert the keys yourself, for example with {int(k): v for k, v in data.items()}.

Q: Is json.loads safe on data from the internet?
A: It builds only dictionaries, lists, strings, numbers, booleans and None, and it does not execute anything, unlike eval(). The documentation still warns that a malicious document can consume considerable CPU and memory, so cap the size of what you parse.