PINGDOM_CHECK

#ExtractSummit2026 The world's largest web scraping conference returns. Austin Oct 7–8 · Dublin Nov 10–11.

Register now
Data Services
Pricing
Login
Try Zyte APIContact Sales
  • Unblocking and Extraction

    Zyte API

    The ultimate API for web scraping. Avoid website bans and access a headless browser or AI Parsing

    Ban Handling

    Headless Browser

    AI Extraction

    SERP

    Enterprise

    DocumentationSupport

    Hosting and Deployment

    Scrapy Cloud

    Run, monitor, and control your Scrapy spiders however you want to.

    Coding Agent Add-Ons

    Agentic Web Data

    Plugins that give coding agents the context to build production Scrapy projects. Starts with Claude Code.

  • Data Services
  • Pricing
  • Browse

    • BlogArticles, podcasts, videos
    • Case studiesCustomer outcomes
    • White papersIn-depth reports
    • DocumentationGuides & API reference
    • EventsConferences, webinars, recordings

    Subscribe

    • NewsletterSwiftly delivered
    • Discord communityExtract Data community
  • Product and E-commerce

    From e-commerce and online marketplaces

    Data for AI

    Collect and structure web data to feed AI

    Job Posting

    From job boards and recruitment websites

    Real Estate

    From Listings portals and specialist websites

    News and Article

    From online publishers and news websites

    Search

    Search engine results page data (SERP)

    Social Media

    From social media platforms online

  • Meet Zyte

    Our story, people and values

    Contact us

    Get in touch

    Support

    Knowledge base and raise support tickets

    Terms and Policies

    Accept our terms and policies

    Open Source

    Our open source projects and contributions

    Web Data Compliance

    Guidelines and resources for compliant web data collection

    Join the team building the future of web data
    We're Hiring
    Trust Center
    Security, compliance & certifications
Login
Try Zyte APIContact Sales

Zyte Developers

Coding tools & hacks straight to your inbox

Become part of the community and receive a bi-weekly dosage of all things code.

Join us
    • Zyte Data
    • News & Articles
    • Search
    • Social Media
    • Product
    • Data for AI
    • Job Posting
    • Real Estate
    • Zyte API - Ban Handling
    • Zyte API - Headless Browser
    • Zyte API - AI Extraction
    • Web Scraping Copilot
    • Zyte API Enterprise
    • Scrapy Cloud
    • Solution Overview
    • Blog
    • Webinars
    • Case Studies
    • White Papers
    • Documentation
    • Web Scraping Maturity Self-Assesment
    • Web Data compliance
    • Meet Zyte
    • Jobs
    • Terms and Policies
    • Trust Center
    • Support
    • Contact us
    • Pricing
    • Do not sell
    • Cookie settings
    • Sign up
    • Talk to us
    • Cost estimator
All articles
AI62, 62 articles
Data quality13, 13 articles
Developer interest57, 57 articles
Integration2, 2 articles
Open-source41, 41 articles
Proxies29, 29 articles
Scraping practice17, 17 articles
Scraping strategy27, 27 articles
Web data60, 60 articles
Web scraping APIs33, 33 articles
Scrapy Cloud48, 48 articles
Web Scraping Copilot11, 11 articles
Zyte API54, 54 articles
AI & Machine Learning2, 2 articles
Automotive2, 2 articles
E-commerce & retail27, 27 articles
Entertainment & Streaming2, 2 articles
Financial Services8, 8 articles
Government2, 2 articles
Market Research & Intelligence3, 3 articles
Media & publishing8, 8 articles
Real Estate2, 2 articles
Recruitment & HR3, 3 articles
Transportation & Logistics2, 2 articles
Travel & hospitality2, 2 articles
Extract Summit25, 25 articles
PyCon1, 1 articles
Large language model23, 23 articles
MCP3, 3 articles
iPaaS2, 2 articles
Python95, 95 articles
Web Scraping Industry Report10, 10 articles
Extract Data podcast1, 1 articles
Zyte Learning Centre1, 1 articles
Zyte YouTube channel1, 1 articles
Zyte blog398, 398 articles
16:9 video4, 4 articles
Article392, 392 articles
Guide1, 1 articles
Build-through2, 2 articles
Curated digest1, 1 articles
Insight and analysis26, 26 articles
Perspective18, 18 articles
Perspectives multi2, 2 articles
Product announcement3, 3 articles
Product demo1, 1 articles
Product update1, 1 articles
Recap3, 3 articles
Research results1, 1 articles
Tutorial7, 7 articles
Viewpoint4, 4 articles
Viewpoint4, 4 articles
Insight and analysis26, 26 articles
Build-through2, 2 articles
Product announcement3, 3 articles
Recap3, 3 articles
Curated digest1, 1 articles

Appearance

Discord Community
BlogAIGPT-5.6, Fable 5, and GLM-5.2 entered a bar “crawl” and got hit by The Rate Limit
ArticleResearch / ReportAI

GPT-5.6, Fable 5, and GLM-5.2 entered a bar “crawl” and got hit by The Rate Limit

When OpenAI shipped GPT-5.6 and Anthropic's Fable 5 was sitting at the top of the price list, my question was not "which one wins a leaderboard." It was the one I actually pay for: for the scraping I do, how much model do I need to buy?

Ayan Pahwa · Developer Advocate

16 min read · July 10, 2026

GPT-5.6, Fable 5, and GLM-5.2 entered a bar “crawl” and got hit by The Rate Limit

A while back I wrote about adding GLM-5.2 to my agentic arsenal, and the part that stuck was not the model, it was the method. I keep a small harness of real scraping tasks and an OpenRouter key so that when something new ships I can throw it at genuine work the same week. GLM-5.2 came out of that as a cheap open-weight workhorse I still reach for.

So when OpenAI shipped GPT-5.6 and Anthropic's Fable 5 was sitting at the top of the price list, my question was not "which one wins a leaderboard." It was the one I actually pay for: for the scraping I do, how much model do I need to buy? GPT-5.6 makes that a sharp question, because it ships as a price ladder, three tiers of the same generation. Add the cheap challenger below it and the frontier model above it and you get a clean spread on the cost that actually dominates a scraping bill, output tokens: from $1.76 per million on GLM-5.2 to $50 on Fable 5, nearly 30 times more. Five contenders walk into the same bar. Who is worth their tab?

The setup, kept light

The mechanics are deliberately simple. I send the identical prompt to each model so nothing is biased by wording, and everything runs through OpenRouter so I am comparing real charged dollars rather than guessing from a rate card. Where I can score a result objectively I do: does the generated code compile, does it run against a real page and return the right values, does the extracted JSON cover the schema. Then I read every output myself, because the automatic scores miss things. Five models, a handful of real scraping tasks, one afternoon. This is a first-hand read, not a benchmark, so take the small sample for what it is.

The five, cheapest to priciest:

  • GLM-5.2 (z-ai) at $0.54 / $1.76 per million in/out tokens, the cheap open-weight challenger.
  • GPT-5.6 Luna at $1 / $6, OpenAI's fast, cost-efficient tier.
  • GPT-5.6 Terra at $2.50 / $15, the balanced middle.
  • GPT-5.6 Sol at $5 / $30, the flagship.
  • Claude Fable 5 at $10 / $50, the most capable, most expensive model on the menu.

The one table that answers the question

Model Price (in/out) Suite cost Tasks passed Cost per success Reasoning tokens Tool loop Suite speed Landing page
GLM-5.2 $0.54 / $1.76 $0.045 3/4 $0.0099 4,585 7/8 176s 10.5 min
GPT-5.6 Luna $1 / $6 $0.044 3/4 $0.0087 1,781 8/8 29s 18s
GPT-5.6 Terra $2.50 / $15 $0.118 4/4 $0.020 1,692 8/8 51s 32s
GPT-5.6 Sol $5 / $30 $0.206 4/4 $0.037 1,716 8/8 68s 36s
Claude Fable 5 $10 / $50 ~$0.38 2/4 $0.119 109 8/8 57s 63s

Read down the "tasks passed" column and the story tells itself. Capability climbs as you pay more, right up to Terra in the middle. Then it stops climbing. Then, at the top, it falls. The curve is not a staircase where more money buys more models. It is a hump, and the peak is in the cheap seats.

image

What every dollar tier gets right: extraction

Start with the good news, which is that most scraping is boring, and boring is where everyone ties. Pulling a fixed schema of clean JSON out of a messy product page, and pulling every book off a listing page into an array, is bread-and-butter structured extraction. All five models did it perfectly. Same valid JSON, same full schema coverage, same 20 books off the listing, no drift.

That is the single most important line in the whole exercise, because extraction is the bulk of real scraping. On the work you do most, the cheapest model in the room is indistinguishable from the one that costs 30 times more. The only thing that separates them here is the bill, and the bill says use the cheap one: Luna and GLM cost under a cent per successful answer, and Sol costs four times that for output you could not pick out of a lineup.

What separates the cheap tier: currency

There is one task where the models split, and it is not the one I expected. I ask each model to wire a Scrapy project to run behind the Zyte API the way Zyte actually recommends today, which is a one-line addon. It is a quiet test of whether a model is working from current knowledge or a stale training snapshot.

The line fell inside a price band, not between vendors. Terra and Sol got it right, reaching straight for the modern one-line addon, no manual plumbing. GLM-5.2 and Luna both got it wrong, hand-wiring the deprecated download handlers and middleware, the textbook symptom of training-data lag. They are the two cheapest models, and they share the same blind spot.

So the freshness you are paying for is real, but it is not much of a moat. Honestly, you can pretty much solve it by giving the agent access to fresh docs through an MCP server like Context7, so it is not a biggie. Out of the box the gap closes at the Terra tier, and you never have to pay flagship prices to clear it.

What the flagship premium buys: nothing I could measure

Here is where the money stops working. Sol passed everything Terra passed, four for four, and matched it on every other axis: same clean addon, same extraction, same eight-for-eight in the agent loop, an indistinguishable landing page. It cost roughly double per successful task for zero additional capability I could detect. On this workload the flagship is a mid-tier model with a bigger price tag. Whatever Sol's extra headroom is for, everyday scraping is not it.

What the frontier model does: it refuses the job

And then the curve falls off a cliff. Fable 5 is a Mythos-class model, Anthropic's most capable tier, the kind of thing you bring in to orchestrate long-horizon, multi-step work and coordinate other models, not to write a parse function. Reaching for it on a scraping task is overkill before you even start. It has the raw capability, and it proved that in the agent loop with a clean sweep. But point it at actual scraping and its safety classifiers start declining the work.

It refused the anti-bot plan outright, returning nothing. Worse, it began writing a perfectly benign CSS-selector function, the kind of "extract the title and price from this product page" code the cheap models wrote without blinking, and then its classifier killed the response mid-function. Even the addon task, which it did complete, came back hedged: it gave the modern addon and then bolted the deprecated manual middleware on beside it, which is exactly the muddle you do not want a junior copying.

So the most expensive model in the field scored the lowest, two of four, and cost the most per successful answer, roughly 14 times Luna. Its reasoning-token count looks impressively low, but that is not efficiency, it is what refusing two tasks looks like. You are paying frontier prices for a model that treats your core workload as a threat. For scraping it is not overqualified. It is unavailable.

The one task I judged by eye: a landing-page build-off

Everything so far has a right answer a script can check. Design does not, so for this one I gave each model the same brief, build a single self-contained landing page for a fictional scraping API, everything inline so it renders offline, and then I put three of them side by side and ranked them blind before revealing which model built which.

image

I ranked them Sol first, Fable a close second, GLM last, and the honest headline is that they were all close. Two things stood out anyway. First, design quality is a wash you cannot buy your way out of: the GPT tiers and Fable all produced clean, shippable, modern pages I genuinely could not tell apart by maker, while GLM went its own way with a warm editorial "field manual" look that has real personality but reads as less of a conversion page. You are not buying a better landing page by spending more.

Second, the cost of getting there was wildly uneven. The two pages I ranked highest took 36 seconds (Sol) and 63 seconds (Fable). The one I ranked last took 10.5 minutes and 22,000 reasoning tokens (GLM). And Fable, which flatly refused to write a scraper a few tasks earlier, cheerfully built a polished page to market one, at roughly double Sol's cost for a page I ranked below it. The frontier model will sell the product it will not build.

The stuff that does not show up in a price table

A few dimensions matter as much as correctness and never appear on the menu.

Speed tracks thinking, and thinking tracks the bill. The GPT tiers ripped through the suite in 29 to 68 seconds; GLM took 176. On the landing page the gap turns absurd: GLM spent 10.5 minutes and 22,000 reasoning tokens building a page the GPT tiers produced in half a minute with almost no reasoning at all. Within the GPT family, cheaper is faster, because cheaper thinks less. One honest caveat on GLM's wall-clock: through OpenRouter, GLM gets routed to whichever third-party host is cheapest at the moment, Morph and StreamLake across my runs, each with its own tokens-per-second, while GPT-5.6 and Fable 5 were served by OpenAI and Anthropic directly, which tend to be faster. So some of GLM's slowness is a routing artifact, not the model itself. Either way, if your scraping is interactive or latency-sensitive, slow is slow at request time.

image

Self-healing is a family trait, not a premium. I gave every model a tool that always fails and watched whether it gave up gracefully or hammered it in a retry storm that burns tokens on every pointless turn. All of them behaved: one attempt, then a plain "this file does not exist," no storm. Good discipline is table stakes now, top to bottom.

Agentic discipline, same story. Every GPT tier, Luna included, went a perfect eight for eight on the tool-calling probe: correct tool selection, multi-hop chains in the right order, parallel fan-out of independent lookups in a single turn, and restraint on the trivial cases (answering "what is 2 plus 2" without reaching for the calculator). GLM was the only one that stumbled, an occasional reach for a tool it did not need. So even the cheapest OpenAI tier is a solid agent-loop citizen. One honest limit on that claim: this probes single-agent tool discipline, not multi-agent orchestration, the coordinating-a-fleet-of-subagents-across-a-long-job skill where a Mythos-class model like Fable is built to shine. That is a real axis, it is just not one a scraping run exercises.

Reporting and restraint differ by personality. GLM writes long and explains everything; the mid GPT tiers are clean and to the point; Fable narrates and hedges, wrapping answers in caveats. On the one judgment task, a build-versus-buy 403 plan, the cheap model gave the sharpest answer: GLM named a concrete threshold for when to stop hand-rolling proxies and reach for a managed API, named the tools, and threw in the trick I would give a junior, which is to check for a hidden JSON endpoint before scaling anything. Sol's plan was thorough but hedged; Fable refused to write one at all.

The five personas

GLM-5.2, the scrappy street-smart veteran. Knows the craft cold and gives the most useful advice in the room, but works from a slightly dated playbook (it missed the modern Zyte addon) and thinks the hardest and slowest of anyone. Cheapest on the menu, genuinely valuable on well-trodden ground, a little undisciplined in an agent loop and a little behind the times.

Luna, the fast, cheap, tidy intern. Quick, lean, does exactly what is asked, perfect in the agent loop, and astonishingly cheap. Its one blind spot is currency, the same stale-wiring gap as GLM. Ideal for high-volume, well-defined work; do not hand it the cutting-edge integration.

Terra, the current, competent mid-level engineer, and the sweet spot. Everything the flagship gets right, at half the price and with no drama: the modern addon, every extraction, eight-for-eight tools, the richest landing page of the bunch, fresh on best practices and frugal on reasoning. If you make one default choice for scraping, this is it.

Sol, the principal engineer you do not need for this work. The most cautious and, on paper, the most capable, with a reasoning-effort dial for genuinely hard problems. But on everyday scraping it is overqualified: the same passes as Terra for double the price, and cranking its effort dial higher on the addon task doubled the cost for zero gain in correctness.

Fable 5, the brilliant specialist who will not take your case. Untouchable on general reasoning and a clean sweep in the agent loop, but its safety classifiers refuse actual scraping: it declined the unblocking plan and killed a harmless selector function mid-sentence. Frontier prices for a model that will not do half the job.

A verdict you can act on

If your job is Reach for Why
High-volume structured extraction Luna or GLM-5.2 Everyone ties on correctness; the cheapest wins on cost per answer. The flagship here is four times the price for identical JSON.
Everyday scraping code that must be current Terra The freshness line switches on at Terra. It writes today's Zyte addon, not last year's middleware, at half Sol's cost.
Agent-loop / tool-calling work Any GPT-5.6 tier, Luna if cost matters Discipline is a family trait; all three went eight-for-eight. Avoid GLM here.
Build-versus-buy and architecture calls GLM-5.2 The cheap model gave the most directly useful plan. Sol over-hedged; Fable refused.
One-shot UI and landing pages Luna Family-level quality; two cents, not 16.
Genuinely frontier-hard, correctness-critical work Sol with effort dialed up The only place its ceiling might earn the premium.
Anything scraping-adjacent on a safety-tuned frontier model not Fable 5 It refuses the work.

The menu lies, and here is how

The pricing page frames the flagship as a 17 times decision over GLM. In real charged dollars it is closer to four, and two things do the compressing.

First, token efficiency. A low per-token rate on a model that thinks twice as hard is not as cheap as it looks. GLM burned 4,585 reasoning tokens across the suite to the GPT tiers' roughly 1,700, and on the landing page it spent 22,000 reasoning tokens to their 50. You pay for tokens, not for the sticker.

Second, the menu is a list price, not your bill. GLM's real charged cost ran above its headline rate once you route it through a real provider, while the GPT tiers' frugality pulled their effective cost well under their sticker multiples.

Two framing points matter more than any per-token number. Measure cost per successful task, not per call, because a cheap wrong answer is not cheap; GLM and Luna "saved" money on the addon and produced code you would have to rewrite. And the effort dial is an invisible multiplier: Sol at high effort cost twice as much as low effort for the same correct answer, and the menu never warns you.

At scale it all gets concrete. If we have to approximate and extrapolate - a million product-page extractions runs about $4,700 on Luna, $6,000 on GLM, $11,000 on Terra, $20,000 on Sol, and roughly $56,000 on Fable, for JSON you cannot tell apart, and Fable would refuse a chunk of the work anyway.

image

Where this falls short

If I stopped at whichever model looked cheapest I would be selling you something, so here is the honest part. This is a small, first-hand sample, one run per task, not a benchmark; a model that wins a task once might not win it reliably, and Fable's refusals in particular could soften or harden with prompt wording. The cost gaps are real but provider routing and token counts move them around, so treat the ranking as directional and the method as the point. And I only tested the scraping work I actually do; your tasks may pull the curve into a different shape, which is exactly why you should run your own.

The real takeaway: know your models, then buy the least you need

The lesson is not "GPT-5.6 beats GLM" or "the frontier model is bad." It is that the price ladder is not a quality ladder. Capability rose to the middle, plateaued at the flagship, and inverted at the frontier, where safety tuning made the most expensive model the least useful for this domain. The smart move is not to buy the best model you can afford. It is to know which model is good for which task and buy the least that clears your bar, which for most scraping is Terra in the middle, and for pure extraction is whatever is cheapest that week.

The durable asset here is the harness, not the verdict. The leaderboard will reshuffle again before this post ages, and when it does the question is not which model I trust today, it is how fast I can prove which one fits which job. If your default model got repriced or deprecated tomorrow, could you answer that in an afternoon?

That harness is nothing fancy, a small custom rig I built for exactly this: it fires the identical prompt at every model through one API, compiles and runs the code they hand back against a real page, checks the JSON, and leaves the taste calls like the landing pages to me. Build your own version, keep it around, and the next launch is an afternoon's work instead of a leap of faith.

Try Zyte API

Build your first scraper in minutes

Free trial, no credit card. From a single request to production in an afternoon.

Get started
AI

Ayan Pahwa

Developer Advocate

More from this author

In this article

  • The setup, kept light
  • The one table that answers the question
  • What every dollar tier gets right: extraction
  • What separates the cheap tier: currency
  • What the flagship premium buys: nothing I could measure
  • What the frontier model does: it refuses the job
  • The one task I judged by eye: a landing-page build-off
  • The stuff that does not show up in a price table
  • The five personas
  • A verdict you can act on
  • The menu lies, and here is how
  • Where this falls short
  • The real takeaway: know your models, then buy the least you need

Follow

Get the latest

Zyte and the data web in your inbox — or wherever you already are.

Subscribe

Or follow elsewhere

The Community · Newsletter

The best of Zyte and the data web, in your inbox.

One curated edition — new articles, product updates, and the stories shaping the data web. No noise.

G2.com

Capterra.com

Proxyway.com

EWDCI logoMost loved workplace certificateZyte rewardISO 27001 iconG2 rewardG2 rewardG2 reward

© Zyte Group Limited 2026