It was a 2025 in which AI-assisted web scraping truly arrived, quantum computing took a leap forward and large parts of the web routinely went dark as sites’ reliance on a few big infrastructure providers became clear to see.
But what will 2026 have in store for the web we know and love? Arguably, the biggest shake-up in its three-decade history.
I asked a variety of web data thinkers to give me their predictions for the year ahead.
Agents, agents everywhere
The rise of AI-assisted web scraping in 2025 sets the stage for a new era. “Agent” hype may already have reached fever pitch. In 2026, though, experts predict a shift toward truly automated web processes, powered by sophisticated AI agents that handle everything.
Jan Seidler, Chief Technology Officer, Zyte
“I think 2026 is the year web data becomes truly automated. Not just faster scrapers, but AI creating, fixing and scaling them - from a site name to working production code, and then keeping it running as the web changes.
“Agent-based workflows will move from demos to everyday tools, including real browser control, app data access and self-healing maintenance.”
Read more: Why your agent deserves a wallet
Ayan Pahwa, Developer Advocate, Zyte
“2026 will be prominent for agentic AI workflows - armies of AI agents doing the heavy lifting - from research to execution to reporting - talking among each other using MCP, A2A etc. Imagine a business running 12 different agents managing leads, accounting, ad ops etc - the founder wakes up to dashboards instead of to-do lists.
“Maintaining these agents will probably be a full-time job. I’m thinking, reliability engineering for AI agents.”
Daniel Cave, Product Marketing Manager, Zyte
“We may see a shift away from building sites exclusively for human consumption and more adoption of agent-first design practices, such as using OpenAI's new shopping cart protocol. How this plays into scraping will be interesting.”
Read more: Why AI agents struggle with web scraping (and how to help them)
Websites tighten up and lock down
As data-gathering technology evolves, so, too, do the defenses. Experts predict new automated solutions and access policies will fuel a different relationship between site and scraper.
Fabien Vauchelles, web scraping expert, Scrapoxy
"The barrier to entry is getting higher and higher every year. I think we will move toward a closed internet in the coming years.
“The future is pretty clear. We will have major websites which will be accessible for AI agents, but everyone else will be locked out."
Akshay Philar, Head of Engineering, Zyte
"The growing adoption of RASP, polymorphic obfuscation, and WebAssembly will further complicate reverse engineering.
“As agents become more adept at handling CAPTCHAs, vendors that currently rely on proof-of-work may pivot toward alternative mechanisms that preserve user experience while distinguishing humans from bots.”
Iain Lennon, Chief Product Officer, Zyte
“Anti-bot solutions will continue to increase the rate of changes to their configurations. Software automation to respond and handle these will increasingly become vital.
“Separately, businesses using web data will increasingly expect to integrate web data pipelines into enterprise architectures, and with enterprise levels of engineering control.”
Reasoning gets a re-think?
Large Language Models, including those with “reasoning” capability, are predicated on “token” prediction - but some experts now think that approach may not catapult AI to the much-vaunted echelons of super-intelligence.
Iván Sánchez, Senior Data Scientist, Zyte
“I'm pretty sure we will get some new AI models from China.
“The only question that I have for 2026 is if the current method of AI reasoning via token prediction will be the paradigm that will stay, the new norm, or will we have something new, a new way of thinking, or maybe parallel thinking?”
Read more: AI and the web: What 2025 changed and what comes next
Market-defining legal cases
While 2025 brought some legal clarity to the issue of scraping for AI systems, experts are looking to 2026 to bring further certainty.
Sanaea Daruwalla, Chief Legal & People Officer, Zyte
“We will continue to see a lot of copyright lawsuits as it relates to data to train AI. Web scrapers will need to continue to follow these cases and ensure compliance as new rulings take place.
“Additionally, we will also see copyright holders try to monetize and license their content, with things like Really Simple Licensing, so we will see more on the legality of this and how it will play out in 2026.”
Read more: Balancing innovation and regulation in data scraping
