Making Your First Zyte API Request

In this video, I'm going to show you how to make your first request with the Z API, and we'll talk about three of the main features that I like the most and that I use the most often. So, I have a blank Python project in front of me. I've got my virtual environment set, and I have installed requests. So, let's import in what we need. So, I'm going to import requests, and I'm also going to import in um OS, which I need, and I'm also going to import in from rich, we're going to import in print. These two are relevant to what I'm doing right now. You probably don't need them. OS is because I have my uh API key stored in an environment variable on my machine. And rich is because what I print out to show you what's coming out of the program, you'll be able to see it a bit nicer. Next thing I'm going to do is I'm just going to set my API key here. So I'm going to say API key is equal to os.getv again because this is stored in my virtual environment on my machine. Uh, and I'm just going to do a quick uh if not API key uh will just raise an exception. This just means that uh if it doesn't find the API key on my machine, it will raise this exception and program will stop. Uh this is probably not relevant to you. Although I would recommend that you store your API key securely, safely in your environment and call it in in a way like this rather than pasting it directly into your code. So now we need to make the request. So this is going to be response is going to equal to requests.post. You need to make a post request to our API. Then we need to give the URL which is http and then it is api.zite.com/v1/ extract like this. That will always be the same for every request. So you can make that a constant if you wanted to. And then we need to authenticate. So our orth is going to be equal to uh and this is where we put our API key. So if you were pasting your API key directly in it would be API key would go here like this. And this needs to be a tupole. However, mine's come from my virtual environment. So, I'm just going to put in the API key variable that I created above. From here, we need to add in a JSON body. So, this is going to be the information for the website and the parameters that we want. So, the first one is obviously the URL. Without a URL, we can't really do anything. So, I'm going to paste in my URL here. And then I'm going to put in HTTP response body. And this is going to be equal to true. This is telling the API that I want the HTML for this and I want it as a plain request and there needs to be a comma there. And that's it. That is the basics of the request itself. Now, there's lots of different parameters you can put in here and we'll talk about a couple more in a minute. U but what I want to do now is I just want to print out the response body for you because I need to show you that when you use HTTP response body, it comes back encoded. So, I'm going to print response.json like so. I'm going to save this and I come over to my other terminal and we're going to run it. UV run main.py assuming I haven't done anything wrong. I should get a load of information back in just a second. There we go. Now you can see here that this is a load of um just random characters and that's because this is base 64 encoded. What that means is we need to decode this when we call it with the HTT with the HTTP response body. Now we can do that very easily in Python. It has its own Python package in the standard library. So we'll do from B 64. We're going to import in B64 decode. Now I can come back to my response. And I'm going to say that the HTML is going to be equal to uh B64 decode on our response.json. And within this response, I want to decode just the HTML part because the rest of it isn't encoded. So I'm going to say HTTP response uh body like so. And this means now we have a bite bytes representation of the HTML. So I can print this out. Print HTML. And I'm only going to print out the first I don't know 400 characters or something like that. So let's save, clear this up, and run again. Now we should be able to see the actual HTML or the first 400 odd characters of it which we have there. Now it's this HTML here which we could then pass out however you want to. Whether you want to use beautiful soup, lxml or parcel or any other HTML parser, this is done here for you. What's worth noting though is that with this one post request that we have made, we have negated the need for us to find any proxies. This API is going to do it all for us. We don't need to worry about the website blocking us in any way, shape or form. That's all handled by the Zite API. So, it's going to make our lives so much easier because we don't have to worry about any of that. But what you might want to do is you might want to actually use a headless browser to render this out for you. Maybe the website that you want is very JavaScript heavy or maybe it just needs JavaScript to run on its own at all. So what we can do here is just change this parameter to browser HTML is equal to true. Now when we do this, we don't need to decode the HTML because it's going to come back to us in a string. So we can just do response.json and then ask for the browser browser HTML like so. Now I'm going to save and we'll come back, clear this up and we'll run again. Now obviously because we're asking for a browser to be spun up to to get this, it's going to take a little bit longer. But what you will be able to do when you actually make projects out of this is not only be able to call this asynchronously. So you have multiple requests going at the same time. You'll also be able to reuse sessions um and you know hold on to IPs which will speed the whole thing up greatly. So there we are. We have the HTML or the start of it. There you can't really tell what it is here but it is the start of the HTML. actually don't really need to print out the whole thing. Um, that's it. And it's rendered through the browser that we've spun up in the background on our system for you. No need for you to worry about installing any extra dependencies. No need to worry about whether playright will work or whether it will be stable or whether it's going to work on the server that you're using. None of that. Just that one line will do it all for you. The last thing I want to show you is the auto extract we have where you can actually tell the API that it this page is a certain type of page and it will use the ML model behind the scenes to extract data from that page in a set schema and return it to you in JSON format. So this is going to negate the need for you to do any HTML passing. And obviously whilst there is a slight more cost associated to this, it makes it very very easy to integrate this into any other system because you know JSON is easy to pass around and almost any language can pass it. So what we're going to do is we're just going to get rid of this and we're going to put the word product here because I know that this is a product page. So I can tell the system I can tell the API rather that hey this is a product page. Give me the data back as a product schema and it will return the information. It will give you a probability. but it's very good at doing it. It's very good at recognizing product pages and understanding where information is and how to get that out. So, I'm going to comment these out because we don't need them. And we're going to do uh print out our response uh response.json. And within this response now, there is a product key like so. So, we save and we're going to come out of here. We're going to clear this up and we're going to run this again. And now we're going to see that when this comes back to us, it's going to have that default schema with all of those common uh data points from the website scrape pulled out for us. And if I turn this into scroll mode and go up, we can see here is a load of product information already. That's the description. There's the the um name, price, uh uh regular price, currency, availability, skew. All of this information has been uh passed out for you by default. Now, there will be cases where maybe this will miss things and I'll show you in a further video how you can actually adjust that. But I really wanted to show you how easy it is to just make one simple request like this with a URL and product is equal to true and to get that JSON data back that you can then push AC push around your system or do whatever you need to. Again, don't have to worry about running proxies, a browser, nothing like that. It's all handled for you with one request. It is worth noting though that when you do it like this, it will default to using browser rendering. Uh you can change it. So you will you can render off of the HTTP response body just by changing some extra parameters. There's product options I think is one of them. And you can tweak it that way too. So what I would recommend you do is go ahead and check out the documentation and have a look around. Get your API key. Go and use the playground and then see where you're at. check out the websites that you want to scrape and see how that works and go ahead and start to build out your first requests like I've done here and start getting that data from the web that you need. So, there'll be links to all of this in the description below. Thank you very much for watching and subscribe because there's going to be a lot more content like this coming up showing you the best way to web scrape using the Zite API.