Now that the contract is signed, we’re officially ready to get started. Here’s a short summary of next steps:
1. Project Setup: Your project has just been handed over to our Data Subscription Setup Team. Your project code is being created (also referred to as spiders) and first data delivery is scheduled.
2. Start Date: Your project will start upon 1st payment receipt. We usually start within 2-3 days of payment receipt, may change depending on work load.
3. Recurring subscription: If you signed up for a recurring subscription, your project will be moved over to our Data Subscription Team once implemented. They will take care of your pre agreed periodical deliveries.
We use a single channel for communications in the Data Subscription team - the Scrapinghub Support Center.
In order to communicate project progress we use a ticketing system as described below. These tickets will be created by us and your team will be copied. You need to create an account to get access to the Support Center.
One overall ticket that tracks the project details. This will include items like:
- The status of the overall project
- Subscription details
- Delivery details
- Project updates
- Anything that is not specific to a certain website/spider
One ticket for each of the spiders/domains being implemented. This will be used for:
- Keeping the conversation focused on one specific topic
- Exchanging sample data & gathering your feedback on it
- Providing you regular updates.
The contract has been signed and the Data Subscription team is aware that a new project in on the way.
We will be in touch once the invoice is paid, to inform you about the estimate project start date.
Sample data is sent to you once the first version of the spider code is ready
We will communicate to clarify any questions about the data to be delivered (schema) or other technical details we need to sort out
Regular updates will be provided in the support centre on both your Project ticket (every 3 days) AND your
Spider ticket (every 2 days)
Schema has been approved by you
We’ve started crawling for the data you have outlined.
- Test the data quality. 'Does it adherence to the schema
- Monitor spider code configuration
- Check for coverage. Ensure that all required records are being extracted
Updates on the full crawl progress will be sent to you every 2 days
Full crawl has been completed.
Internal QA consisting of automated and manual testing is performed by the Data Subscription QA team on the full crawl.
Legitimate reported issues are fixed; the Data Subscription QA team determines that the dataset is of acceptable quality to be delivered.
All Data has been delivered to a the agreed destination.
You will receive an email notification with a list of urls to download the data.
If you have a Data subscription with us, your work will be managed with the same care and attention by our Data Subscription Delivery team.
All future work will be handled through raising tickets. This is the ticket lifecycle...
- Developer allocation
- First analysis
- Next communication
- Next communication
- Update in ETA
- Problem resolution
- Code review
Validation from our QA team
Close the ticket