Download copy of a website - protected by User-agent: Disallow:
£20-250 GBP
Completed
Posted almost 11 years ago
£20-250 GBP
Paid on delivery
Can you download a copy of a website?
I've been asked to take over a project that is going wrong, and they want a copy of the development website downloading for security for all the text ( and images if possible) before they move the job away.
I've tried to download it using sitesucker and its failed because of the [login to view URL] file
The robots txt file that blocks my download software with a code:
User-agent: *
Disallow: /
The site is on a subdomain of the domain name
Is it possible? and how much?
I'm expert in web scraping / data mining. I'm mass scraper and I can scrape many pages for short time.
Proofs of the quality of my work are in my profile.
More details in PM.