I am searching for a skilled freelancer who is able to automatically (NO MANUAL) grab a decent amount of products from an online shop, over 100k items. The only part of the item's information needed is it's part number. Simple so far...
These part numbers are scraped from a results page that is listed from selecting a series of filters. I will instruct you what filters to choose. The subsequent results pages are brand specific (depending on what brand is selected) the results page only lists 10 items. I need all the part numbers that may lie multiple pages deep. Once all the part numbers from one brand are collected, the automated crawl must then go onto the next brand. There are about 1000 brands and about 100,000 part numbers to collect. The site is clever and will inhibit you from crawling too much and too quickly, so you will have to utilize some of your disguising tools.
We will need the data in csv format in one file.
Selenium is no good for this project.
Your solution must be script driven.
If you and your code is successful, we will contract you to run the code once a month from here on in.
If you think that you are skilled enough for this project feel free to bid!
Please see the attached document which will show you the web site URL and instructions on what is needed.
At the bottom of this document is a code / keyword / text . You must tell me this keyword in your bid please. This will stop time wasters
517-DP270CL200ML Hi. I am a very experienced web developer with many project hours of web scraping. I have developed my own private scraping tools which helps me deliver top quality results for my clients.
38 freelancers are bidding on average $187 for this job
Hi sir, I'm professional web data extractor, I have had a look on this site, I can scrape it by script. wait for your reply, I can start work now. thanks. Regards Jianhua.
517-DP270CL200ML yes i understood you need all the part no in csv file, well there is many ways to get all 100k codes, i'm not going yo use selenium, please pm so that we can discuss more....thanks.
Hi, I have checked the requirements. Will do this scraping script using PHP. I'm available to start now. Please share additional details. Regards, Mohan Test keyword: 517-DP270CL20ML