Find Jobs
Hire Freelancers

Crawler for expired domains

$30-250 USD

Closed
Posted about 10 years ago

$30-250 USD

Paid on delivery
I am looking for a guy who can create a polite python crawler to find expired domains. Following functionality: (1) Indexing domains - Crawler crawls through the web and saves domainnames in a database - There shall be the setting opportunity just to crawl domains with a certain ending (2) Checking website status - Another crawler regularly and automatically visits the domainnames saved in database - By using HTTP-Header-Status or other ways, it shall be checked whether is domain is indexed in the nameservers (3) Displayment of domains - In case the domain is not in the DNS entries anymore, the pagerank of the domain is checked - Furthermore, the validity of the pagerank is checked These two parameters are then set in a list. PLEASE JUST BID IF YOU CAN COMPLETE FULFILL THE REQUIREMENTS!
Project ID: 5469240

About the project

4 proposals
Remote project
Active 10 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
4 freelancers are bidding on average $264 USD for this job
User Avatar
I have made web crawlers in past. I have very good understanding of requests and https communication. I have more than 4 years of professional experience. I can do this job
$250 USD in 5 days
4.8 (11 reviews)
4.7
4.7
User Avatar
I have a lot of experience in Python scraping and MySQL. I think we can obtain the pagerank through some online service that doesn't employ bot-protection, or some other way. I am really interested and I'll do my best to get it done. Thanks
$222 USD in 3 days
5.0 (1 review)
2.2
2.2
User Avatar
We are working for last 5 years in this field. Many best Web Developers are working as a team. We will delivered the best quality product on time. If you want see our work we will send you the sample. Please give us a chance to work with you
$222 USD in 7 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of GERMANY
Lüneburg, Germany
5.0
105
Payment method verified
Member since Jul 21, 2013

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.