Find Jobs
Hire Freelancers

Develop website scraping code

$750-1500 USD

Cancelled
Posted over 10 years ago

$750-1500 USD

Paid on delivery
I need to develop code that can be placed on a server to scrape information from a website. The website is a public site which posts energy prices. Specifically, it is www.newyorkpowertochoose.com. I all providers and other fields to be scrapped from the website in regular intervals (e.g. every 6 hours, every day) and placed into a MS SQL Server database. The website requests specific zip codes to provide information on pricing. I will provide approximately 16 target zipcodes. Each scrub would take all information for all specified zip codes at the interval specified. The data sought form the site includes: - Service Type (Electricity or Gas)* - Zip Code* - Utility Name* - Supplier Name - Rate Type (Fixed or Variable) - Rate (Numeric Only) - Green Offer (Yes/No) - Minimum Term (Months Only) - Cancellation Fee (Numeric Only) - Cancellation Fee Text (used only when qualifying text is present) - Comments (Text) - Date/Time Stamp of Scrub Lastly, I would also like to have an image of the page scrubbed saved in an image file. I will provide the Microsoft SQL Server database IP address and login credentials, a directory to save the page images, and the zip codes to be used for each scrub (you can use 10003 as a test zip code). I need a way to adjust the scrub increment (e.g. days / every 6 hours) and the zip codes in the future if necessary. This adjusting mechanism doesn’t need to be sophisticated it only needs to be able to be done quickly and efficiently. To be specific, I am reasonably tech savvy and could go into the code and adjust it if it’s only located in one or two places though a database driven interval and zip code table might work best. To help ensure we remain aligned throughout the development process, I propose the following gates, but I am open to your input: Phase 1: Developer Review Target Site & Verify Feasibility of Scrubs (each service type, each zip code, at specified intervals) Phase 2: Agree on Target Database Structure Phase 3: Test Single Scrub (1 zip code) with input sent to target Microsoft SQL Server database & page image sent to specified directory Phase 4: Full Scrub Code Finalized The budget is moderately flexible. I am willing to pay a reasonable price to get it done right, in a timely manner, and by a reliable coder with whom I could do business with in the future but without overpaying. I am estimating this project to be in the "Very Small" to "Small" category ($250-$1500 USD) but please indicate if you feel reasonable compensation should be higher or lower. I will consider all reasonable bids.
Project ID: 4940532

About the project

7 proposals
Remote project
Active 11 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
7 freelancers are bidding on average $980 USD for this job
User Avatar
Hi.. Expert web scraper/Data Minor here. Interested in your project. I assure you 100% accurate and good quality work. Ready to start. have a look at PM. Regards
$947 USD in 3 days
4.9 (40 reviews)
5.6
5.6
User Avatar
Hi, I already finished few such projects.
$750 USD in 3 days
4.5 (4 reviews)
3.7
3.7
User Avatar
Let me discuss on the project , please do consider us.
$1,100 USD in 15 days
5.0 (1 review)
3.0
3.0
User Avatar
Hi, Please check PM. Thanks
$1,250 USD in 30 days
0.0 (0 reviews)
0.0
0.0
User Avatar
php/mysql/ajax team
$789 USD in 5 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Client is always right as we believe. Expert in PHP, Wordpress Script Install, Software Development, Web Scraping, Website Design, web scraping with more than 10+years experience and day by day updated ourselves with latest Technology. we are ISO 9001-2008 esteemed company. we have done similar work if you choose me I can start instantly. kindly ,Regards,
$777 USD in 10 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
New York, United States
5.0
1
Payment method verified
Member since Sep 18, 2013

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.