Find Information from Websites


This is a project to scrap websites and to load results in a MySQL database.

This database will then be used by our content managers to write short descriptions of results and route users to the original website.

The data consists in interesting activities to be performed in a given city (restaurants, bars, visits, highlights...with addresses, descriptions, pictures...).

The developer will have to:

- Develop a web scraper scrapping 3 different web sites (these sites will be provided once the freelancer selected)

- Load HTML pages into a collection [url removed, login to view] class (Object Oriented programming, class to be developed)

- Load the Activity objects into a MySQL database via an [url removed, login to view] object

- When loading the content in the database, the code will have to check if a similar activity name already exists in the database for this city.

If the activitiy already exists, from the same website, the existing entry is updated

If the activity already exists, from another website, a new entry is created by it is linked to the existing one

If no matching name is found, a new entry is created.

- IMPORTANT note: the scraper MUST respect the [url removed, login to view] policy of the website. If the [url removed, login to view] of the provide websites prevent from doing this project, we will provide you alternative websites

- Pictures will be downloaded into a local folder (the URL to this folder will be loaded in the database)

- Build a simple HTML page to test the crawler. The HTML page will allow the tester to do the following things:

1. Select on of the 3 web sites

2. Manually enter the name of a city

3. The page will build the corresponding URL and start scarping it and loading in the database

- The source code will have to be commented


- Programming language: PHP 5.5.3

No PHP framework is authorized on this project, only standard PHP function and packages have to be used.

Programming must be object oriented.

PDO must be used for data access layer.

- Database : MySQL 5.5

- The deliverables will be tested using a MAMP apache and mysql installation on a Mac OS X environment

The developer will be provided with:

- a graphical datamodel

- an SQL file to create the corresponding data structure

- the URL of the 3 web sites to scrap

The project will consist in 3 steps, which will divide the payment:

- Delivery of the scraper + loading in the database for the 1st website : 50%

- Delivery of the scraper + loading in the database for the 2nd website: 25%

- Delivery of the scraper + loading in the database for the 3rd website: 25%

Good luck!

Skills: Data Entry, Microsoft, Microsoft Office, Photography, Windows Desktop

See more: we find you web developer, we develop websites, websites programming, websites develop, websites created by freelancer, websites collection, website for page managers, web scarping, web page scrapping, web object oriented programming, web developer freelancer sites, web developer collection, web crawler developer, tester it freelancer, steps to develop website, sql programming language, sql developer freelancer, sql database developer freelancer, sites to find web developer, simple programming websites

About the Employer:
( 0 reviews ) Nepal

Project ID: #6251972

12 freelancers are bidding on average $160 for this job


A proposal has not yet been provided

$250 USD in 5 days
(2 Reviews)

I fully understood your job description. I'm really interested and ready to work as soon as possible. I hope you consider me as part of your staff. I'm a fast-learner, speed typist, flexible and dedicated. I am loo More

$50 USD in 1 day
(3 Reviews)

Dear Hiring Manager, We are ready to work with you, Our company is expert in all type of Web Scrap, Data entry and Data Research work in excel, word and csv. which can use various functions like Vlookup, concatenate More

$150 USD in 5 days
(1 Review)

A proposal has not yet been provided

$155 USD in 3 days
(0 Reviews)

I am a sincere and hard working person.I am willing to work in this field.I ready to work any [login to view URL] you give a chance to me.I am surely prove my sincerity.

$200 USD in 3 days
(0 Reviews)

Hello! I am a Software Engineer from Spain, I want to help you with your project. I have experience scrapping web sites with PHP, in fact, in my last projet ( [login to view URL]) I have to do this in many web sites, f More

$166 USD in 4 days
(0 Reviews)

I am MBA holder , I have efficiency in Microsoft Office 2010.I am a talented and experienced Data Entry Operator. I have good experience with Data Entry and I have been working as a Data Entry Operator for many years. More

$155 USD in 3 days
(0 Reviews)

A proposal has not yet been provided

$155 USD in 3 days
(0 Reviews)

Hello! Good day! I am Hannah Andrea Tallo. I believe that I am fit for this job and able to meet your standards. I am hardworking, reliable and persistent when it comes to my work. I hope I'll be a given a chance. Than More

$161 USD in 3 days
(0 Reviews)

A proposal has not yet been provided

$166 USD in 3 days
(0 Reviews)

A proposal has not yet been provided

$155 USD in 3 days
(0 Reviews)

I have done B. Technology in Computer Science Also I work on many regular expressions for scrapping data. I am having more then 5 years experience working with php scripts and mysql queries, I am also very much expe More

$155 USD in 3 days
(0 Reviews)