In Progress

Website Scraper

Need to scrape [url removed, login to view] and store various fields to a mongoDB. Specifically, it will have the ability to scrape the RSS feed, link below:

[url removed, login to view]

The RSS feed has numerous categories, example one above, but interested in scraping the below one:

1. BreakingNewsReleasesEnglish

a. [url removed, login to view]

The code needs to have the ability to scrape any category via just a variable change in code via a look-up table. For example, if a user wants to scrape “BreakingNewsReleasesEnglish”, a look-up table needs to match a dynamic user entered variable to the proper RSS feed, in this example:

[url removed, login to view]

And scrape that category. Provide a section in a readme file where this variable resides.

Only scrape articles that are in English, i.e. <language>en</language>

For the link immediately found in the main RSS feed and the link leading to the FULL article, the following MUST be stored to a mongoDB (allow NO duplicates via upsert or other method based on some unique parameter to each article)

Skills: NoSQL Couch & Mongo, PHP

See more: php mongo, mongodb php, mongodb c++, mongo or, xml scrape, nosql php, mongo, couch, leading article, feed full article, xml file language english, mongodb fields, mongodb com, table scraper, php xml table example, english articles website, xml category rss, scraping xml file php, php scraper rss, website scraping code, articles scraper, scraper xml, scrape full article, xml feed table, need change english

About the Employer:
( 7 reviews ) New York, United States

Project ID: #5482675

Awarded to:

kadukeitor

Hi .. can do this job quickly .. the specifications are very clear .. . we will in constant communication ...

$200 USD in 7 days
(39 Reviews)
5.8

9 freelancers are bidding on average $163 for this job

rajeshsonisl

Hello, With 99% completion rate, 650+ successfully completed projects, and a 5.00 reputation (maximum possible, 5.0) (Yes, not even 4.99 average rating, can be verified on my profile page !!)... you can never go wro More

$309 USD in 1 day
(804 Reviews)
8.3
ebson

I can easily do this script as long as rss fields remain the same for different categories. You can pass the category as a parameter to the script example: ./[url removed, login to view] BreakingNewsReleasesEnglish I see php men More

$148 USD in 3 days
(29 Reviews)
5.7
anuyadav1

i am well experienced with scraping websites , i can easily scrape this one and save in mongodb . .

$200 USD in 5 days
(31 Reviews)
5.4
iautomationus

For the variable, there could be something simular to [url removed, login to view] I can provide the script, to take that variable, and scrape the appropriate feed into the database with no duplicates. Just take note, that e More

$144 USD in 1 day
(42 Reviews)
4.8
Dhruvika111

Dear aptocap,Greetings! As per our previous and current working experiences samples is the best and perfect way to judge the work quality and accuracy of service provider and its also allowed us to calculate the exa More

$67 USD in 1 day
(5 Reviews)
4.3
BrothersTeam

A proposal has not yet been provided

$155 USD in 3 days
(17 Reviews)
4.1
Toperfection

Dear "aptocap" Hope you are doing well. I have reviewed the project details and would like to offer our services. We have completed many Research/Data collection/Product add/Data mining assignments on [url removed, login to view] More

$78 USD in 1 day
(1 Review)
0.8
ale3an

A proposal has not yet been provided

$166 USD in 3 days
(0 Reviews)
0.0