API Expert to grab Millions of Items from external JSON API and save in my SQL or NoSQL DB

Hello Good Day I know how to process large JSON files and storing in database. I also have a great track record here. Looking forward to work with you. Thanks Rinsad

Hello sir , For the things you mentioned , to achieve this I can make efficient php script that will cinnect through curl to the api and will fetch the response from it and will make some udf fubctions that will check whether the items returned from api are already db if yes then we will update otherwise we will do insert. Waiting for your positive response so we can start the work soon. Thanks

I have read your project details, and I am interested in offering my services. I provide 100% satisfaction guaranteed on all my work. Milestone is required to begin; to be released when the job has been satisfactorily completed. I estimate a 4-5 day turnaround (including weekends) on this project. ftp details to your server will be required. Contact me at your soonest convenience so we may get your solution delivered quickly. Thanks, coderXO

Hi, I have 5 years and 3 months experience in HTML, CSS, Javascript, JQuery, PHP, MYSQL, AJAX, Wordpress, Drupal, ZenCart, OSCommerce, OpenCart, Magento, Joomla, Prestashop and Smarty I am expert in this kind of work and can complete all your requirements. I will use CorePHP, jquery and javascript methods do this. I assure you that i will provide you best quality work and you will definitely hire me again for future work. I am mostly available online if you want to discuss anything or have any questions and you will get reply quickly. Get quality work from me within your time. So please award me this project and give me chance to work with you. Looking forward for reply. Thanks Kishan

Hi Sir, I can write an API in golang so parse your json and do the bulk insert to mysql db. golang run in parallel very fast so it can handle big of records to mysql. Thanks Thanh

Hi, I can help you to create a PHP script (no framework at all) to do that and save the data into MySQL DB. But firstly, I need to know the returned data structure of JSON and those external API URL/service(s) so I can review it better and adjust my bid & completion time precisely. For its procedure, briefly, each received items (from external json) will be checked in the database first, if not found then it will simply inserted to the DB, but if found, then it will update the item data. After all items from that external json has been processed then the update date (or time if necessary) from all items in the database will be checked and any not-yet-processed item will be marked as deleted (since it is not provided by that json). Regards, Ibnu

Hi there! I've been working with the PHP stack for over a decade, and that is the stack I'd be using for this project. I'd probably just use one script to read the api output, then fork the php process to run the mysql statements to test the data and update/insert. Would love a few minutes to chat about what columns you'd be wanting to test incoming data against. Regards, Joseph Cardwell

Hi, This looks an easy task. I'll be using PHP code to read the json data which can be initiated with click of a button OR we can setup cron job to run the code at a fixed time each day. We can then store the data in MySQL. kindly ping to discuss further on your project.

Hello, As you mentioned, need to fetch data using JSON and item count to be 100K-1M, using framework would be an overhead for the system. I would prefer: - Core PHP + JSON - MySQLi - Using cron job About Myself: I have been in the industry for a long time now and worked on an e-commerce solution that used to get feed from the vendor on as xml. So more or less, the job is similar and I know the issues that can crop up in such assignments. I am working alone under 'TrickMyIdea' (all-in-one) and would provide the best solution. Thanks & Regards Hardeep Singh TrickMyIdea

Hello, I have worked on many API integrations and consumed the API calls in a right manner. I will use .NET(C#) technologies to do this job. Im over 8 years experience in .NET. If you are interested, Please let me know. Thanks, Sheik

Hi There, I can do this job for you. Thank You

i have done a similar project and can help in implementing the same.

The tech i will use is. 1. use python to fetch data from the web(we should see how they send data, I hope they send data in bulk because requsting for each record is a pain, if they are providing us a file of 10M records in json data, thats OK) 2. python or php to upload data to mysql db 3. Setup cron tab to run every day 15 days is what you need to build test and make sure it is working fine. you will definitely need support till that time. Thanks, Raj

I am interested in this project as it is exactly within my scope of expertise. I am an 10-year experienced web/ app design and engineering consultant. I have very good experience in development of applications, websites and web portals. Worked on integration of various API, Payment Gateways into websites and implemented open source products. I am very anxious to develop a relationship with you and I would be grateful for any opportunity to enable me to earn your confidence in our services. I can personally assure you that your business is very meaningful to me and will receive the highest degree of attention and that your confidence in me will not be misplaced.

hello sir i have 4 years of experience in web development specially in large scale application which process lots of data at a time . as per your requirement I will do this project in hadoop, big data and Java. because that much of data can not be processed using Php and nosql . big data and hadoop can process this data in less time. drop me a message for more discussion. looking forward to hear from you. thanks for reading. have a good day ahead !!

Hi, this is Vikas . I have been into Web Development and Design since past 7 years. I have been creating awesome websites for myself and various clients around the world. I focus on great user experience and efficient code in order to optimize the sites both for speed and performance. I also have hands on open source frameworks like Wordpress, Drupal, Joomla, Laravel, Zend, Symphony, Django, Express NodeJS, MVC, ASP.Net etc. Here is a list of my skills- 1. Custom Website building from scratch 2. Website Performance improvment for faster page load 3. PSD to HTML and HTML to Dynamic Website 4. Building Scalable E-commerce Marketplaces 5. Building Great UI and User Experience 6. API Based Integration into websites. I am good at - PHP, HTML, CSS, Javascript, Jquery, Angular JS, Node JS. I have read your project description and I think I would be the right person to do your project. I will ensure great communication throughout the project with timely updates about the progress of the project. I would like to have a chat with you about the project and discuss more about how I will be approaching this task. Ping me and lets get started. Thanks :)

Hi, As per the requirement, we can use Laravel 5.3 framework which has awesome queue worker which will handle job scheduling. By using job, we can call that JSON API and grap the item and then insert ino mysql database. We can schedule this job for running in specified interval . I recently completed an app in laravel which will grap info from PODIO api and then generate pdf . I can do this job with best quality.

Trabajo con QCodo, puedo hacer perfectamente un proceso vía CronTab que se ejecute cada cierto tiempo y mantenga actualizada la base de datos luego de recibir el JSON.

Hi, I would be using Java with Spring Framework (if it is not too memory intensive for your operation). Initially 2 threads will be started - Thread 1 will begin reading the existing data from the DB into a map wherein ID against the respective product object would be cached Thread 2 will start loading objects from the JSON file into a map using a key. Once both these threads have finished we would compare both these objects maps for changes in attrbutes, addition or deletion of products. Based on the result we would maintain the operations to be carried out as results and perform the same Reading from JSON file will be done serially. However writing to database can be performed in parallel using multiple threads - configurable Since multithreading comes into picture for writing to the DB we would be using connection pooling thus making writing to DB operations faster

Hi, I have 5+ years on working experience with PHP and most of its frameworks, MySql and web services. Working on yii since 2 years , creating high end listing applications for the clients backed by Yii2 and Mongodb. Based on your description, Bigdata will we beneficial to you. Yii2 + Mongodb will give you an fast and efficient solution for the project. Moreover time based cron jobs will look after the daily updates of the data.