Find Jobs
Hire Freelancers

Write a parallel processing program in Python

$30-250 USD

Closed
Posted over 6 years ago

$30-250 USD

Paid on delivery
Suppose I have a lot of zip files, and each zip file contains a folder which has tens of thousands files in it. I need to process those files in all of the zip files, and extract the content to save into a MongoDB collection. I need your help to write in Python, a parallel program which can open 4 or 5 threads (configurable) to process the files within the folder of a zip file. Requirements: 1. The threads work together to process one zip file. If one process finishes its task while the others are still working, it will be on hold and do nothing. 2. Only when all threads finish processing the files in a zip file, they will start working on a new zip file. The zip files should be processed in a sequential order. The file names of the zip files contain dates. 3. Each thread reads files, extracts the content, and saves it to Mongo. I have the code to do this work and I can share it with you. Please demonstrate you have the expertise to finish this work when you bid the project. I'll send you a few zip files once you are awarded with the project. thanks, Richard
Project ID: 15649127

About the project

17 proposals
Remote project
Active 6 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
17 freelancers are bidding on average $188 USD for this job
User Avatar
Hi I checked the requirements carefully. I have rich experiences of multi-threading and multi-processing. I am confident with this job. I am ready to start the work. Best, Zhang Relevant Skills and Experience python, multi-threading, multi-processing Proposed Milestones $200 USD - making the script for extracting the data and saving into mongodb.
$200 USD in 3 days
4.9 (125 reviews)
7.6
7.6
User Avatar
Hello, I'd be glad to implement the parallel processing script for you .To have an idea about my experience in python, i invite you to take a look on my previous project feedbacks on my profile page Relevant Skills and Experience Python Proposed Milestones $255 USD - upon completion looking forward to chat with you soon for more details. Best regards,
$255 USD in 3 days
5.0 (149 reviews)
7.2
7.2
User Avatar
Hello sir, to be very specific, i can write a program which will first extract zip file and divide the files in parallel threads and every thread is responsible for processing set of files. Relevant Skills and Experience every thread will also responsible for saving data in mongodb and my program will take care of the sequential order of zip files and work accordingly. Thanks Proposed Milestones $246 USD - final milestone
$246 USD in 7 days
4.9 (75 reviews)
7.0
7.0
User Avatar
Hello I am glad to see such a project here, involving both Python & parallel programming seems to be good fit from my background, your objective seems clear to me so I feel comfortable to take the job. Feel free to contact me and share the code that saves content in Mongo Thanassis
$155 USD in 3 days
4.9 (110 reviews)
6.3
6.3
User Avatar
Sir/Madam, I am an experienced Python developer with familiarity with OOPS concept and core Python with multiprocessing. I have worked on projects involving complex data structures. I can write highly optimized code for you. Looking forward to working with you on this project. Yash
$222 USD in 5 days
5.0 (42 reviews)
5.2
5.2
User Avatar
I am a parallel programmer, and know how to handle threading. Relevant Skills and Experience Python, parallel processing Proposed Milestones $140 USD - final milestone
$140 USD in 4 days
4.9 (40 reviews)
5.0
5.0
User Avatar
Hi, I should be able to help you with this. Relevant Skills and Experience Have years of experience with Python and multi threading/processing programming. Proposed Milestones $50 USD - Upon the delivery of the script. $116 USD - Pay the rest after testing it yourself. I have a couple of questions, please reach me through chat.
$166 USD in 2 days
5.0 (3 reviews)
3.4
3.4
User Avatar
A proposal has not yet been provided
$150 USD in 5 days
5.0 (2 reviews)
2.4
2.4
User Avatar
I will need some more details about the OS to be run on, and the kind of data to be processed, and I will do it asap Relevant Skills and Experience I've already made a lot of projects using python, that contains threads and multiprocessing Proposed Milestones $222 USD - Finish the task
$222 USD in 10 days
4.4 (5 reviews)
2.4
2.4
User Avatar
Hi, I can help you with distributed/parallel processing of these zipped files and importing them into MongoDB with your rules regarding the threads sequence and zip file sequence. Relevant Skills and Experience I am an expert Python Developer and i have experience with multiprocessing module and parallel programming in Python. I am also experienced with MongoDB. Proposed Milestones $388 USD - Delivery of the Processing Script Additional Services Offered $150 USD - MongoDB optimizations $100 USD - Linux server setup Let's discuss some more details about the file structure and their equivalent collection in MongoDB
$388 USD in 6 days
5.0 (1 review)
1.8
1.8
User Avatar
A proposal has not yet been provided
$133 USD in 3 days
5.0 (2 reviews)
1.6
1.6
User Avatar
Hey ! I'm SAYAN PROGRAMMER I've reviewed your complete job description, and I fulfill all the qualifications required for this project. I always try to provide good quality work to my clients. I have more than 15 years of experience in this field. I have completed 200 projects with clients 100% satisfaction and I am sure if you will respond me then I will be able to explain my skills as well. My key skills are: Python, C #, C + +, C programming ,JAVA, Software Development, Software Architecture, PHP, Java script, Software Engineering, and much more. Looking forward to work with you on this project. THANKS Regards SAYAN PROGRAMMER
$30 USD in 2 days
0.0 (0 reviews)
0.0
0.0
User Avatar
A proposal has not yet been provided
$166 USD in 4 days
0.0 (0 reviews)
0.0
0.0
User Avatar
HI, one year ago I coded threads for parallel processing of images comparison ( using libs from Anaconda). Now I'm seeing at that code, in wich i pass as parameter a # of threads to run at same time.. Relevant Skills and Experience ..I run it to process thousands of images and worked very very well. In the threads were included access to files and database with no conflicts. skills: Python during the last 3 years. See my resume Proposed Milestones $10 USD - define where to store temp unzipped files for processing $50 USD - Define Threads classes to process info and main algorithm $30 USD - Process sample zips showing ouputs as to save to Mongo $65 USD - Work done with your approval Additional Services Offered $1 USD - Anything you need from functional analysis to develoment I think everything is well defined. So no additional questions. Thank You.
$155 USD in 3 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hello I am a master student in computer science with some experience in python, java and C++. I am specializing in machine learning and use especially Python. Therefore I think this task is perfect for me. Bests Hugues
$244 USD in 6 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Herndon, United States
5.0
27
Payment method verified
Member since Jul 7, 2012

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.