We need to parse a large file (20GB-25Gb) what is ';' separated and has from 2-??? columns per row
Message_type is in first column the rest is the message itself nad we need this to be in high performance
We need to generate a new fil for every message_type (24 types) and of that 20Gb of data roughly 3Gb are data with a tag we are interested in.
The logic for the tags needs to be dynamic (if we add a file load_whattoload that needs to be imported dynamically, code for that that is somewhat wokring is provided, but can be changed if needed) We don't wan to have 24 IF IFEL IFEL ELSE statements
Since we are using pandas we the file needs to parsed with pandas, unless there is a really good reason why not to.
Wokring simple code with IF IFEL is provided, with a few tags completed and a small and medium dataset sample aswell
We need this to be done ASAP and needs to be done within 12h after acceptance.
IF you know SQL that is big plus since if you are a dedicated coder, you can keep converting the current SQL based parser functions to Python.
33 freelancers are bidding on average €509 for this job
PYTHON developer I have read your job description and I am pretty sure that I can complete every bit of your requirements. Further details and cost can be discussed in chat
Hello? How are you? Thanks for posting this job. I am a Senior Python developer with 7+ years. I have been doing this kind of python projects many times. Please let me know. Best Regards
Hello! I'm a pandas expert! All my scrapping project has a pandas feature on it. I can help you finish this task , with the best performance. Contact me. Thanks.
Hi, I have 6+ years of experience in application development. I can parse the doc as you need with Pandas but 12 hr time is very short. At least should be 3 days.
Hello, I have been did the same project before and so my capabilities and experiences is the best thing for this project. Let's discuss the detail. Regard.