Find Jobs
Hire Freelancers

Programatically searching a huge database and sorting results

$10-40 USD / hour

Completed
Posted over 3 years ago

$10-40 USD / hour

Programatically searching a huge database and sorting results The goal is to find all of the most common strings that precede each word or phrase in domain names. This is NOT a manual data entry job! We have a very large database of ALL .com domain names and another list containing thousands of unique words or phrases. We need to take each word or phrase from the second list and search the whole domain database to find every instance of that word or phrase ONLY IF it starts within the first 7 digits of the domain name. The goal is to create a list of the most common words that precede each term, in order of most popular to the least popular, and show the quantity of each. To give you an example, if the original term is “REBATE” (term number 1753) you would search the database of nearly 100 million domain names, finding any domain name that contains the term REBATE starting in the first 7 digits of the number. Then list all of the occurrences sorted by the first 7 digits in order of most frequent to least. So the outcome should look something like this (JUST AN EXAMPLE!) 1753,REBATE THE-REBATE,18 YOUR-REBATE,9 CASH-REBATE,8 EASY-REBATE,8 RAPID-REBATE,7 HOME-REBATE,5 MAILIN-REBATE,5 TAX-REBATE,4 NOWAIT-REBATE,3 GETA-REBATE,2 ETC… You also have to be able to open a rar file. The output should be single text file listing multiple terms, one after another like the above. We have thousands of terms to search. Our goal is to find who can do this the most efficiently, so we will award this to multiple freelancers asking them each to put in ONE HOUR worth of time. Then we will select whoever completes the most terms (and does it properly) to continue. We may work with more than one or may find one freelancer stands out and give them all the work. I have a good track record as an employer with Freelancer and an even bigger positive record with Upwork which I’ve used for several years. I’m doing it this way because it’s impossible to tell from someone’s profile how efficient and accurate of a worker someone is just from their profile or talking to them. This will be a lot of work for whoever is best at it. The worst case, if someone else is faster and gets more done, you’ll get a positive review and a completed job. The first step is to answer some questions. ***No automatic proposals. If you don’t answer these questions fully, you won’t be considered. What tools would you use to do this? Have you worked with a database of 100 million records before? What’s the biggest database you’ve worked with before? What would be your goal or expectation in: Terms Searched and Sorted / hour?
Project ID: 27394396

About the project

26 proposals
Remote project
Active 4 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
Hello. I've checked your description in detail. ***** I'm using Python for this, and have deal with oracle phone number database. ***** I am very experienced, honest, have good skills, and also have much availability to work at anytime. I am looking for long-term relationship with my customers. I wish to work for you, please open chat with me. Thank you.
$35 USD in 40 days
0.0 (0 reviews)
0.0
0.0
User Avatar
$0 USD in 40 days
5.0 (16 reviews)
6.0
6.0
26 freelancers are bidding on average $25 USD/hour for this job
User Avatar
Hello there! Happy to bid here since I have the capability to build your project. I am a database expert and have rich experience in manipulating, sorting data. So I think you’d better discuss with me for clear requirements, save your time, excellent result, reasonable price, good maintenance and so on. I’ll give it my best shot and devote you. Looking for your good reply. Yours Faithfully. Yana.
$25 USD in 40 days
4.9 (6 reviews)
5.1
5.1
User Avatar
Hi there Just as a short introduction, we have a team of experienced Machine Learning and Deep Learning Experts who would be glad to help you shape your requirements into products. They have built many machine learning models (like face recognition app and Histopathologic Cancer Detection) before and they are professional in Python, Data Segmentation, OCR, Image and Video Processing, Matlab, Octave, , MySQL, node.js, android and IOS programming. Please contact me for more details. Hope to hear from you soon.
$25 USD in 40 days
5.0 (4 reviews)
4.1
4.1
User Avatar
Greetings. Thanks for your post. As a senior fullstack developer who has 10+ years of experience, I can deliver satisfactory product in a high quality. What tools would you use to do this? I will use c/c++ because it's the fastest and best solution and fully customized. Have you worked with a database of 100 million records before? Yes, using c++ in QT framework. What’s the biggest database you’ve worked with before? MySQL What would be your goal or expectation in: Terms Searched and Sorted / hour? Maybe 10000~30000/hr. But it will get faster going on.. I am very confident with my skills and I would like to help your business by doing my best. Please contact me for further discussion. Thanks & Regards.
$25 USD in 40 days
5.0 (3 reviews)
4.1
4.1
User Avatar
you don't need a database for this type of job. if you have a dB you need just a plain csv output and some Linux bash command to filter and sort string inside your file. as long as the file is smaller than available ram, parsing it will be really fast without wasting time with mysql. parsing and sorting will be blazing fast... just few lines of code in order to format the result as you like will be needed at the end. Sorry to not put here more details but I don't reveal everything here. I will share the Linux script with you at the very end by the way. and if you haven't it available I can provide a simple web interface to make searches with some API rest ps. I managed a 100gb mysql database so I know a bit about these things best regards
$28 USD in 20 days
5.0 (6 reviews)
4.2
4.2
User Avatar
Hello there, I throughly checked the requirements and really well understand it. First I need to know the type of DB we're using like: Mysql, Oracle, or any other. Frankly says, It can't be possible to traverse 1 Million db iteration by one script. What I suggest is to use Python like programming langualge with splitting the search into different number of jobs and execute them in paraller. Here is the solutions, I found best for this task: What tools would you use to do this? -All the above mentioned automation can be effectively achive in Python. What’s the biggest database you’ve worked with before? -Yes I worked on sites like mawjuud which is a property site and have 5k listings post daily with my cron job. What would be your goal or expectation in: Terms Searched and Sorted / hour? -We can't says the exact time estimation but, dividing the search result into multiple chunks and execute in paraller will definately reduce the search time. Talking about myself, I've great knowledge and experience in Scripting, Automation, Google Sheets, Python, Automate testing, etc. Even I have designed 100+ highly technical chatbots for Hybrid chat. Apart from that, you can also see my profile for the feedback from my past clients. Let me know if you are willing to discuss things further with me. I'm excited to work with you. Cheers, Rishab Singla
$10 USD in 40 days
5.0 (3 reviews)
3.3
3.3
User Avatar
I have very good parallel programming skills, which we can use to solve your task. What tools would you use to do this? - C/C++ and OpenMP or Pthreads Have you worked with a database of 100 million records before? - No. What’s the biggest database you’ve worked with before? - Maybe < 1 milion. What would be your goal or expectation in: Terms Searched and Sorted / hour ? Don't know. Have to run and see.
$20 USD in 40 days
5.0 (5 reviews)
3.2
3.2
User Avatar
Nice to meet you I am a Machine Learning expert In all domains such as industry, economy and biomedicine, any hard problem can be resolved using Artificial Intelligence Techniques. In my PhD research, I have developed a Hybrid Intelligent Method which can be used for identification and modeling of complex and nonlinear not systems or for prediction of chaotic behavior. After ten years of manipulating some paradigms of machine learning techniques (such as Support Vector Machine and Neural Networks) with some computational approaches (such as fuzzy logic and wavelet analysis) and other theories (such as kernel methods and chaotic theory), I am sure that if the opportunity is offered to me, I will be able to tackle any problem using my rigor of mathematician, my varied knowledge of computer science as well as my great passion for the modern physical sciences. I am good at big-data-sales,data-analytics,data-extraction,data-mining,data-processing,data-scraping,web-scraping Thank you
$34 USD in 40 days
5.0 (1 review)
3.0
3.0
User Avatar
Please give me a chance i am beginner i will try my best i will make u happy to my work i alaways do my best in everything work
$25 USD in 40 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi I checked your post with title "Programatically searching a huge database and sorting results ". I am familiar to python. I want to discuss your project in detail. please contact me thanks
$15 USD in 38 days
0.0 (0 reviews)
0.0
0.0
User Avatar
I am a professional data entry expert. I can assure you, that I shall give you top class work. You will like to come back to me again and again.
$25 USD in 40 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Dear Sir / Mam, I am interested in your job vacancy. I have bundle of experience of research base projects in the past. I am very confident to provide you accurate results as I understand the basic requirements. I would like to take the test as well and I assure you quality plus quantity. I beleive that communication is the key to success. Looking Forward to serving you, Regards Waqar What tools would you use to do this? I will use tools like similar website, google extensive research. Have you worked with a database of 100 million records before? No What’s the biggest database you’ve worked with before? 90k database management of the emails. I worked as a recruiter and I manage huge database, adding the data, updating the records etc. What would be your goal or expectation in: Terms Searched and Sorted / hour? I think 20 or more leads per hour.
$10 USD in 40 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hello! I provide data collection services, through web scraping and text mining, for data interpretation, comparison, composition, distribution and relationship. Feel free to contact me!
$25 USD in 40 days
0.0 (0 reviews)
0.0
0.0
User Avatar
I have 9 years of experience in software development. I recently worked on something similar with client where they will search address based on some keywords like 'street 9' searched within 100million property addresses, giving top 10 results with matching percentage. I would create a website or desktop application using .net and for db I already know how to get most optimized query to get the result.
$22 USD in 10 days
0.0 (0 reviews)
0.0
0.0
User Avatar
I will be using SAS to perform the work I have got more 15 years in data analytics and data quality
$22 USD in 10 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi, I like your approach, nice. The will answers your questions are as follows: I would use python, I am learning HTML, php and javascript at the moment, but like coding at a lower level. I coded in ruby for Standard Bank South Africa. I am enjoying python now. I worked at Standard Bank, the databases were in Oracle, but for my last stint we were using mongoDB and I like it. I use a PC, but it isn't the slowest and would have to clean up a bit of space, but I could make some space. I worked for corporate an investment banking, the databases were huge. We had three sites with the databases backed up using veritas cluster, which I did. I have no idea of the expectations per hour, but it wont be that slow. I have an i7 at 2.80Gig with 16Gig of RAM. I am also using linux mint 19.3. Storage may always be cleaned up and sorted. Once I have some data we can agree on what needs to be done. Once I understand what needs to be done I will see it as a challenge and go for it. I can always email my CV. This does sound like fun. The sorting and comparison of such large data will prove to be a challenge. Off the top of my head I would extract the data first and create a new DB with it and then sort it. Well thats me, I am working at learning the HTML php relationship for passing variables from forms at the moment. Let me know, it sounds like a fun challenge. I would just have to focus on python for a little to get my mind back into it. Thank you.
$20 USD in 50 days
0.0 (0 reviews)
0.0
0.0
User Avatar
I've skills in data entry , data processing , MS-Excel , MS-Word and many more. Plus I complete my work right on time and if you hire me I'll never let you down . It would be pleasure to work with you . Ping me if you want to work with me. Thanks Deepu Sinha
$40 USD in 72 days
0.0 (0 reviews)
0.0
0.0
User Avatar
i read your contents and understand what is work. i will do this job. i will filter the data and find all of the most common strings that precede each word or phrase. i am expert in Ms Excel. I seen your contents. I will do this job. I can do this work deliberately with accuracy and speed. Although I am new in this website but I have a vast experience of 20+ years of with various public organizations as well as individual throughout my career deliberately, i can work on following areas: MS EXCEL: I am expert of EXCEL, Data entry as well as creation of database files/Lead lists, statements, Ledgers, I can do cross matching of data with more than one files. I am expert in data analysys, formatting, sorting and filtration of data. I can also do data cleansing, I can do many tasks by using different formulas, sub-total, Vlookup etc. i can add tables & graphs, MS WORD: I can do many jobs in Ms Word as like composing and formatting letters, books, summary, press release, student notes, examination papers etc. I can also convert data form PDF to word/excel. I also did project of Copy/Cut & paste of data with good speed. I am sure, my work will reflect about my expertise with accuracy and speed. During project I will be in contact with you for guideline/suggestions. This will be helpful to improve the quality of work. I will try my best to make you satisfy from my work.
$23 USD in 42 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Salt Point, United States
5.0
7
Payment method verified
Member since Dec 12, 2019

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.