Lean Certified and Automation Expert with over 10+ years of experience; which includes 3+ Years of experience in Big Data using HDFS, Hive, PIG, Sqoop Hbase, Oozie, MapReduce, SparkSQL Programing.
Handled two projects on Hadoop Distributed Technology.
Strong Knowledge on Java, NO SQL databases like MongoDB and HBASE.
Used Sqoop to transfer data between MYSQL and HDFS.
Designed and implemented custom writable, custom input formats, custom partitions and custom comparators in Mapreduce.
Converted existing SQL queries into Hive QL queries.
Implemented UDFs, UDAFs, UDTFs in java for hive to process the data that can't be performed usingHive inbuilt functions
Effectively used Oozie to develop automatic workflows of Sqoop, Mapreduce and Hive jobs.
Exported the analyzed data into mysql using Sqoop for visualization and to generate reports for the BI team
Utilized Agile Methodologies to manage full life-cycle development of the project
Weekly meetings with technical collaborators and active participation in code review sessions with senior and junior developers.
Responsible for creating Hive tables based on business requirements
Implemented Partitioning, Dynamic Partitions and Buckets in HIVE for efficient data access.
Involved in NoSQL database design, integration and implementation.
Loaded data into NoSQL database HBase
Developed PIG UDFs for manipulating the data according to Business Requirements
Optimized HIVE Queries efficiently by using various compression mechanisms