About

Rajendra Singh is a highly skilled data engineer with over 6 years of experience in the software industry. He has worked in various domains including telecom, sales & marketing, electrical manufacturing, and automobiles industry. Rajendra is proficient in developing end-to-end applications using technologies such as Spring Boot, Java, REST, Microservices, Hibernate, and SQL. He has a solid understanding of CI/CD pipelines with Git, Jenkins, Docker, Kubernetes, and Terraform. Rajendra is also familiar with cloud services such as AWS stack - EMR, Lambda, SQS, Kinesis, EC2, S3, ECS, and CloudWatch. He has experience with BigData technologies such as Spark and EMR, and has worked with job orchestrator/schedulers like Airflow. Rajendra has experience with messaging queue systems like AWS Kinesis and SQS, and query engines, schema cataloging engines, and warehousing tools like Athena, Glue, and Snowflake. He is skilled in writing automation scripts mainly in Python language (pyarrow, boto, pandas). Currently, Rajendra is an Associate Staff Engineer at Nagarro, part of a Data Platform team under Nagarro Centre of Excellence - BigData/AI Practice. He is working on a project titled "Platform & Data Management (Data Ingestion)" where he is configuring new and migrating old Airflow Ingestion pipeline Dags to a new Airflow Data Platform. Rajendra is also performing data ingestion over different channels Glue/Athena and Snowflake to support various Hadoop file formats. He is developing and managing microservices that play an important role in building Airflow Dags. Rajendra is writing/modifying/performance-tuning Spark transformation jobs and process automation/validation scripts/lambdas in Python. He is also working on new ingestion-pipeline, supplier change requests, managing and giving production support for the ETL pipelines. Rajendra is experienced in performing historical loads (TBs), writing SparkSQL queries, and Pyspark redaction scripts to remove PII data from different channels. He is configuring snowflake external tables and snow pipes. Rajendra has also developed an Airflow pipeline profiler that captures data including Dag execution timing and cost incurred per Dag run. In the past, Rajendra has worked as a Senior Software Engineer at Mindfire Solutions and as a Software Engineer at EDS Technologies. At Mindfire Solutions, he worked on a project titled "Phototype Suite" where he was responsible for