Senior Data Engineer – Apache Nifi

Job Summary:
We are seeking a highly experienced and results-driven Senior Data Engineer with 8+ years of relevant experience in building and maintaining scalable ETL data pipelines. The ideal candidate must have strong expertise in Databricks, Python, ETL tools, and cloud platforms like AWS or Azure. Exposure to Apache NiFi, KNIME, and modern data architectures is a strong advantage. This is a 100% onsite role based in Bangalore or Hyderabad.

Key Responsibilities:
Design, develop, and maintain scalable ETL pipelines using Databricks, Apache NiFi, KNIME, and other integration tools.
Build data workflows to integrate data from diverse sources including APIs, databases, flat files, and cloud storage platforms.
Write clean and optimized Python and SQL scripts to process, transform, and analyze data.
Leverage cloud platforms (AWS/Azure/GCP/Snowflake) for data storage, orchestration, and performance optimization.
Collaborate with cross-functional teams including data analysts and scientists to understand and deliver data solutions.
Ensure high data quality, consistency, and integrity across all systems.
Implement data governance, lineage, and documentation practices.
Monitor production data pipelines, troubleshoot issues, and ensure system reliability.

Mandatory Skills & Qualifications:
8+ years of relevant experience in Data Engineering.
5+ years of hands-on experience with Databricks.
4+ years of professional Python scripting experience in data transformation.
3–4 years of experience with ETL development and pipeline orchestration.
Hands-on experience with cloud platforms (AWS or Azure required).
Proficiency in SQL and data modeling concepts.
Experience with data warehousing tools like Snowflake, Redshift, or BigQuery.
Solid understanding of data quality, governance, and security principles.

Preferred Skills:
Working knowledge of Apache NiFi or KNIME for data integration.
Experience with workflow orchestration tools such as Apache Airflow or Luigi.
Exposure to real-time processing frameworks like Apache Kafka or Spark Streaming.
Experience with CI/CD pipelines and version control systems (Git).

Qualifications:
Bachelor’s or Master’s degree in Computer Science, Information Systems, or a related field.

Why Join Us?
Opportunity to work on enterprise-scale data platforms and innovative cloud projects
Exposure to cutting-edge technologies like Databricks, AWS, NiFi, and modern ETL tools
Collaborative work environment with cross-functional teams and domain experts
Long-term project with career growth and skill enhancement