Evnek - Azure Data Engineer - SQL/PySpark

Job Description : Data Engineer. Location : Pune, India (Work from Office). Experience : 5 Years. Notice Period : Immediate Joiner. Job Type : Contract. Role Overview : We are seeking an experienced Data Engineer with over 5 years of experience to join our team in Pune. The ideal candidate will be an expert in SQL, with at least 3 years of hands-on experience in PySpark, and a solid 2 years of experience working with Azure cloud services. You will play a critical role in designing, developing, and maintaining our data infrastructure to support various business needs. Key Responsibilities : - Data Pipeline Development : Design, develop, and maintain scalable and efficient data pipelines using PySpark and SQL to process large volumes of data. - Data Integration : Integrate data from various sources into a unified data warehouse on Azure, ensuring data accuracy, consistency, and reliability. - Data Modeling : Develop and maintain complex data models to support data analysis and reporting needs. - Performance Optimization : Optimize and tune SQL queries and PySpark jobs for performance, ensuring minimal latency and high efficiency. - Azure Cloud Management : Utilize Azure services such as Azure Data Factory, Azure Databricks, and Azure Storage to manage and orchestrate data workflows. - Collaboration : Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs. - Data Quality : Implement data quality checks and monitoring systems to ensure data integrity and accuracy. - Documentation : Create and maintain comprehensive documentation of data processes, systems, and architecture. Required Skills and Qualifications : - SQL Expertise : Advanced proficiency in SQL, with experience in writing complex queries, stored procedures, and performance tuning. - PySpark Experience : At least 3 years of experience working with PySpark, with a strong understanding of Spark architecture and distributed computing. - Azure Proficiency : A minimum of 2 years of experience in Azure cloud services, including Azure Data Factory, Azure Databricks, and other related services. - Data Warehousing : Experience with data warehousing concepts and ETL processes. - Problem-Solving : Strong analytical and problem-solving skills with the ability to work independently and as part of a team. - Communication Skills : Excellent verbal and written communication skills, with the ability to explain complex technical concepts to non-technical stakeholders. Educational Background : - Bachelor's degree in Computer Science, Information Technology, or a related field. Preferred Qualifications : - Experience with other cloud platforms like AWS or Google Cloud. - Knowledge of data governance and security practices. - Familiarity with CI/CD processes and tools. (ref:hirist.tech)

Location: Pune, IN

Posted Date: 10/9/2024