Azure Data Engineer - Data Factory/Data Pipeline

Job Title : Azure Data Engineer Job Type : Full-time / Contract Experience : 5 Years About the Role : We are seeking a skilled Azure Data Engineer with hands-on experience in Azure Data Factory (ADF), Power BI, Azure Synapse Analytics, and PySpark to join our growing data engineering team. The ideal candidate will have a strong background in cloud-based data engineering, data transformation, and data visualization, with a passion for building scalable and efficient data pipelines. As an Azure Data Engineer, you will be responsible for developing and managing data integration workflows, designing and optimizing data models, and ensuring the successful delivery of data solutions. You will collaborate with data scientists, business analysts, and other stakeholders to create end-to-end data solutions that support analytics, reporting, and business intelligence. Key Responsibilities : - Design, implement, and maintain data pipelines using Azure Data Factory (ADF) to move and transform data across various sources and destinations. - Leverage Azure Synapse Analytics for building and optimizing data warehousing solutions, and implement real-time and batch data processing strategies. - Develop data models, structures, and workflows using PySpark for large-scale data processing and analytics. - Create interactive Power BI reports and dashboards for data visualization and business insights. - Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications for data solutions. - Ensure data quality, integrity, and security across all stages of the data pipeline. - Optimize and tune data pipeline performance, monitor data pipeline jobs, and troubleshoot issues. - Implement best practices for data engineering processes, including data governance, scalability, and automation. - Provide technical leadership and guidance on Azure Data Engineering technologies to junior engineers and other team members. Required Skills and Qualifications : - 5 years of experience as a Data Engineer or in a related role. - Strong expertise in Azure Data Factory (ADF), with hands-on experience in designing and orchestrating data pipelines. - Proven experience with Azure Synapse Analytics, including data warehousing, analytics, and integration with other Azure services. - Proficiency in PySpark for distributed data processing and building scalable data processing solutions. - Hands-on experience with Power BI for building interactive reports and dashboards for data-driven insights. - Strong SQL skills and experience with data modeling, transformations, and querying large datasets. - Solid understanding of ETL processes, data integration, and data governance. - Familiarity with Azure services like Azure Blob Storage, Azure SQL Database, Azure Data Lake, and Azure Databricks is a plus. - Experience with data version control, deployment, and automation tools. - Excellent problem-solving skills and the ability to troubleshoot complex data pipeline issues. - Strong communication skills with the ability to work effectively with both technical and non-technical teams. Preferred Qualifications : - Experience with Azure Databricks and Spark clusters for big data processing. - Familiarity with Azure Machine Learning or other advanced analytics tools. - Knowledge of DevOps practices and CI/CD pipelines for automating data pipeline deployments. - Experience with agile methodologies and working in agile teams. (ref:hirist.tech)

Location: hyderabad, IN

Posted Date: 11/14/2024