Arthya Tech Solutions - Senior Developer - Azure Databricks

Job Title : Senior Developer - Azure Databricks Company : Arthya Solutions Position Summary : - At Arthya Solutions, we are building a Global Competency Center (GCC) for our client, a US-based producer of dairy products. - As a Senior Developer, you will provide technical leadership to Agile teams within our Business Analytics & Insights (BAI) team. - You will work closely with cross-functional teams to deliver high-quality solutions across domains including Supply Chain, Finance, Operations, Customer Experience, HR, Risk Management, and Global IT. Responsibilities : - Develop a comprehensive technical plan for migration, covering data ingestion, transformation, storage, and access control using Azure Data Factory and Azure Data Lake. - Design and implement scalable and efficient data pipelines using Azure Databricks to facilitate seamless data movement from multiple sources. - Develop scalable and reusable frameworks for ingesting data sets. - Ensure data quality and integrity throughout the entire data pipeline by implementing robust data validation and cleansing mechanisms. - Work with event-based and streaming technologies to ingest and process data. - Provide technical guidance and support to the team, addressing any technical challenges or issues that arise during and after the migration phase. - Stay up-to-date with advancements in cloud computing, data engineering, and analytics technologies, recommending best practices and industry standards for implementing the data lake solution. Requirements : - 10 years of IT experience. - Minimum of 4 years of experience with Azure Databricks. - Experience in Data Modeling and Source System Analysis. - Proficiency in PySpark. - Strong SQL skills. - Familiarity with Azure components such as Azure Data Factory, Azure Data Lake, and Azure SQL. - Experience with Python for data engineering purposes. - Ability to conduct data profiling, cataloging, and mapping for technical design and construction of data flows. - Experience with data visualization and exploration tools. Preferred Qualifications : Microsoft Certified : - Azure Data Engineer Associate. - Experience preparing data for Data Science and Machine Learning. - Knowledge of Jupyter Notebooks or Databricks Notebooks for Python development. Personal Attributes : - Proven ability to provide technical leadership and guidance to teams. - Strong problem-solving skills with the capability to translate business requirements into actionable data solutions. - Excellent communication skills to effectively convey complex ideas to both technical and non-technical stakeholders. - Strong team player with excellent interpersonal and collaboration skills. - Ability to manage multiple projects simultaneously and deliver high-quality results within specified timelines. Educational Qualifications : - A Bachelor's or Master's degree in Computer Science, Information Technology, or a related field (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 10/24/2024