Hero FinCorp - Data Architect - Python/Spark

Responsibilities for the job : Key Responsibilities : - Architectural Leadership: Design and implement scalable data architectures using Databricks, ensuring they meet the specific needs of financial services such as risk management, compliance, and customer analytics. - Conduct assessment of existing data architectures, identify areas of improvement, and recommend remediation plans to optimise data management process and systems. - Develop data migration strategies and oversee the execution of data migration activities, ensuring data integrity and minimal disruption to client operations. - Data Pipeline Development: Build and optimize data ingestion and processing pipelines to facilitate real-time analytics and reporting. - Collaboration with Stakeholders: Work closely with business units to understand their data requirements and translate them into technical specifications that align with business goals. - Compliance and Governance: Ensure data solutions adhere to financial regulations and best practices in data governance, including data security and privacy standards. - Performance Optimization: Monitor and tune the performance of data systems to ensure efficiency and reliability, addressing any bottlenecks or issues that arise. Mentorship and Training: Provide guidance to junior data engineers and analysts, sharing best practices and promoting a culture of continuous improvement Other Responsibilities (If Any):- Availability during month-end Deck generation, may be sometime during week-end/holidays. Eligibility Criteria for the Job Education : B.E/B. Tech in any specialization, BCA, M.Tech in any specialization, MCA Age : No Bar Work Experience : - Proven experience, 7 years as a Data Architect, Data Solutions Architect, or similar role in a consulting environment. - Strong hands-on experience on various cloud services like ADF/Lambda, ADLS/S3, Security, Monitoring, Governance & Compliance. - Hands-on Experience to design platform and build- Databricks- based solution on any cloud platform as well as design and build solution powered by- DBT- models and integrate with databricks. - Must have good: - End-to-End design solution on cloud platform. - Knowledge of Data Engineering concept and related services of cloud. - Experience in Python and Spark and setting up development best practices. - Knowledge of docker and Kubernetes. - Experience with claims-based authentication (SAML/OAuth/OIDC), MFA,- RBAC, SSO etc. - Knowledge of cloud security controls including tenant isolation, encryption at rest, encryption in transit, key management, vulnerability assessments, application firewalls, SIEM, etc. Primary Skill : - Effectively communicate complex technical concepts and recommendations to non-technical stakeholders, including C-level executives and business leaders, to facilitate decision-making and secure buy-in. - The ability to lead the design, implementation, and optimization of our Databricks platform. - Work closely with our data engineering team to ensure that our Databricks platform is optimized for performance, scalability, and reliability. - Develop and maintain a comprehensive understanding of our data pipeline and data architecture. - Develop and maintain documentation for our Databricks platform, including architecture diagrams, deployment guides, and operational procedures. - Experience in a Platform architecture role using service and hosting solutions such as private/public cloud IaaS, PaaS, and SaaS platforms. - Provide guidance and support to our data engineering team on Databricks-related issues. - Maintains close awareness of new and emerging technologies and their potential application for service offerings and products. Technical Skills : - Must have orchestrated at least 3 projects using any of the cloud platforms (GCP, Azure, AWS etc.) is a must. - Must have orchestrated at least 2 projects using Databricks over AWS/Azure - Mush have worked on any cloud PaaS/SaaS database/DWH such as AWS redshift/ Big Query/ Snowflake - Expertise in programming languages like Python or Pyspark or Scala or SQL/PLSQL. - Lead the design and implementation of data artefacts, including data modelling, data integration, data storage, and data governance based on industry best practices and client-specific requirements. - Hands-on experience on data modelling techniques to create Enterprise Data Models - Implementing data management concepts (Metadata Management, Data Quality, Data Testing) - Python/Spark Hands -on Exp from data engineering perspective is a must - Experience in at least one of the major ETL tools (Talend TAC, SSIS, Informatica) will be added advantage - Experience with Github and Jenkins CI/CD or similar Devops tools. - Experience with ETL tooling, migrating ETL code from one technology to another will be a benefit - Hands-on experience in Visualization/Dashboard tools (PowerBI, Tableau, Qliksense) (ref:hirist.tech)

Location: in, IN

Posted Date: 10/27/2024