Data Engineer - Google Cloud Platform

Job specification : Skill : GCP Data Engineering Work Location : Chennai (Ascendas Tech Park) Salary : 10 - 28 LPA Experience : 3 to 8 Years. Notice Period : Max 60 Days Job Description : - For Data Engineering - Good Experience or strong understanding of GCP (Big query) - Good Experience on Bigtable, Pub/Sub, Data proc, Data Flow - Good Experience on SQL, Python/Airflow Roles & Responsibilities: - Design, build, and maintain robust data pipelines using Google Cloud Platform (GCP) tools, including Dataflow and Dataproc, to ensure seamless data ingestion and transformation. - Data Modeling and Architecture - Create and maintain efficient data models in BigQuery and Bigtable, optimizing for performance and scalability to support analytical workloads. - Real-time Data Processing - Implement real-time data processing solutions using Pub/Sub, ensuring timely availability of data for analytics and reporting. - SQL Development - Write and optimize complex SQL queries for data extraction, transformation, and loading (ETL) processes, ensuring data accuracy and integrity. - Python and Workflow Management - Develop and manage workflows using Python and Apache Airflow, orchestrating various data processes and ensuring smooth execution of data tasks. Data Quality Assurance - Monitor and validate data pipelines to ensure high data quality and reliability, implementing data quality checks and automated testing frameworks. - Collaboration and Communication - Work closely with data analysts, data scientists, and other stakeholders to understand data requirements and provide necessary data support. Performance Tuning - Analyze and optimize the performance of data pipelines and queries, recommending improvements as necessary to enhance efficiency. - Documentation and Best Practices - Document data processes, architecture, and best practices to ensure knowledge sharing and maintainability across the team. - Cloud Infrastructure Management - Manage and monitor GCP resources, ensuring cost-effective utilization and adherence to security best practices. - Continuous Learning and Improvement - Stay updated with the latest trends and technologies in data engineering and GCP, applying new knowledge to improve data solutions. - Troubleshooting and Support - Provide technical support and troubleshooting for data-related issues, identifying root causes and implementing solutions. Total Experience : Relevant Experience : CTC : ECTC : Holding offer: Notice Period: If serving NP Last working date: Current location: Availability date & time for discussion : (ref:hirist.tech)

Location: chennai, IN

Posted Date: 10/9/2024