StaidLogic - Data Engineer - Google Cloud Platform

Role: CLOUD DATA ENGINEER. Location : Pune. Experience : total 10 years. Role Description : - This is a contract role for 2 Cloud Data Engineers located on-site in Pune. - The Cloud Data Engineers will be responsible for data engineering, data modeling, ETL processes, data warehousing, and data analytics on a day-to-day basis. - Proficient in working with Azure Blob Storage, Azure Data Lake Storage, Azure Data Factory, Azure SQL Data Warehouse, Azure Data Bricks, and on Python, SQL, and PL/SQL concepts. - Worked on migrating and transforming data from on-premises to cloud and between cloud services by creating Azure data factory pipelines & data flows using different ADF activities and components. - Experience in writing Spark Applications using (Python/Scala) to connect to different cloud services like Azure SQL DB, Azure Postgres, Azure Synapse Analytics, ADLS, and AWS S3 and performing different data transformations based on business requirements. - Familiar with Azure DevOps to deploy ADF pipelines and ARM Templates into other environments by creating release pipelines. - Experience in legacy data migration projects such as Big Data to AWS Redshift migration, i.e, from on-premises to AWS Cloud and Snowflake Data warehouse. - Experience with Snowflake SQL and Snowflake pipelines to pull data from AWS S3 and traditional DB. - Worked in a source-consumer application in Google Cloud using different GCP services like Google Cloud Storage, Cloud Data proc, Big Query, Google Cloud PUB/SUB, Google Cloud Composer, etc. - Experienced in creating airflow DAGs using the most common Operators in Airflow - Python Operator, Bash Operator, and Google Operators to orchestrate the data flow using Google Cloud services. - Experienced in writing Spark Applications using (Python/Scala) to connect to different cloud services like Google Cloud Storage, and Cloud SQL and performing different data transformations based on business requirements. - Proficient in working with AWS S3, AWS Glue, Redshift Data Warehouse, and AWS Ec2 and on Python, SQL, and PL/SQL concepts. - Responsible for designing and implementing data pipelines using AWS services such as S3, Glue, EC2, and EMR. - Building and maintaining data warehouses and lakes using AWS Redshift and data security and access controls using AWS IAM. - Design, develop, and maintain ETL (Extract, Transform, Load) processes using AWS Glue to extract data from various sources, transform it to meet business requirements, and load it into target data sources. - Worked with the DevOps team to create environment-specific Configuration yaml files to deploy code through CI/CD process by creating artifacts using a central repository. - Familiar with Processing Real-Time Streaming data using Azure Event Hubs and Azure Stream Analytics and visualizing the results using Power BI and Tableau. - Experienced in Fact-Dimensional modeling (Star Schema, Snowflake Schema), transactional modeling, and SCD (Slowly Changing Dimension). - Experienced in designing and creating RDBMS Tables and views, User Created Data Types, Indexes, Stored Procedures, Cursors, Triggers, and transactions. - Had good knowledge and Hands-on with the ETL tool Informatica, Talend to understand the existing flows, modify the flow, and create the data flow based on the requirement. - worked extensively with source code management and version control tools like Git, GitHub, and GitLab. - Familiar with the Agile working method, used JIRA to track work progress and Confluence to prepare and manage technical documentation. - Familiarity and knowledge of components of the Hadoop Ecosystem like HDFS and HIVE. (ref:hirist.tech)

Location: visakhapatnam, IN

Posted Date: 1/22/2025