Bitkraft Technologies - Big Data Engineer - Hadoop/Spark

Summary : Bitkraft Technologies LLP is looking for Big Data Engineer to join our software engineering team. You will be working across the stack on cutting edge web development projects for our custom services business. As a Big Data Engineer, you play a pivotal role in designing, developing, and implementing cutting-edge data processing solutions. The ideal candidate will have a deep understanding of distributed systems, big data technologies, and real-time data processing frameworks. If you love solving problems, are a team player and want to work in a fast paced environment with core technical and business challenges, we would like to meet you. Essential Skills : - Deep understanding of big data technologies, including Hadoop, Spark, Kafka, and Flink. - Proven experience in designing and implementing scalable, high-performance data processing solutions. - Strong knowledge of real-time data processing concepts and frameworks. - Familiarity with cloud platforms (AWS, Azure, GCP) and container orchestration (Kubernetes). - Experience working in a fast-paced, data-driven environment and handling large-scale data. Other Essential Skills / Requirements : - Great attention to detail - Strong work ethic and commitment to meet deadlines and support team members meet goals - Be flexible with working across time zones with overseas customers if required - Ability to work independently and as part of a team. Desirable Skills : - Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes). - Knowledge of machine learning pipelines or data science workflows. - Experience in real-time data processing and stream processing using technologies like Apache Kafka or Apache Flink. Key Responsibilities : - Design, develop, and maintain scalable and efficient data pipelines to process and transform large datasets from diverse sources (structured, unstructured, and semi-structured). - Architect and optimize big data platforms, databases, and technologies (e.g., Hadoop, Spark, Kafka, AWS, GCP, etc.) for high performance data processing and storage. - Develop and optimize ETL (Extract, Transform, Load) processes to move data across systems and ensure data integrity, quality, and accuracy. - Integrate data from various sources (internal and external) into a centralized data warehouse or lake. - Optimize data processing and storage performance for large datasets, ensuring scalability and efficiency. - Work closely with data scientists, analysts, and business teams to understand data requirements and deliver solutions to meet business needs. - Implement automation for data workflows and set up monitoring systems to ensure the reliability and performance of data pipelines. - Ensure proper data governance and security practices are in place, adhering to organizational and regulatory compliance requirements. - Investigate and resolve issues related to data integrity, pipeline failures, and system performance. Experience : 5 to 10 years Location : Fort, Mumbai Why join Bitkraft : - Your inputs and opinions are valued - Exposure to latest technologies - Working directly with client teams - International project experience - You get to see the big picture on the project - Fast paced environment with quick project completions - Manage your own time for efficient working - A friendly and easy going work environment About Bitkraft Technologies LLP : Bitkraft Technologies LLP is an award winning Software Engineering Consultancy focused on Enterprise Software Solutions, Mobile Apps Development, ML/AI Solution Engineering, Extended Reality, Managed Cloud Services and Technology Skill-sourcing, with an extraordinary track record. We are driven by technology and push the limits of what can be done to realise the business needs of our customers. Our team is committed towards delivering products of the highest standards and we take pride in creating robust user-driven solutions that meet business needs. Bitkraft boasts of clients across over 10 countries including US, UK, UAE, Oman, Australia and India to name a few. To know more about Bitkraft visit our website bitkraft.co.in (ref:hirist.tech)

Location: mumbai, IN

Posted Date: 11/27/2024