Reubro International - Python Developer - Web Scraping

Job Description : Python Developer for Web Scraping Project with Admin Panel At Reubro Interntional, we're building human-focused technology solutions that capture efficiency, performance, and user-friendly designs. We're seeking a python developer to develop web applications and build thoughtful experiences centered around our customers. During a typical day, you'll be working alongside our web team to determine customer needs, brainstorm solutions, generate website mockups and prototypes, present these prototypes to customers, and both develop and optimize live platforms. We're looking for someone who is quick on their feet, possesses strong technical skills, and isn't afraid to ask questions. An understanding of not just how to code but why certain code works is a must. Responsibilities : Web Scraping Development : - Design and implement web scraping scripts using Python libraries - Scrape product information such as names, descriptions, prices, SKU numbers, stock availability, and images from various e-commerce platforms. - Ensure the system handles dynamic content loading and avoids anti-scraping measures like CAPTCHA and rate-limiting. Admin Panel Development : - Build an admin panel using frameworks like Flask or Django for managing scraping tasks. - Provide functionality for scheduling scraping jobs, viewing collected data, and exporting it to CSV or Excel. Data Processing & Storage : - Design and implement a robust data storage solution using relational databases like PostgreSQL or MySQL. - Ensure the scraped data is stored in a structured format and can be retrieved efficiently for display or export. Scalability & Cloud Deployment : - Deploy the solution on a scalable cloud platform to handle growing data and traffic. - Implement automated scheduling and performance monitoring for scraping jobs. Required Skills : - Proficiency in Python, with experience in web scraping using libraries such as Scrapy, BeautifulSoup, and Selenium. - Experience with RESTful APIs and web scraping strategies for dynamic websites. - Strong understanding of web technologies (HTML, CSS, JavaScript) to analyze website structures for scraping. - Experience with Flask/Django for developing admin interfaces. - Familiarity with relational databases (PostgreSQL, MySQL) for data storage and retrieval. - Experience with cloud platforms for deployment and scalability. - Familiarity with scheduling tools for automating scraping tasks. - Experience handling anti-scraping measures - Knowledge of distributed scraping solutions and load balancing. - Experience in building admin panels for data management and error monitoring. - Understanding of data export formats such as CSV and Excel. Preferred Qualifications : - Bachelor's degree in Engineering, Computer Science or related field. Candidates having a Master's degree in Software Engineering will also be considered (ref:hirist.tech)

Location: ernakulam, IN

Posted Date: 10/9/2024