Description:
As a Python Developer within the scraping team, you will be among our talented and passionate team helping us optimize data collection!
-
- Design, implement, and document robust Scrapy spiders, ensuring resilience against future website changes while maintaining existing ones.
- Conduct comprehensive code review and data validation to maintain the highest standards of code and data quality.
- Leverage in-depth knowledge of the HTTP protocol, the mechanisms of modern web browsers, and our own internal tools to design sophisticated web crawling solutions that bypass advanced anti-bot countermeasures.
- Architect and contribute to significant new developments across multiple codebases, providing clear technical documentation and conducting training sessions for relevant teams when necessary.
- Qualifications
-
- Master's degree in Computer Science, IT, or a related field.
- At least 3 years of professional experience in software engineering.
- Knowledge of BeautifulSoup or Scrapy framework
- Familiarity with HTML and JavaScript, with a solid understanding of single-page applications.
- Experience with RESTful and/or GraphQL APIs.
- Hands-on experience with major Python web frameworks such as Django, FastAPI, or similar.
- Knowledge of the scraping framework, Scrapy is a plus.
- Experience with time series databases, such as InfluxDB, is advantageous.
- Strong skills in Docker, Git, pandas, regular expressions, Linux, and bash scripting.
- Proven experience with major cloud providers such as AWS, GCP, or Azure.
- You are enthusiastic, motivated, autonomous.
- You like tackling mentally-stimulating software engineering challenges.
- You are passionate about tech and learning new things.