As a Web Crawler Plugin Developer, you will be responsible for developing and maintaining plugins to be used with our in-house crawler. Your role will involve creating plugins in PHP and JavaScript that enable our crawler to collect and parse data from various websites effectively.
- Plugin Development: Develop high-quality plugins for our in-house crawler using PHP.
- Parsing and Processing: Develop parsing logic in PHP, HTML, and Regular Expression to extract relevant information from HTML, XML, JSON, and other web formats.
- Testing: Conduct thorough testing to ensure the seamless functionality of plugins.
- Collaboration: Work closely with other team members to ensure the smooth operation of the crawler and development of plugins.
Bachelor's degree in Computer Science, Software Engineering, or a related field (or equivalent practical experience).Proficiency in PHP and JavaScript.Solid understanding of web technologies, including HTML, CSS, JavaScript, and HTTP protocols.Good knowledge of front-end frameworks such as React, Angular, or Vue.js.Proficiency in writing regular expressions (regex)Familiarity with Chrome/Firefox Developer Tools and their usage for debugging, analyzing network traffic, and inspecting HTML/CSS elements during the web scraping process to facilitate efficient plugin development.