The "Remote Crawler" project represents a significant achievement in the realm of job searching and automation.
This endeavor involved the development and deployment of a highly robust remote job crawler, harnessing the power of web scraping techniques and a scalable architectural framework. The system was designed to not only gather job opportunities efficiently but also to enhance the user experience by automatically tweeting newly discovered opportunities using Twitter APIs. To ensure the timeliness and relevance of job updates, a sophisticated cron job schedule was implemented, guaranteeing bi-daily crawler runs. Furthermore, the project's infrastructure was hosted on AWS, including an orchestrated RDS setup for optimal data management. This project stands as a testament to innovation and efficiency in the world of remote job searching and automation.
- Golang
- Gin Framework
- Docker, Docker Compose
- Postgres, GORM
- REST APIs
Feel Free to create new issues and raise PR's