Skip to content

Commit 5488e37

Browse files
ceroberozceroberoz
authored
Update README.md
1 parent 6f2043d commit 5488e37

File tree

1 file changed

+33
-36
lines changed

1 file changed

+33
-36
lines changed

‎README.md

Lines changed: 33 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,40 @@
11
# id-jobs: Indonesian Job Market Data Aggregator 💼🇮🇩
22

3-
[![Daily Job Data Update](https://github.com/ceroberoz/id-jobs/actions/workflows/scrape.yml/badge.svg)](https://github.com/ceroberoz/id-jobs/actions/workflows/scrape.yml)
4-
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
5-
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
6-
![Powered by Scrapy](https://img.shields.io/badge/Powered%20by-Scrapy-green.svg)
3+
[![Daily Job Data Update](https://github.com/ceroberoz/id-jobs/actions/workflows/scrape.yml/badge.svg)](https://github.com/ceroberoz/id-jobs/actions/workflows/scrape.yml)
4+
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
5+
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
6+
![Powered by Scrapy](https://img.shields.io/badge/Powered%20by-Scrapy-green.svg)
77
![Enhanced by Playwright](https://img.shields.io/badge/Enhanced%20by-Playwright-orange.svg)
88

99
## 🆕 Latest Updates
1010

11-
- Added TechInAsia spider to collect job data from Tech in Asia Jobs portal
12-
- Implemented Algolia API integration for efficient data retrieval from TechInAsia
13-
- Enhanced data sanitization to ensure CSV-friendly output
14-
- Improved error handling and logging for the new spider
15-
- Updated documentation to reflect the addition of TechInAsia as a data source
11+
- Added **TechInAsia** spider for job data collection
12+
- Integrated **Algolia API** for efficient data retrieval
13+
- Improved data sanitization and CSV export
14+
- Enhanced error handling and logging
15+
- Updated docs with new data source details
1616

1717
## 📊 Overview
1818

19-
id-jobs collects job listings from Indonesian job portals and company websites, respecting each site's terms of service.
19+
id-jobs gathers job listings from Indonesian job portals and company websites, following each site's terms of service.
2020

21-
**View the Data on Google Sheets:** [https://s.id/id-jobs-v2](https://s.id/id-jobs-v2)
22-
23-
**View the Dashboard on LookerStudio by Google:** [https://s.id/id-jobs-dashboard](https://s.id/id-jobs-dashboard)
21+
**View Data on Google Sheets:** [https://s.id/id-jobs-v2](https://s.id/id-jobs-v2)
22+
**View Dashboard on Looker Studio:** [https://s.id/id-jobs-dashboard](https://s.id/id-jobs-dashboard)
2423

2524
## 🎨 Job Age Colors
2625

27-
| Age | Time | Color |
28-
|-----|------|-------|
29-
| New | ≤ 1 day | ![#00CC00](https://via.placeholder.com/15/00CC00/000000?text=+) Bright Green |
30-
| Hot | 1-7 days | ![#FF6600](https://via.placeholder.com/15/FF6600/000000?text=+) Bright Orange |
31-
| Recent | 8-15 days | ![#FFFF00](https://via.placeholder.com/15/FFFF00/000000?text=+) Bright Yellow |
32-
| Aging | 16-21 days | ![#E6E6E6](https://via.placeholder.com/15/E6E6E6/000000?text=+) Light Gray |
33-
| Old | 22-30 days | ![#CCCCCC](https://via.placeholder.com/15/CCCCCC/000000?text=+) Medium Gray |
34-
| Expired | > 30 days | ![#B3B3B3](https://via.placeholder.com/15/B3B3B3/000000?text=+) Dark Gray |
26+
| Age | Time | Color |
27+
|------------|------------|----------------|
28+
| New | ≤ 1 day | ![#00CC00](https://via.placeholder.com/15/00CC00/000000?text=+) Green |
29+
| Hot | 1-7 days | ![#FF6600](https://via.placeholder.com/15/FF6600/000000?text=+) Orange |
30+
| Recent | 8-15 days | ![#FFFF00](https://via.placeholder.com/15/FFFF00/000000?text=+) Yellow |
31+
| Aging | 16-21 days | ![#E6E6E6](https://via.placeholder.com/15/E6E6E6/000000?text=+) Gray |
32+
| Old | 22-30 days | ![#CCCCCC](https://via.placeholder.com/15/CCCCCC/000000?text=+) Dark Gray |
33+
| Expired | > 30 days | ![#B3B3B3](https://via.placeholder.com/15/B3B3B3/000000?text=+) Very Dark Gray |
3534

3635
## 🔧 How It Works
3736

38-
id-jobs automatically collects job data from various websites, cleans the information, and compiles it into a single spreadsheet. We use Scrapy for most sites and Playwright for sites with complex JavaScript rendering.
37+
id-jobs scrapes job data from multiple sites, cleans it, and compiles it into a single spreadsheet. We use **Scrapy** for most sites and **Playwright** for JavaScript-heavy sites.
3938

4039
![Scraping Process](how-scraper-works.gif)
4140

@@ -45,36 +44,34 @@ id-jobs automatically collects job data from various websites, cleans the inform
4544

4645
## 🌟 Why Use id-jobs?
4746

48-
id-jobs simplifies job searching by gathering information from multiple sources into one place, providing insights on work arrangements, job levels, and application deadlines.
47+
id-jobs aggregates job listings in one place, offering insights like work arrangements, job levels, and deadlines.
4948

5049
## 📚 Data Sources
5150

52-
We collect data from various job portals and company websites, including:
51+
We collect data from several job portals and company websites, including:
5352
Blibli, Dealls, Evermos, Flip, GoTo, Glints (Lite), Jobstreet, Kalibrr, Karir.com, Kredivo, Mekari, SoftwareOne, Tiket, Tech in Asia Jobs, and more.
5453

5554
## 🚀 Features
5655

5756
- Daily updates
58-
- Work arrangement identification
59-
- Job level detection
60-
- Application deadline calculation
61-
- Improved data accuracy
57+
- Identifies work arrangements & job levels
58+
- Tracks application deadlines
59+
- Accurate data
6260
- User-friendly Google Sheets interface
63-
- Job age tracking
64-
- JavaScript-rendered content handling with Playwright
65-
- Efficient pagination across multiple pages
66-
- Integration with Algolia API for improved data retrieval
61+
- Tracks job age
62+
- Handles JavaScript-rendered content
63+
- Efficient pagination
64+
- Integrated with Algolia API
6765

6866
## 🏁 Getting Started
6967

70-
For a quick guide, see our [Quickstart Guide](QUICKSTART.md).
68+
For a quick guide, check the [Quickstart Guide](QUICKSTART.md).
7169

7270
## ❓ FAQ
7371

74-
Check our [FAQ](FAQ.md) for common questions.
72+
See our [FAQ](FAQ.md) for common questions.
7573

7674
## 📄 License
7775

78-
id-jobs is open source under the GPL-3.0 license. You can use, modify, and share the code, as long as you keep it open source.
79-
76+
id-jobs is open source under the GPL-3.0 license. You can use, modify, and share it, as long as it remains open source.
8077
We respect website terms of service when collecting data.

0 commit comments

Comments
 (0)