Skip to content

Latest commit

 

History

History
22 lines (18 loc) · 1.41 KB

develop.md

File metadata and controls

22 lines (18 loc) · 1.41 KB
Raw

How to implement your feeds

Choose your feed ID

Within the project scope, feed ID is used to locate the implementation python file under src/scraper/feeds and metadata json file in config/feeds.

ID specification

  • Every ID should be unique.
  • It must not contain underscore _. Use - instead
  • Suggested to match the ID in mobilitydatabase if your feed is defined in their database already.

Setup metadata file

Metadata file must be named as {FEED-ID}.json and placed under config/feeds.

While there is no constraints to fields in the json file, it is suggested to follow mobilitydatabase's schema. If your feed exists in their database, their api endpoint will return the json file which you can just copy and paste.

Implement the logic

Implementation python file must be named as {FEED-ID}.py and placed under src/scraper/feeds.

You must define a class with following specification

  • inherit src/scraper/feeds
  • class name must be same as the feed ID, in capitalized letter and underscore _ instead of -.
  • implement def __init__(self):
  • implement def scrape(self): which return the scraped binary result