Skip to content

Latest commit

 

History

History
365 lines (291 loc) · 15.7 KB

api.md

File metadata and controls

365 lines (291 loc) · 15.7 KB
Raw

Application Programming Interface (API)

BadgeApp is a relatively simple web application, and its external application programming interface (API) is simple too. The BadgeApp API is a simple REST API that follows Ruby on Rails conventions. We want people to use our data; please do so!

This document provides a quickstart, legal information (the data is released under at least the Creative Commons Attribution License version 3.0 (CC-BY-3.0)), a discussion of the most common requests, a pointer to a sample analysis program, how to query, how to download the database, and then various kinds of more specialized information.

Quickstart

Like any REST API, use an HTTP verb (like GET) on a resource.

For example, you can get the JSON data for project #1 from the real production site (https://bestpractices.coreinfrastructure.org) by retrieving (a GET) data from this URL:

https://bestpractices.coreinfrastructure.org/projects/1.json

Note that you can ask for a particular result data format (where supported) by adding a period and its format (e.g., ".json", ".csv", and ".svg") to the URL before the parameters (if any). When using "." + format, the format name must be all lowercase. You can also request a data format by including an "Accept" statement in the requesting HTTP header.

A GET just retrieves information, and since most information is public, in most cases you don't need to log in for a GET. Other operations, like POST or DELETE, require logging in first (as is typical, cookies are used to track logged-in sessions).

From here on we'll omit the scheme (https) and hostname, and we'll indicate variables by beginning their name with ":" (colon). So the URL above is an example of this pattern, which retrieves information about project :id in a given :format (HTML by default):

GET /projects/:id(.:format)

Legal information

We want people to use the data we supply, so please do so! The main requirement is that you provide attribution.

More specifically: As noted on the website front page, all publicly-available non-code content managed by the badging application is released under at least the Creative Commons Attribution License version 3.0 (CC-BY-3.0); newer non-code content is released under CC-BY version 3.0 or later (CC-BY-3.0+). If referencing collectively or not otherwise noted, please credit the CII Best Practices badge contributors.

If you use the data for research, we'd love to hear about your results, so please do share the results with us if you can. That said, you are not legally required to share any results.

Most common requests

Here are the most common requests that an external system might request.

  • GET /(:locale/)projects/:id(/:level)(.:format)

    Request data for project :id in :format (default html, json also supported). External interfaces should normally request format "json". If "level" is given (0, 1, or 2), that level is shown (level is ignored if json format is requested, because we just provide all the data when JSON is requested).

  • GET /(:locale/)projects/:id/badge(.:format)

    Request the badge display for project :id in :format (default SVG; JSON is also supported). If you just want to the badge status of a project, retrieve this as JSON and look at badge_level. The SVG badges are specially and rapidly served by the Content Delivery Network (CDN) that we use, so feel free to using "img src" to embed them, since they will be returned especially rapidly.

  • GET /(:locale/)projects(.:format)(?:query)

    Perform a query on the projects to return a list of the matching projects, up to the maximum number allowed in a page. The format is html by default; json and csv are also supported. See below for more about the query.

Sample program

See the best-practices.py program to see an example of how to download and analyze data. Notice that since we supply JSON data in pages, you need to retrieve all the pages if you want the entire dataset. If you retrieve the entire database, store it locally (in a file or database system); it intentionally takes some time to download all of it.

Query (Search)

The "/projects" URL supports various searches. For example, retrieving this URL:

/projects.json?gteq=90&lteq=99&page=2

Will retrieve a list of project data in JSON format, but only for projects with 90% or better passing and less than or equal to 99% passing (that is, not completely passing), and will retrieve the second page of this list (by default the first page is returned). (Beware: If you embed the URL in an HTML document, you must as always write & as &.

We reserve the right to change the details, but we do try to provide a reasonable interface. Most parameters can be combined (in which case all criteria must be met). The following search parameters are supported:

  • status: "passing", "in_progress", "silver", or "gold"
  • gteq: Integer, % greater than or equal of passing criteria
  • lteq: Integer, % less than or equal of passing criteria. Can be combined with gteq
  • pq: Text, "prefix query" - matches against prefix of URL or name
  • q: Text, "normal query" - match against parsed name, description, URL This is implemented by PostgreSQL, so you can use "&" (and), "|" (or), and "'text...*'" (prefix). Note that URLs cannot include spaces, and & has a special meaning, so you typically will need to use URL encoding in a query. For example, /en/projects?q=R%20%26%20analysis is a search that requires both "R" and "analysis". This search request system breaks URLs into parts, so you can't use "q" to search on a whole URL (use pq instead).
  • page: Page to display (starting at 1)

See app/controllers/project_controllers.rb if you want to see the implementation's source code.

Downloading the database

We encourage analysis of OSS trends. We do provide some search capabilities, but for most analysis you will typically need to download the database. We can't anticipate all possible uses, and we're trying to keep the software relatively small & focused.

You can download the project data in JSON and CSV format using typical Rails REST conventions. Just add ".json" or ".csv" to the URL (or include an Accept statement, like "Accept: application/json", in the HTTP header). You can even do this on a search if we already support the search (e.g., by name). Similarly, you can download user data in JSON format using ".json" at the end of the URL.

There is a current technical limitation in that you must request project and user data page-by-page. This isn't hard, just provide a page parameter (e.g., "page=2"). This is because Rails does not stream JSON or CSV data by default, so if we allowed this the application would download the entire database into memory to process it. Rails applications can stream data (there are even web pages explaining how to do it), but the call for it is rare and there are some complications in its implementation, so we just haven't implemented it yet.

So you can download the projects by repeatedly requesting this (changing "1" into 2, 3, etc. until all is loaded):

https://bestpractices.coreinfrastructure.org/projects.json?page=1

You can similarly load the user data starting from here:

https://bestpractices.coreinfrastructure.org/users.json?page=1

To directly download the user list you must be logged in, and we intentionally restrict the information we share about users. We only provide basic public information such as name, nickname, and such. In particular, we only provide email addresses to BadgeApp administrators, because we value the privacy of our users.

As we note about privacy and legal issues, please see our privacy policy and terms of use. All publicly-available non-code content is released under at least the Creative Commons Attribution License version 3.0 (CC-BY-3.0); newer non-code content is released under CC-BY version 3.0 or later (CC-BY-3.0+). If referencing collectively or not otherwise noted, please credit the "CII Best Practices badge contributors" and note the license. You should also note the website URL as the data source, which is https://bestpractices.coreinfrastructure.org.

If you are doing research, we urge you do to it responsibly and reproducibly. Please be sure to capture the date and time when you began and completed capturing this dataset (you need both, because the data could be changing while you're downloading it). Do what you can to ensure that your research can be replicated. Consider the points in Good Enough Practices in Scientific Computing, a "set of computing tools and techniques that every researcher can and should adopt." For example, "Where possible, save data as originally generated (i.e. by an instrument or from a survey." These aren't requirements for using this data, but they are well worth considering.

Full entry point list

The REST interface supports the following interfaces, which is enough to programmatically create a new user, login and logout, create project data, edit it, and delete it (subject to the authorization rules). In particular, viewing with a web browser (which by default emits 'GET') a URL with the absolute path "/projects/:id" (where :id is an id number) will retrieve HTML that shows the status for project number id. A URL with absolute path "/projects/:id.json" will retrieve just the status data in JSON format (useful for further programmatic processing).

The following shows the HTTP verb (e.g., GET), the URI pattern, and the controller#action in the code (important if you need to examine the source code itself which is in directory app/controllers/).

Verb   URI Pattern                        Controller#Action
GET    /projects/:id(.:format)            projects#show # .json supported.
GET    /projects/:id/badge(.:format)      projects#badge {:format=>"svg"}

GET    /projects(.:format)                projects#index
POST   /projects(.:format)                projects#create
GET    /projects/new(.:format)            projects#new
GET    /projects/:id/edit(.:format)       projects#edit
PATCH  /projects/:id(.:format)            projects#update
PUT    /projects/:id(.:format)            projects#update
DELETE /projects/:id(.:format)            projects#destroy

GET    /users/new(.:format)               users#new
GET    /signup(.:format)                  users#new
GET    /users/:id/edit(.:format)          users#edit
GET    /users/:id(.:format)               users#show

GET    /sessions/new(.:format)            sessions#new
GET    /login(.:format)                   sessions#new
POST   /login(.:format)                   sessions#create
DELETE /logout(.:format)                  sessions#destroy
GET    /signout(.:format)                 sessions#destroy

If you install the application you can have it report the routes by running "rake routes".

Rails _method parameter in POST

As explained in the Rails Guide Form Helpers:

The Rails framework encourages RESTful design of your applications, which means you'll be making a lot of "PATCH" and "DELETE" requests (besides "GET" and "POST"). However, most browsers don't support methods other than "GET" and "POST" when it comes to submitting forms. Rails works around this issue by emulating other methods over POST with a hidden input named "_method", which is set to reflect the desired method.

This works in the BadgeApp application too. However, note that most requests other than GET (and a few GET requests) require a logged-in session, so while this is alternative way to make the request, for the request to succeed you still have to use a logged-in session with adequate authorization.

Cross-Origin Resource Sharing (CORS)

The BadgeApp permits some access by client-side JavaScript programs that originate from other sites.

Client-side JavaScript programs are, by default, subject to the "same origin" policy, which prevents them from arbitrarily accessing sites other than where they came from. The standard way to give a client-side JavaScript program additional privilege is through Cross-Origin Resource Sharing (CORS) HTTP headers.

The BadgeApp provides CORS headers in certain cases when an "Origin" is provided. When a client-side JavaScript program makes a request to a different origin, it provides its "origin", and that allows the BadgeApp to decide what it wants to allow.

The CORS header expressly does not share credentials, and only allows GET (or OPTIONS) for a few specific resources. If someone provides a good reason to allow more accesses from JavaScript clients, we'll gladly consider it. For details, see the BadgeApp source code config/initializers/cors.rb.

Locale

Users indicate their locale via the URL. We use one of the standard approaches, where the locale is the first part of the path. For example, https://bestpractices.coreinfrastructure.org/fr/projects/ selects the locale "fr" (French) when displaying "/projects". This even works at the top page, e.g., https://bestpractices.coreinfrastructure.org/fr. The application also supports the locale as a query parameter, e.g., https://bestpractices.coreinfrastructure.org/projects?locale=fr, though that is not the canonical URL format.

At one time, when a locale was not provided, English (en) was assumed. However, that made it impossible to distinguish between "use the browser's preferred locale" and "use English".

Now, if no locale is selected, the application will redirect requestors to the best matching locale (as recommended by the accept-language value provided by the browser, with English as the last resort). The API has an important exception: if JSON format is requested, no locale redirection occurs (because JSON isn't locale-sensitive). The request for the badge image is also not redirectored (again, because it isn't locale-sensitive).

Locales are always exactly 2 lowercase letters, or 2 lowercase letters followed by an dash and then more alphanumerics. For example, "fr" is French, while "zh-cn" is Chinese (Simplified). This convention lets you syntactically distinguish between locales and other possible meanings of a URL's prefix; non-locale pages never match the locale syntax.

Canonical URLs

The canonical URL format for pathnames is a single "/" for the topmost root (which will redirect to the best-matching locale), otherwise there is no trailing slash. This is the canonical format generated by the Rails router helpers.

See also

Project participation and interface:

Criteria:

  • criteria.md - Criteria for "passing" badge
  • other.md - Criteria for other badges (silver and gold)

Development processes and security: