Release
Quality
Repository
Dependencies

---

Rating Tracker

A web service fetching and providing financial and ESG ratings for stocks.

Features

Stock List with sorting and filtering

Stocks and their information are presented in a paginated table which offers comprehensive and in-depth sorting and filtering by many of the available attributes.

Automatic and scheduled data fetching from several providers

By providing identifiers for stocks from Yahoo Finance, Morningstar, MarketScreener, MSCI, LSEG Data & Analytics, Standard & Poor’s and Sustainalytics in the “Add Stock” dialog, Rating Tracker can automatically fetch financial data as well as financial and ESG ratings. The identifiers to use can be found in the provider’s URL for the stock as shown in the following examples:

• Yahoo: https://finance.yahoo.com/quote/AAPL (This ticker symbol is also the primary identifier for stocks in URLs, database tables etc. If a Yahoo Finance ticker is not available for a stock, an arbitrary ticker can be prefixed with an underscore _ to indicate that no prices must be fetched for this stock.)
• Morningstar: https://tools.morningstar.it/it/stockreport/default.aspx?Site=it&id=0P000000GY&LanguageId=it-IT&SecurityToken=0P000000GY]3]0]E0WWE$$ALL
• MarketScreener: https://www.marketscreener.com/quote/stock/APPLE-INC-4849
• MSCI: https://www.msci.com/our-solutions/esg-investing/esg-ratings-climate-search-tool/issuer/apple-inc/IID000000002157615
• LSEG Data & Analytics: https://www.lseg.com/bin/esg/esgsearchresult?ricCode=AAPL.O (see also Refinitiv Identification Code)
• Standard & Poor’s: https://www.spglobal.com/esg/scores/results?cid=4004205
• Sustainalytics: https://www.sustainalytics.com/esg-rating/apple-inc/1007903183

The fetching can be scheduled by providing a Cron-like specifier in an environment variable. See below for details.

Stock Logos

When providing an ISIN for a stock, its logo is automatically fetched and cached from TradeRepublic broker.

Rating Scores

The fetched ratings of a stock are aggregated to both a financial and ESG score using the average values of all ratings, such that a score of 0 is assigned to an average stock and a score of 100 is assigned to a stock with perfect scores in all underlying ratings.

The financial and ESG score are used to compute a total score using the harmonic mean of both numbers, so that a stock has to perform well in both financial and ESG ratings to obtain a good total score.

Watchlists

Stocks noteworthy to a user can be organized in watchlists. A dedicated Favorites watchlist is provided by default and can easily be maintained by clicking the star icon for a stock.

Users can subscribe to a watchlist, so they receive notifications when a stock’s rating is updated.

Portfolios

Stocks weighted by a given amount of currency can be aggretated in portfolios. The average rating scores of the portfolio can be displayed, as well as the distribution of regions, industry sectors, company sizes and styles of the stocks in the portfolio.

Portfolio Builder

This tool can provide optimal weights of stocks in a portfolio based on the preferred proportions of regions, sectors and other factors.

Stocks can be taken from an existing portfolio, a watchlist, or searched for manually:

Constraints for regions, sectors, sizes and styles can then be provided, as well as a currency for the portfolio and other settings, such as the total amount to invest, the minimum currency amount per stock or the algorithm to use for proportional representation:

The resulting stock distribution is presented in a histogram-like chart, and differences between target and actual percentages are highlighted:

The dialog to save the weighted stocks to a new or existing portfolio transparently lists all updates to be made as well as the progress in transferring them to the server:

User Management

The Rating Tracker supports multiple users, who can self-register via WebAuthn or OpenID Connect and access the application after being granted fine-grained access by an administrator, for whom a “User Management” web interface is provided.

Notification Messages via Signal

Based on their access rights, users can subscribe to updates of stock ratings, fetch error reports, or new user registrations by providing a phone number capable of receiving messages via the instant messenger Signal.

Error reports with screenshots

When fetching a stock fails, a screenshot of the page the fetch was attempted from is stored and a link to them is sent to stock maintainers who subscribed to error reports, so they can analyze and fix the issue.

Structured logging

JSON-formatted logs are printed to stdout as well as rotating log files. For better readability, a console-based prettifier or a separate log viewing service like Dozzle can be used. The prettifier pino-pretty comes pre-installed in the container and can be utilized by modifying the container command to pipe the server output: node ./server.mjs | pino-pretty.

You can also use an existing Rating Tracker container to prettify log files by aliasing pino-pretty in your shell like:

# ~/.zshrc
alias pino-pretty="docker exec -i \$(docker compose -f <path to docker compose file> ps -q <container name>) pino-pretty -c

# To view (and follow) a log file:
tail -n +1 -f <...>/rating-tracker.log | pino-pretty | less

…and more to come!

Planned features are documented here. If you feel that something is missing, feel free to request a feature!

Demo

An instance of the Rating Tracker is publicly available at https://ratingtracker.mruder.dev, for which access is granted at request.

Deployment

Rating Tracker is built to be deployed using Docker or a similar container platform.

Prerequisites

To run Rating Tracker, the following services must be available:

• PostgreSQL, storing information related to stocks and users
• Signal Messenger REST API, sending notifications via the Signal messenger
• nginx or a different reverse proxy, providing SSL encryption (required for most WebAuthn clients)

Minimal Example Setup using Docker Compose

Docker Compose is the preferred way to run Rating Tracker together with all the services it depends on. The following configuration file shows an exemplary setup.

View Docker Compose configuration

services:
  postgres:
    image: postgres:alpine
    ports:
      - "127.0.0.1:5432:5432"
    environment:
      POSTGRES_DB: "rating-tracker"
      POSTGRES_USER: "rating-tracker"
      POSTGRES_PASSWORD: "********"
      PGDATA: /var/lib/postgresql/data
    volumes:
      - ./postgresql/data:/var/lib/postgresql/data
    shm_size: '256mb'

  signal:
    image: bbernhard/signal-cli-rest-api
    environment:
      MODE: json-rpc
    ports:
      - "127.0.0.1:8080:8080"
    volumes:
      - ./signal-cli:/home/.local/share/signal-cli

  rating-tracker:
    image: marvinruder/rating-tracker
    tty: true # required for colored output to stdout
    init: true # required for graceful shutdown
    environment:
      PORT: 21076
      DOMAIN: "example.com"
      SUBDOMAIN: "ratingtracker"
      TRUSTWORTHY_PROXY_COUNT: 1
      LOG_FILE: "/app/logs/rating-tracker-log-(DATE).log" # (DATE) is replaced by the current date to support log rotation
      DATABASE_URL: "postgresql://rating-tracker:********@postgres:5432/rating-tracker?schema=rating-tracker"
      MAX_FETCH_CONCURRENCY: 4
      AUTO_FETCH_SCHEDULE: "0 0 0 * * *" # this format includes seconds
      SIGNAL_URL: "http://signal:8080"
      SIGNAL_SENDER: "+12345678900"
    ports:
      - "127.0.0.1:443:21076" # not required if nginx runs in same Docker Compose setup
    volumes:
      - ./logs/rating-tracker:/app/logs
    depends_on:
      - postgres
      - signal
    restart: unless-stopped

The port bindings are optional but helpful to connect to the services from the host, e.g. for debugging purposes.

Setup steps

Initialize database setup

Rating Tracker uses Prisma to interact with the PostgreSQL database. At every startup, Prisma Migrate will automatically compare your database with the schema included in the image and create all tables and indexes that are not present yet.

Create Signal account

Run a shell in the Signal REST API container and proceed with this excellent documentation.

Configure webserver as reverse proxy

After setting up NGINX as a webserver with SSL, the following virtual host configuration can be used to run a reverse proxy:

View NGINX configuration

resolver 127.0.0.11 valid=15s; # DNS resolver from Docker to resolve Docker Compose container names

location / {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    set $target_host rating-tracker; # use 127.0.0.1 here if nginx runs outside of Docker Compose setup
    proxy_pass http://$target_host:21076;
}

Initial admin registration

After setting up your Rating Tracker instance, navigate to its URL and register, creating WebAuthn credentials. The first user registered will be granted ultimate access rights automatically. After registering, you can log in using your credentials.

Configure OpenID Connect provider

To use Rating Tracker with an OpenID Connect provider, set the following mandatory environment variables:

• OIDC_ISSUER_URL: The issuer identifier (i.e., the base URL) of the OpenID Connect provider
• OIDC_CLIENT_ID: The client identifier configured for Rating Tracker at the OpenID Connect provider
• OIDC_CLIENT_SECRET: The client secret related to the Client ID above

Given that your Rating Tracker instance resides at https://ratingtracker.example.com, the client at your OpenID Connect provider must be configured with:

• the root or home URL: https://ratingtracker.example.com
• the redirect URI: https://ratingtracker.example.com/login
• the post-logout redirect URI: https://ratingtracker.example.com/login?origin=oidc_post_logout
• the Authorization Code Flow enabled
• Front channel logout enabled
• optionally and if supported, the logo URL: https://mruder.internal.mruder.dev/assets/images/favicon-dev/favicon-192.png

To read all supported user information from the claims, the scopes openid profile email phone should be requested using the OIDC_SCOPES environment variable.

To manage access rights using OpenID Connect roles, create the following roles in your OpenID Connect provider:

Role name	Description
administrative_access	Administrative access to the application. May add, edit and delete users.
write_stocks_access	Write access to stocks. May add, edit and delete stocks, as well as fetch data from providers.
general_access	General access to the application. May log in and view stocks.

Next, assign users to the roles as needed, then add the roles to a scope so that they are included in the ID token. To have Rating Tracker extract the roles from the ID token, set the OIDC_ROLE_CLAIM_PATH environment variable to the JMES path referencing the array of roles within the ID token claims, such as resource_access."rating-tracker".roles.

Supported environment variables

Variables in bold font are mandatory.

View complete list of environment variables

Variable	Example Value	Explanation
PORT	21076	The TCP port Rating Tracker is served on.
DOMAIN	example.com	The domain Rating Tracker will be available at. This is especially important for WebAuthn, since credentials will only be offered to the user by their client when the domain provided as part of the registration or authentication challence matches the domain of the URL the user navigated to.
SUBDOMAIN	ratingtracker	An optional subdomain. Credentials created for one domain can be used to authenticate to different Rating Tracker instances served on all subdomains of that domain, making it easy to use multiple deployment stages, development servers etc.
TRUSTWORTHY_PROXY_COUNT	1	A number of trusted proxies that are allowed to set the X-Forwarded-For header to identifier the real IP of a client. If unset, the header is not trusted.
DATABASE_URL	postgresql://rating-tracker:********@127.0.0.1:5432/rating-tracker?schema=rating-tracker	The connection URL of the PostgreSQL instance, specifying username, password, host, port, database and schema. Can also use the PostgreSQL service name (e.g. postgres in this configuration) as hostname if set up within the same Docker Compose file.
LOG_FILE	/var/log/rating-tracker-(DATE).log	A file path for storing Rating Tracker log files. The string (DATE) will be replaced by the current date. If unset, logs are stored in the /tmp directory.
LOG_LEVEL	debug	The level for the log output to stdout. Can be one of fatal, error, warn, info, debug, trace. If unset, info will be used.
AUTO_FETCH_SCHEDULE	0 30 2 * * *	A Cron-like specification of a schedule for when to fetch all stocks from all providers. The format in use includes seconds, so the example value resolves to “every day at 2:30:00 AM”. If unset, no automatic fetching will happen.
MAX_FETCH_CONCURRENCY	4	The number of fetcher instances used concurrently when fetching information for multiple stocks. If unset, no concurrent fetches will be performed.
SIGNAL_URL	http://127.0.0.1:8080	The URL of the Signal REST API. Can also use the Signal REST API service name (e.g. signal in this configuration) as hostname if set up within the same Docker Compose file. If unset, no Signal notification messages will be sent.
SIGNAL_SENDER	+12345678900	The phone number of the Signal account registered with the Signal CLI service, which will be used to send notification messages. Read more here on how to register a Signal account. If unset, no Signal notification messages will be sent.
SMTP_HOST	smtp.example.com	The hostname of the SMTP server to use for sending emails. If unset, no emails will be sent.
SMTP_PORT	587	The port of the SMTP server to use for sending emails. If unset, the default port will be used based on the security settings.
SMTP_SECURITY	tls	The security method to use for the SMTP connection. Can be one of none, tls, ssl. If unset, none will be used.
SMTP_USER	ratingtracker	The username to use for the SMTP connection. If unset, no authentication will be used.
SMTP_PASS	********	The password to use for the SMTP connection. If unset, no authentication will be used.
SMTP_FROM	ratingtracker@example.com	The email address to use as the sender of emails. If unset, no emails will be sent.
OIDC_ISSUER_URL	https://sso.example.com	The issuer identifier (i.e., the base URL) of the OpenID Connect provider to use for authentication. If unset, no OpenID Connect authentication will be used.
OIDC_CLIENT_ID	rating-tracker	The client identifier to use for the OpenID Connect provider. If unset, no OpenID Connect authentication will be used.
OIDC_CLIENT_SECRET	********	The client secret to use for the OpenID Connect provider. If unset, no OpenID Connect authentication will be used.
OIDC_SCOPES	openid profile email	A space-delimited list of scopes to request from the OpenID Connect provider. If unset, a default set of scopes will be used.
OIDC_ROLE_CLAIM_PATH	resource_access."rating-tracker".roles	A JMES path referencing an array of roles within the ID token claims. If unset, no roles will be extracted from the ID token.

API Reference

Any Rating Tracker instance’s API is self-documented, its OpenAPI web interface is hosted at /api-docs. The complete OpenAPI specification document can be downloaded at /api-spec/v3.1.

Development

Create an environment for developing

An environment with all tools required for developing Rating Tracker and the services it depends on can quickly be created using the VS Code development container configuration included in this repository. The scripts section in the package.json provides helpful commands:

• Clone the repository and open it in Visual Studio Code. When prompted, select “Reopen in Container”. This will create a Docker container with all required tools, recommended extensions and dependencies installed.
• Check your environment. SSL Certificates must be provided to Vite beforehand, and a Signal account must be created before starting the server (see section Setup steps for details).
• Run yarn prisma:migrate:deploy to initialize the PostgreSQL database.
• Run yarn dev to start the backend server as well as the Vite frontend development server.

Environment variables not included in the development container configuration can easily be defined in an .env file:

See recommended development environment variables

# .env
# # Those variables are already defined in the development container configuration. You only need to define them here if you want to override them.
# DATABASE_URL="postgresql://rating-tracker:rating-tracker@postgres:5432/rating-tracker?schema=rating-tracker"
# SIGNAL_URL="http://signal:8080"
# NODE_ENV="development"
# PORT="3001"
# MAX_FETCH_CONCURRENCY="4"
# POSTGRES_USER="rating-tracker"
# POSTGRES_PASS="rating-tracker"
# LOG_LEVEL="trace"

DOMAIN=example.com
SUBDOMAIN=ratingtracker
SIGNAL_SENDER="+12345678900"
# AUTO_FETCH_SCHEDULE=" 0 * * * * *" # This would run every minute, activate for debugging only.

Run tests

The VS Code development container configuration includes an additional PostgreSQL instance for running tests. To use them, configure your development environment as described above and run yarn test to run all tests from all packages. Additionally, the packages’ package.json configurations contain a test:watch script to run tests in watch mode.

Contribute

Contributions are welcome!

Disclaimer

This software is provided under the conditions of the MIT License.

Important

Use this tool at your own risk. Excessive data fetching from providers, publishing or selling the information obtained by fetching is not recommended. Your actions may have consequences… 🦋

Authors

• Marvin A. Ruder (he/him)