Skip to content

Commit

Permalink
jwt validation v1 (#57)
Browse files Browse the repository at this point in the history
* jwt validation v1
  • Loading branch information
Marketen authored May 29, 2024
1 parent e6c7191 commit b87d9b6
Show file tree
Hide file tree
Showing 15 changed files with 331 additions and 25 deletions.
3 changes: 2 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ MAX_ENTRIES_PER_BSON=
BEACON_NODE_URL_MAINNET=
BEACON_NODE_URL_HOLESKY=
BEACON_NODE_URL_GNOSIS=
BEACON_NODE_URL_LUKSO=
BEACON_NODE_URL_LUKSO=
JWT_USERS_FILE=
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
.env
./listener/tmp/*
./listener/bin/*
./listener/bin/*
jwt
private.pem
public.pem
71 changes: 55 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@

This repository contains the code for the validator monitoring system. The system is designed to listen signatures request from different networks validate them and store the results in a database.

It also contains a simple JWT generator that can be used to easily generate the JWT token necessary to access some API endpoints. More on this on the [API](#api) section.

In dappnode, the signature request and validation flow is as follows:

1. The staking brain sends the signature request of type `PROOF_OF_VALIDATION` to the web3signer (see https://github.com/Consensys/web3signer/pull/982). The request has the following format:
**1.** The staking brain sends the signature request of type `PROOF_OF_VALIDATION` to the web3signer (see <https://github.com/Consensys/web3signer/pull/982>). The request has the following format:

```json
{
Expand All @@ -16,17 +18,54 @@ In dappnode, the signature request and validation flow is as follows:
}
```

2. The web3signer answers with the `PROOF_OF_VALIDATION` signature. Its important to notice that the order of the items in the JSON matters.
**2.** The web3signer answers with the `PROOF_OF_VALIDATION` signature. Its important to notice that the order of the items in the JSON matters.

3. The staking brain sends back all the `PROOF_OF_VALIDATION` signatures to the the signatures monitoring system. The listener will validate the requests, the validators and the signatures and finally store the result into a mongo db.
**3.** The staking brain sends back all the `PROOF_OF_VALIDATION` signatures to the the signatures monitoring system. The listener will validate the requests, the validators and the signatures and finally store the result into a mongo db.

## API
##  API

- `/signatures?network=<network>`:
- `POST`: TODO
- `GET`: TODO

## Validation
### Authentication

The `GET /signatures` endpoint is protected by a JWT token, which must be included in the HTTPS request. This token should be passed in the Authorization header using the Bearer schema. The expected format is:

```text
Bearer <JWT token>
```

#### JWT requirements

To access the `GET /signatures` endpoint, the JWT must meet the following criteria:

- **Key ID** (`kid`): The JWT must include a kid claim in the header. It will be used to identify which public key to use to verify the signature.

As a nice to have, the JWT can also include the following claims as part of the payload:

- **Expiration time** (`exp`): The expiration time of the token, in Unix time. If no `exp` is provided, the token will be valid indefinitely.
- **Subject** (`sub`): Additional information about the user or entity behind the token. (e.g. an email address)

#### Generating the JWT

To generate a JWT token, you can use the `jwt-generator` tool included in this repository. The tool requires an RSA private key in PEM format to sign the token.
A keypair in PEM format can be generated using OpenSSL:

```sh
openssl genrsa -out private.pem 2048
openssl rsa -in private.pem -pubout -out public.pem
```

Once you have the private key, you can generate a JWT token using the `jwt-generator` tool:

```sh
./jwt-generator --private-key=path/to/private.pem --kid=your_kid_here --exp=24h --output=path/to/output.jwt
```

Only JWT tokens with whitelisted "kid" and pubkey will be accepted. Please contact the dappnode team for more information on this.

##  Validation

The process of validating the request and the signature follows the next steps:

Expand All @@ -35,30 +74,30 @@ The process of validating the request and the signature follows the next steps:

```go
type SignatureRequest struct {
Payload string `json:"payload"`
Pubkey string `json:"pubkey"`
Signature string `json:"signature"`
Tag Tag `json:"tag"`
Payload string `json:"payload"`
Pubkey string `json:"pubkey"`
Signature string `json:"signature"`
Tag Tag `json:"tag"`
}
```

The payload must be encoded in base64 and must have the following format:

```go
type DecodedPayload struct {
Type string `json:"type"`
Platform string `json:"platform"`
Timestamp string `json:"timestamp"`
Type string `json:"type"`
Platform string `json:"platform"`
Timestamp string `json:"timestamp"`
}
```

3. The validators must be in status "active_on_going" according to a standard beacon node API, see https://ethereum.github.io/beacon-APIs/#/Beacon/postStateValidators:
3. The validators must be in status "active_on_going" according to a standard beacon node API, see <https://ethereum.github.io/beacon-APIs/#/Beacon/postStateValidators>:
3.1 The signatures from the validators that are not in this status will be discarded.
3.2 If in the moment of querying the beacon node to get the validator status the beacon node is down the signature will be accepted storing the validator status as "unknown" for later validation.
4. Only the signatures that have passed the previous steps will be validated. The validation of the signature will be done using the pubkey from the request.
5. Only valid signatures will be stored in the database.

## Crons
##  Crons

There are 2 cron to ensure the system is working properly:

Expand Down Expand Up @@ -88,12 +127,12 @@ bson.M{
"timestamp": req.DecodedPayload.Timestamp,
},
},
}
}
```

**Mongo db UI**

There is a express mongo db UI that can be accessed at `http://localhost:8080`. If its running in dev mode and the compose dev was deployed on a dappnode environment then it can be access through http://ui.dappnode:8080
There is a express mongo db UI that can be accessed at `http://localhost:8080`. If its running in dev mode and the compose dev was deployed on a dappnode environment then it can be access through <http://ui.dappnode:8080>

## Environment variables

Expand Down
1 change: 1 addition & 0 deletions docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ services:
volumes:
- ./listener/cmd:/app/cmd
- ./listener/internal:/app/internal
- ./jwt:/app/jwt
networks:
dncore_network:
aliases:
Expand Down
4 changes: 3 additions & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,13 @@ services:
BEACON_NODE_URL_LUKSO: ${BEACON_NODE_URL_LUKSO}
BEACON_NODE_URL_GNOSIS: ${BEACON_NODE_URL_GNOSIS}
MAX_ENTRIES_PER_BSON: ${MAX_ENTRIES_PER_BSON}
JWT_USERS_FILE: ${JWT_USERS_FILE}
depends_on:
- mongo
container_name: listener
restart: always

volumes:
- ./jwt:/app/jwt ## listener expects /app/jwt to exist, careful when changing this path
ui:
build:
context: ui
Expand Down
6 changes: 6 additions & 0 deletions jwt/users.json.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"stader": {
"publicKey": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAq81M9pHCZEExzJFgWEXK\navIs0AexsLlP6CIGkbvfe/GX+kIjP28kkXYGCJlUuVhGYa8wU2mBYeXTbtvi9OR9\ndmKTOzsl3QzIKVd5BqXqbTmQxGp0S6ShujK6LHTOELxwYhFKulx2ls2DSyXhqOGx\nyh0Gm/3H7CiCgNHMJWUUiy5Xyp71vtimzDM+OniUVQE/ZjPg5WG+cM536Ms8XcK1\nNIN0z8ovgAibHqw8jEljxM89Sn9XD3mQo8kBTG+3dLsjUbHZDiJZogNgeXsOrM7m\nh3YtIwMvr5YEWUR7ON7ST5Wrwx14uF6YDE0yo6nb/cmmSSUJ/cdX36dNK3dGrYhB\nywIDAQAB\n-----END PUBLIC KEY-----",
"tags": ["solo"]
}
}
76 changes: 76 additions & 0 deletions listener/cmd/jwt-generator/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
package main

import (
"flag"
"fmt"
"os"
"time"

"github.com/dappnode/validator-monitoring/listener/internal/logger"

"github.com/golang-jwt/jwt/v5"
)

func main() {
// Define flags for the command-line input
privateKeyPath := flag.String("private-key", "", "Path to the RSA private key file (mandatory)")
subject := flag.String("sub", "", "Subject claim for the JWT (optional)")
expiration := flag.String("exp", "", "Expiration duration for the JWT in hours (optional, e.g., '24h' for 24 hours). If no value is provided, the generated token will not expire.")
kid := flag.String("kid", "", "Key ID (kid) for the JWT (mandatory)")
outputFilePath := flag.String("output", "token.jwt", "Output file path for the JWT. Defaults to ./token.jwt")

flag.Parse()

// Check for mandatory parameters
if *kid == "" || *privateKeyPath == "" {
logger.Fatal("Key ID (kid) and private key path must be provided")
}

// Read the private key file
privateKeyData, err := os.ReadFile(*privateKeyPath)
if err != nil {
logger.Fatal(fmt.Sprintf("Failed to read private key file: %v", err))
}

// Parse the RSA private key
privateKey, err := jwt.ParseRSAPrivateKeyFromPEM(privateKeyData)
if err != nil {
logger.Fatal(fmt.Sprintf("Failed to parse private key: %v", err))
}

// Prepare the claims for the JWT. These are optional
claims := jwt.MapClaims{}
if *subject != "" {
claims["sub"] = *subject
}
if *expiration != "" {
duration, err := time.ParseDuration(*expiration)
if err != nil {
logger.Fatal(fmt.Sprintf("Failed to parse expiration duration: %v", err))
}
claims["exp"] = time.Now().Add(duration).Unix()
}

// Create a new token object, specifying signing method and claims
token := jwt.NewWithClaims(jwt.SigningMethodRS256, claims)

// Set the key ID (kid) in the token header
token.Header["kid"] = *kid

// Sign the token with the private key
tokenString, err := token.SignedString(privateKey)
if err != nil {
logger.Fatal(fmt.Sprintf("Failed to sign token: %v", err))
}

// Output the token to the console
fmt.Println("JWT generated successfully:")
fmt.Println(tokenString)

// Save the token to a file
err = os.WriteFile(*outputFilePath, []byte(tokenString), 0644)
if err != nil {
logger.Fatal(fmt.Sprintf("Failed to write the JWT to file: %v", err))
}
fmt.Println("JWT saved to file:", *outputFilePath)
}
1 change: 1 addition & 0 deletions listener/cmd/listener/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ func main() {
dbCollection,
config.BeaconNodeURLs,
config.MaxEntriesPerBson,
config.JWTUsersFilePath,
)

// Start the API server in a goroutine. Needs to be in a goroutine to allow for the cron job to run,
Expand Down
1 change: 1 addition & 0 deletions listener/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ module github.com/dappnode/validator-monitoring/listener
go 1.22.0

require (
github.com/golang-jwt/jwt/v5 v5.2.1
github.com/gorilla/mux v1.8.1
github.com/herumi/bls-eth-go-binary v1.35.0
github.com/robfig/cron v1.2.0
Expand Down
2 changes: 2 additions & 0 deletions listener/go.sum
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/golang-jwt/jwt/v5 v5.2.1 h1:OuVbFODueb089Lh128TAcimifWaLhJwVflnrgM17wHk=
github.com/golang-jwt/jwt/v5 v5.2.1/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/golang/snappy v0.0.1 h1:Qgr9rKW7uDUkrbSmQeiDsGa8SjGyCOGtuasMWwvp2P4=
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/go-cmp v0.5.2 h1:X2ev0eStA3AbceY54o37/0PQ/UWqKEiiO2dKL5OPaFM=
Expand Down
6 changes: 4 additions & 2 deletions listener/internal/api/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,18 @@ type httpApi struct {
dbCollection *mongo.Collection
beaconNodeUrls map[types.Network]string
maxEntriesPerBson int
jwtUsersFilePath string
}

// create a new api instance
func NewApi(port string, dbClient *mongo.Client, dbCollection *mongo.Collection, beaconNodeUrls map[types.Network]string, maxEntriesPerBson int) *httpApi {
func NewApi(port string, dbClient *mongo.Client, dbCollection *mongo.Collection, beaconNodeUrls map[types.Network]string, maxEntriesPerBson int, jwtUsersFilePath string) *httpApi {
return &httpApi{
port: port,
dbClient: dbClient,
dbCollection: dbCollection,
beaconNodeUrls: beaconNodeUrls,
maxEntriesPerBson: maxEntriesPerBson,
jwtUsersFilePath: jwtUsersFilePath,
}
}

Expand All @@ -41,7 +43,7 @@ func (s *httpApi) Start() {

s.server = &http.Server{
Addr: ":" + s.port,
Handler: routes.SetupRouter(s.dbCollection, s.beaconNodeUrls, s.maxEntriesPerBson),
Handler: routes.SetupRouter(s.dbCollection, s.beaconNodeUrls, s.maxEntriesPerBson, s.jwtUsersFilePath),
}

// ListenAndServe returns ErrServerClosed to indicate that the server has been shut down when the server is closed gracefully. We need to
Expand Down
48 changes: 48 additions & 0 deletions listener/internal/api/handlers/getSignatures.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
package handlers

import (
"context"
"encoding/json"
"fmt"
"net/http"

"github.com/dappnode/validator-monitoring/listener/internal/api/middleware"
"github.com/dappnode/validator-monitoring/listener/internal/logger"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/mongo"
)

func GetSignatures(w http.ResponseWriter, r *http.Request, dbCollection *mongo.Collection) {
logger.Debug("Received new GET '/signatures' request")
// Get tags from the context
tags, ok := r.Context().Value(middleware.TagsKey).([]string)
// middlewware already checks that tags is not empty. If something fails here, it is
// because middleware didnt pass context correctly
if !ok || len(tags) == 0 {
http.Error(w, "Internal server error", http.StatusInternalServerError)
return
}

// Query MongoDB for documents with tags matching the context tags
var results []bson.M
filter := bson.M{
"tag": bson.M{"$in": tags},
}
cursor, err := dbCollection.Find(context.Background(), filter)
if err != nil {
http.Error(w, fmt.Sprintf("Failed to query MongoDB: %v", err), http.StatusInternalServerError)
return
}
defer cursor.Close(context.Background())

if err := cursor.All(context.Background(), &results); err != nil {
http.Error(w, fmt.Sprintf("Failed to read cursor: %v", err), http.StatusInternalServerError)
return
}

// Return the results as JSON
w.Header().Set("Content-Type", "application/json")
if err := json.NewEncoder(w).Encode(results); err != nil {
http.Error(w, fmt.Sprintf("Failed to encode results: %v", err), http.StatusInternalServerError)
}
}
Loading

0 comments on commit b87d9b6

Please sign in to comment.