Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reorganize database migrations #1881

Merged
merged 5 commits into from
Jan 31, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
[submodule "h5ai-nginx"]
path = h5ai-nginx
url = https://github.com/sourcifyeth/h5ai-nginx
url = https://github.com/sourcifyeth/h5ai-nginx
[submodule "services/database/database-specs"]
path = services/database/database-specs
url = https://github.com/verifier-alliance/database-specs
26 changes: 18 additions & 8 deletions services/database/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,45 +6,55 @@ Sourcify's database is an extension of the [Verifier Alliance](https://verifiera

The migrations can be run to set up the Sourcify database.

## Running the migrations
## Database migrations

### Prerequisites

Please initialize the [Verifier Alliance database-specs](https://github.com/verifier-alliance/database-specs) submodule before moving on with the migrations:

```
git submodule update --init
```

### Running the migrations

- Copy paste `.env.template` in `.env` and set the variables.
- Run `npm run migrate:up` to update the database to the latest version. This will run the `dev` config in [database.json](./database.json) with `localhost`. To run the migrations in production with `POSTGRES_HOST` use `npm run migrate:up -- --env production`

For convenience, you can run the Postgres container in `docker-compose.yml` with `docker-compose up`.

## Reset the database
### Reset the database

- Run `npm run migrate:reset` to reverse all the executed migrations

## Creating a change to the schema with migrations
### Creating a change to the schema with migrations

- Run `npm run migrate:create` to create a new database migration, a new file under `migrations/` will appear, use that file to alter the database.

# Migrating from the legacy repository (RepositoryV1) to the database
## Migrating from the legacy repository (RepositoryV1) to the database

Following v2.0.0, Sourcify no longer uses the filesystem as its source of truth. To switch from the legacy repository to the new database, contracts need to be re-compiled and verified with a new Sourcify instance.

## Synchronization process
### Synchronization process

The synchronization process takes two steps, in the first one we are going to store all the contracts from the repov1 into `sourcify_sync`, a table used to keep track of the to-be-synced contracts. In the second step we are using the `sourcify_sync` table to re-verify all the contracts on a new sourcify instance marking every successful synced contract into `sourcify_sync` as `synced`.

> **Note**
> Use `npm run sourcify:database -- --help` for a full list of options and parameters

## 1. Import the repository in the `sourcify_sync` table
### 1. Import the repository in the `sourcify_sync` table

```
npm run sourcify:database import-repo /home/app/repository/contracts
```

## 2. Start synchronization from `sourcify_sync` to a Sourcify instance
### 2. Start synchronization from `sourcify_sync` to a Sourcify instance

```
npm run sourcify:database sync https://sourcify.dev/server /home/app/repository/contracts -- --chains 1,5,11155111 --limit 2 --start-from <timestamp> --until <timestamp>
```

## 3. Verifying deprecated chains
### 3. Verifying deprecated chains

If there are chains that have been deprecated, their RPCs will not be available anymore so there's no way to fetch the deployment information for these contracts. We had verified these contracts so we might want to have these contracts regardless in our DB. To achieve that we need to put placeholders for the data related to the contract deployment, mostly on the `contract_deployments` table.

Expand Down
1 change: 1 addition & 0 deletions services/database/database-specs
Submodule database-specs added at 5d86aa
Loading