small updates

.env.sample

@@ -0,0 +1,5 @@
+DATABASE_URL=postgres://postgres:password@postgres:5433/namada-indexer
+TENDERMINT_URL=http://host.docker.internal:26657
+CACHE_URL=redis://dragonfly:6379
+WEBSERVER_PORT=5001
+DATABASE_URL_TEST=postgres://postgres:password@0.0.0.0:5433

Dockerfile

@@ -1,5 +1,5 @@
 FROM lukemathwalker/cargo-chef:latest-rust-1.79-bookworm AS chef
-RUN apt-get update && apt-get install -y protobuf-compiler build-essential clang-tools-14 
+RUN apt-get update && apt-get install -y protobuf-compiler build-essential clang-tools-14
 
 FROM chef AS planner
 WORKDIR /app
@@ -15,7 +15,7 @@ ARG PACKAGE
 RUN cargo build --release --bin ${PACKAGE}
 
 FROM debian:bookworm-slim AS runtime
-RUN apt-get update && apt-get install -y libpq5 ca-certificates
+RUN apt-get update && apt-get install -y libpq5 ca-certificates curl
 WORKDIR /app
 ARG PACKAGE
 COPY --from=builder /app/target/release/${PACKAGE} ./

README.md

@@ -1,71 +1,139 @@
-# Namada Interface Indexer
+# 🟡 Namada Indexer
 
-A set of microservices that crawler data from a namada node, store them in a postgres database and serve them via a REST api.
+## Status
 
-> 🔧 This is currently being worked on. Don't expect things to work! 🔧
+- 🔧 - This project is a work in progress. 
+- 🚧 - Functionality is not guaranteed at this stage. 
+- ⚠️ - Use at your own risk.
 
-## Namadillo integration
+##  About 
 
-When using this project as a backend for [Namadillo](https://github.com/anoma/namada-interface), always checkout the latest tag, as the `main` branch could have an incompatible set of APIs.
+This repository, **Namada Indexer**, is distinct from and incomparable to the similarly named [Namada MASP Indexer](https://github.com/anoma/namada-masp-indexer).
 
-## Contributing
+Note that: `Namada Indexer != Namada MASP Indexer && Namada MASP Indexer != Namada Indexer`.
 
-If you feel like contributing to the this project, feel free to choose an issued labeled as `bug` or `good-first-issue`. If you want to add a new feature, we ask you to first open an issue so we discuss about it. 
+The **Namada Indexer** is a collection of microservices that crawls data from a Namada Node, stores it in a PostgreSQL database, and makes it accessible via a REST API.
 
-Our git process requires you to target `main` if developing a new feature. Instead, if fixing a bug, you should checkout the latest tag maintaining branch (e.g `1.0.0-maint`).
+The primary goal of the indexer is to retrieve and store the data necessary for [Namadillo](https://github.com/anoma/namada-interface) (the Namada Interface) to operate. Consequently, the indexer does not store historical data, except for the `Transactions Service`, which indexes all transactions starting from block 1 and is not used by Namadillo.
+
+## Namadillo Integration
+
+When using this project as a backend for [Namadillo](https://github.com/anoma/namada-interface), always ensure you check out the latest tag. The `main` branch may contain an incompatible set of APIs.
+
+## Contribution
+
+We welcome contributions to this project! If you'd like to help, feel free to pick an issue labeled as `bug` or `good-first-issue`. If you want to propose a new feature, please open an issue first so we can discuss it.
+
+- For **new features**, target the `main` branch.
+- For **bug fixes**, check out the latest maintenance branch (e.g., `1.0.0-maint`) and target your changes there.
 
 ## Architecture
 
-The indexer is composed of a set microservices and a webserver, each one of these lives in his own crate. Each microservice is responsible of indexing some data from the chain and store them in the postgres database. Right now, there are 4 microservices:
+The Namada Indexer is composed of a set of microservices, with each component residing in its own crate. Each microservice is responsible for indexing specific data from the blockchain and storing it in the PostgreSQL database.
+
+### Microservices & Containers
+- `namada/chain-indexer`: Processes blocks sequentially and extracts information from transactions (e.g., balances).
+
+- `namada/governance-indexer`: Tracks new proposals and their corresponding votes.
+
+- `namada/parameters-indexer`: Retrieves the chain parameters.
+
+- `namada/pos-indexer`: Retrieves the validator set at the start of each new epoch.
+
+- `namada/rewards-indexer`: Fetches Proof-of-Stake rewards for each new epoch.
 
-- `chain`: goes block by block and fetches information from transactions (e.g balances)
-- `pos`: fetches the validator set each new epoch.
-- `rewards`: fetches PoS rewards each new epoch.
-- `governance`: fetches new proposal and the corresponding votes.
-- `parameters`: fetches the chain parameters.
-- `transactions`: fetches transaction starting from block height 0 (or the latest processed block height).
+- `namada/transactions-indexer`: Processes transactions starting from block height 0 (or the last successfully processed block height).
 
-The `webserver` is responsible to serve the data via a REST API, which are described in the `swagger.yml` file in the project root.
-We host a HTML version of the swagger at [https://anoma.github.io/namada-indexer/](https://anoma.github.io/namada-indexer/).
+- `namada/webserver-indexer`: The `webserver` serves indexed data via a REST API, enabling external applications and users to access blockchain data in a structured and accessible way. It listens on port `5001`.
 
-![Namada indexer architecture](docs/architecture.png "Architecture")
+- `docker.dragonflydb.io/dragonflydb/dragonfly`: This container runs a DragonflyDB instance, an advanced in-memory key-value store that acts as a caching layer. It listens on port `6379` and stores frequently accessed or temporary data, improving system performance by reducing the need for repeated database queries.
 
-# How to run
+- `postgres:16-alpine`: This container runs a PostgreSQL instance, serving as the primary database for storing indexed data fetched by the microservices. It listens on port `5433` and provides a reliable and scalable storage backend for the project.
 
-## Prerequisites
+<p align="center">
+  <img src="docs/architecture.png" alt="Architecture" title="Architecture" width="500">
+</p>
 
-- Create the `.env` file in the root of the project. You can use the `.env_sample` file as a reference:
+
+# 🚀 Getting Started
+
+Follow these instructions to set up the project locally. The steps below will guide you through the process of getting a local copy up and running.
+
+It is strongly recommended to change the default username and password for your PostgreSQL database for security purposes. Update these credentials in both the `.env` file and the `docker-compose.yml` file to reflect the changes.
+
+## 🐳 Installation with Docker 
+
+### Prerequisites
+
+Before starting, ensure you have the necessary tools and dependencies installed. Below are the steps to set up the required environment.
+
+- **Packages**: Install prerequisite packages from the APT repository.
 
 ```sh
-cp .env_sample .env
+apt-get install -y curl apt-transport-https ca-certificates software-properties-common git nano just build-essential
 ```
 
-- Set the `TENDERMINT_URL` with the Namada RPC url:
-  - [Either create a local chain](https://docs.namada.net/operators/networks/local-network)
-  - Or use a Public RPC
+- **Docker**: Follow the official instructions provided by Docker to install it: [Install Docker Engine](https://docs.docker.com/engine/install/).
 
-## With docker
+- **Just**: Refer to the official documentation to install `just`: [Just Installation Guide](https://github.com/casey/just).
 
-- Install [just](https://github.com/casey/just)
-- Run `just docker-up`
+### Usage
+Ensure you have the latest repository cloned to maintain compatibility with other Namada interfaces. Use the following commands to clone the repository and navigate into its directory.
 
-## Without docker
+```sh
+git clone https://github.com/anoma/namada-indexer.git
+cd namada-indexer
+```
 
-- Install rust/cargo
-- Update the `.env` values to match your setup, for example:
-  ```env
-  DATABASE_URL=postgres://postgres:password@0.0.0.0:5433/namada-indexer
-  TENDERMINT_URL=http://127.0.0.1:27657
-  CACHE_URL=redis://redis@0.0.0.0:6379
-  PORT=5001
-  ```
-- Use the `run.sh` script inside each package. Keep in mind that PoS package have to be run always while other service might not
+Create the `.env` file in the root of the project. You can use the `.env.sample` file as a reference. 
 
-## Testing via seeder
+```sh
+cp .env.sample .env
+```
+- The `TENDERMINT_URL` variable must point to a Namada RPC URL, which can be either public or local. For a public RPC URL, refer to the [Namada Ecosystem Repository](https://github.com/Luminara-Hub/namada-ecosystem/tree/main/user-and-dev-tools/mainnet). If running the Namada Node locally, use the preconfigured `http://host.docker.internal:26657`.
+- When running locally, ensure that CometBFT allows RPC calls by setting the the configuration in your `config.toml` file.
+
+Build the required Docker containers for the project.
+```sh
+docker compose build
+```
+
+Launch the Namada Indexer using the `just` command, which orchestrates the Docker containers.
+```sh
+# Run the Docker containers in the foreground, displaying all logs and keeping the terminal active until stopped.
+just docker-up
+
+# Run the Docker containers in detached mode, starting them in the background without showing logs in the terminal.
+just docker-up-d
+```
+
+## Installation without Docker
+
+If you prefer not to use Docker, you can follow the instructions below to set up and run the services manually.
 
-Instead of fetching data from a running network, for testing porpuses it's also possible to populate the databse with some random data.
+- Install **Rust** and **Cargo** on your system. Refer to the [official Rust installation guide](https://www.rust-lang.org/tools/install).
 
-- `cargo build`
-- `cd seeder && cargo run -- --database-url postgres://postgres:password@0.0.0.0:5433/namada-indexer`
+- Update the `.env` file with values that match your setup.
 
-It's possible to only run the webserver and have access to the data via API.
+- Use the `run.sh` script located inside each package to start the services.  
+   - The **PoS** package must always be running.  
+   - Other services can be run as needed based on your requirements.
+
+## REST API
+The API endpoints are described in the `swagger.yml` file located in the project root. A hosted HTML version of the API documentation is available at [Namada Interface Indexer REST API](https://anoma.github.io/namada-indexer).
+
+## Populating the Database for Testing
+
+Instead of fetching data from a running network, you can populate the database with random data for testing purposes. Build the project using the following command.
+
+```sh
+cargo build
+# Run the seeder script to populate the database
+cd seeder && cargo run -- --database-url postgres://postgres:password@0.0.0.0:5433/namada-indexer
+```
+
+After populating the database, you can run the webserver to access the data via the API. To query your PostgreSQL database, ensure the PostgreSQL client is installed.
+
+```sh
+apt-get install -y postgresql-client
+```

docker-compose-db.yml

@@ -3,7 +3,7 @@ services:
     image: postgres:16-alpine
     command: ["postgres", "-c", "listen_addresses=0.0.0.0", "-c", "max_connections=200", "-p", "5433"]
     expose:
-      - "5433" # Publishes 5432 to other containers but NOT to host machine
+      - "5433"
     ports:
       - "5433:5433"
     environment:
@@ -12,10 +12,11 @@ services:
       PGUSER: postgres
       POSTGRES_DB: namada-indexer
     healthcheck:
-      test: ["CMD-SHELL", "pg_isready", "-d", "namada-indexer"]
+      test: ["CMD-SHELL", "pg_isready -U postgres -d namada-indexer -h localhost -p 5433"]
       interval: 5s
       timeout: 5s
       retries: 5
+    restart: unless-stopped
 
   dragonfly:
     image: docker.dragonflydb.io/dragonflydb/dragonfly
@@ -26,7 +27,7 @@ services:
       - "6379:6379"
     healthcheck:
       test: ["CMD-SHELL", "redis-cli ping | grep PONG"]
-      interval: 3s
-      timeout: 3s
+      interval: 5s
+      timeout: 5s
       retries: 5
-
+    restart: unless-stopped

docker-compose.yml

@@ -1,7 +1,10 @@
 x-defaults: &defaults
-  restart: on-failure
+  restart: unless-stopped
   depends_on:
-    - postgres
+      postgres:
+        condition: service_healthy
+      dragonfly:
+        condition: service_healthy
   build: &build
     context: .
     dockerfile: Dockerfile
@@ -15,6 +18,8 @@ x-defaults: &defaults
   command: "./service \
     --tendermint-url ${TENDERMINT_URL} \
     --database-url ${DATABASE_URL}"
+  extra_hosts:
+  - "host.docker.internal:host-gateway"
 
 include:
   - docker-compose-db.yml
@@ -80,10 +85,6 @@ services:
 
   webserver:
     image: namada/webserver-indexer
-    restart: on-failure
-    depends_on:
-      - postgres
-      - dragonfly
     build:
       context: .
       dockerfile: Dockerfile
@@ -96,7 +97,8 @@ services:
       <<: *env-vars
     healthcheck:
       test: curl --fail http://localhost:5001/health || exit 1
-      interval: 15s
-      timeout: 10s
-      retries: 3
-      start_period: 10s
+      interval: 5s
+      timeout: 5s
+      retries: 5
+    restart: unless-stopped
+

docs/README.md

@@ -1,49 +1,47 @@
-# Namada Indexer
+# 🟡 Namada Indexer
 
-## Goal
+## Architecture
 
-Main purpose of the Namada Indexer is to retrieve and store information required for the Namada Interface(Namadillo) to operate.
-Because of this, the Namada Indexer does not store historical data.
-Exception to this rule is the `Transactions Service` which is not used by the Namadillo and it indexes all transactions starting from block 1.
-
-### Overview:
+An overview of the Namada Indexer's system design and how data flows through its components.
 
 ![Namada indexer architecture](architecture.png "Architecture")
 
-### Flow:
+## Data Flow
+
+This diagram illustrates the high-level flow of data within the Namada Indexer.
 
 ![High Level Flow](high_level_flow.png "High Level Flow")
 
 ## Components
 
 Namada Indexer consists of the following services:
-
 - Chain Service
-- PoS Service
-- Parameters Service
 - Governance Service
+- Parameters Service
+- PoS Service
 - Rewards Service
 - Transactions Service
-- Webserver
+- Webserver Service
 
-### SRC(Service, Responsibility, Collaborator) cards for each service:
+## SRC Cards for Each Service
+
+An overview of each service with its **Service**, **Responsibility**, and **Collaborator** details.
 
 ![Cards](cards.png "Cards")
 
+
 ## Database
 
-We use PostgreSQL and diesel as ORM. You can find the schema in the [schema.rs](../orm/src/schema.rs) file.
+The Namada Indexer utilizes PostgreSQL with Diesel as the ORM. The database schema is defined in the [schema.rs](../orm/src/schema.rs) file.
 
-![DB](db.png "DB")
+![Database Schema](db.png "DB")
 
-## Service communication/orchestration
+## Service Communication/Orchestration
 
-At this point only by checking the state of the database. In the future we might use a simple message broker like Redis streams.
+Currently, service communication is achieved by checking the database state. In the future, a message broker like Redis Streams might be introduced.
 
 ## API
 
-For the API documentation, please refer to the [swagger.yml](../swagger.yml) file.
-We generate the client using [OpenApi generator](https://github.com/OpenAPITools/openapi-generator).
-You can find the published versions [here](https://www.npmjs.com/package/@namada/indexer-client).
+API documentation is available in the [swagger.yml](../swagger.yml) file. The client is generated using the [OpenAPI Generator](https://github.com/OpenAPITools/openapi-generator), and the published versions can be found on [npm](https://www.npmjs.com/package/@namada/indexer-client).
 
-Graphs/cards thanks to [excalidraw <3](docs_indexer_2024_09_20.excalidraw).
+Graphs and cards are created with ❤️ using [Excalidraw](https://excalidraw.com/).