forked from humanprotocol/human-protocol
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Human APP backend - 2nd milestone (humanprotocol#1751)
* Merge pull request #17 from blockydevs/error_handling Error handling - exception filters * Merge pull request #15 from blockydevs/10-jobs-discovery Endpoint to fetch jobs discovery * Merge pull request #16 from blockydevs/11-job-assignment Endpoint to assign a job to user and to retrieve assigned jobs * Merge pull request #18 from blockydevs/authorization-token-handling Middleware to pass the auth token * Merge pull request #21 from blockydevs/unit_tests_fix fix: unit tests * Merge pull request #14 from blockydevs/#13-Oracles-discovery Oracles-discovery * Merge pull request #20 from blockydevs/#Statistics-Exchange-oracle # Statistics exchange oracle * Naming covention alignemnt (#27) * Alignment of naming convention and endpoint structure * Alignment of naming convention and endpoint structure * ExchangeOracleApi->ExchangeOracle * externalApiGateway-> gateway * Coverage improvement * CI/CD workflow * fix(unit tests): Remove validationSchema from ConfigModule * Alignment after testing with stage exchange oracle (#29) * Fixes after check * Removes unnecessary comment * Fixes SDK limitation * Removes unnecessary interceptor, changes Body to Query in request for oracle discovery * Adds statistics related DTO interfaces with api mapping * Changes in interface naming convention of received data * Review fixes * Review fixes * Review fixes * Review fixes * Changes logic of calling exchange oracle: url is obtained based on the address provided in the enpoint. Adds CORS * Adds test coverage * Adds test coverage * Adds kv store to the modules that use it * minor fixes * minor fixes * Alignment of the swagger and interface used in the project * Minor fixes * Minor fixes * Minor fixes * Minor fixes * Removes todo * Review fixes * Review fixes * Review fixes * Review fixes * name: human-app -> human-app-server in package.json (#63) * Fixes after code review (2024-05-07) (#75) * File `global-common.interface.ts` has been renamed to `global-common.ts`. * Renamed and moved EnvironmentVariableMissingError class. * Renamed and moved Pageable classes to utils directory. * Remove unused import in exchange-oracle.gateway.spec.ts * Remove console.log from fetchAssignedJobs method. * Update dependencies in package.json. * Update terminology in README.md * Remove unused Headers import from job-assignment.controller.ts * Removed OnModuleInit import from kv-store-gateway.service.ts * Corrected the data types for REDIS_PORT and REDIS_HOST in .env.example file. * Remove redundant volume configuration in Docker-compose * Add RPC_URL environment variable to docker-compose * Replace hardcoded URL_KEY in KvStoreGateway with imported constant. * fix: Update KVStoreClient import and test in KvStoreGateway * Add Reputation Oracle address to .env and refactor Oracle Discovery. * Add token authentication for Oracle statistics. * Remove manual environment variable check. * Refactor address to use escrowAddress in job assignment service. * Ensure environment variables validation takes place before module loading. * Revert "Add token authentication for Oracle statistics." * Update terminology from 'Interfaces' to 'Model' in README * Refactor PageableClasses to PageableDto and the file renamed from pageable-classes.ts to pageable.model.ts * Update KV store gateway test to use actual SDK. * Added REPUTATION_ORACLE_ADDRESS in app configuration and docker compose file. * Integrate and use EscrowUtils for job assignment processing. * Added chainId parameter to `getExchangeOracleAddressByEscrowAddress` method. * Remove redundant code KV-store gateway test * Update EscrowUtilsGateway tests. * Add mock ConfigService to oracle-discovery service test. * Revert "Add mock ConfigService to oracle-discovery service test." This reverts commit a2134ad. * Update oracle-discovery.service.spec.ts tests. * Fixes CR 2024-05-10 (#78) * The KVStoreKeys.url constant is now directly called. * Changed Redis data volume path and updated gitignore. * Refactor oracle discovery and statistics modules. * Add CHAIN_IDS_ENABLED to .env.example * Fix typo in oracle-discovery.service file name and references. --------- Co-authored-by: MWBlocky <137490000+MWBlocky@users.noreply.github.com> Co-authored-by: macnablocky <143715306+macnablocky@users.noreply.github.com> Co-authored-by: MWBlocky <michal.walczak@blockydevs.com> Co-authored-by: maciek.nabialek <maciej.nabialek@blockydevs.com>
- Loading branch information
5 people
authored
May 13, 2024
1 parent
872a8bd
commit 8788c80
Showing
91 changed files
with
3,450 additions
and
134 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| name: Human App Check | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - "main" | ||
| pull_request: | ||
| paths: | ||
| - "packages/core/**" | ||
| - "packages/sdk/typescript/human-protocol-sdk/**" | ||
| - "packages/apps/human-app/**" | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| job-app-server-test: | ||
| name: Human App Server Test | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - run: npm install --global yarn && yarn | ||
| name: Install dependencies | ||
| - run: yarn human-app-server:test | ||
| name: Run Job Human App unit tests |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| Dockerfile | ||
| .dockerignore | ||
| node_modules | ||
| npm-debug.log | ||
| dist |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,19 @@ | ||
| HOST= | ||
| PORT= | ||
| REPUTATION_ORACLE_URL= | ||
| E2E_TESTING_EMAIL_ADDRESS= | ||
| E2E_TESTING_PASSWORD= | ||
| HOST= # string, example: localhost | ||
| PORT= # number, example: 5010 | ||
| REPUTATION_ORACLE_URL= # string | ||
| REPUTATION_ORACLE_ADDRESS= # string | ||
| REDIS_HOST= # string, example: localhost | ||
| REDIS_PORT= # number, example: 6379 | ||
| CACHE_TTL_ORACLE_DISCOVERY= # number, example: 43200 | ||
| CACHE_TTL_ORACLE_STATS= # number, example: 900 | ||
| CACHE_TTL_USER_STATS= # number, example: 86400 | ||
| E2E_TESTING_EMAIL_ADDRESS= # string | ||
| E2E_TESTING_PASSWORD= # string | ||
| E2E_TESTING_EXCHANGE_ORACLE_URL= # string | ||
| E2E_TESTING_ESCROW_ADDRESS= # string | ||
| E2E_TESTING_ESCROW_CHAIN_ID= # number | ||
| RPC_URL= # string | ||
| CORS_ENABLED= # boolean, example: true | ||
| CORS_ALLOWED_ORIGIN= # string example: http://localhost:5173 | ||
| CORS_ALLOWED_HEADERS= # string, example: 'Content-Type,Accept' | ||
| CHAIN_IDS_ENABLED= # number array, example: 80002,80001 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Base image | ||
| FROM node:18 | ||
|
|
||
| # Create app directory | ||
| WORKDIR /usr/src/app | ||
|
|
||
| # Bundle app source | ||
| COPY . . | ||
|
|
||
| # Install app dependencies | ||
| RUN yarn install | ||
|
|
||
| # Creates a "dist" folder with the production build | ||
| RUN yarn workspace @human-protocol/human-app-server build | ||
|
|
||
| # Start the server using the production build | ||
| CMD [ "node", "packages/apps/human-app/server/dist/src/main.js" ] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| version: '3.8' | ||
|
|
||
| services: | ||
| human-app: | ||
| container_name: human-app | ||
| restart: unless-stopped | ||
| build: | ||
| context: ../../../../ | ||
| dockerfile: packages/apps/human-app/server/Dockerfile | ||
| expose: | ||
| - '${PORT}' | ||
| ports: | ||
| - '${LOCAL_PORT}:${PORT}' | ||
| environment: | ||
| NODE_ENV: ${NODE_ENV} | ||
| HOST: ${HOST} | ||
| PORT: ${PORT} | ||
| REPUTATION_ORACLE_URL: ${REPUTATION_ORACLE_URL} | ||
| REPUTATION_ORACLE_ADDRESS: ${REPUTATION_ORACLE_ADDRESS} | ||
| REDIS_PORT: ${REDIS_PORT} | ||
| REDIS_HOST: redis | ||
| CACHE_TTL_ORACLE_DISCOVERY: ${CACHE_TTL_ORACLE_DISCOVERY} | ||
| RPC_URL: ${RPC_URL} | ||
| depends_on: | ||
| - redis | ||
| redis: | ||
| image: redis:latest | ||
| container_name: human_app_cache | ||
| ports: | ||
| - '${REDIS_PORT}:6379' | ||
| volumes: | ||
| - ./redis_data:/data |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
20 changes: 20 additions & 0 deletions
20
packages/apps/human-app/server/src/common/config/cache-factory.config.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| import { CacheModuleAsyncOptions } from '@nestjs/common/cache'; | ||
| import { ConfigModule } from '@nestjs/config'; | ||
| import { EnvironmentConfigService } from './environment-config.service'; | ||
| import { redisStore } from 'cache-manager-redis-store'; | ||
| export const CacheFactoryConfig: CacheModuleAsyncOptions = { | ||
| isGlobal: true, | ||
| imports: [ConfigModule], | ||
| useFactory: async (configService: EnvironmentConfigService) => { | ||
| const store = await redisStore({ | ||
| socket: { | ||
| host: configService.cacheHost, | ||
| port: configService.cachePort, | ||
| }, | ||
| }); | ||
| return { | ||
| store: () => store, | ||
| }; | ||
| }, | ||
| inject: [EnvironmentConfigService], | ||
| }; |
72 changes: 60 additions & 12 deletions
72
packages/apps/human-app/server/src/common/config/environment-config.service.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,23 +1,71 @@ | ||
| import Joi from 'joi'; | ||
| import { ConfigService } from '@nestjs/config'; | ||
| import { Injectable } from '@nestjs/common'; | ||
|
|
||
| const DEFAULT_CACHE_TTL_ORACLE_STATS = 12 * 60 * 60; | ||
| const DEFAULT_CACHE_TTL_USER_STATS = 15 * 60; | ||
| const DEFAULT_CACHE_TTL_ORACLE_DISCOVERY = 24 * 60 * 60; | ||
| const DEFAULT_CORS_ALLOWED_ORIGIN = 'http://localhost:5173'; | ||
| const DEFAULT_CORS_ALLOWED_HEADERS = 'Content-Type, Accept'; | ||
| @Injectable() | ||
| export class EnvironmentConfigService { | ||
| constructor(private configService: ConfigService) {} | ||
| get host(): string { | ||
| return this.configService.get<string>('HOST', 'localhost'); | ||
| return this.configService.getOrThrow<string>('HOST'); | ||
| } | ||
| get port(): string { | ||
| return this.configService.get<string>('PORT', '5010'); | ||
| get port(): number { | ||
| return this.configService.getOrThrow<number>('PORT'); | ||
| } | ||
| get reputationOracleUrl(): string { | ||
| return this.configService.get<string>('REPUTATION_ORACLE_URL', ''); | ||
| return this.configService.getOrThrow<string>('REPUTATION_ORACLE_URL'); | ||
| } | ||
| get reputationOracleAddress(): string { | ||
| return this.configService.getOrThrow<string>('REPUTATION_ORACLE_ADDRESS'); | ||
| } | ||
| get cachePort(): number { | ||
| return this.configService.getOrThrow<number>('REDIS_PORT'); | ||
| } | ||
| get cacheHost(): string { | ||
| return this.configService.getOrThrow<string>('REDIS_HOST'); | ||
| } | ||
| get cacheTtlOracleStats(): number { | ||
| return this.configService.get<number>( | ||
| 'CACHE_TTL_ORACLE_STATS', | ||
| DEFAULT_CACHE_TTL_ORACLE_STATS, | ||
| ); | ||
| } | ||
|
|
||
| get cacheTtlUserStats(): number { | ||
| return this.configService.get<number>( | ||
| 'CACHE_TTL_USER_STATS', | ||
| DEFAULT_CACHE_TTL_USER_STATS, | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| export const envValidator = Joi.object({ | ||
| HOST: Joi.string().default('localhost'), | ||
| PORT: Joi.string().default(5010), | ||
| REPUTATION_ORACLE_URL: Joi.string().required(), | ||
| }); | ||
| get cacheTtlOracleDiscovery(): number { | ||
| return this.configService.get<number>( | ||
| 'CACHE_TTL_ORACLE_DISCOVERY', | ||
| DEFAULT_CACHE_TTL_ORACLE_DISCOVERY, | ||
| ); | ||
| } | ||
| get rpcUrl(): string { | ||
| return this.configService.getOrThrow<string>('RPC_URL'); | ||
| } | ||
| get isCorsEnabled(): boolean { | ||
| return this.configService.get<boolean>('CORS_ENABLED', false); | ||
| } | ||
| get corsEnabledOrigin(): string { | ||
| return this.configService.get<string>( | ||
| 'CORS_ALLOWED_ORIGIN', | ||
| DEFAULT_CORS_ALLOWED_ORIGIN, | ||
| ); | ||
| } | ||
| get corsAllowedHeaders(): string { | ||
| return this.configService.get<string>( | ||
| 'CORS_ALLOWED_HEADERS', | ||
| DEFAULT_CORS_ALLOWED_HEADERS, | ||
| ); | ||
| } | ||
| get chainIdsEnabled(): string[] { | ||
| const chainIds = this.configService.getOrThrow<string>('CHAIN_IDS_ENABLED'); | ||
| return chainIds.split(',').map((id) => id.trim()); | ||
| } | ||
| } |
Oops, something went wrong.