Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-manifest setup for subgraph deployments #978

Merged
merged 29 commits into from
Jul 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
b511924
basin support for multiple manifest files
soilking Jul 23, 2024
a250de7
Merge branch 'feat/sk/subgraphs-wsteth' into feat/sk/multimanifest
soilking Jul 23, 2024
7487b70
Update manifest path for tests
soilking Jul 23, 2024
b8a69b8
multi manifest setup for bean subgraph
soilking Jul 23, 2024
08e4d2c
multi manifest setup for beanstalk subgraph
soilking Jul 23, 2024
e80b96a
multi manifest for beanft
soilking Jul 23, 2024
fb15be2
updated readmes
soilking Jul 23, 2024
36f28df
Merge branch 'master' into feat/sk/multimanifest
soilking Jul 26, 2024
ea19e62
update github workflows
soilking Jul 26, 2024
abd6a4f
Update graph-cli version
soilking Jul 29, 2024
eac68f7
Merge branch 'master' into feat/sk/multimanifest
soilking Jul 29, 2024
d9b2163
fix missing whitelisted token
soilking Jul 29, 2024
9477f3e
update abi reference names
soilking Jul 29, 2024
27e8676
update abi reference
soilking Jul 30, 2024
4fa0481
Revert "update abi reference"
soilking Jul 30, 2024
2c2e590
switch abi for transferbatch
soilking Jul 30, 2024
0467835
add abi + temp graft
soilking Jul 30, 2024
db3ca87
update window column name
soilking Jul 30, 2024
b7460eb
Graft
soilking Jul 30, 2024
b36a477
Revert "Graft"
soilking Jul 30, 2024
50b54b8
Graft
soilking Jul 30, 2024
f41588b
temp disable apy
soilking Jul 30, 2024
0f41f15
from -> fromFarmer
soilking Jul 30, 2024
c79b3c6
refactor to -> toFarmer
soilking Jul 30, 2024
fc7b137
add graft
soilking Jul 30, 2024
4bb31a5
fix multi gauge lp apy issue
soilking Jul 30, 2024
6bc70d3
ui graphql codegen
soilking Jul 30, 2024
be35a0f
enable ema and graft
soilking Jul 31, 2024
57d8b49
disable grafting
soilking Jul 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.subgraph-basin.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
run: echo "There are uncommitted changes - execute 'yarn codegen' locally and commit the generated files!"

- name: Build Subgraph
run: yarn build
run: yarn build -- ethereum
working-directory: projects/subgraph-basin
test:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.subgraph-bean.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
run: echo "There are uncommitted changes - execute 'yarn codegen' locally and commit the generated files!"

- name: Build Subgraph
run: yarn build
run: yarn build -- ethereum
working-directory: projects/subgraph-bean
test:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.subgraph-beanft.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
run: echo "There are uncommitted changes - execute 'yarn codegen' locally and commit the generated files!"

- name: Build Subgraph
run: yarn build
run: yarn build -- ethereum
working-directory: projects/subgraph-beanft
test:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.subgraph-beanstalk.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
run: echo "There are uncommitted changes - execute 'yarn codegen' locally and commit the generated files!"

- name: Build Subgraph
run: yarn build
run: yarn build -- ethereum
working-directory: projects/subgraph-beanstalk
test:
runs-on: ubuntu-latest
Expand Down
4 changes: 4 additions & 0 deletions projects/subgraph-basin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,7 @@ All currently used subgraphs live on a centralized host controlled by beanstalk
### Testing

To test with Docker, the first time you will need to run `yarn run graph test -d`. This will build the `matchstick` Docker image. Then, you can use the `yarn testd` script to run all tests. Alternatively, use `yarn testd-named <TestName1> ...` to run specific tests. I have found running in Docker to be preferred since otherwise there can be issues with console output and some test cases fail silently.

### Deploying

When using graph cli commands, you will often need to specify which manifest file should be used. This is necessary to support multiple chains in the same codebase. The commands which need it will be evident - as they will fail when unable to find a `subgraph.yaml` file. In those commands, include `./manifest/${chain}.yaml` as the final argument to the command. See scripts inside `package.json` for examples.
58 changes: 58 additions & 0 deletions projects/subgraph-basin/manifests/codegen-abis.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# This file exists solely for the purpose of facilitating all codegen in a shared location such that all ABIs
# or templates are expanded independently of being used in all chains. Most of the information here is irrelevant,
# the only important part is in the `abis` and `templates` sections.
# - For abis, its only the list of abis that is relevant. The name of the dataSource is also visible.
# - For templates, it is only the name of the template that is relevant.
specVersion: 0.0.4
schema:
file: ../schema.graphql
dataSources:
- kind: ethereum/contract
name: Basin-ABIs
network: not_relevant
source:
address: "0xBA51AAAA95aeEFc1292515b36D86C51dC7877773"
abi: Aquifer
startBlock: 17977922
mapping:
kind: ethereum/events
apiVersion: 0.0.7
language: wasm/assemblyscript
entities:
- Aquifer
abis:
- name: Aquifer
file: ../../subgraph-core/abis/Aquifer.json
- name: ERC20
file: ../../subgraph-core/abis/ERC20.json
- name: Beanstalk
file: ../../subgraph-core/abis/Beanstalk/Beanstalk-Replanted.json
- name: Well
file: ../../subgraph-core/abis/Well.json
- name: CurvePrice
file: ../../subgraph-core/abis/CurvePrice.json
- name: BeanstalkPrice
file: ../../subgraph-core/abis/BeanstalkPrice.json
eventHandlers:
- event: BoreWell(address,address,address[],(address,bytes),(address,bytes)[],bytes)
handler: handleBoreWell
file: ../src/templates/AquiferHandler.ts
templates:
- kind: ethereum/contract
name: Well
network: not_relevant
source:
abi: Well
mapping:
kind: ethereum/events
apiVersion: 0.0.7
language: wasm/assemblyscript
entities:
- Well
abis:
- name: Well
file: ../../subgraph-core/abis/Well.json
eventHandlers:
- event: Sync(uint256[],uint256,address)
handler: handleSync
file: ../src/WellHandler.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
specVersion: 0.0.4
schema:
file: ./schema.graphql
file: ../schema.graphql
dataSources:
- kind: ethereum/contract
name: Aquifer
Expand All @@ -17,13 +17,13 @@ dataSources:
- Aquifer
abis:
- name: Aquifer
file: ../subgraph-core/abis/Aquifer.json
file: ../../subgraph-core/abis/Aquifer.json
- name: ERC20
file: ../subgraph-core/abis/ERC20.json
file: ../../subgraph-core/abis/ERC20.json
eventHandlers:
- event: BoreWell(address,address,address[],(address,bytes),(address,bytes)[],bytes)
handler: handleBoreWell
file: ./src/templates/AquiferHandler.ts
file: ../src/templates/AquiferHandler.ts
- kind: ethereum/contract
name: Beanstalk
network: mainnet
Expand All @@ -39,11 +39,11 @@ dataSources:
- Beanstalk
abis:
- name: Beanstalk
file: ../subgraph-core/abis/Beanstalk/Beanstalk-Replanted.json
file: ../../subgraph-core/abis/Beanstalk/Beanstalk-Replanted.json
eventHandlers:
- event: Sunrise(indexed uint256)
handler: handleSunrise
file: ./src/BeanstalkHandler.ts
file: ../src/BeanstalkHandler.ts
templates:
- kind: ethereum/contract
name: Well
Expand All @@ -58,13 +58,13 @@ templates:
- Well
abis:
- name: Well
file: ../subgraph-core/abis/Well.json
file: ../../subgraph-core/abis/Well.json
- name: CurvePrice
file: ../subgraph-core/abis/CurvePrice.json
file: ../../subgraph-core/abis/CurvePrice.json
- name: BeanstalkPrice
file: ../subgraph-core/abis/BeanstalkPrice.json
file: ../../subgraph-core/abis/BeanstalkPrice.json
- name: ERC20
file: ../subgraph-core/abis/ERC20.json
file: ../../subgraph-core/abis/ERC20.json
eventHandlers:
- event: AddLiquidity(uint256[],uint256,address)
handler: handleAddLiquidity
Expand All @@ -78,7 +78,7 @@ templates:
handler: handleShift
- event: Sync(uint256[],uint256,address)
handler: handleSync
file: ./src/WellHandler.ts
file: ../src/WellHandler.ts
# features:
# - grafting
# graft:
Expand Down
2 changes: 1 addition & 1 deletion projects/subgraph-basin/matchstick-docker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
# subgraph has some dependencies on other projects in the repo.
testsFolder: repo-mounted/projects/subgraph-basin/tests
libsFolder: repo-mounted/node_modules
manifestPath: repo-mounted/projects/subgraph-basin/subgraph.yaml
manifestPath: repo-mounted/projects/subgraph-basin/manifests/ethereum.yaml
1 change: 1 addition & 0 deletions projects/subgraph-basin/matchstick.yaml
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
libsFolder: ../../node_modules
manifestPath: ./manifests/ethereum.yaml
12 changes: 6 additions & 6 deletions projects/subgraph-basin/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,17 @@
"directory": "projects/subgraph-basin"
},
"scripts": {
"codegen": "rm -rf ./generated && graph codegen",
"build": "yarn codegen && graph build",
"codegen": "rm -rf ./generated && graph codegen ./manifests/codegen-abis.yaml",
"build": "echo Using manifest at ./manifests/$1.yaml && yarn codegen && graph build ./manifests/$1.yaml",
"test": "graph test",
"testd": "docker run -it --rm --mount type=bind,source=\"$(pwd)\"/matchstick-docker.yaml,target=/matchstick/matchstick.yaml --mount type=bind,source=\"$(pwd)\"/../../,target=/matchstick/repo-mounted/ matchstick",
"testd-named": "../subgraph-core/tests/scripts/docker-run-named.sh",
"create-local": "graph create --node http://127.0.0.1:8020/ basin",
"remove-local": "graph remove --node http://127.0.0.1:8020/ basin",
"deploy-local": "yarn codegen && graph deploy --node http://127.0.0.1:8020/ --ipfs http://127.0.0.1:5001 basin"
"create-local": "echo Using manifest at ./manifests/$1.yaml && graph create --node http://127.0.0.1:8020/ basin_$1",
"remove-local": "echo Using manifest at ./manifests/$1.yaml && graph remove --node http://127.0.0.1:8020/ basin_$1",
"deploy-local": "echo Using manifest at ./manifests/$1.yaml && yarn codegen && graph deploy --node http://127.0.0.1:8020/ --ipfs http://127.0.0.1:5001 basin_$1 ./manifests/$1.yaml"
},
"dependencies": {
"@graphprotocol/graph-cli": "0.69.0",
"@graphprotocol/graph-cli": "0.79.2",
"@graphprotocol/graph-ts": "0.34.0"
},
"devDependencies": {
Expand Down
2 changes: 1 addition & 1 deletion projects/subgraph-basin/src/BeanstalkHandler.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { Address } from "@graphprotocol/graph-ts";
import { AQUIFER } from "../../subgraph-core/utils/Constants";
import { Sunrise } from "../generated/Beanstalk/Beanstalk";
import { Sunrise } from "../generated/Basin-ABIs/Beanstalk";
import { loadOrCreateAquifer } from "./utils/Aquifer";
import { checkForSnapshot } from "./utils/Well";

Expand Down
2 changes: 1 addition & 1 deletion projects/subgraph-basin/src/WellHandler.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { AddLiquidity, RemoveLiquidity, RemoveLiquidityOneToken, Shift, Swap, Sync, Transfer } from "../generated/templates/Well/Well";
import { AddLiquidity, RemoveLiquidity, RemoveLiquidityOneToken, Shift, Swap, Sync, Transfer } from "../generated/Basin-ABIs/Well";
import { loadOrCreateAccount } from "./utils/Account";
import { deltaBigIntArray, emptyBigIntArray, ZERO_BI } from "../../subgraph-core/utils/Decimals";
import { recordAddLiquidityEvent, recordRemoveLiquidityEvent, recordRemoveLiquidityOneEvent, recordSyncEvent } from "./utils/Liquidity";
Expand Down
4 changes: 2 additions & 2 deletions projects/subgraph-basin/src/templates/AquiferHandler.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { Address, Bytes, log } from "@graphprotocol/graph-ts";
import { BoreWell } from "../../generated/Aquifer/Aquifer";
import { ERC20 } from "../../generated/Aquifer/ERC20";
import { BoreWell } from "../../generated/Basin-ABIs/Aquifer";
import { ERC20 } from "../../generated/Basin-ABIs/ERC20";
import { Well } from "../../generated/templates";
import { loadOrCreateAquifer } from "../utils/Aquifer";
import { loadOrCreatePump } from "../utils/Pump";
Expand Down
2 changes: 1 addition & 1 deletion projects/subgraph-basin/src/utils/BeanstalkPrice.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Unfortunately this file must be copied across the various subgraph projects. This is due to the codegen
import { Address, BigInt } from "@graphprotocol/graph-ts";
import { BeanstalkPrice } from "../../generated/templates/Well/BeanstalkPrice";
import { BeanstalkPrice } from "../../generated/Basin-ABIs/BeanstalkPrice";
import { BEANSTALK_PRICE_1, BEANSTALK_PRICE_2, PRICE_2_BLOCK } from "../../../subgraph-core/utils/Constants";

// Gets the BeanstalkPrice contract, bound to the appropriate instance of the contract.
Expand Down
2 changes: 1 addition & 1 deletion projects/subgraph-basin/src/utils/Liquidity.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { BigInt } from "@graphprotocol/graph-ts";
import { Deposit, Withdraw } from "../../generated/schema";
import { AddLiquidity, RemoveLiquidity, RemoveLiquidityOneToken, Sync } from "../../generated/templates/Well/Well";
import { AddLiquidity, RemoveLiquidity, RemoveLiquidityOneToken, Sync } from "../../generated/Basin-ABIs/Well";
import { getBigDecimalArrayTotal } from "../../../subgraph-core/utils/Decimals";
import { getCalculatedReserveUSDValues, loadWell } from "./Well";

Expand Down
2 changes: 1 addition & 1 deletion projects/subgraph-basin/src/utils/Pump.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { Address } from "@graphprotocol/graph-ts";
import { BoreWellPumpsStruct } from "../../generated/Aquifer/Aquifer";
import { BoreWellPumpsStruct } from "../../generated/Basin-ABIs/Aquifer";
import { Pump } from "../../generated/schema";

export function loadOrCreatePump(pumpData: BoreWellPumpsStruct, wellAddress: Address): Pump {
Expand Down
2 changes: 1 addition & 1 deletion projects/subgraph-basin/src/utils/Swap.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Shift, Swap } from "../../generated/templates/Well/Well";
import { Shift, Swap } from "../../generated/Basin-ABIs/Well";
import { Swap as SwapEvent } from "../../generated/schema";
import { Address, BigInt } from "@graphprotocol/graph-ts";

Expand Down
4 changes: 2 additions & 2 deletions projects/subgraph-basin/src/utils/Token.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { Address, BigDecimal, BigInt, log } from "@graphprotocol/graph-ts";
import { ERC20 } from "../../generated/Aquifer/ERC20";
import { ERC20 } from "../../generated/Basin-ABIs/ERC20";
import { Token } from "../../generated/schema";
import { CurvePrice } from "../../generated/templates/Well/CurvePrice";
import { CurvePrice } from "../../generated/Basin-ABIs/CurvePrice";
import { BEAN_ERC20, CURVE_PRICE } from "../../../subgraph-core/utils/Constants";
import { toDecimal, ZERO_BD, ZERO_BI } from "../../../subgraph-core/utils/Decimals";
import { getBeanstalkPrice } from "./BeanstalkPrice";
Expand Down
4 changes: 2 additions & 2 deletions projects/subgraph-basin/src/utils/Well.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { Address, BigDecimal, BigInt, Bytes, log } from "@graphprotocol/graph-ts";
import { BoreWellWellFunctionStruct } from "../../generated/Aquifer/Aquifer";
import { BoreWellWellFunctionStruct } from "../../generated/Basin-ABIs/Aquifer";
import { Well, WellDailySnapshot, WellFunction, WellHourlySnapshot } from "../../generated/schema";
import { ERC20 } from "../../generated/templates/Well/ERC20";
import { ERC20 } from "../../generated/Basin-ABIs/ERC20";
import { BEAN_ERC20 } from "../../../subgraph-core/utils/Constants";
import { dayFromTimestamp, hourFromTimestamp } from "../../../subgraph-core/utils/Dates";
import {
Expand Down
2 changes: 1 addition & 1 deletion projects/subgraph-basin/tests/helpers/Aquifer.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { Address, Bytes, ethereum } from "@graphprotocol/graph-ts";
import { newMockEvent } from "matchstick-as/assembly/index";
import { BoreWell } from "../../generated/Aquifer/Aquifer";
import { BoreWell } from "../../generated/Basin-ABIs/Aquifer";
import { handleBoreWell } from "../../src/templates/AquiferHandler";
import { BEAN_ERC20, WETH } from "../../../subgraph-core/utils/Constants";
import { AQUIFER, IMPLEMENTATION, PUMP, WELL, WELL_DATA, WELL_FUNCTION } from "./Constants";
Expand Down
2 changes: 1 addition & 1 deletion projects/subgraph-basin/tests/helpers/Well.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { Address, BigInt, ethereum } from "@graphprotocol/graph-ts";
import { newMockEvent } from "matchstick-as/assembly/index";
import { AddLiquidity, RemoveLiquidity, RemoveLiquidityOneToken, Shift, Swap, Sync } from "../../generated/templates/Well/Well";
import { AddLiquidity, RemoveLiquidity, RemoveLiquidityOneToken, Shift, Swap, Sync } from "../../generated/Basin-ABIs/Well";
import { CURRENT_BLOCK_TIMESTAMP } from "./Constants";

export function createAddLiquidityEvent(well: Address, account: Address, lpAmountOut: BigInt, tokenAmountsIn: BigInt[]): AddLiquidity {
Expand Down
4 changes: 4 additions & 0 deletions projects/subgraph-bean/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,7 @@ All currently used subgraphs live on a centralized host controlled by beanstalk
### Testing

To test with Docker, the first time you will need to run `yarn run graph test -d`. This will build the `matchstick` Docker image. Then, you can use the `yarn testd` script to run all tests. Alternatively, use `yarn testd-named <TestName1> ...` to run specific tests. I have found running in Docker to be preferred since otherwise there can be issues with console output and some test cases fail silently.

### Deploying

When using graph cli commands, you will often need to specify which manifest file should be used. This is necessary to support multiple chains in the same codebase. The commands which need it will be evident - as they will fail when unable to find a `subgraph.yaml` file. In those commands, include `./manifest/${chain}.yaml` as the final argument to the command. See scripts inside `package.json` for examples.
49 changes: 49 additions & 0 deletions projects/subgraph-bean/manifests/codegen-abis.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# This file exists solely for the purpose of facilitating all codegen in a shared location such that all ABIs
# or templates are expanded independently of being used in all chains. Most of the information here is irrelevant,
# the only important part is in the `abis` and `templates` sections.
# - For abis, its only the list of abis that is relevant. The name of the dataSource is also visible.
# - For templates, it is only the name of the template that is relevant.
specVersion: 0.0.4
schema:
file: ../schema.graphql
dataSources:
- kind: ethereum/contract
name: Bean-ABIs
network: not_relevant
source:
address: "0xDC59ac4FeFa32293A95889Dc396682858d52e5Db"
abi: ERC20
startBlock: 12974077
mapping:
kind: ethereum/events
apiVersion: 0.0.7
language: wasm/assemblyscript
entities:
- Bean
abis:
- name: ERC20
file: ../../subgraph-core/abis/ERC20.json
- name: UniswapV2Pair
file: ../../subgraph-core/abis/UniswapV2Pair.json
- name: Bean3CRV
file: ../../subgraph-core/abis/Bean3CRV.json
- name: Well
file: ../../subgraph-core/abis/Well.json
- name: PreReplant
file: ../../subgraph-core/abis/Beanstalk/Beanstalk-Pre-Replant.json
- name: Beanstalk
file: ../../subgraph-core/abis/Beanstalk/Beanstalk-Replanted.json
- name: BIP37
file: ../../subgraph-core/abis/Beanstalk/Beanstalk-BIP37.json
- name: SeedGauge
file: ../../subgraph-core/abis/Beanstalk/Beanstalk-BIP45.json
- name: BeanstalkPrice
file: ../../subgraph-core/abis/BeanstalkPrice.json
- name: CurvePrice
file: ../../subgraph-core/abis/CurvePrice.json
- name: CalculationsCurve
file: ../../subgraph-core/abis/CalculationsCurve.json
eventHandlers:
- event: Transfer(indexed address,indexed address,uint256)
handler: handleTransfer
file: ../src/BeanHandler.ts
Loading
Loading