[LUM-754] - Batch withdrawals and restore stuck deposits (#45)

* Handle pool paused state

* WIP import updates and project clean up

* Allow native zone toAddress

* Fix all errors - not compiling yet

* Use new ICA server implem - to be thoroughly tested

* Fix missing crisis key in store

* Fix go mod according to upgrade guide

* Prepare buf config

* Update to new makefile base proto gen

* Update workflow

* Fix workflow

* Fix lint workflow

* Handle re-delegate assets from a disabled validator

* Adjust return type

* Exclude rand error from gosec

* Add new makefile cmd to check for formatting issues

* Add new workflows and associated cmds and scripts

* Fix workflow

* Add missing goimports and update readme

* Use official lint workflow

* Remove unused code

* Add new auto formatted proto generated files

* Apply new formatting rules

* clean up linting

* Fix new IBC msg server use of timeout timestamp

* Rollback ci lint version

* Fix dfract linting

* Fix beam lint issues

* Remove use of legacy fundraiser path in favour of new config

* Bump lint ci

* Attempt to fix CI

* Another attemp to fix this lint ci

* Bump golangci-lint version

* Add improvement for test proposal handler and redelegate retry

* Add edit deposit feature

* Fix format check

* Fix linting errors

* Update metadata for EditDeposit and ValidateBasic static check

* Fix import ordering

* Review proto folders structure

* Remove unused simulation code

* Add proper query srv implem

* Revamp RedelegateRetry and migrate to UpdatePool handler

* Migrate Redelegate to UpdatePool and update tests

* Cleanup of unused code and proto

* Add pool security check for UpdateRedelegateStatus

* Use utils function instead of hardcoded disabledValidators in pool_test

* Verify redelegate with deposit success and avoid double funding for ica pool address

* Fix typo in comment

* Fix cmd new commands

* Introduce RebalanceValidatorsBondings and Remove Redelegate State

* Improve RebalanceValidatorsBondings and Redelegate handling

* Use inactiveVal for OnRedelegateToRemoteZoneCompleted

* Merge the 2 unit tests together

* Add airdrop tx proto and chainID in root for ignite compatibility on sdk.047

* Add RebalanceValidatorsBondings condition, params improvement for RedelegateToActiveValidators and helper functions

* Add valSrcAddr for ValidateUnbondAmount

* Set default modules key tables

* Added missing IBC migrations and fixed types naming

* Prune expired client states

* Register the tendermint light client

* Fix codecs registration

* Revert "Fix codecs registration"

This reverts commit 148b730.

* Move back proto to their original path for backward compatibility

* Use store key for consistence

* Unused ClaimCapability anymore

* Remove capabilities for ownership transfer

* Install more dependencies

* Redundant implementation

* IBC Stack revamp

* Add comment

* Add new ProposalUpdateParams and WithdrawAndMint tx

* Revert to all codec path to avoid changing current deposit path

* Add missing end of comment

* Fix lint and import ordering

* Fixed ignite gov module config

* Remove unused errors

* Fix typo and add one more test for proposal handler

* Fix usage between port owner name and port ID

* Fixed ICA callbacks handlers registration

* Formatting

* Add RegisterMsgServer for RegisterServices in x/millions/module.go

* Add RegisterMsgServer to dfract module

* Add RegisterMsgServer for RegisterServices in x/airdrop/module.go

* Add new NewMsgServerImpl for beam module

* Add format for import ordering

* Fix draw prizes fees implementation

* Apply feedback changes

* Add additional tests for invalid depositDenoms

* Migrate consensus params and properly register keeper

* Fix consensus module registration

* Fix implementation following 0.47 sdk upgrade

* Add more comment and unify ValidateBasics

* Fix comment duplicate wording and test description

* [LUM-752] Migrate port IDs (#42)

* Added store migrator for port IDs

* Fix imports

* Fix comment

* Migrate DFract params to the new system

* formatting

* Add config.yml default params for dfract for ignite compatibility

* Improve error message for min_deposit_amount check

* Add epoch implementation for withdrawals

* Rename variables

* Remove legacy callback

* Remove DS_Store

* Remove useless key

* fix linter

* Fix lint and format check

* Fix comments and variables

* Update events

* update store variable name

* same for hasEpochPoolUnbonding

* Decrease the limit per withdrawal

* Remove useless comments

* Add missing genesis

* Add migration for failed ICAUndelegations

* Added missing upgrade handler

* Fix proto implementation

* Update store_test.go

* Update keepers.go

* Bump version

* Remove Epochunbonding from genesis and related endpoints

* Bump SDK version

* Code cleaning

* Reduce hardcap limit

* use MaxAcceptableWithdrawalIDsCount instead of hardcoded int

* Revert the unappropriate split delegation in case of error callback

* Fix withdrawal error split computation

* Remove glitch in .gitignore

* remove useless space

* [LUM-789] Patch broken deposit entities (#47)

* Patch broken deposit entities

* format store migration

---------

Co-authored-by: Fabrice Bascoulergue <lebascou@users.noreply.github.com>
Co-authored-by: Fabrice Bascoulergue <3663159+lebascou@users.noreply.github.com>
Co-authored-by: Segfault <5221072+Segfaultd@users.noreply.github.com>

app/app.go

@@ -106,13 +106,16 @@ import (
 	"github.com/lum-network/chain/x/dfract"
 	dfractclient "github.com/lum-network/chain/x/dfract/client"
 	dfracttypes "github.com/lum-network/chain/x/dfract/types"
+	"github.com/lum-network/chain/x/epochs"
 	"github.com/lum-network/chain/x/icacallbacks"
 	icacallbackstypes "github.com/lum-network/chain/x/icacallbacks/types"
 	"github.com/lum-network/chain/x/icqueries"
 	icqueriestypes "github.com/lum-network/chain/x/icqueries/types"
 	"github.com/lum-network/chain/x/millions"
 	millionsclient "github.com/lum-network/chain/x/millions/client"
 	millionstypes "github.com/lum-network/chain/x/millions/types"
+
+	epochstypes "github.com/lum-network/chain/x/epochs/types"
 )
 
 var (
@@ -162,6 +165,7 @@ var (
 		airdrop.AppModuleBasic{},
 		dfract.AppModuleBasic{},
 		millions.AppModuleBasic{},
+		epochs.AppModuleBasic{},
 	)
 
 	// module account permissions
@@ -265,7 +269,7 @@ func New(
 		minttypes.StoreKey, distrtypes.StoreKey, slashingtypes.StoreKey,
 		govtypes.StoreKey, paramstypes.StoreKey, ibchost.StoreKey, ICAControllerCustomStoreKey, icahosttypes.StoreKey, upgradetypes.StoreKey, feegrant.StoreKey,
 		evidencetypes.StoreKey, ibctransfertypes.StoreKey, capabilitytypes.StoreKey, authzkeeper.StoreKey, crisistypes.StoreKey, consensusparamtypes.StoreKey,
-		icacallbackstypes.StoreKey, icqueriestypes.StoreKey,
+		icacallbackstypes.StoreKey, icqueriestypes.StoreKey, epochstypes.StoreKey,
 		beamtypes.StoreKey, airdroptypes.StoreKey, dfracttypes.StoreKey, millionstypes.StoreKey,
 	)
 	tkeys := sdk.NewTransientStoreKeys(paramstypes.TStoreKey)
@@ -326,6 +330,7 @@ func New(
 		millions.NewAppModule(appCodec, *app.MillionsKeeper),
 		icacallbacks.NewAppModule(appCodec, *app.ICACallbacksKeeper, app.AccountKeeper, app.BankKeeper),
 		icqueries.NewAppModule(appCodec, *app.ICQueriesKeeper),
+		epochs.NewAppModule(appCodec, *app.EpochsKeeper),
 	)
 
 	// During begin block slashing happens after distr.BeginBlocker so that
@@ -354,11 +359,12 @@ func New(
 		ibctransfertypes.ModuleName,
 		icqueriestypes.ModuleName,
 		icacallbackstypes.ModuleName,
+		epochstypes.ModuleName,
+		consensusparamtypes.ModuleName,
 		beamtypes.ModuleName,
 		airdroptypes.ModuleName,
 		dfracttypes.ModuleName,
 		millionstypes.ModuleName,
-		consensusparamtypes.ModuleName,
 	)
 
 	app.mm.SetOrderEndBlockers(
@@ -383,11 +389,12 @@ func New(
 		ibctransfertypes.ModuleName,
 		icqueriestypes.ModuleName,
 		icacallbackstypes.ModuleName,
+		epochstypes.ModuleName,
+		consensusparamtypes.ModuleName,
 		beamtypes.ModuleName,
 		airdroptypes.ModuleName,
 		dfracttypes.ModuleName,
 		millionstypes.ModuleName,
-		consensusparamtypes.ModuleName,
 	)
 
 	// NOTE: The genutils module must occur after staking so that pools are
@@ -417,11 +424,12 @@ func New(
 		authz.ModuleName,
 		icqueriestypes.ModuleName,
 		icacallbackstypes.ModuleName,
+		epochstypes.ModuleName,
+		consensusparamtypes.ModuleName,
 		beamtypes.ModuleName,
 		airdroptypes.ModuleName,
 		dfracttypes.ModuleName,
 		millionstypes.ModuleName,
-		consensusparamtypes.ModuleName,
 	)
 
 	app.mm.RegisterInvariants(app.CrisisKeeper)
@@ -877,6 +885,13 @@ func (app *App) registerUpgradeHandlers() {
 		return app.mm.RunMigrations(ctx, app.configurator, fromVM)
 	})
 
+	app.UpgradeKeeper.SetUpgradeHandler("v1.5.2", func(ctx sdk.Context, plan upgradetypes.Plan, fromVM module.VersionMap) (module.VersionMap, error) {
+		app.Logger().Info("Starting v1.5.2 upgrade")
+
+		app.Logger().Info("v1.5.2 upgrade applied")
+		return app.mm.RunMigrations(ctx, app.configurator, fromVM)
+	})
+
 	upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk()
 	if err != nil {
 		panic(fmt.Sprintf("failed to read upgrade info from disk %s", err))
@@ -951,4 +966,11 @@ func (app *App) registerUpgradeHandlers() {
 		storeUpgrades := storetypes.StoreUpgrades{}
 		app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades))
 	}
+
+	if upgradeInfo.Name == "v1.5.2" && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height) {
+		storeUpgrades := storetypes.StoreUpgrades{
+			Added: []string{epochstypes.StoreKey},
+		}
+		app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades))
+	}
 }

app/keepers.go

@@ -67,6 +67,9 @@ import (
 	"github.com/lum-network/chain/x/millions"
 	millionskeeper "github.com/lum-network/chain/x/millions/keeper"
 	millionstypes "github.com/lum-network/chain/x/millions/types"
+
+	epochskeeper "github.com/lum-network/chain/x/epochs/keeper"
+	epochstypes "github.com/lum-network/chain/x/epochs/types"
 )
 
 type AppKeepers struct {
@@ -106,6 +109,7 @@ type AppKeepers struct {
 	BeamKeeper         *beamkeeper.Keeper
 	DFractKeeper       *dfractkeeper.Keeper
 	MillionsKeeper     *millionskeeper.Keeper
+	EpochsKeeper       *epochskeeper.Keeper
 }
 
 // InitSpecialKeepers Init the "special" keepers in the order of definition
@@ -319,6 +323,9 @@ func (app *App) InitNormalKeepers() {
 	// Initialize our ICQueries keeper
 	app.ICQueriesKeeper = icquerieskeeper.NewKeeper(appCodec, keys[icqueriestypes.StoreKey], app.IBCKeeper)
 
+	// Initialize custom epochs
+	app.EpochsKeeper = epochskeeper.NewKeeper(appCodec, keys[epochstypes.StoreKey])
+
 	// Initialize our custom beam keeper
 	app.BeamKeeper = beamkeeper.NewKeeper(
 		appCodec,
@@ -453,6 +460,8 @@ func (app *App) SetupHooks() {
 	app.GovKeeper.SetHooks(govtypes.NewMultiGovHooks(
 		govtypes.NewMultiGovHooks(app.AirdropKeeper.Hooks()),
 	))
+
+	app.EpochsKeeper.SetHooks(epochstypes.NewMultiEpochHooks(app.MillionsKeeper.Hooks()))
 }
 
 // InitParamsKeeper init params keeper and its subspaces
@@ -478,6 +487,7 @@ func (app *App) InitParamsKeeper(appCodec codec.BinaryCodec, legacyAmino *codec.
 	// Custom modules
 	paramsKeeper.Subspace(icacallbackstypes.ModuleName)
 	paramsKeeper.Subspace(icqueriestypes.ModuleName)
+	paramsKeeper.Subspace(epochstypes.ModuleName)
 	paramsKeeper.Subspace(beamtypes.ModuleName)
 	paramsKeeper.Subspace(dfracttypes.ModuleName)
 	paramsKeeper.Subspace(millionstypes.ModuleName)

go.mod

@@ -3,12 +3,12 @@ module github.com/lum-network/chain
 go 1.18
 
 require (
-	cosmossdk.io/errors v1.0.0-beta.7
+	cosmossdk.io/errors v1.0.0
 	cosmossdk.io/math v1.0.1
 	github.com/cometbft/cometbft v0.37.2
 	github.com/cometbft/cometbft-db v0.8.0
 	github.com/cosmos/cosmos-proto v1.0.0-beta.2
-	github.com/cosmos/cosmos-sdk v0.47.3
+	github.com/cosmos/cosmos-sdk v0.47.4
 	github.com/cosmos/gogoproto v1.4.10
 	github.com/cosmos/ibc-go/v7 v7.2.0
 	github.com/cosmos/ics23/go v0.10.0
@@ -22,22 +22,22 @@ require (
 	github.com/spf13/cobra v1.7.0
 	github.com/spf13/pflag v1.0.5
 	github.com/stretchr/testify v1.8.4
-	google.golang.org/genproto/googleapis/api v0.0.0-20230530153820-e85fd2cbaebc
-	google.golang.org/grpc v1.55.0
-	google.golang.org/protobuf v1.30.0
+	google.golang.org/genproto/googleapis/api v0.0.0-20230629202037-9506855d4529
+	google.golang.org/grpc v1.56.2
+	google.golang.org/protobuf v1.31.0
 	gopkg.in/yaml.v2 v2.4.0
 )
 
 require (
-	cloud.google.com/go v0.110.0 // indirect
-	cloud.google.com/go/compute v1.19.0 // indirect
+	cloud.google.com/go v0.110.4 // indirect
+	cloud.google.com/go/compute v1.20.1 // indirect
 	cloud.google.com/go/compute/metadata v0.2.3 // indirect
-	cloud.google.com/go/iam v0.13.0 // indirect
-	cloud.google.com/go/storage v1.29.0 // indirect
+	cloud.google.com/go/iam v1.1.0 // indirect
+	cloud.google.com/go/storage v1.30.1 // indirect
 	cosmossdk.io/api v0.3.1 // indirect
 	cosmossdk.io/core v0.5.1 // indirect
 	cosmossdk.io/depinject v1.0.0-alpha.3 // indirect
-	cosmossdk.io/log v1.1.0 // indirect
+	cosmossdk.io/log v1.1.1-0.20230704160919-88f2c830b0ca // indirect
 	cosmossdk.io/tools/rosetta v0.2.1 // indirect
 	filippo.io/edwards25519 v1.0.0 // indirect
 	github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4 // indirect
@@ -87,10 +87,10 @@ require (
 	github.com/google/btree v1.1.2 // indirect
 	github.com/google/go-cmp v0.5.9 // indirect
 	github.com/google/orderedcode v0.0.1 // indirect
-	github.com/google/s2a-go v0.1.3 // indirect
+	github.com/google/s2a-go v0.1.4 // indirect
 	github.com/google/uuid v1.3.0 // indirect
 	github.com/googleapis/enterprise-certificate-proxy v0.2.3 // indirect
-	github.com/googleapis/gax-go/v2 v2.8.0 // indirect
+	github.com/googleapis/gax-go/v2 v2.11.0 // indirect
 	github.com/gorilla/handlers v1.5.1 // indirect
 	github.com/gorilla/websocket v1.5.0 // indirect
 	github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
@@ -117,7 +117,7 @@ require (
 	github.com/magiconair/properties v1.8.7 // indirect
 	github.com/manifoldco/promptui v0.9.0 // indirect
 	github.com/mattn/go-colorable v0.1.13 // indirect
-	github.com/mattn/go-isatty v0.0.18 // indirect
+	github.com/mattn/go-isatty v0.0.19 // indirect
 	github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
 	github.com/mimoo/StrobeGo v0.0.0-20210601165009-122bf33a46e0 // indirect
 	github.com/minio/highwayhash v1.0.2 // indirect
@@ -149,18 +149,18 @@ require (
 	github.com/zondax/ledger-go v0.14.1 // indirect
 	go.etcd.io/bbolt v1.3.7 // indirect
 	go.opencensus.io v0.24.0 // indirect
-	golang.org/x/crypto v0.9.0 // indirect
+	golang.org/x/crypto v0.11.0 // indirect
 	golang.org/x/exp v0.0.0-20230515195305-f3d0a9c9a5cc // indirect
-	golang.org/x/net v0.10.0 // indirect
+	golang.org/x/net v0.12.0 // indirect
 	golang.org/x/oauth2 v0.8.0 // indirect
-	golang.org/x/sys v0.8.0 // indirect
-	golang.org/x/term v0.8.0 // indirect
-	golang.org/x/text v0.9.0 // indirect
+	golang.org/x/sys v0.10.0 // indirect
+	golang.org/x/term v0.10.0 // indirect
+	golang.org/x/text v0.11.0 // indirect
 	golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
-	google.golang.org/api v0.122.0 // indirect
+	google.golang.org/api v0.126.0 // indirect
 	google.golang.org/appengine v1.6.7 // indirect
-	google.golang.org/genproto v0.0.0-20230526203410-71b5a4ffd15e // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20230530153820-e85fd2cbaebc // indirect
+	google.golang.org/genproto v0.0.0-20230706204954-ccb25ca9f130 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98 // indirect
 	gopkg.in/ini.v1 v1.67.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 	nhooyr.io/websocket v1.8.6 // indirect