📝 Add docstrings to jt/dbmigrate
#1909
Closed
+3,819
−3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Docstrings generation was requested by @JayT106. * #1908 (comment) The following files were modified: * `cmd/cronosd/cmd/database.go` * `cmd/cronosd/cmd/migrate_db.go` * `cmd/cronosd/cmd/patch_db.go` * `cmd/cronosd/cmd/root.go` * `cmd/cronosd/dbmigrate/height_filter.go` * `cmd/cronosd/dbmigrate/migrate.go` * `cmd/cronosd/dbmigrate/migrate_no_rocksdb.go` * `cmd/cronosd/dbmigrate/migrate_rocksdb.go` * `cmd/cronosd/dbmigrate/patch.go` * `cmd/cronosd/dbmigrate/swap-migrated-db.sh` * `x/e2ee/client/cli/encrypt.go`
coderabbitai
bot
requested review from
a team,
JayT106,
XinyuCRO and
calvinaco
and removed request for
a team and
XinyuCRO
Contributor
Author
|
Important Review skippedCodeRabbit bot authored PR detected. To trigger a single review, invoke the You can disable this status message by setting the Comment |
|
Hey there and thank you for opening this pull request! 👋🏼 We require pull request titles to follow the Conventional Commits specification and it looks like your proposed title needs to be adjusted. Details: |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| "target_keys_checked", targetKeys, | ||
| "mismatches", mismatchCount, | ||
| ) | ||
| lastProgressReport = time.Now() |
Check warning
Code scanning / CodeQL
Calling the system time Warning
|
|
||
| logger := opts.Logger | ||
| stats := &MigrationStats{ | ||
| StartTime: time.Now(), |
Check warning
Code scanning / CodeQL
Calling the system time Warning
| } | ||
| } | ||
|
|
||
| stats.EndTime = time.Now() |
Check warning
Code scanning / CodeQL
Calling the system time Warning
Comment on lines
+737
to
+870
| for ethTxHash, info := range ethTxInfos { | ||
| // Create specific prefix for this transaction to minimize iteration range | ||
| // Format: ethereum_tx.ethereumTxHash/0x<eth_txhash>/<height>/<txindex> | ||
| // This will match both keys with and without "$es$<eventseq>" suffix | ||
| // Note: ethTxHash already includes the 0x prefix | ||
| prefix := fmt.Sprintf("ethereum_tx.ethereumTxHash/%s/%d/%d", ethTxHash, info.Height, info.TxIndex) | ||
| prefixBytes := []byte(prefix) | ||
|
|
||
| // Create end boundary by incrementing the prefix (exclusive upper bound) | ||
| endBytes := incrementBytes(prefixBytes) | ||
|
|
||
| // Create bounded iterator with [start, end) | ||
| it, err := sourceDB.Iterator(prefixBytes, endBytes) | ||
| if err != nil { | ||
| logger.Error("Failed to create iterator for ethereum event keys", "error", err, "eth_txhash", ethTxHash) | ||
| stats.ErrorCount.Add(1) | ||
| continue | ||
| } | ||
|
|
||
| eventKeysFound := 0 | ||
| for it.Valid() { | ||
| key := it.Key() | ||
| value := it.Value() | ||
|
|
||
| // Stop if we're past the prefix | ||
| if !bytes.HasPrefix(key, prefixBytes) { | ||
| break | ||
| } | ||
|
|
||
| eventKeysFound++ | ||
| keyStr := string(key) | ||
|
|
||
| logger.Debug("Found ethereum event key in source", | ||
| "event_key", keyStr, | ||
| "eth_txhash", ethTxHash, | ||
| "height", info.Height, | ||
| "tx_index", info.TxIndex, | ||
| ) | ||
|
|
||
| // Check for conflicts | ||
| shouldWrite := true | ||
| if !opts.SkipConflictChecks { | ||
| existingValue, err := targetDB.Get(key) | ||
| if err != nil { | ||
| stats.ErrorCount.Add(1) | ||
| logger.Error("Failed to check existing ethereum event key", "error", err) | ||
| it.Next() | ||
| continue | ||
| } | ||
|
|
||
| if existingValue != nil { | ||
| switch currentStrategy { | ||
| case ConflictSkip: | ||
| shouldWrite = false | ||
| skippedCount++ | ||
| logger.Debug("Skipping existing ethereum event key", | ||
| "event_key", keyStr, | ||
| ) | ||
|
|
||
| case ConflictReplace, ConflictReplaceAll: | ||
| shouldWrite = true | ||
| logger.Debug("Replacing existing ethereum event key", | ||
| "event_key", keyStr, | ||
| ) | ||
|
|
||
| case ConflictAsk: | ||
| // Use replace strategy for event keys to avoid excessive prompting | ||
| shouldWrite = true | ||
| logger.Debug("Patching ethereum event key (using current strategy)", | ||
| "event_key", keyStr, | ||
| ) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if shouldWrite { | ||
| // Make a copy of the value since iterator reuses memory | ||
| valueCopy := make([]byte, len(value)) | ||
| copy(valueCopy, value) | ||
|
|
||
| if opts.DryRun { | ||
| logger.Debug("[DRY RUN] Would patch ethereum event key", | ||
| "event_key", keyStr, | ||
| "value_preview", formatKeyPrefix(valueCopy, 80), | ||
| ) | ||
| } else { | ||
| if err := batch.Set(key, valueCopy); err != nil { | ||
| stats.ErrorCount.Add(1) | ||
| logger.Error("Failed to set ethereum event key in batch", "error", err) | ||
| it.Next() | ||
| continue | ||
| } | ||
| logger.Debug("Patched ethereum event key", | ||
| "event_key", keyStr, | ||
| "value_preview", formatKeyPrefix(valueCopy, 80), | ||
| ) | ||
| } | ||
|
|
||
| batchCount++ | ||
| processedCount++ | ||
|
|
||
| // Write batch when full | ||
| if batchCount >= opts.BatchSize { | ||
| if !opts.DryRun { | ||
| if err := batch.Write(); err != nil { | ||
| it.Close() | ||
| return fmt.Errorf("failed to write ethereum event batch: %w", err) | ||
| } | ||
| logger.Debug("Wrote ethereum event batch", "batch_size", batchCount) | ||
| batch.Close() | ||
| batch = targetDB.NewBatch() | ||
| } | ||
| stats.ProcessedKeys.Add(int64(batchCount)) | ||
| batchCount = 0 | ||
| } | ||
| } | ||
|
|
||
| it.Next() | ||
| } | ||
|
|
||
| if err := it.Error(); err != nil { | ||
| it.Close() | ||
| return fmt.Errorf("iterator error for eth_txhash %s: %w", ethTxHash, err) | ||
| } | ||
|
|
||
| it.Close() | ||
|
|
||
| if eventKeysFound > 0 { | ||
| logger.Debug("Processed event keys for transaction", | ||
| "eth_txhash", ethTxHash, | ||
| "event_keys_found", eventKeysFound, | ||
| ) | ||
| } | ||
| } |
Check warning
Code scanning / CodeQL
Iteration over map Warning
|
|
||
| batchCount := 0 | ||
| skippedCount := int64(0) | ||
| lastLogTime := time.Now() |
Check warning
Code scanning / CodeQL
Calling the system time Warning
| "progress", fmt.Sprintf("%.2f%%", progress), | ||
| "errors", stats.ErrorCount.Load(), | ||
| ) | ||
| lastLogTime = time.Now() |
Check warning
Code scanning / CodeQL
Calling the system time Warning
| } | ||
|
|
||
| // If more than 80% is printable, treat as text | ||
| if float64(printableCount)/float64(len(value)) > 0.8 { |
Check notice
Code scanning / CodeQL
Floating point arithmetic Note
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Docstrings generation was requested by @JayT106.
The following files were modified:
cmd/cronosd/cmd/database.gocmd/cronosd/cmd/migrate_db.gocmd/cronosd/cmd/patch_db.gocmd/cronosd/cmd/root.gocmd/cronosd/dbmigrate/height_filter.gocmd/cronosd/dbmigrate/migrate.gocmd/cronosd/dbmigrate/migrate_no_rocksdb.gocmd/cronosd/dbmigrate/migrate_rocksdb.gocmd/cronosd/dbmigrate/patch.gocmd/cronosd/dbmigrate/swap-migrated-db.shx/e2ee/client/cli/encrypt.goThese files were ignored
cmd/cronosd/cmd/migrate_db_test.gocmd/cronosd/dbmigrate/height_filter_test.gocmd/cronosd/dbmigrate/height_parse_test.gocmd/cronosd/dbmigrate/migrate_basic_test.gocmd/cronosd/dbmigrate/migrate_dbname_test.gocmd/cronosd/dbmigrate/migrate_rocksdb_test.gocmd/cronosd/dbmigrate/migrate_test.gocmd/cronosd/dbmigrate/patch_test.goThese file types are not supported
CHANGELOG.mdMakefilecmd/cronosd/dbmigrate/QUICKSTART.mdcmd/cronosd/dbmigrate/README.mdℹ️ Note