Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Off-chain] feat: in-memory query cache(s) #994

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

bryanchriswhite
Copy link
Contributor

@bryanchriswhite bryanchriswhite commented Dec 11, 2024

Summary

Adds the QueryCache[T any] and HistoricalQueryCache[T any] interfaces, InMemoryCache[T any] implementation, configurations, and options functions.

---
title: Legend
---

classDiagram-v2

class GenericInterface__T__any {
    <<interface>>
    GenericMethod() T
}

class Implemenetation {
    ExportedField FieldType
    unexportedField FieldType
}

Implemenetation --|> GenericInterface__T__any: implements

%% class Embedder__T__any {
%%     <<interface>>
%%     GenericInterface[T]
%% }

%% Embedder__T__any ..|> GenericInterface__T__any: embeds
Loading
---
title: Query Caches
---

classDiagram-v2


class KeyValueCache__T__any {
    <<interface>>
    Get(key string) (value T, err error)
    Set(key string, value T) (err error)
    Delete(key string)
    Clear()
}

class HistoricalKeyValueCache__T__any {
    <<interface>>
    GetLatestVersion(key string) (value T, err error)
    GetVersion(key string, version int64) (value T, err error)
    SetVersion(key string, value T, version int64) (err error)
}

class inMemoryKVCache__T__any:::cacheImpl
inMemoryKVCache__T__any --|> KeyValueCache__T__any
inMemoryKVCache__T__any --|> HistoricalKeyValueCache__T__any
Loading

Issue

Type of change

Select one or more from the following:

Testing

  • Documentation: make docusaurus_start; only needed if you make doc changes
  • Unit Tests: make go_develop_and_test
  • LocalNet E2E Tests: make test_e2e
  • DevNet E2E Tests: Add the devnet-test-e2e label to the PR.

Sanity Checklist

  • I have tested my changes using the available tooling
  • I have commented my code
  • I have performed a self-review of my own code; both comments & source code
  • I create and reference any new tickets, if applicable
  • I have left TODOs throughout the codebase, if applicable

@bryanchriswhite bryanchriswhite added the off-chain Off-chain business logic label Dec 11, 2024
@bryanchriswhite bryanchriswhite self-assigned this Dec 11, 2024
@bryanchriswhite bryanchriswhite linked an issue Dec 11, 2024 that may be closed by this pull request
4 tasks
@bryanchriswhite bryanchriswhite marked this pull request as ready for review December 12, 2024 11:20
* pokt/main:
  [Relayminer, Bug] fix: sessiontree logger never initialized (#993)
  fix: E2E tests - RPC URL path (#1008)
  Updated cheat sheat docs with an example after installation (#1004)
  fix: Nil session tree logger (#1007)
@bryanchriswhite bryanchriswhite changed the base branch from fix/sessiontree to main December 13, 2024 14:24
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bryanchriswhite I did a superficial review but did not dive into the validation of the business logic line-by-line.

Is there any section where you'd want another pair of 👀 ?

pkg/client/interface.go Outdated Show resolved Hide resolved
pkg/client/query/cache/config.go Outdated Show resolved Hide resolved
pkg/client/query/cache/config.go Outdated Show resolved Hide resolved
pkg/client/query/cache/config.go Outdated Show resolved Hide resolved
pkg/client/query/cache/config.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory_test.go Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory_test.go Outdated Show resolved Hide resolved
Co-authored-by: Daniel Olshansky <olshansky.daniel@gmail.com>
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the iteration. Thanks @bryanchriswhite!

Left a few new comments, a few replies to older threads, but it should be g2g after the next round 🙌

pkg/client/query/cache/config.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
Copy link
Contributor

@red-0ne red-0ne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great caching design.

Left some comments but did not tend to pkg/client/query/cache/memory_test.go yet. Which I'll do right after this one.

pkg/client/query/cache/config.go Show resolved Hide resolved
pkg/client/query/cache/memory.go Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
Copy link
Contributor

@red-0ne red-0ne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Asking for a small change, but otherwise LGTM.

Not approving yet as I want review the tests too.

pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/query/cache/memory.go Show resolved Hide resolved
pkg/client/interface.go Outdated Show resolved Hide resolved
pkg/client/query/cache/config.go Show resolved Hide resolved
pkg/client/query/cache/config.go Show resolved Hide resolved
pkg/client/query/cache/memory.go Outdated Show resolved Hide resolved
pkg/client/interface.go Outdated Show resolved Hide resolved
@Olshansk
Copy link
Member

@bryanchriswhite Could we get this over the finish line ASAP so it can be used in #1050?

@Olshansk Olshansk marked this pull request as ready for review February 12, 2025 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
off-chain Off-chain business logic
Projects
Status: 👀 In review
Development

Successfully merging this pull request may close these issues.

[Off-Chain] ModuleParamsClient & Historical Params
3 participants