Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement dynamodb based request and metric stores #26

Merged
merged 2 commits into from
Oct 1, 2024

Conversation

cfkanesan
Copy link
Contributor

  • implemented dynamodb backend for request and metric stores
  • float type is not supported by dynamodb, values are cast to and from Decimal type, precision is not important because it's mainly timestamps
  • the request table has secondary indexes on the user and status fields to enable querying (constant lookup time)
  • the metrics table has a secondary index on the request_id
  • the index definitions are motivated by access patterns observed elsewhere in the codebase
  • user_id field is duplicated at the top level to use as the hash key for a global secondary index for the request table
  • request_id field on the metrics table is dropped if NULL, this makes the index sparse
  • the user index query takes precedence on the status index query since the former is more restrictive
  • the wipe functions are not implemented because the testing is done in units against moto and are thus not needed
  • the endpoint url config can be used to run against a local instance of dynamodb
  • collect_metric_info is not yet implemented, dynamodb reports metrics directly to Cloudwatch, could be needed if another monitoring system is used

@FussyDuck
Copy link

FussyDuck commented Oct 1, 2024

CLA assistant check
All committers have signed the CLA.

@jameshawkes jameshawkes requested review from peshence and sametd October 1, 2024 09:56
Copy link
Collaborator

@peshence peshence left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

Copy link
Member

@sametd sametd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@sametd sametd merged commit 1d1d608 into ecmwf:develop Oct 1, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants