Skip to content
This repository has been archived by the owner on Jan 29, 2019. It is now read-only.

API: Expose index stats #30

Closed
abitrolly opened this issue Dec 19, 2017 · 11 comments
Closed

API: Expose index stats #30

abitrolly opened this issue Dec 19, 2017 · 11 comments

Comments

@abitrolly
Copy link
Contributor

abitrolly commented Dec 19, 2017

We need to see through cyber-ui when indexator is not functional. For example, when it can not access underlying Parity server.

Possible implementation is that the error should propagate to http://search.cyber.fund/cybernode status page and change the cyber-search status to yellow with exact message "ETH node (Parity) is not accessible for indexing".

@hleb-albau
Copy link
Contributor

Is this functionality really needed? We will have page with latest blocks for all systems, we will have page with index size, count and etc cluster info.

@abitrolly
Copy link
Contributor Author

abitrolly commented Dec 20, 2017

Yes. This is really needed.

The story is that person wants to find his transaction that was just posted on the chain. He opens search, enters transaction and gets nothing. Status shows green. The person leaves.

If indexator does not function, the error message could be exposed in http://search.cyber.fund/cybernode status page together with yellow status for Search API component. This creates a feedback loop with a user and tells him what's going on. User can report specific error to us and track the progress. Makes it cybernetic. )

@abitrolly
Copy link
Contributor Author

We will have page with latest blocks for all systems, we will have page with index size, count and etc cluster info.

How do you see API that returns cluster info page? The critical part is only the service health check with error message in UI, so maybe leave cluster info and status checks separated.

@cyborgshead
Copy link
Member

@YodaMike please comment on API for state of indexation story

  • API with stats from Elastic (from Elassandra)
  • API for indexation status (how many blocks indexed and how many are left)

@YodaMike
Copy link
Contributor

  1. API for indices stats is implemented, there are pretty big json with a lot of stats so it is necessary to identify which data is needed.
  2. we need to implement API for information about not indexed data (remained blocks)

@hleb-albau
Copy link
Contributor

@SiarheiSadau we should discuss, where and what info we will show to the users and then define goals for this task

@abitrolly
Copy link
Contributor Author

abitrolly commented Mar 24, 2018 via email

@hleb-albau
Copy link
Contributor

@abitrolly It's not wrong, we should just explicit declare what to expose according to cyber-browser requirements. I just want more info here.

@hleb-albau hleb-albau changed the title API: Expose indexator status API: Expose index stats Apr 6, 2018
@hleb-albau
Copy link
Contributor

Rename task
In scope of this taks should be done:

  • Entry point returning information about elassanda index state[sum tx number,blockhains numer,db size in gb]
  • Reflect changes to the search-api-docs

All other staff related to search healthy will be done in separate issue.

All required info can be obtained by processing result of elastic search default stats endpoint.

@hleb-albau
Copy link
Contributor

http://browser.cybersearch.io/

https://github.com/cybercongress/cyber-browser/blob/master/docs/Software_Requirements.md#1-main-page

Search in [134 M] transactions in [2] blockchain systems with [135] parsed tokens. Database size : [369 GBs]

@arturalbov
Copy link
Contributor

Endpoint for statistics: http://api.cybersearch.io/search/stats

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants