Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db_schema: system_info.results.overall -> results #459

Merged
merged 3 commits into from
Oct 28, 2022
Merged

db_schema: system_info.results.overall -> results #459

merged 3 commits into from
Oct 28, 2022

Conversation

lyuyangh
Copy link
Member

@lyuyangh lyuyangh commented Oct 27, 2022

part of #433
fixes #320

This is the last PR for the first step. It moves overall metrics out of system_info and stores something like this for each system.

"results": {
  "example": { "rouge1": 0.33, ...},
  "token": { "F1": 0.35, ... }
}

Previously, the endpoints and the UI do not specify the analysis level for each metric which can be confusing because metrics can have the same name for different analysis levels. This PR fixes this issue partially.

  • Now, the systems table page displays the levels. The corresponding endpoint (GET /systems) is also updated to include this information.

image

The headers become even more verbose. We can group the column headers to make it easier to read but I'll leave it for another PR.

image

  • The system submission UI and the endpoint (POST /systems) remain the same because updating those would break the python client. Also, the SDK does not support the configuration of other analysis levels at the moment.

@lyuyangh lyuyangh requested a review from neubig as a code owner October 27, 2022 00:46
Copy link
Contributor

@neubig neubig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Just noticed a potential version number clash.

openapi/openapi.yaml Outdated Show resolved Hide resolved
lyuyangh and others added 2 commits October 27, 2022 21:36
Co-authored-by: Graham Neubig <neubig@gmail.com>
@lyuyangh lyuyangh merged commit 269b373 into main Oct 28, 2022
@lyuyangh lyuyangh deleted the caching-5 branch October 28, 2022 01:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Systems page- Created Atdropdown is incorrect
2 participants