Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

~0.5 % improvement #119

Merged
merged 1 commit into from
Jul 3, 2024
Merged

Conversation

jyrkialakuijala
Copy link
Collaborator

Score type MSE Min score Max score Mean score
Zimtohrli 0.077647743787295 0.608127552926780 0.829014091535645 0.728679787505096
ViSQOL 0.115330916105424 0.520833375452983 0.801480831107469 0.675101633981268
2f 0.129541391104905 0.484687555319526 0.797475783883375 0.661870345773127
PESQ 0.147425552045669 0.342342966279351 0.841271127756762 0.647128996775172
CDPAM 0.153471222942756 0.441558428344727 0.728779141125759 0.620699318941738
PARLAQ 0.185057687192323 0.445261140223642 0.784370761057963 0.587162756572532
AQUA 0.223207996944378 0.331645933512413 0.739286336419790 0.547804951221731
PEAQB 0.225217321572038 0.278744167467764 0.851011116004117 0.553935720513487
DPAM 0.315810440183130 0.186717781679534 0.690564701717118 0.460415212267967
WARP-Q 0.339686211572685 0.067600137543649 0.777119464646524 0.475793617709890
GVPMOS 0.412937133868407 0.006851162794410 0.783946603687895 0.412912222208318

|Score type |MSE               |Min score         |Max score         |Mean score        |
|-----------|------------------|------------------|------------------|------------------|
|Zimtohrli  |0.077647743787295 |0.608127552926780 |0.829014091535645 |0.728679787505096 |
|ViSQOL     |0.115330916105424 |0.520833375452983 |0.801480831107469 |0.675101633981268 |
|2f         |0.129541391104905 |0.484687555319526 |0.797475783883375 |0.661870345773127 |
|PESQ       |0.147425552045669 |0.342342966279351 |0.841271127756762 |0.647128996775172 |
|CDPAM      |0.153471222942756 |0.441558428344727 |0.728779141125759 |0.620699318941738 |
|PARLAQ     |0.185057687192323 |0.445261140223642 |0.784370761057963 |0.587162756572532 |
|AQUA       |0.223207996944378 |0.331645933512413 |0.739286336419790 |0.547804951221731 |
|PEAQB      |0.225217321572038 |0.278744167467764 |0.851011116004117 |0.553935720513487 |
|DPAM       |0.315810440183130 |0.186717781679534 |0.690564701717118 |0.460415212267967 |
|WARP-Q     |0.339686211572685 |0.067600137543649 |0.777119464646524 |0.475793617709890 |
|GVPMOS     |0.412937133868407 |0.006851162794410 |0.783946603687895 |0.412912222208318 |
@jyrkialakuijala jyrkialakuijala merged commit b9fbd50 into google:tabuli Jul 3, 2024
1 of 2 checks passed
@jyrkialakuijala
Copy link
Collaborator Author

jyrkialakuijala commented Jul 3, 2024

It looks to me that the internal values are out of ranges where they are supposed to be and while this metric correlates fantastically with this corpora, there is likely a more fundamental issue here

it is possible that the loudness corrected spectra shouldn't be L2/NSIM matched but another distance metric should be used

eventual L2/NSIM matching likely drives the internal representations to be very strange and it is highly likely that the system generalizes relatively poorly in its current state to much louder or much more silent sounds as well as spectral extremes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant