Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tabuli #103

Merged
merged 37 commits into from
Jun 14, 2024
Merged

Tabuli #103

merged 37 commits into from
Jun 14, 2024

Conversation

jyrkialakuijala
Copy link
Collaborator

update tabuli and make it faster and simpler

Martin Bruse and others added 30 commits June 13, 2024 18:27
@jyrkialakuijala
Made it possible to define the loudness coeffs in the Go Zimtohrli wr…
d191a9b 
…apper.
@jyrkialakuijala
Refactored CutFullyMasked to avoid an extra loop.
1b2540b
@zond @jyrkialakuijala
Added an f2 variant metric.
25d8c81
@jyrkialakuijala
Refactored CutFullyMasked to avoid an extra loop.
d37e18d 
Also refactored and cleaned up some naming and other small details.
@jyrkialakuijala
Removed unused variable.
618494d
@jyrkialakuijala
Fixed small error.
8b7368b
@jyrkialakuijala
Added the AQUA metric.
80fd0d9
@jyrkialakuijala
Added the 2f metric.
b2bc6af
@jyrkialakuijala
More doc comments.
0689066
@jyrkialakuijala
Fixed Go lint issues.
d640340
@jyrkialakuijala
Fixed Go lint issues.
a0e128a
@jyrkialakuijala
IWYU fixes.
08cce16
@jyrkialakuijala
Made the goohrli test more forgiving of architectural float differences.
87cd6a7
@jyrkialakuijala
Fixed the naming of the 2f metric.
9640544
@zond @jyrkialakuijala
Update sebass_db.go
3ace8e6 
Fixed typo.
@jyrkialakuijala
Replaced the 2f coeffs.
54e36c6 
Replaced the coeffs from https://ieeexplore.ieee.org/document/8937179 with coeffs from https://www.audiolabs-erlangen.de/resources/2019-WASPAA-SEBASS/#NewModelParams, which provided slightly better mean performance.
@zond @jyrkialakuijala
More documentation.
bc66cf6
@zond @jyrkialakuijala
Made the progress bar wait at most 100ms for writing to stderr.
9ee2144
@jyrkialakuijala
New COMPARISON.md with latest version of 2f, PARLAQ, and GVPMOS.
470f450
@jyrkialakuijala
Removed redundant function.
9c9453d
@jyrkialakuijala
Made --report and --leaderboard use different precision.
5793352 
Also hopefully fixed the bug where goohrli.a isn't always recreated when needed.
@jyrkialakuijala
Fixed the DTW benchmark.
c4df855
@jyrkialakuijala
Fixed some performance issues in DTW.
5dd9b68 
Co-authored-by: Jyrki Alakuijala <jyrki.alakuijala@gmail.com>
@jyrkialakuijala
Fixed the slightly different behaviour of the DTW after the optimizat…
aa9dd9b 
…ion.
@jyrkialakuijala
special casing norm=2 for speedup
8668e92 
Before:

|Score type |MSE               |Min score         |Max score         |Mean score        |
|-----------|------------------|------------------|------------------|------------------|
|Zimtohrli  |0.101641872045228 |0.563959418943879 |0.762245848804069 |0.686398636984063 |
|ViSQOL     |0.115330916105424 |0.520833375452983 |0.801480831107469 |0.675101633981268 |
|2f         |0.129541391104905 |0.484687555319526 |0.797475783883375 |0.661870345773127 |
|PESQ       |0.147425552045669 |0.342342966279351 |0.841271127756762 |0.647128996775172 |
|CDPAM      |0.153471222942756 |0.441558428344727 |0.728779141125759 |0.620699318941738 |
|PARLAQ     |0.185057687192323 |0.445261140223642 |0.784370761057963 |0.587162756572532 |
|AQUA       |0.223207996944378 |0.331645933512413 |0.739286336419790 |0.547804951221731 |
|PEAQB      |0.225217321572038 |0.278744167467764 |0.851011116004117 |0.553935720513487 |
|DPAM       |0.315810440183130 |0.186717781679534 |0.690564701717118 |0.460415212267967 |
|WARP-Q     |0.339686211572685 |0.067600137543649 |0.777119464646524 |0.475793617709890 |
|GVPMOS     |0.412937133868407 |0.006851162794410 |0.783946603687895 |0.412912222208318 |

real	44m52.908s # not indicative as the terminal was blocked by
another window and consequently blocking the computation on my desktop

user	748m34.686s
sys	219m0.831s

After:

|Score type |MSE               |Min score         |Max score         |Mean score        |
|-----------|------------------|------------------|------------------|------------------|
|Zimtohrli  |0.101642192840456 |0.563959418943879 |0.762245848804069 |0.686398039616168 |
|ViSQOL     |0.115330916105424 |0.520833375452983 |0.801480831107469 |0.675101633981268 |
|2f         |0.129541391104905 |0.484687555319526 |0.797475783883375 |0.661870345773127 |
|PESQ       |0.147425552045669 |0.342342966279351 |0.841271127756762 |0.647128996775172 |
|CDPAM      |0.153471222942756 |0.441558428344727 |0.728779141125759 |0.620699318941738 |
|PARLAQ     |0.185057687192323 |0.445261140223642 |0.784370761057963 |0.587162756572532 |
|AQUA       |0.223207996944378 |0.331645933512413 |0.739286336419790 |0.547804951221731 |
|PEAQB      |0.225217321572038 |0.278744167467764 |0.851011116004117 |0.553935720513487 |
|DPAM       |0.315810440183130 |0.186717781679534 |0.690564701717118 |0.460415212267967 |
|WARP-Q     |0.339686211572685 |0.067600137543649 |0.777119464646524 |0.475793617709890 |
|GVPMOS     |0.412937133868407 |0.006851162794410 |0.783946603687895 |0.412912222208318 |

real	13m52.362s
user	625m20.070s
sys	279m59.803s

17 % reduction in user time

some more speedup on the gammatone filter side
@jyrkialakuijala
further speedup
1837a0a 
Before:

Score type	MSE	Min score	Max score	Mean score
Zimtohrli	0.101642192840456	0.563959418943879	0.762245848804069	0.686398039616168
ViSQOL	0.115330916105424	0.520833375452983	0.801480831107469	0.675101633981268
2f	0.129541391104905	0.484687555319526	0.797475783883375	0.661870345773127
PESQ	0.147425552045669	0.342342966279351	0.841271127756762	0.647128996775172
CDPAM	0.153471222942756	0.441558428344727	0.728779141125759	0.620699318941738
PARLAQ	0.185057687192323	0.445261140223642	0.784370761057963	0.587162756572532
AQUA	0.223207996944378	0.331645933512413	0.739286336419790	0.547804951221731
PEAQB	0.225217321572038	0.278744167467764	0.851011116004117	0.553935720513487
DPAM	0.315810440183130	0.186717781679534	0.690564701717118	0.460415212267967
WARP-Q	0.339686211572685	0.067600137543649	0.777119464646524	0.475793617709890
GVPMOS	0.412937133868407	0.006851162794410	0.783946603687895	0.412912222208318
real 13m52.362s
user 625m20.070s
sys 279m59.803s

After:

|Score type |MSE               |Min score         |Max score         |Mean score        |
|-----------|------------------|------------------|------------------|------------------|
|Zimtohrli  |0.101642192138072 |0.563959418943879 |0.762245848804069 |0.686398040909622 |
|ViSQOL     |0.115330916105424 |0.520833375452983 |0.801480831107469 |0.675101633981268 |
|2f         |0.129541391104905 |0.484687555319526 |0.797475783883375 |0.661870345773127 |
|PESQ       |0.147425552045669 |0.342342966279351 |0.841271127756762 |0.647128996775172 |
|CDPAM      |0.153471222942756 |0.441558428344727 |0.728779141125759 |0.620699318941738 |
|PARLAQ     |0.185057687192323 |0.445261140223642 |0.784370761057963 |0.587162756572532 |
|AQUA       |0.223207996944378 |0.331645933512413 |0.739286336419790 |0.547804951221731 |
|PEAQB      |0.225217321572038 |0.278744167467764 |0.851011116004117 |0.553935720513487 |
|DPAM       |0.315810440183130 |0.186717781679534 |0.690564701717118 |0.460415212267967 |
|WARP-Q     |0.339686211572685 |0.067600137543649 |0.777119464646524 |0.475793617709890 |
|GVPMOS     |0.412937133868407 |0.006851162794410 |0.783946603687895 |0.412912222208318 |

real	13m46.277s
user	611m8.295s
sys	281m14.419s

a few percent more speedup on user time
@jyrkialakuijala
Tiny change.
7b84dd8
@zond @jyrkialakuijala
Hacked together a replacement of the ellitic filters with the tabuli
d84a7f8 
rotator filters.
@jyrkialakuijala
..
68b7e00
@jyrkialakuijala
...
f930b9c
Martin Bruse and others added 7 commits June 13, 2024 18:31
@jyrkialakuijala
Made it possible to define the loudness coeffs in the Go Zimtohrli wr…
96fe87b 
…apper.
@jyrkialakuijala
Refactored CutFullyMasked to avoid an extra loop.
6e4780d 
Also refactored and cleaned up some naming and other small details.
@jyrkialakuijala
Added the AQUA metric.
a559150
@jyrkialakuijala
Added the 2f metric.
86bc878
@jyrkialakuijala
New COMPARISON.md with latest version of 2f, PARLAQ, and GVPMOS.
3cdb083
@jyrkialakuijala
stash
9fc866a
@jyrkialakuijala
refreshing tabuli to changes in main
e5142d4 
plus further simplifications

~2-3 % speedup by avoiding one sqrt-square pair that did nothing

Before:

|Score type |MSE               |Min score         |Max score         |Mean score        |
|-----------|------------------|------------------|------------------|------------------|
|Zimtohrli  |0.101331579109522 |0.567006001464443 |0.774702668464531 |0.690847270049710 |
|ViSQOL     |0.115330916105424 |0.520833375452983 |0.801480831107469 |0.675101633981268 |
|2f         |0.129541391104905 |0.484687555319526 |0.797475783883375 |0.661870345773127 |
|PESQ       |0.147425552045669 |0.342342966279351 |0.841271127756762 |0.647128996775172 |
|CDPAM      |0.153471222942756 |0.441558428344727 |0.728779141125759 |0.620699318941738 |
|PARLAQ     |0.185057687192323 |0.445261140223642 |0.784370761057963 |0.587162756572532 |
|AQUA       |0.223207996944378 |0.331645933512413 |0.739286336419790 |0.547804951221731 |
|PEAQB      |0.225217321572038 |0.278744167467764 |0.851011116004117 |0.553935720513487 |
|DPAM       |0.315810440183130 |0.186717781679534 |0.690564701717118 |0.460415212267967 |
|WARP-Q     |0.339686211572685 |0.067600137543649 |0.777119464646524 |0.475793617709890 |
|GVPMOS     |0.412937133868407 |0.006851162794410 |0.783946603687895 |0.412912222208318 |

After:

|Score type |MSE               |Min score         |Max score         |Mean score        |
|-----------|------------------|------------------|------------------|------------------|
|Zimtohrli  |0.101180623602238 |0.563709113431054 |0.773872237318597 |0.691051313421366 |
|ViSQOL     |0.115330916105424 |0.520833375452983 |0.801480831107469 |0.675101633981268 |
|2f         |0.129541391104905 |0.484687555319526 |0.797475783883375 |0.661870345773127 |
|PESQ       |0.147425552045669 |0.342342966279351 |0.841271127756762 |0.647128996775172 |
|CDPAM      |0.153471222942756 |0.441558428344727 |0.728779141125759 |0.620699318941738 |
|PARLAQ     |0.185057687192323 |0.445261140223642 |0.784370761057963 |0.587162756572532 |
|AQUA       |0.223207996944378 |0.331645933512413 |0.739286336419790 |0.547804951221731 |
|PEAQB      |0.225217321572038 |0.278744167467764 |0.851011116004117 |0.553935720513487 |
|DPAM       |0.315810440183130 |0.186717781679534 |0.690564701717118 |0.460415212267967 |
|WARP-Q     |0.339686211572685 |0.067600137543649 |0.777119464646524 |0.475793617709890 |
|GVPMOS     |0.412937133868407 |0.006851162794410 |0.783946603687895 |0.412912222208318 |

real	8m5.190s
user	337m41.200s
sys	163m45.641s
@google-cla Google CLA
Copy link

google-cla bot commented Jun 14, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@jyrkialakuijala jyrkialakuijala merged commit 67449be into google:tabuli Jun 14, 2024
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jyrkialakuijala @zond