Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Activation unit #1

Merged
merged 210 commits into from
Sep 24, 2024
Merged

Activation unit #1

merged 210 commits into from
Sep 24, 2024

Conversation

gamzeisl
Copy link
Collaborator

This PR cleans and extends the merge request !10 on GitLab.

Changes:

  • Implementation of integer-only GELU and RELU activation unit for ITA.
  • Feedforward and linear layers are added to the FSM.
  • FFN requant params are reserved in the register file.

@gamzeisl gamzeisl self-assigned this Sep 11, 2024
Xeratec
Xeratec requested changes Sep 24, 2024
View reviewed changes
Copy link
Collaborator

@Xeratec Xeratec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of changes but look good to me. I only have a few minor comments about the golden model and the CI. Furthermore, I suggest squashing the changes to keep a clean commit history.

.gitlab-ci.yml Outdated Show resolved Hide resolved
PyITA/ITA.py Outdated Show resolved Hide resolved
PyITA/ITA.py Outdated Show resolved Hide resolved
PyITA/ITA.py Outdated Show resolved Hide resolved
PyITA/test_ITA.py Outdated Show resolved Hide resolved
gamzeisl and others added 27 commits September 24, 2024 18:22
@gamzeisl
build(requirements): add pytorch
c087fb8
@gamzeisl
build(requirements): add pytest
a17a7ee
@TimonFercho @gamzeisl
test(py-ita): add feedforward size parameter
840f9f7
@TimonFercho @gamzeisl
test(test-generator): add feedforward parameter and adjust output path
5b273e4
@TimonFercho @gamzeisl
feat(py-ita): extend print_properties with feedforward size
43671fe
@TimonFercho @gamzeisl
feat(py-ita). implement floating point i_poly
cb9838d
@TimonFercho @gamzeisl
feat(py-ita): add i_poly_wrapper function
ac44139
@TimonFercho @gamzeisl
build(requirements): add pytest-check
57a2520
@TimonFercho @gamzeisl
feat(py-ita): add symmetric (de-)quantization functions
6babe4b
@TimonFercho @gamzeisl
feat(py-ita): add functions for floating point poly and erf
963995b
@TimonFercho @gamzeisl
test(py-ita): add test for quantization, poly and erf functions
7718585
@TimonFercho @gamzeisl
fix(py-ita): fix q_c typo from definition
c7b7cde
@TimonFercho @gamzeisl
feat(py-ita): implement floating point i_gelu
9eacf08
@TimonFercho @gamzeisl
test(py-ita): add i_gelu tests
1a89be3
@TimonFercho @gamzeisl
fix(py-ita): fix ipoly types to avoid overflow
0362520
@TimonFercho @gamzeisl
refactor(py-ita): type i-gelu and i-erf
2e2fd90
@TimonFercho @gamzeisl
test(py-ita): clean up and pretty print
f9701e2
@TimonFercho @gamzeisl
refactor(py-ita): fix typo
1e460c6
@TimonFercho @gamzeisl
test(py-ita): make scaling factor output more readable
13d7e4e
@TimonFercho @gamzeisl
test(py-ita): verify i-gelu domain interval [b, -b]
b8298a8
@TimonFercho @gamzeisl
test(py-ita): add error plotting
bd3d588
@TimonFercho @gamzeisl
refactor(py-ita): rename variable
0555a27
@TimonFercho @gamzeisl
feat(py-ita): add missing imports
bd005fe
@TimonFercho @gamzeisl
feat(py-ita): add gelu golden model based on random preactivation values
e5588a7
@TimonFercho @gamzeisl
feat(ita-tb): adjustments for feedforward parameter
ab46f67
@TimonFercho @gamzeisl
feat(return-status): adjust parameters and check number of outputs
2cd0b5f
@TimonFercho @gamzeisl
build(makefile): add feedforward parameter
0c96449
TimonFercho and others added 23 commits September 24, 2024 18:22
@TimonFercho @gamzeisl
test(activation): add extra calc_en input
46120f2
@TimonFercho @gamzeisl
refactor(accel-pkg): removed unused fields in control_t
ea4d762
@TimonFercho @gamzeisl
test(ita-tb): don't reference unused control_t fields
866db19
@TimonFercho @gamzeisl
refactor(gelu): merge gelu one and gelu b constants
9c0421f
@TimonFercho @gamzeisl
test(activation): removed unused signal
22686fe
@gamzeisl
fix(py-ita): correctly compute L2 error
b264e91
@TimonFercho @gamzeisl
refactor(ita-package): remove unused gelu one constant
9573935
@TimonFercho @gamzeisl
chore(accel-pkg): increase fifo depth
fcda25d
@TimonFercho @gamzeisl
build(ci): fix mismatch of generated testvectors
beb4cfc
@gamzeisl
feat(return-status): add ff checks
90e4d13
@gamzeisl
change(hwpe-pkg): do not reuse regs for activations
774ab10
@gamzeisl
fix(ita_tb): lower input valid signal after handshake
f4c85d3
@gamzeisl
feat: add support for two layer ffn
00555ef
@gamzeisl
fix(PyITA): correct random vector gen for ffn
29dce4c
@gamzeisl
feat(PyITA): write hwpe files for ffn
6b18f84
@gamzeisl
feat(hwpe_tb): extend to test ffn
e1941c5
@gamzeisl
fix(PyITA): correct typecast in gelu
82f56d9
@gamzeisl
change(ci): add activation to hwpe sim
a7a6e79
@gamzeisl
feat(PyITA): add separate requantization params for ffn
18bee9d
@gamzeisl
feat(hw): add separate ffn requant params
ac33aa5
@gamzeisl
[PyITA] Move GELU functions
2ab2114
@gamzeisl
[ci] Add tests with relu
7a22227
@gamzeisl
[PyITA] Modify license headers
8ebfda6
@gamzeisl gamzeisl merged commit 2e6e8d8 into main Sep 24, 2024
6 checks passed
@gamzeisl gamzeisl deleted the activation-unit branch September 24, 2024 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Xeratec Xeratec Xeratec requested changes

Assignees

@gamzeisl gamzeisl

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gamzeisl @Xeratec @TimonFercho