Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

a = K.permute_dimensions(a, (0, 3, 2, 1)) #19

Open
monkeyshichi opened this issue Mar 9, 2020 · 2 comments
Open

a = K.permute_dimensions(a, (0, 3, 2, 1)) #19

monkeyshichi opened this issue Mar 9, 2020 · 2 comments

Comments

@monkeyshichi
Copy link

monkeyshichi commented Mar 9, 2020

ValueError: Dimension must be 5 but is 4 for 'attention_1/transpose_7' (op: 'Transpose') with input shapes: [?,8,?,8,?], [4].

请问这个怎么解决 run example
https://kexue.fm/archives/4765
代码测试
在Keras上对IMDB进行简单的测试

@chaoswjc
Copy link

chaoswjc commented Apr 16, 2021

个人经验:
a = K.batch_dot(qw, kw, [3, 3]) / self.key_size**0.5
改为a = tf.einsum('bjhd,bkhd->bhjk', qw, kw) / self.key_size**0.5

o = K.batch_dot(a, vw, [3, 2])
改为o = tf.einsum('bhjk,bkhd->bjhd', a, vw)
主要是keras版本变动导致广播方式不一样了

@Yumeka999
Copy link

个人经验: 将a = K.batch_dot(qw, kw, [3, 3]) / self.key_size**0.5 改为a = tf.einsum('bjhd,bkhd->bhjk', qw, kw) / self.key_size**0.5

o = K.batch_dot(a, vw, [3, 2]) 改为o = tf.einsum('bhjk,bkhd->bjhd', a, vw) 主要是keras版本变动导致广播方式不一样了

有用

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants