student base

[jorgtied]

What would be the slightly bigger student base model configuration?

[graemenail]

This was our 6-2 student base system from En-De WMT22.

# Model options
dec-cell-base-depth: 2
dec-cell-high-depth: 1
dec-cell: ssru
dec-depth: 2
dim-emb: 512
enc-cell-depth: 1
enc-cell: gru
enc-depth: 6
enc-type: bidirectional
tied-embeddings-all: true
transformer-decoder-autoreg: rnn
transformer-dim-ffn: 2048
transformer-ffn-activation: relu
transformer-ffn-depth: 2
transformer-guided-alignment-layer: last
transformer-heads: 8
transformer-postprocess: dan
transformer-preprocess: ""
transformer-tied-layers: []
transformer-train-position-embeddings: false
type: transformer

Which is essentially: https://github.com/browsermt/students/blob/master/train-student/models/student.base/student.base.yml

For comparison our 6-2 student tiny, which looks to be the same as the tiny11 you posted in #3

6-2 student tiny

# Model options
dec-cell-base-depth: 2
dec-cell-high-depth: 1
dec-cell: ssru
dec-depth: 2
dim-emb: 256
enc-cell-depth: 1
enc-cell: gru
enc-depth: 6
enc-type: bidirectional
tied-embeddings-all: true
transformer-decoder-autoreg: rnn
transformer-dim-ffn: 1536
transformer-ffn-activation: relu
transformer-ffn-depth: 2
transformer-guided-alignment-layer: last
transformer-heads: 8
transformer-postprocess: dan
transformer-preprocess: ""
transformer-tied-layers: []
transformer-train-position-embeddings: false
type: transformer

And deciding between tied-embeddings-all: true or tied-embeddings: true as required