Pinned Loading

1. llama3-Chinese llama3-Chinese Public

   pre-training llama3 using chinese

   Python 12

2. llama3-8x8b-MoE llama3-8x8b-MoE Public

   Copy the MLP of llama3 8 times as 8 experts , created a router with random initialization,add load balancing loss to construct an 8x8b MoE model based on llama3.

   Python 25 3

3. DIE-EC DIE-EC Public

   Python 4