-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Description
Fine-tuned DeekSeek-Coder 6.7B's results look too good when compared to GPT-4 turbo
CodeBLUE is 0.149 < 0.262.
However, the compilation and execution results are the same or better than GPT-4
- compilation: 0.70 == 0.70
- execution: 0.65 > 0.52
When I try to reproduce the results using .cpp files under Fortran2Cpp/data/deepseek_coder_base_6.7B/readable_format_HPEC/cpp
None of them can be compiled. The cpp files contain natural language text and code blocks within ...
It looks like these files still need to be processed to extract cpp content.
Where are the extracted cpp files used to generate the results in Table 3? @bin123apple Can you upload them into the git repo's right folder?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels