Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload missing scan results used for Table 4 in the paper #42

Merged
merged 2 commits into from
Jul 3, 2024

Conversation

wtwhite
Copy link
Collaborator

@wtwhite wtwhite commented Jul 2, 2024

Resolves #41.

TODO before committing: These files were reported as untracked by git status. I'm 99% sure they match what was used to generate Table 4, but rerun that generation using ReportResultsByCVE.java to double-check first. Verified.

@wtwhite wtwhite self-assigned this Jul 2, 2024
@wtwhite
Copy link
Collaborator Author

wtwhite commented Jul 3, 2024

It seemed that ReportResultsByCVE might generate Table 4, but it generates a different LaTeX table (and some other files).

run-summary.sh already generates the "right" data as output.txt in a different format (CSV). I'll write a quick script to convert output.txt into Table 4.

…tex) from output.csv (which is produced by run-summary.sh)
@wtwhite
Copy link
Collaborator Author

wtwhite commented Jul 3, 2024

The new Perl script to generate cves-detected-by-sca-tools-in-tpovs.tex from output.csv produces an equivalent LaTeX table up to whitespace changes -- except that it fixes the missing sum in the rightmost column:

wtwhite@wtwhite-vuw-vm:~/code/xshady$ ./create-table-4.pl < output.csv > new_table_4.tex
wtwhite@wtwhite-vuw-vm:~/code/xshady$ diff -w ~/code/shading-study-paper/tables/cves-detected-by-sca-tools-in-tpovs.tex new_table_4.tex 
34,35c34,36
< 		CVE-2022-45688   & $\checkmark $    & $\checkmark $    & $\checkmark $   & $\times$      & $\checkmark$   & $\times$   \\  \hline
< 		sum              & 27    & 26    & 25   & 16     & 29  &    \\  \hline
---
> 		CVE-2022-45688  & $\checkmark $ & $\checkmark $ & $\checkmark $ & $\times $ & $\checkmark $ & $\times $ \\
> 		\hline
> 		sum & 27 & 26 & 25 & 16 & 29 & 13 \\ \hline

(Note that \hline being moved to the next line is an unimportant syntactic difference that simplifies the script.)

I earlier confirmed that the uncommitted output.csv in that directory dated October 5, 2023 is identical to what we get if we rerun run-summary.sh (files renamed around since that script always writes to output.csv):

wtwhite@wtwhite-vuw-vm:~/code/xshady$ ls -ltra output*.csv
-rw-rw-r-- 1 wtwhite wtwhite 1802 Oct  5  2023 output.csv
-rw-rw-r-- 1 wtwhite wtwhite 1802 Jul  2 18:52 output_AFTER_RERUNNING_ON_2024-07-02_IS_IDENTICAL.csv
wtwhite@wtwhite-vuw-vm:~/code/xshady$ md5sum output*.csv
d70f33f0ff8491885145dfe6050326ea  output_AFTER_RERUNNING_ON_2024-07-02_IS_IDENTICAL.csv
d70f33f0ff8491885145dfe6050326ea  output.csv

Conclusion: I'm now confident that the files added in bfabdbc are what was used to generate the current Table 4. Will merge.

@wtwhite wtwhite marked this pull request as ready for review July 3, 2024 02:57
@wtwhite wtwhite merged commit 27ead1b into main Jul 3, 2024
2 checks passed
@wtwhite wtwhite deleted the upload-missing-scan-results branch July 3, 2024 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

for some CVEs /scan-results are missing
1 participant