This repository contains the files used for the manual validation of code clone pairs generated by GPTCloneBench, a benchmark for semantic clones and cross-language clones, utilizing the GPT-3 model and SemanticCloneBench. The validation process involved the use of Jeffrey Svajlenko's Java GUI for assessing the accuracy and precision of the clone pairs. The manual validation process involved multiple validators who cross-checked and ensured agreement on the classification of each clone pair. Any discrepancies, ambiguities, or disputes in the classification were resolved through consensus.
Code clones, which refer to identical or nearly identical code snippets in software systems, have long been studied in software engineering. Cloning is a common practice among programmers during the development process, but it can harm the quality of software in the long run. Various clone detection tools have been developed to identify and categorize different types of clones, including syntactic and semantic clones.
The manual validation of code clone pairs is crucial to ensure the benchmark's dataset's quality and reliability. While the GPTCloneBench dataset was automatically generated using the GPT-3 model, manual validation adds an extra layer of accuracy and precision to the clones present in the benchmark.
The validation process involved the following steps:
-
Java GUI: Jeffrey Svajlenko's Java GUI was utilized to visually inspect the generated code clone pairs. The GUI allowed the validator to view each clone pair side by side, compare their similarities and differences, and mark each pair as "True Positive" or "False Positive."
-
Semantic Validation: For semantic clones, the validator assessed whether the generated clones indeed represented true semantic similarities. This involved understanding the intended functionality of the code snippets and ensuring they perform the same or similar tasks.
-
Cross-Language Validation: Cross-language clones were verified to ensure they represent actual similarities between code snippets in different programming languages.
-
False Positive Marking: For clone pairs that were not true semantic clones, they were marked as "False Positive" using the Java GUI.
-
Cross-Check and Consensus: Validators cross-checked the classification of each clone pair and engaged in discussions to resolve any discrepancies, ambiguities, or disputes in the classification. Consensus was reached through collective agreement.
The repository contains Java files for running a GUI tool used for manual validation of code clone pairs generated by GPTCloneBench. The code clone pairs are grouped into batches, each in its own folder, with corresponding input and output files for the validation results.