Fault localization tool that implements and analyzes contextualization strategies.
Before running Road2Fault, you must run InSS to instrument and collecting spectra data from the subject program.
Get the InSS project, available at https://github.com/higoramario/mis-mcp.git
This version of InSS contains the MCP and MCT strategies. README file has the instructions to compile the project and for its initial settings.
- Run the Ant script (build.xml) to build the project.
ant
- Create an enviroment variable for the 'build' folder.
export ROAD2FAULT_HOME=/home/user/apps/road2fault/build
- Give execution permissions for the files in the ${ROAD2FAULT_HOME}/bin folder.
sudo chmod +x ${ROAD2FAULT_HOME}/bin/*
- set the InSS root folder.
export INSS_HOME=/home/user/apps/mis-mcp/build/mis
- Instrument the faulty program.
${INSS_HOME}/bin/instrumenter --instrument ${PROGRAM_DIR}/target/classes/ --dest ${PROGRAM_DIR}/instrument/
--instrument: program's class folder
--dest: destination folder
- Generate the program's data-flow graph.
${INSS_HOME}/bin/defuse --file ${PROGRAM_DIR}/target/classes/ --dest ${PROGRAM_DIR}/gxl/
--file: program's class folder
--dest: destination folder
- Set the CLASSPATH, including the instrumented classes, test classes, junit library, inss libraries (in ${INSS_HOME}/libs folder), and other libraries and folders from which the faulty program depends.
LIBS=
for i in `ls $INSS_HOME/libs/*.jar`
do
LIBS=${LIBS}:${i}
done
export CLASSPATH=${PROGRAM_DIR}:${PROGRAM_DIR}/instrument/:${PROGRAM_DIR}/target/test-classes/:${PROGRAM_DIR}/lib/junit.jar${LIBS}
- Collect the spectra for each test class running the following command:
java -Dgxl.dir=${PROGRAM_DIR}/gxl \
-Drequirement.determination=br.usp.each.opal.requirement.NodeDetermination \
-Dinstrumentation.strategy=br.usp.each.inss.instrumentation.AllNodes \
-Dsimulator.strategy=br.usp.each.inss.executor.SimulatorMethodCallPair \
br.usp.each.inss.InSSTestRunner package.TestClass
-Dgxl.dir: directory that contains the data-flow graphs (gxl files) previously created
-Drequirement.determination: choose one of the following options
- NodeDetermination for node, mcp, or mct spectra
- DuaDetermination for dua spectra
- EdgeDetermination for branch spectra
-Dinstrumentation.strategy: choose one of the following options
- AllNodes for node, mcp, or mct spectra
- CorrectedDemandDriven for dua spectra
- AllEdges for branch spectra
-Dsimulator.strategy: choose one of the following options
- SimpleSimulator for node, dua, or branch spectra
- SimulatorMethodCallPair for mcp spectra
- SimulatorMethodCallTriple for mct spectra
Insert the test class' name after br.usp.each.inss.InSSTestRunner.
The output is a file with the corresponding spectra extension (e.g., package.TestClass.mcp).
- Save the spectra files in a specific folder.
- Run the following command to execute Road2Fault.
${ROAD2FAULT_HOME}/bin/road2fault_mcp -r mcpspectra -rt mcp -hr ochiai -c ${PROGRAM_DIR}/target/classes -lt requirement
- Set the desired road2fault program to collect spectra data: ..- road2fault_mcp for MCP spectra and the MCP roadmap ..- road2fault_unit for node, branch or dua spectra ..- road2fault_mct for MCT spectra ..- road2fault_rmcp for the unit list with method position
- -r: the folder that contains the spectra files
- -rt: type of requirement: mcp, mct, node, branch, or dua
- -hr: type of heuristic used to calculate the suspiciousness: ochiai, tarantula, drt, jaccard, zoltar, op, minus, kulczynski2, mccon, wong3
- -c: path of the original classes of the subject program
- -lt: type of list: requirement or csv for mcp or mct spectra; requirement, class, package or csv for node, branch or dua spectra
The output is an XML suspiciousness list organized by requirement, classes, or packages. The csv options generates the suspiciousness list in the csv format. To run Road2Fault, it is necessary to remove files which the extension finish with '*-out'. These files contain the junit's output information and must not be processed by Road2Fault.
- To generate the new unit list with the method position attribute, run:
$ROAD2FAULT_HOME/bin/road2fault_rmcp -uf list_dci_NODE_TARANTULA_BY_PACKAGE.xml-debug -rf list_dci_RMCP_TARANTULA.xml-debug
- -uf: previous unit list created by the 'road2fault_unit' program
- -rf: roadmap
The position attribute was included for using in CodeForest.
Road2Fault extracts the fault localization results of debugging filtering strategies from the suspiciousness lists (.xml-debug extension). The tool generates the results in csv files and also the log files to check if the strategies are working well. It is also possible to plot bar charts with the results.
Currently, it is possible to extract results only for the faults used in the dataset we used in our experiment. New faults can be added following the structure of BatchExecutor, since the faulty sites must be informed to calculate the results.
InspectionStrategyLevelScorePerMethod: Get level scores based on levels of each method. This strategy gets each score from a method, at most 15 score levels.
InspectionStrategyDeltaBudget: it uses the effort budget as a delta. If the budget is 5, this strategy checks in the block list (BL) the score which contains 5 blocks, and this score is used to inspect the blocks inside the methods indicated by the roadmap. If the 1st score (highest) contains more blocks than the budget, such score is returned.
To run the extractor, the report files must be placed in the following structure:
/home/user/experiments/{program-name}/{version_fault-name}
- /home/user/experiments: your local folder that contains the report files of all programs.
- program-name: one folder for each program named with the program's name in lower case
- version_fault-name: one folder containing the .xml-debug files for each fault.
The folder must be named with the version number preceded by letter v, and followed by '_' and the fault's description.
/home/user/experiments/ant/v1_CLJ_HD_1
The tool runs results for two ranking heuristics: Ochiai and Tarantula. Three report files for each heuristic are needed to execute extract the results:
- list_dci_MCP_OCHIAI.xml-debug
- list_dci_NODE_OCHIAI_BY_PACKAGE.xml-debug
- list_dci_NODE_OCHIAI_BY_REQUIREMENT.xml-debug
In case of Tarantula, the files has the description TARANTULA instead of OCHIAI. The report files of all 62 faults from both heuristics must be inside the folder to execute the extractor.
- Run the following commands to extract the results for the LS and FB strategies. There are different programs to extract the results for single- and multiple-fault versions. You must pass the path to the root folder that contains the report files (e.g., /home/user/experiments/).
${ROAD2FAULT_HOME}/bin/r2f_ls "/home/user/experiments/"
${ROAD2FAULT_HOME}/bin/r2f_ls_mult "/home/user/experiments/"
${ROAD2FAULT_HOME}/bin/r2f_fb "/home/user/experiments/"
${ROAD2FAULT_HOME}/bin/r2f_fb_mult "/home/user/experiments/"
-
A csv file (ch-icd-output-fb.csv or ch-icd-output-ls.csv) with the number of blocks and methods inspected until reach the fault by technique (ICD-LS, CH-LS, ICD-FB, CH-FB, and BL), by effort budget (1 - 100) and by different values for LS (1 - 15) and FB (5 - 100). The file contains the values for each program and fault, and summarized values by program and for all programs using Ochiai and Tarantula. The csv files also contain the percentage of examined code to reach the faults. We considered the worst case scenario to deal with ties, i.e., all blocks with score equal to the faulty one are included to determine the position of the faulty one block.
-
A log file containing all blocks and methods visited for all strategies per program, fault and heuristic.
-
Bar plot charts with the results by program and for all programs, comparing CH-FB, ICD-FB, and BL, and also for CH-LS, ICD-LS, and BL.
InspectionStrategy: original strategy used in the 1st version of ICD. Methods with the same name are included at once in the roadmap, no matter their signatures or classes. The score precision is fully.
We use 62 bugs in our experiments from the following projects. For the multiple-fault versions we combined these bugs, creating 62 versions, 25 with 2 faults and 37 with 4 faults.
Project | # Single faults | # Two faults | # Four faults | # Two and four faults |
---|---|---|---|---|
Ant | 18 | 16 | 2 | 18 |
Commons-Math | 20 | 3 | 21 | 24 |
HSQLDB | 4 | 3 | 0 | 3 |
JTopas | 4 | 1 | 0 | 1 |
PMD | 2 | 1 | 0 | 1 |
XML-security | 13 | 1 | 14 | 15 |
XStream | 1 | 0 | 0 | 0 |
Ant, JTopas, and XML-security were obtained from SIR (http://sir.unl.edu). Commons-Math, HSQLDB, PMD, and XStream were obtained from their repositories. We created a SIR-like structure to ease the insertion of faults for experiments. These latter ones are available here, along with instructions to insert the faults.
The scripts we used in the experiments for single-fault versions are available here. For multiple-fault versions, the scripts are available here.
The main results of our experiments for single-fault versions are available here. For multiple-fault versions are available here. The csv files and logs generated by the calculator are also available for single and multiple faults.
@article{
title={Contextualizing Spectrum-based Fault Localization},
author={Higor A. de Souza and Danilo Mutti and Marcos L. Chaim and Fabio Kon},
journal={Information and Software Technology},
volume={94},
number={},
pages={245--261},
year={2017}
}