Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ClassFormatError and NoSuchMethodError when using GZoltar to execute tests for JxPath #67

Open
Instein98 opened this issue Apr 21, 2023 · 3 comments
Open

ClassFormatError and NoSuchMethodError when using GZoltar to execute tests for JxPath #67

Instein98 opened this issue Apr 21, 2023 · 3 comments

Comments

@Instein98
Copy link

Instein98 commented Apr 21, 2023 •
edited
Loading

Context

When using GZoltar to execute tests for JxPath-1, most of the tests failed with unexpected errors, i.e., 201 ClassFormatError and 83 NoSuchMethodError.

Steps to Reproduce

  1. Create a script localize.sh with the following content. Please change the value of "work_dir" and "D4J_HOME" correspondingly. 
    #!/bin/bash

work_dir="/tmp/test"
export D4J_HOME="/Users/xxx/defects4j/"
rm -rf "$work_dir"; mkdir "$work_dir"

export _JAVA_OPTIONS="-Xmx6144M -XX:MaxHeapSize=4096M"
export MAVEN_OPTS="-Xmx1024M"
export ANT_OPTS="-Xmx6144M -XX:MaxHeapSize=4096M"

#
# Get GZoltar
#

cd "$work_dir"
if [ ! -d "$work_dir/gzoltar" ]; then
    git clone https://github.com/GZoltar/gzoltar.git
    cd "$work_dir/gzoltar"
    mvn clean package
fi

export GZOLTAR_AGENT_JAR="$work_dir/gzoltar/com.gzoltar.agent.rt/target/com.gzoltar.agent.rt-1.7.4-SNAPSHOT-all.jar"
export GZOLTAR_CLI_JAR="$work_dir/gzoltar/com.gzoltar.cli/target/com.gzoltar.cli-1.7.4-SNAPSHOT-jar-with-dependencies.jar"

# #
# # Get D4J
# #

# cd "$work_dir"
# git clone https://github.com/rjust/defects4j.git
# cd "$work_dir/defects4j"
# ./init.sh


export TZ='America/Los_Angeles' # some D4J's requires this specific TimeZone
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8

#
# Checkout JxPath-1b, compile it, and get its metadata
#

PID="JxPath"
BID="1"

# Checkout
cd "$work_dir"
rm -rf "$PID-${BID}b"; "$D4J_HOME/framework/bin/defects4j" checkout -p "$PID" -v "${BID}b" -w "$PID-${BID}b"

# Compile
cd "$work_dir/$PID-${BID}b"
"$D4J_HOME/framework/bin/defects4j" compile

# Collect metadata
cd "$work_dir/$PID-${BID}b"
test_classpath=$($D4J_HOME/framework/bin/defects4j export -p cp.test)
src_classes_dir=$($D4J_HOME/framework/bin/defects4j export -p dir.bin.classes)
src_classes_dir="$work_dir/$PID-${BID}b/$src_classes_dir"
test_classes_dir=$($D4J_HOME/framework/bin/defects4j export -p dir.bin.tests)
test_classes_dir="$work_dir/$PID-${BID}b/$test_classes_dir"
echo "$PID-${BID}b's classpath: $test_classpath" >&2
echo "$PID-${BID}b's bin dir: $src_classes_dir" >&2
echo "$PID-${BID}b's test bin dir: $test_classes_dir" >&2

#
# Collect unit tests to run GZoltar with
#

cd "$work_dir/$PID-${BID}b"
unit_tests_file="$work_dir/$PID-${BID}b/unit_tests.txt"
relevant_tests="*"  # Note, you might want to consider the set of relevant tests provided by D4J, i.e., $D4J_HOME/framework/projects/$PID/relevant_tests/$BID

java -cp "$D4J_HOME/framework/projects/lib/junit-4.11.jar:$test_classpath:$test_classes_dir:$GZOLTAR_CLI_JAR" \
  com.gzoltar.cli.Main listTestMethods \
    "$test_classes_dir" \
    --outputFile "$unit_tests_file" \
    --includes "$relevant_tests"
head "$unit_tests_file"

#
# Collect classes to perform fault localization on
# Note: the `sed` commands below might not work on BSD-based distributions such as MacOS.
#

cd "$work_dir/$PID-${BID}b"

loaded_classes_file="$D4J_HOME/framework/projects/$PID/loaded_classes/$BID.src"
if [[ "$OSTYPE" == "darwin"* ]]; then
    normal_classes=$(cat "$loaded_classes_file" | gsed 's/$/:/' | gsed ':a;N;$!ba;s/\n//g')
    inner_classes=$(cat "$loaded_classes_file" | gsed 's/$/$*:/' | gsed ':a;N;$!ba;s/\n//g')
else
    normal_classes=$(cat "$loaded_classes_file" | sed 's/$/:/' | sed ':a;N;$!ba;s/\n//g')
    inner_classes=$(cat "$loaded_classes_file" | sed 's/$/$*:/' | sed ':a;N;$!ba;s/\n//g')
fi
classes_to_debug="$normal_classes$inner_classes"
echo "Likely faulty classes: $classes_to_debug" >&2

#
# Run GZoltar
#

cd "$work_dir/$PID-${BID}b"

ser_file="$work_dir/$PID-${BID}b/gzoltar.ser"
java -XX:MaxPermSize=4096M -javaagent:$GZOLTAR_AGENT_JAR=destfile=$ser_file,buildlocation=$src_classes_dir,includes=$classes_to_debug,excludes="",inclnolocationclasses=false,output="FILE" \
  -cp "$D4J_HOME/framework/projects/lib/junit-4.11.jar:$src_classes_dir:$test_classpath:$test_classes_dir:$GZOLTAR_CLI_JAR" \
  com.gzoltar.cli.Main runTestMethods \
    --testMethods "$unit_tests_file" \
    --collectCoverage

#
# Generate fault localization report
#

cd "$work_dir/$PID-${BID}b"

java -cp "$D4J_HOME/framework/projects/lib/junit-4.11.jar:$src_classes_dir:$test_classpath:$GZOLTAR_CLI_JAR" \
    com.gzoltar.cli.Main faultLocalizationReport \
      --buildLocation "$src_classes_dir" \
      --granularity "line" \
      --inclPublicMethods \
      --inclStaticConstructors \
      --inclDeprecatedMethods \
      --dataFile "$ser_file" \
      --outputDirectory "$work_dir/$PID-${BID}b" \
      --family "sfl" \
      --formula "ochiai" \
      --metric "entropy" \
      --formatter "txt"
  2. Execute the script and redirect the output to a log file: 
    bash localize.sh |& tee localize.log
  3. In the localize.log file, we can observe that there are 83 NoSuchMethodError and 201 ClassFormatError. However, JxPath-1b should only fail 2 tests with NullPointerException.

Expected behaviour

The test execution should only fail 2 tests with NullPointerException.

Environment (please complete the following information, if relevant):

I have reproduced the problem on both of my MacOS and Linux machines.

  1. MacOS 12.0.1 (21A559)
  2. Ubuntu 18.04.6 LTS
@jose
Copy link
Member

jose commented Apr 24, 2023

Hi @Instein98,

Thanks for taking the time to report this issue, I will try to debug it on my end. But before I go ahead,

  1. Defects4J requires Java-8, what's the output of the following commands on your box?
javac -version
java -version
  1. After the checkout/compile step, could you please run
cd "$work_dir/$PID-${BID}b"
"$D4J_HOME/framework/bin/defects4j" test

and let me know the content of the failing_tests file? Do you get the same content as in here?

--
Best,
Jose

@Instein98
Copy link
Author

Thanks for your prompt reply!
On my macOS machine:

% java -version 
openjdk version "1.8.0_292"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_292-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.292-b10, mixed mode)
% javac -version
javac 1.8.0_292

On my Linux (Ubuntu) machine:

$ javac -version
javac 1.8.0_362
$ java -version
openjdk version "1.8.0_362"
OpenJDK Runtime Environment (build 1.8.0_362-8u362-ga-0ubuntu1~18.04.1-b09)
OpenJDK 64-Bit Server VM (build 25.362-b09, mixed mode)

The output of running defects4j test in the /tmp/test/JxPath-1b/:

Running ant (compile.tests)................................................ OK
Running ant (run.dev.tests)................................................ OK
Failing tests: 2
  - org.apache.commons.jxpath.ri.model.dom.DOMModelTest::testGetNode
  - org.apache.commons.jxpath.ri.model.jdom.JDOMModelTest::testGetNode

Content in failing_tests file:

--- org.apache.commons.jxpath.ri.model.dom.DOMModelTest::testGetNode
java.lang.NullPointerException
	at org.apache.commons.jxpath.JXPathTestCase.assertXPathNodeType(JXPathTestCase.java:224)
	at org.apache.commons.jxpath.ri.model.dom.DOMModelTest.testGetNode(DOMModelTest.java:68)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at junit.framework.TestCase.runTest(TestCase.java:176)
	at junit.framework.TestCase.runBare(TestCase.java:141)
	at junit.framework.TestResult$1.protect(TestResult.java:122)
	at junit.framework.TestResult.runProtected(TestResult.java:142)
	at junit.framework.TestResult.run(TestResult.java:125)
	at junit.framework.TestCase.run(TestCase.java:129)
	at junit.framework.TestSuite.runTest(TestSuite.java:255)
	at junit.framework.TestSuite.run(TestSuite.java:250)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:520)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeInVM(JUnitTask.java:1484)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:872)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeOrQueue(JUnitTask.java:1972)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute1(JUnitTask.java:824)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:2277)
	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
	at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
	at org.apache.tools.ant.Task.perform(Task.java:348)
	at org.apache.tools.ant.Target.execute(Target.java:392)
	at org.apache.tools.ant.Target.performTasks(Target.java:413)
	at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
	at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
	at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
	at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
	at org.apache.tools.ant.Main.runBuild(Main.java:811)
	at org.apache.tools.ant.Main.startAnt(Main.java:217)
	at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
	at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
--- org.apache.commons.jxpath.ri.model.jdom.JDOMModelTest::testGetNode
java.lang.NullPointerException
	at org.apache.commons.jxpath.JXPathTestCase.assertXPathNodeType(JXPathTestCase.java:224)
	at org.apache.commons.jxpath.ri.model.jdom.JDOMModelTest.testGetNode(JDOMModelTest.java:65)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at junit.framework.TestCase.runTest(TestCase.java:176)
	at junit.framework.TestCase.runBare(TestCase.java:141)
	at junit.framework.TestResult$1.protect(TestResult.java:122)
	at junit.framework.TestResult.runProtected(TestResult.java:142)
	at junit.framework.TestResult.run(TestResult.java:125)
	at junit.framework.TestCase.run(TestCase.java:129)
	at junit.framework.TestSuite.runTest(TestSuite.java:255)
	at junit.framework.TestSuite.run(TestSuite.java:250)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:520)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeInVM(JUnitTask.java:1484)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:872)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeOrQueue(JUnitTask.java:1972)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute1(JUnitTask.java:824)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:2277)
	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
	at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
	at org.apache.tools.ant.Task.perform(Task.java:348)
	at org.apache.tools.ant.Target.execute(Target.java:392)
	at org.apache.tools.ant.Target.performTasks(Target.java:413)
	at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
	at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
	at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
	at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
	at org.apache.tools.ant.Main.runBuild(Main.java:811)
	at org.apache.tools.ant.Main.startAnt(Main.java:217)
	at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
	at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)

Please let me know if you need more information.

@jose
Copy link
Member

jose commented Jul 3, 2023

Hi @Instein98,

This issue has been confirmed and reproduced on my end and, unfortunately, there are also other unexpected failing tests on Defects4J v2.0.0 (see #71 and #72. I'm not sure when I will have the time to debug and fix those.

--
Best,
Jose

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jose @Instein98