Update LibAFL, remove old version

[domenukk]

No description provided.

[tokatoka]

also you can add in fuzzer.py/build()

    cxxflags = ['--libafl', '--std=c++14']

and append export it to fix the builds

[tokatoka]

it's llvm.sh that doesn't work :<
(this was working last week!)

24.61 + LLVM_VERSION_STRING=-16
24.61 + [[ -n focal ]]
24.61 + REPO_NAME='deb http://apt.llvm.org/focal/  llvm-toolchain-focal-16 main'
24.61 + wget -q --method=HEAD http://apt.llvm.org/focal
25.13 + [[ -n '' ]]
25.13 + echo 'Distribution '\''ubuntu'\'' in version '\''20.04.5 LTS (Focal Fossa)'\'' is not supported by this script.'
25.13 + exit 2
25.13 Distribution 'ubuntu' in version '20.04.5 LTS (Focal Fossa)' is not supported by this script.

[tokatoka]

@domenukk
can you apply this patch

diff --git a/fuzzers/libafl/builder.Dockerfile b/fuzzers/libafl/builder.Dockerfile
index d9631e5..d49990d 100644
--- a/fuzzers/libafl/builder.Dockerfile
+++ b/fuzzers/libafl/builder.Dockerfile
@@ -24,8 +24,9 @@ RUN apt-get update && \
     apt-get install -y wget libstdc++5 libtool-bin automake flex bison \
         libglib2.0-dev libpixman-1-dev python3-setuptools unzip \
         apt-utils apt-transport-https ca-certificates joe curl && \
-    wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && ./llvm.sh 16 \
-    wget https://gist.githubusercontent.com/tokatoka/26f4ba95991c6e33139999976332aa8e/raw/20996c7348bcbab115b79c315cee964f1a19e5f3/createAliases.sh && chmod u+x createAliases.sh && ./createAliases.sh
+    wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && ./llvm.sh 16
+
+RUN wget https://gist.githubusercontent.com/tokatoka/26f4ba95991c6e33139999976332aa8e/raw/698ac2087d58ce5c7a6ad59adce58dbfdc32bd46/createAliases.sh && chmod u+x ./createAliases.sh && ./createAliases.sh
 
 # Uninstall old Rust & Install the latest one.
 RUN if which rustup; then rustup self uninstall -y; fi && \
@@ -46,4 +47,4 @@ RUN cd /libafl && \
     unset CFLAGS CXXFLAGS && \
     export LIBAFL_EDGES_MAP_SIZE=2621440 && \
     cd ./fuzzers/fuzzbench && \
-    PATH="/root/.cargo/bin/:$PATH" cargo build --profile release-fuzzbench
\ No newline at end of file
+    PATH="/root/.cargo/bin/:$PATH" cargo build --profile release-fuzzbench

[tokatoka]

nono

    wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && ./llvm.sh 16
RUN wget https://gist.githubusercontent.com/tokatoka/26f4ba95991c6e33139999976332aa8e/raw/698ac2087d58ce5c7a6ad59adce58dbfdc32bd46/createAliases.sh && chmod u+x ./createAliases.sh && ./createAliases.sh

here

[domenukk]

Don't get why it would make a difference, but will try

[domenukk]

Still doesn't build everything..

[tokatoka]

/usr/bin/ld: cannot find -lFuzzingEngine

so perhaps you shouldn't delete FUZZER_LIB?

[domenukk]

WARNING:root:Removed seed file /out/seeds/Mplus1p-Regular.ttf as it exceeds 1 Mb limit.
['/out/hb-shape-fuzzer', '-o', '/out/corpus', '-i', '/out/seeds']
hb-shape-fuzzer: ../../src/harfbuzz/test/fuzzing/main.cc:11: int main(int, char **): Assertion `blob' failed.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/src/fuzzers/libafl/fuzzer.py", line 72, in fuzz
    subprocess.check_call(command, cwd=os.environ['OUT'], env=fuzzer_env)
  File "/usr/local/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/out/hb-shape-fuzzer', '-o', '/out/corpus', '-i', '/out/seeds']' died with <Signals.SIGABRT: 6>.
ERROR:root:Executed command: "nice -n 5 python3 -u -c from fuzzers.libafl import fuzzer; fuzzer.fuzz("/out/seeds", "/out/corpus", "/out/hb-shape-fuzzer")" returned: 1.
ERROR:root:Fuzz process returned nonzero. Extras: {'traceback': 'Traceback (most recent call last):\n  File "/src/experiment/runner.py", line 200, in run_fuzzer\n    new_process.execute(command,\n  File "/src/common/new_process.py", line 125, in execute\n    raise subprocess.CalledProcessError(retcode, command)\nsubprocess.CalledProcessError: Command \'[\'nice\', \'-n\', \'5\', \'python3\', \'-u\', \'-c\', \'from fuzzers.libafl import fuzzer; fuzzer.fuzz("/out/seeds", "/out/corpus", "/out/hb-shape-fuzzer")\']\' returned non-zero exit status 1.\n'}
INFO:root:Doing final sync.
make: *** [docker/generated.mk:143277: test-run-libafl-harfbuzz_hb-shape-fuzzer] Error 1

and for libjpeg

#22 6.184 -- FUZZ_BINDIR = /out
#22 6.184 CMake Error at fuzz/CMakeLists.txt:18 (message):
#22 6.184   FUZZ_LIBRARY must be specified.
#22 6.184 
#22 6.184 
#22 6.185 -- Configuring incomplete, errors occurred!

[domenukk]

#22 0.271 AFL++ build: 
#22 0.271 ['tracepc', 'cmplog', 'dict2file']
#22 0.889 Building benchmark openh264_decoder_fuzzer with fuzzer aflrustrust
#22 0.891 + svn export https://github.com/mozillasecurity/fuzzdata.git/trunk/samples/h264 corpus/
#22 0.988 svn: E170013: Unable to connect to a repository at URL 'https://github.com/mozillasecurity/fuzzdata.git/trunk/samples/h264'
#22 0.988 svn: E215004: No more credentials or we tried too many times.
#22 0.988 Authentication failed
#22 1.202 Traceback (most recent call last):
#22 1.202   File "<string>", line 1, in <module>
#22 1.202   File "/src/fuzzers/aflrustrust/fuzzer.py", line 28, in build
#22 1.202     aflplusplus_fuzzer.build('tracepc', 'cmplog', 'dict2file')
#22 1.202   File "/src/fuzzers/aflplusplus/fuzzer.py", line 178, in build
#22 1.202     utils.build_benchmark()
#22 1.202   File "/src/fuzzers/utils.py", line 81, in build_benchmark
#22 1.202     subprocess.check_call(['/bin/bash', '-ex', build_script], env=env)
#22 1.203   File "/usr/local/lib/python3.10/subprocess.py", line 369, in check_call
#22 1.203     raise CalledProcessError(retcode, cmd)
#22 1.203 subprocess.CalledProcessError: Command '['/bin/bash', '-ex', '/src/build.sh']' returned non-zero exit status 1.
#22 ERROR: process "/bin/sh -c echo \"Run fuzzer_build to build the target\" && if [ -z \"$debug_builder\" ] ; then fuzzer_build; fi" did not complete successfully: exit code: 1
------
 > [stage-1 15/15] RUN echo "Run fuzzer_build to build the target" && if [ -z "$debug_builder" ] ; then fuzzer_build; fi:
1.202   File "<string>", line 1, in <module>
1.202   File "/src/fuzzers/aflrustrust/fuzzer.py", line 28, in build

@jonathanmetzman is that a bug on github CI end or my fault?

[tokatoka]

@jonathanmetzman is that a bug on github CI end or my fault?

i think it's just a random network error

[addisoncrump]

Nope, fuzzdata is gone.

[tokatoka]

the error seems consistent

[addisoncrump]

Yeah, Mozilla has been warning people for months to drop dependency on it. There may be an alternative available, but that's probably better in a separate PR with rebase 🙂

[jonathanmetzman]

OK so this is good to merge right?

[domenukk]

Yes :)