roadmap for testing compiler versions (and results over tinyconfig with gcc 6.3 and 8)

[FAMILIAR-project]

some ad-hoc experiments and manual compilation for gcc 8 (instead of gcc 6.3) using a custom Docker image (the old tuxml/tuxml3:dev)

root@ad1cc9a80f87:/linux-4.17# ls -alrth /TuxML/linux-4.13.3/vmlinux
-rwxr-xr-x. 1 root root 3.1M Jan  7 17:01 /TuxML/linux-4.13.3/vmlinux
root@ad1cc9a80f87:/linux-4.17# ls -alrth /linux-4.15/vmlinux
ls: cannot access '/linux-4.15/vmlinux': No such file or directory
root@ad1cc9a80f87:/linux-4.17# ls -alrth /linux-4.17/vmlinux
-rwxr-xr-x. 1 root root 12M Jan  7 18:09 /linux-4.17/vmlinux
root@ad1cc9a80f87:/linux-4.17# ls -alrth /linux-5.0/vmlinux
-rwxr-xr-x. 1 root root 12M Jan  7 17:16 /linux-5.0/vmlinux
root@ad1cc9a80f87:/linux-4.17# ls -alrth /linux-5.4/vmlinux
-rwxr-xr-x. 1 root root 12M Jan  7 16:57 /linux-5.4/vmlinux
root@ad1cc9a80f87:/linux-4.17# ls -alrth /linux-5.8/vmlinux
-rwxr-xr-x. 1 root root 12M Jan  7 16:05 /linux-5.8/vmlinux
root@ad1cc9a80f87:/linux-4.17# gcc --version
gcc (Debian 8.3.0-6) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

there is a "surprising" and noticeable decrease for 4.13.3 (3Mb for gcc 8 instead of 7Mb for gcc 6)...
it actually motivated this experiment

but then for other kernel versions, gcc 8 is very similar to gcc 6.3 (12Mb all along)

see data that have been gathered for gcc 6.3
https://github.com/TuxML/evolution-showcase
https://github.com/TuxML/evolution-showcase/blob/master/evo-kernel-linux.png

these results suggest two hypothesis that are worth investigating in the future:

• it might be the case that gcc version does not influence much binary size
• why then this strong decrease specifically for 4.13.3?

It calls to implement the following:

• systematize the experiments with TuxML machinery (kernel_generator, etc.) instead of ad-hoc experiments
• experiments with gcc 10 (not only gcc 6 and 8 as did above) and several kernel versions
• experiments with defconfig and gcc 6, 9, and 10 on different kernel versions
• experiments with random configurations and gcc 6, 9, and 10 on different kernel versions (note: given a random config, we should give the same one for gcc versions/kernel versions for comparing over the same config)
• analyze the data and conclude about hypothesis