ASAN: VPP silently quits after 41560

[vrpolakatcisco]

This is the other issue affecting usage of address sanitizer (asan) in CSIT when examining VPP crashes. Contrary to #3624 that can be circumvented by removing perfmon plugin (as it is not helpful when VPP crashes), this one limits usefullness of asan as 41560 [0] has been merged 2024-09-26. Bisect job quite reliably converges to 41560 being the first build where VPP vanishes without anything in logs [1]. I have not tried reverting yet, so there may be other causes preventing usage on newer VPP builds.

[0] 41560: vlib: process node scheduler rework | https://gerrit.fd.io/r/c/vpp/+/41560
[1] https://logs.fd.io/vex-yul-rot-jenkins-1/vpp-csit-bisect-master-ubuntu2404-x86_64-2n-icx/35/csit_middle/10/log.html.gz#s1-s1-s1-s1-s1-t1-k2-k5-k1-k1-k1-k1

[vrpolakatcisco]

VPP crashes

That is not confirmed. When I tried to start VPP in nodaemon mode, the CSIT code returned [2] code -1, which is a paramiko feature that means the SSH channel went down before a true error code could be determined.

Also, 3n-snr testbed got broken [3] during investigation of ASAN, so I will stop experimenting with it on production servers for now. (Even though I doubt the breakage has been caused by ASAN, as previous runs since #25 were fine.)

[2] https://logs.fd.io/vex-yul-rot-jenkins-1/vpp-csit-verify-perf-master-ubuntu2404-x86_64-2n-icx/61/csit_current/0/log.html.gz#s1-s1-s1-s1-s1-t1-k2-k5-k1-k1-k1-k1
[3] https://s3-logs.fd.io/vex-yul-rot-jenkins-1/vpp-csit-verify-perf-master-ubuntu2404-x86_64-3n-snr/49/csit_current/0/log.html.gz#s1-s1-s1-s1-s1-t1-k3-k3