Fix double exit bug in parallel pipelines (#739)

* Fix double-exit bug in parallel pipelines

Signed-off-by: Bolun Thompson <bolunthompson@ucla.edu>

* Assert running_procs is positive

Signed-off-by: Bolun Thompson <bolunthompson@ucla.edu>

* Fix: don't use exit handler

The reason why it worked on Ubuntu 20.04 was that the EXIT handler was
not called, while on Ubuntu 24.04 it correctly is.

Signed-off-by: Bolun Thompson <bolunthompson@ucla.edu>

* Add comment explanation

Signed-off-by: Bolun Thompson <bolunthompson@ucla.edu>

---------

Signed-off-by: Bolun Thompson <bolunthompson@ucla.edu>

compiler/pash_compilation_server.py

@@ -379,6 +379,7 @@ def remove_process(self, process_id):
                 ]
             )
 
+        assert self.running_procs > 0
         self.running_procs -= 1
         if self.running_procs == 0:
             self.unsafe_running = False

compiler/pash_runtime.sh

@@ -141,7 +141,9 @@ else
         pash_redir_output echo "$$: (5) BaSh script exited with ec: $pash_runtime_final_status"
     else 
         function run_parallel() {
-            trap inform_daemon_exit SIGTERM SIGINT EXIT
+            # ideally we'd call inform_daemon_exit with the EXIT handler,
+            # but it isn't called on Ubuntu 20.04 (for no clear reason)
+            trap inform_daemon_exit SIGTERM SIGINT
             export SCRIPT_TO_EXECUTE="$pash_script_to_execute"
             source "$RUNTIME_DIR/pash_restore_state_and_execute.sh"
             inform_daemon_exit