Depending on multiple parallel tasks being done?

[boenrobot]

I originally made this StackOverflow thread but I think what I'm asking is just not possible with the current gulp API, which is why I'm making this issue.

What I want ultimately is to start tasks just once, and as soon as all of their dependencies have finished executing.

You can see a more concrete example in the link, but I'll abstract it a bit here...

I have tasks A, B, C, D, E, F, G
G depends on E and C.
F depends on E and B.
C depends on A.
B depends on A.
E depends on D and A.
A and D have no dependencies.

It seems to be impossible to model a hierarchy where all of those would be executed just once, as soon as possible.

I can do

series(parallel(A, D), parallel(B, E), parallel(C, F), G)

but while that would "get the job done"™, it's not optimal.

• G is waiting on everything else, even though it could begin as soon as E and C are done, without caring about F.
• B could begin as soon as A is done, without caring for D.

I could also do

parallel(
  series(parallel(series(parallel(A, D), E), series(A, C)), G),
  series(parallel(series(parallel(A, D), E), series(A, B)), F)
)

but that's even worse, as A, D and E are executed multiple times. It would've been optimal if the completion in any one one of the instances was signaled through the others.

I think earlier gulp versions solved this with a 3rd argument to task() that was a list of dependencies... I'm not suggesting for this to be brought back though. I quite like this new approach for the simpler cases. I realize cases like the above aren't exactly common place... Though they do happen from time to time, so they should be addressed IMO.

I'd like to suggest a new task class that would allow projects to register such dependencies.

Here's an example:

export class G extends GulpTask {
  public constructor() {
    super();
    // Provided by GulpTask. Is a variable list of TaskFunction object types, NOT instances. Executed once, in parallel, skipped if already executed.
    this.addDependencies(E, C);
  }
  run(): TaskFunction {
    return /* return an actual type function, to be executed once all dependencies have finished */;
  }
}

In this case, task G would be able to be executed like any other task. run() will be called only after E and C have finished, and it will be called only once. Its result will then be executed like any other task.

The class should be instanciated once by gulp at startup, and similarly, its dependencies should be instanciated once if not already instanciated. Then, after figuring out the dependency tree of the targeted task, the inner most task functions should be ran in parallel, and after that (i.e. serially), the next layer of dependencies should be executed in parallel, and so on. If a dependency has already finished executing, it should not be executed a second time. Most important in this whole thing though... Gulp should be smart enough to not wait for an unrelated dependency before starting an upper layer dependency.

Under the hood, I believe this would require for GulpTask to listen to completion events by its GulpTask dependencies, and maybe wrap normal tasks in a GulpTask class, so that they can emit such events on completion.

NOTE: If a GulpTask class is passed to other APIs, f.e. to watch(), series() or parallel(), then the specified dependencies should be executed once more, as well their run() methods. The instance should still be started just the first time, and remain live for the duration of the gulp process.

i.e.

• series(G, G) will call the constructors of G, E and C once if not already started, then parallel the run() of E and C, then the run() of G, then parallel the run() of E and C again, then the run() of G again.
• parallel(G, G) will call the constructors of G, E and C once if not already started, then parallel the run() of E and C from the first argument, as well as the run() of E and C of the second argument, then after the E and C of the first argument are done, do the G of the first argument, and similarly, do the G of the second argument after its own E and C are done.
• watch('*', G) will call the constructors of G, E and C once if not already started, and for every change to a file, it will call the run() of E and C in parallel, and after that, the run() of G.

[neunato]

I originally made this StackOverflow thread but I think what I'm asking is just not possible with the current gulp API, which is why I'm making this issue.

What I want ultimately is to start tasks just once, and as soon as all of their dependencies have finished executing.

You could wrap your tasks in a function that returns a promise to execute the task. That way, the task gets executed only once, as soon as the first call, and all .series/parallel the task is a part of will wait for the same promise.

Unfortunately this leaves a mess in the console output, but you could easily implement your own logging inside the wrapper and use the --silent flag.

"use strict"

let {series, parallel} = require("gulp")
let async_done = require("async-done")
let sleep = async (ms) => new Promise((resolve) => setTimeout(resolve, ms))
let log = console.log

let tasks = {
   A: async () => { log("A started"); await sleep(1000); log("\nA finished") },
   B: async () => { log("B started"); await sleep(1000); log("\nB finished") },
   C: async () => { log("C started"); await sleep(1000); log("\nC finished") },
   D: async () => { log("D started"); await sleep(1500); log("\nD finished") },
   E: async () => { log("E started"); await sleep(1000); log("\nE finished") },
   F: async () => { log("F started"); await sleep(1000); log("\nF finished") },
   G: async () => { log("G started"); await sleep(1000); log("\nG finished") }
}

let entries = Object.entries(tasks)
for (let [name, task] of entries) {
   tasks[name] = async () => {
      if (!task.done)
         task.done = new Promise((resolve) => async_done(task, resolve))
      return task.done
   }
}

let A = tasks.A
let D = tasks.D
let E = series(parallel(D, A), tasks.E)
let B = series(A, tasks.B)
let C = series(A, tasks.C)
let F = series(parallel(E, B), tasks.F)
let G = series(parallel(E, C), tasks.G)
let H = parallel(A, B, C, D, E, F, G)

module.exports = {
   default: H
}