Trigger a job using REST/HTTP POST with the job params

[dyasny]

Hi,

I am currently using argo workflows for $subj and I wonder whether Furiko can do the same? Essentially, Another app triggers a job by sending a HTTP POST to argo WF with a JSON blob that contains the arguments/params the job will use. Same idea as triggering parameterized jobs in Jenkins via REST.

Another approach I was looking into would be a gitops approach, basically a poller for git (as well as a receiver for github webhooks), which would check out a config file for a job from git, and trigger the jobs using that file

[irvinlim]

Hey @dyasny, thanks for your interest in the project!

I am currently using argo workflows for $subj and I wonder whether Furiko can do the same?

We're actually in the process of planning Argo Workflows support, you're in luck! However, it's likely going to be a full integration rather than a simple HTTP API call, since Furiko would prefer to handle and watch the full lifecycle of the Workflow, rather than possibly cause it to be run forever.

For triggering arbitrary HTTP calls, it might actually be slightly more complicated, since we need to dispatch HTTP tasks to a pool of workers, in order to avoid single-point-of-failure issues.

If you are interested in the full design details, you can check out #63. For the roadmap, you can check out https://furiko.io/docs/contributing/roadmap/

Another approach I was looking into would be a gitops approach

Unfortunately I don't think Furiko would be the best tool for GitOps, there are a considerable amount of tools already out there that can handle this better, such as ArgoCD. Nevertheless, natively supporting HTTP webhooks to trigger a job could be a good idea as well.

[dyasny]

I'm not looking to integrate argo workflows, I'm looking to replace it, since my use case is pretty narrow and I'd rather maintain a small task runner than a massive project like argo. All I need is a listener that receives a payload that contains some key:value(s) and translates it into a job that has those same key:value pairs as ENV vars.

[irvinlim]

Apologies for misunderstanding. That is completely feasible using job options (https://furiko.io/docs/execution/jobconfig/job-options) in your JobConfig (i.e. the job template).

However, the part on the setting up on an endpoint isn't supported directly out-of-the-box, though it is an interesting feature for us to support natively. In the meantime, I think that you can just write a simple K8s Service that listens for HTTP requests, parses the request JSON body, and simply creates a new Job (using the generated Go client), which should be doable in an hour or two.

We could support this natively but I think some design may be required (e.g. a separate service, CRD design, etc). I will add it to our roadmap first: https://furiko.io/docs/contributing/roadmap

EDIT: Maybe you could just use kube-apiserver (see comments below)

[dyasny]

Thanks, that answers my question. I'd rather avoid writing (and then maintaining) my own stuff when there are working options available out there. I'll keep an eye on Furiko for now then, maybe get back to it when the feature is available.

[irvinlim]

Just circling back, actually if all you need is a simple REST API, you could also just call the K8s apiserver directly to create a Job, which would look pretty much the same if you added a service in between to send the request to kube-apiserver. Basically, it is the same as doing kubectl create, which is outlined here: https://furiko.io/docs/execution/job/adhoc-execution

You'd lose the customizability to have "pretty" webhook URLs, and figuring out the RBAC may be a bit of a pain, but it's another possible alternative if you'd like to give it a shot. Cheers

[dyasny]

Thank you, I'll take a look