This repo demonstrates how to:
- Set up a scheduled activity (using Cloud Scheduler) that pulls data (an array of objects) from local data in a Cloud Function that offloads the data to a Pub/Sub topic...
- Which in turn has a Cloud Function trigger on the topic; the function divides the retrieved data into individual Cloud Tasks, so that the individual pieces of data are handled uniquely, then...
- Runs a service on Cloud Run to do something with the data from each individual task.
- Bonus: Creating and using separate service accounts for each service and using a completely private setup.
- You have a GCP account
- You are logged in through your environment
- You have set your variables as needed in
init.shand the respectivedeploy.shfiles
- Install dependencies with
npm installoryarn install. - Run
npm startoryarn start - Call or visit endpoint; should be
http://localhost:8080/
First of all, go through all scripts with names like init.sh, deploy.sh and index.js and ensure you set any relevant values that you are asked for.
Run init.sh. You should run this in steps since some things need to be done dynamically (like adding IDs).
Functions will have deployment URLs in the format https://{REGION}-{PROJECT}.cloudfunctions.net/{FUNCTION}. You'll also get a unique URL to the Cloud Run service.
Logs are available in the Logs view or through the Functions view.
The Cloud Run service is available at the Services view.
Cloud Scheduler will display the job in the Jobs view. For the Cloud Tasks queue, it's visible at Queues, which is good to have a look at if your calls are getting stuck (i.e. not working as intended).
Run teardown.sh. This will destroy the entire project, so know what you are getting into!