What's the best way to setup and initiate a Sidekiq worker on a MRKS deployed service?

[PabloC]

What's your approach?
What do you recommend?
I'am using the example in the Readme to starting a job with a new host with same IP address and a different role.

[yoelcabo]

I'am using the example in the Readme to starting a job with a new host with same IP address and a different role.

That works like a charm for us :)

servers:
  web:
    hosts:
      - [REDACTED]
    cmd: bundle exec puma -C config/puma.rb
  workers:
    hosts:
      - [REDACTED]
      - [REDACTED]
      - [REDACTED]
    cmd: bundle exec sidekiq -q my_queue -c 8 -t 600

[Michoels]

I tried this, and had a terrible time getting Sidekiq to connect to my Redis container.

After at least 6 hours of banging my head against the wall, I found this solution.

Adding the special host-gateway value with --add-host allows the containers to see each other's ports.

Here's the complete deploy.yml:

# deploy.yml

service: "app"
image: username/app

servers:
  web:
    hosts:
      - [REDACTED_SERVER_IP_ADDRESS]
    options:
      "add-host": host.docker.internal:host-gateway # This does the trick

  sidekiq:
    cmd: bundle exec sidekiq
    hosts:
      - [REDACTED_SERVER_IP_ADDRESS]
    options:
      "add-host": host.docker.internal:host-gateway # Add it here too

env:
  secret:
    - REDIS_URL

accessories:
  redis:
    image: redis:7.0
    host: [REDACTED_SERVER_IP_ADDRESS]
    port: 6379
    directories:
      - data:/data

After deploying, all containers on the same server can now see each other's ports as host.docker.internal:PORT

Finally I set the environment variable REDIS_URL to redis://host.docker.internal:6379, and it all works fine now.

[krtschmr]

what's the current best practice to start a standard solid_que worker on single instance VPS?

the deployment is good, web is served.
now if you want to start a worker with 4 threads, how's that done? docs are pretty poor on that topic, since that word worker isn't mentioned at a..

[Michoels]

You can configure the worker count in config/queue.yml (docs)

Regarding deployment, the official docs say:

For small projects, you can run Solid Queue on the same machine as your webserver. When you're ready to scale, Solid Queue supports horizontal scaling out-of-the-box. You can run Solid Queue on a separate server from your webserver, or even run bin/jobs on multiple machines at the same time. Depending on the configuration, you can designate some machines to run only dispatchers or only workers. See the configuration section for more details on this.

If you want to run Solid Queue in your main Rails container, you probably need to use the Puma plugin.
If you want to run it in a separate container, you'd probably do something like this:

solid_queue:
    cmd: bin/jobs
    hosts:
      - [REDACTED_SERVER_IP_ADDRESS]

@dhh demonstrated this on stage at the Rails World 2024 keynote starting at 49:33