crew package not available in background task launched by crew

[yogat3ch]

Prework

• Read and agree to the Contributor Code of Conduct and contributing guidelines.
• Confirm that your issue is a genuine bug in the crew package itself and not a user error, known limitation, or issue from another package that crew depends on. For example, if you get errors running tar_make_clustermq(), try isolating the problem in a reproducible example that runs clustermq and not crew. And for miscellaneous troubleshooting, please post to discussions instead of issues.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• Post a minimal reproducible example like this one so the maintainer can troubleshoot the problems you identify. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.

Description

crew seems to be working fine in all of our AWS environments except for one in which very bizarre behavior is happening. The background tasks launched by crew give an error indicating that the crew package doesn't exist. Here's the logs written by crew that show this anomaly:

{

R version 4.3.0 (2023-04-21) -- "Already Tomorrow"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: aarch64-unknown-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> crew::crew_worker(settings = list(url = "ws://127.0.0.1:37039/02fe7f0a3bb57842a2e17c32", autoexit = 15L, cleanup = 1L, output = TRUE, maxtasks = Inf, idletime = 10000, walltime = Inf, timerstart = 0L, tls = NULL, rs = c(10407L, 1461037220L, 2123128661L, -694900718L, 161841259L, -1585953616L, -1384386319L)), launcher = "email@email.com", worker = 1L, instance = "02fe7f0a3bb57842a2e17c32")
Error in loadNamespace(x) : there is no package called ‘crew’
Calls: loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted,

Reproducible example

I'm unable to do so as it involves running in a remote arm64 linux container on AWS.

Expected result

On all of our other environments, crew seems to function. On this environment, on occasion, crew will cause intermittent errors with these logs.
Have you encountered anything like this? How might crew launch a worker, and then launch a task that is unable to find the crew package on that worker when trying to run the task? Perhaps it's somehow loading with a different library path?
I'm stumped.

Diagnostic information

R. 4.3.0 on aarch64-unknown-linux-gnu (64-bit)

[yogat3ch]

I think I know why this is now.
We're using renv to restore all of the dependencies for our golem app. renv writes all the packages to it's own library directory, rather than the standard R library directory.
I think the R session that launches in the background is likely using the default library path, and isn't inheriting from the parent session.
Does the background R session loaded by mirai load the local .Rprofile as a typical R session? I could add code to the .Rprofile to solve this if so. I'm going to test that out

[shikokuchuo]

Yes, the background R sessions should read .Rprofile.

[wlandau]

You can also manually supply library paths to the library argument of controller$push(): https://wlandau.github.io/crew/reference/crew_class_controller.html#method-crew_class_controller-push

[yogat3ch]

@wlandau , ahh that library argument is helpful. I'm going to try that. I managed to get renv::restore to restore to the default R library path and verified that newly launched R sessions on the container are indeed using the default(/usr/local/lib/R/site-library) as the library path. However, it seems like the workers are still dying immediately with each task because they are unable to access the crew package. I'm not sure why those tasks are using a different library path from the default or if even if they are. It's difficult to debug a background task on a remote container!
I'll mark an answer once this is solved. Thank you for the assistance!

[yogat3ch]

The fix required forcing the instances of renv::restore and install_local in the Dockerfile that built the server to install to the R default R library path on linux (/usr/local/lib/R/site-library), rather than the renv custom path (/usr/local/lib/R/site-library/R-4.3/aarch-arm64.../)
This ensures that background tasks launched by crew , which use the default R library path on linux (/usr/local/lib/R/site-library), will have the dependencies necessary to run code and not crash the app.

Thanks for the assist on this @shikokuchuo & @wlandau!