Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin install: use micromamba to make sure git is available #1733

Merged
merged 4 commits into from
Jan 27, 2025
Merged

Plugin install: use micromamba to make sure git is available #1733

Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
44371cf
plugin install: create base env with git and run git commands there
emlys Jan 10, 2025
b9a8c7d
clean up - remove commented code
emlys Jan 10, 2025
05ecf09
read plugin's conda dependencies from pyproject.toml rather than hard…
emlys Jan 10, 2025
cfa5f15
include python in plugin env; allow undefined conda deps
emlys Jan 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 24 additions & 15 deletions workbench/src/main/setupAddRemovePlugin.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,15 +56,26 @@ export function setupAddPlugin() {
try {
logger.info('adding plugin at', pluginURL);
const micromamba = settingsStore.get('micromamba');
const rootPrefix = upath.join(process.resourcesPath, 'micromamba_envs')
const baseEnvPrefix = upath.join(rootPrefix, 'invest_base');
// Create invest_base environment, if it doesn't already exist
// The purpose of this environment is just to ensure that git is available
if (!fs.existsSync(baseEnvPrefix)) {
await spawnWithLogging(
micromamba,
['create', '--yes', '--prefix', `"${baseEnvPrefix}"`, '-c', 'conda-forge', 'git']
dcdenu4 marked this conversation as resolved.
Show resolved Hide resolved
);
}
// Create a temporary directory and check out the plugin's pyproject.toml
const tmpPluginDir = fs.mkdtempSync(upath.join(tmpdir(), 'natcap-invest-'));
await spawnWithLogging(
'git',
['clone', '--depth', '1', '--no-checkout', pluginURL, tmpPluginDir]
micromamba,
['run', '--prefix', `"${baseEnvPrefix}"`,
'git', 'clone', '--depth', '1', '--no-checkout', pluginURL, tmpPluginDir]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't really tested this out myself on a bare bones VM, but do you think we'll need to set any basic git configuration, like username, to avoid any git prompts when calling git clone?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would that be the same as #1712? I tried to reproduce the issue by temporarily renaming my .gitconfig and confirming that my user.name was no longer set in the output of git config --list. I didn't get any prompts during plugin installation. Maybe it only affects windows?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, thanks for trying to test that out. For this PR I think it's okay to see if we come across it in the wild and can take it up from there.

);
await spawnWithLogging(
'git',
['checkout', 'HEAD', 'pyproject.toml'],
micromamba,
['run', '--prefix', `"${baseEnvPrefix}"`, 'git', 'checkout', 'HEAD', 'pyproject.toml'],
{ cwd: tmpPluginDir }
);
// Read in the plugin's pyproject.toml, then delete it
Expand All @@ -77,25 +88,23 @@ export function setupAddPlugin() {
const pluginID = pyprojectTOML.tool.natcap.invest.model_id;
const pluginName = pyprojectTOML.tool.natcap.invest.model_name;
const pluginPyName = pyprojectTOML.tool.natcap.invest.pyname;
const condaDeps = pyprojectTOML.tool.natcap.invest.conda_dependencies;
dcdenu4 marked this conversation as resolved.
Show resolved Hide resolved

// Create a conda env containing the plugin and its dependencies
const envName = `invest_plugin_${pluginID}`;
await spawnWithLogging(
micromamba,
['create', '--yes', '--name', envName, '-c', 'conda-forge', '"python<3.12"', '"gdal<3.6"']
);
const pluginEnvPrefix = upath.join(rootPrefix, envName)
const createCommand = [
'create', '--yes', '--prefix', `"${pluginEnvPrefix}"`,
'-c', 'conda-forge']; // include dependencies read from pyproject.toml
condaDeps.forEach((dep) => createCommand.push(`"${dep}"`))
await spawnWithLogging(micromamba, createCommand);
logger.info('created micromamba env for plugin');
await spawnWithLogging(
micromamba,
['run', '--name', envName, 'pip', 'install', `git+${pluginURL}`]
['run', '--prefix', `"${pluginEnvPrefix}"`, 'pip', 'install', `git+${pluginURL}`]
);
logger.info('installed plugin into its env');
// Write plugin metadata to the workbench's config.json
const envInfo = execSync(`${micromamba} info --name ${envName}`, { windowsHide: true }).toString();
logger.info(`env info:\n${envInfo}`);
const regex = /env location : (.+)/;
const envPath = envInfo.match(regex)[1];
logger.info(`env path:\n${envPath}`);
logger.info('writing plugin info to settings store');
settingsStore.set(
`plugins.${pluginID}`,
Expand All @@ -104,7 +113,7 @@ export function setupAddPlugin() {
pyname: pluginPyName,
type: 'plugin',
source: pluginURL,
env: envPath,
env: pluginEnvPrefix,
}
);
logger.info('successfully added plugin');
Expand Down
Loading