-
Notifications
You must be signed in to change notification settings - Fork 1
Using WSL
Note steps 3 onward are all in a WSL terminal:
- Install WSL
- Open a WSL terminal
- Go to the WSL home directory (
cd ~
), if you thenpwd
it should be/home/<username>/
- From there clone the repository (within WSL, you can put it wherever
relative to your
~
directory if you want)git clone --recurse-submodules https://github.com/byu-transpolab/wfrc_asim_scenario
cd wfrc_asim_scenario
- Make sure R is installed (
sudo apt install r-base r-base-dev
) - Open R from the terminal (
R
) - You'll probably have to install the R packages since WSL can't use the
windows-installed ones. The packages are:
tidyverse
,sf
,tigris
,tidycensus
,targets
,tarchetypes
, andxml2
. There's a good chance you'll run into errors with the installation due to missing packages etc. in WSL. Most of the errors will look something like the following:
------------ Missing dependencies ----------
The library libgdal is missing. The library can be found
in one of the following:
deb: libgdal-devel libgdal (debian, ubuntu, etc.)
rpm: lib-gdal (fedora, rhel, etc.)
etc.
---------------------------------------------
The exact wording is probably wrong since I'm typing this from memory, but
there will be a text block that looks generally like this. It may be in the
middle of the install text, so I'd watch the process and stop it with ctrl-c
(you may need to push that a few times) once you see something like this. The
only important line is the deb:
line: install these library(ies) with:
sudo apt install <copy everything after 'deb:' and before the parenthesis>
(you'll
need to do this outside an R session; it may be helpful to have two WSL windows
side by side for this). Then try installing the R packages again. This is an
iterative process and you will probably get different errors like this each
time; just keep installing the dependencies and eventually all the R packages
will install. You might also need to run sudo apt install build-essential
; in
fact I'd recommend doing this first. You can exit out of R for now (q()
).
Now install conda (still from within WSL):
cd ~
sudo apt install wget
wget <https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh>
sh Miniconda3-latest-Linux-x86_64.sh
The defaults for the installation are probably fine (don't have it run conda init
yet). After the install finishes, see if you can run conda
. It will
probably give you an error that the command can't be found. Note where the
installation path was (probably /home/<username>/miniconda3
). Now:
cd ~
, nano .bashrc
, and add the following line somewhere:
export PATH=$PATH:<conda install path>/bin
. Save with ctrl-o , exit with
ctrl-x. Now run nano .bash_profile
and add source .bashrc
to it (the file
may be empty to start, that's fine). Exit, then source .bash_profile
. See
if conda
works now, it should. Then run conda init
. It should list a bunch
of files and say some are modified. Exit out of any WSL window(s) and re-open
them, you should see a (base)
at the start of the terminal prompt. If so,
then conda is good to go!
With WSL open, now set up the two conda environments:
cd ~/wfrc_asim_scenario #Change this to be wherever the repo is
#create activitysim conda environment
conda create -y -n ASIM_DEV python=3.9
conda activate ASIM_DEV
conda install -y pytables
pip install -r reference/asim_requirements.txt
#create populationsim conda environment
conda create -y -n popsim python=3.8
conda activate popsim
conda install -y pytables
pip install scipy
pip install populationsim
conda activate base
This may take some time. If the ASIM_DEV
environment above doesn't work when
you're running scripts, create it with
conda env create --file=activitysim/conda-environments/activitysim-dev.yml --name ASIM_DEV
instead (make sure you cd
to wherever the repository is first).
You'll have to conda remove --name ASIM_DEV --all
before remaking it. This
way of creating the environment will take probably 10-15 minutes to run.
Now you
need a Census API key. https://api.census.gov/data/key_signup.html. You might
get two emails, and the link to activate the key might be finnicky, but it
worked after a few tries for me. Copy this key. In WSL cd
to the repo
directory, and nano .Renviron
. Write CENSUS_API_KEY="<your key here>"
(with
the quotes) and save and exit.
Now start R (R
) from the terminal in the root
of the repository (probably cd ~/wfrc_asim_scenario
). Run tar_make()
and
see what happens. It *should *make it to I think skims_file
and then error
(if it keeps going and doesn't error, then great! That should be fully
working). Ideally the targets pipeline would run the .sh files itself, but I
couldn't get it to outside of linux. So we need to do that part manually.
There
are a few .sh scripts that need to be run at certain points in the targets
pipeline, and the process will be similar for all of them. Find what the
errored target does (for skims_file
, it runs prepare_skims()
). We want to
find which .sh file is being run (sh/build_skims.sh
) and then run those
commands manually. The if
statements about
sourcing .bashrc
or .bash_profile
can be skipped, as can
any printf
lines. For build_skims.sh
, that means we need to run conda activate ASIM_DEV
and the python
line (conda info
just shows info about
the current env, you can run it if you want). For the python
line, though, we
need to replace the $
arguments with actual folder paths. What those mean is
the nth argument of the command for $n
(counting from 0). So we go back
to prepare_skims()
and see that the args
are the script and three
directories. So python py/build_omx.py "$1" "$2" "$3"
becomes python py/build_omx.py inputs/skims inputs/skims data_activitysim
(we don't need to
add the || exit 1
). Run this in WSL (outside of R) and it *should *work.
If it
works, great! You will need to edit the targets pipeline a bit to essentially
bypass running the build_skims.sh
file (and the other script files once
you've gotten to them). Since the script file targets return a file path in the
targets pipeline, you can just hard-code the path (in the skims case,
it's data_activitysim/skims.omx
, from the return statement in
the prepare_skims()
function). I believe this process needs to be done
for build_skims.sh
, run_popsim.sh
, and run_activitysim.sh
. Unfortunately,
you'll have to alternate re-running tar_make()
and these scripts manually
each time you change something upstream.
I'm now looking at the code more closely, and I'm noticing a couple things that I think are incorrectly written. I'm not entirely sure since it's been a while, but you could try these changes if you run into errors.
- In
run_popsim.sh
, theASIM_DEV
environment is activated instead ofpopsim
. This might work, but you might need to usepopsim
. - You could try adding the line
#!/bin/bash -v
to the top of the three scripts, there's a chance this will make it work within the targets pipeline. Not sure though. - The sourcing of
.bashrc
and.bash_profile
(lines 5 and 8) are backwards based on theif
statements in the scripts. This shouldn't matter since you made both anyway, and if you're running the scripts manually you can skip these lines entirely, but fixing this may make the scripts work in the targets pipeline.
Not sure why these discrepancies, but it was probably just oversight on my part that happened to still work with the supercomputer setup.