Skip to content

Commit

Permalink
Fix mfc.bat run for MS Windows (#533)
Browse files Browse the repository at this point in the history
Co-authored-by: ChrisZYJ <chriszhangyanjun@gmail.com>
  • Loading branch information
henryleberre and ChrisZYJ authored Jul 24, 2024
1 parent a0781f8 commit b4dc8ad
Show file tree
Hide file tree
Showing 8 changed files with 208 additions and 124 deletions.
48 changes: 36 additions & 12 deletions docs/documentation/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Please select your desired configuration from the list bellow:
. ./mfc.sh load
```

<a id="via-aptitude"></a>
- **Via [Aptitude](https://wiki.debian.org/Aptitude):**

```shell
Expand Down Expand Up @@ -59,29 +60,52 @@ On Windows, you can either use Intel Compilers with the standard Microsoft toolc
[Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/) for a Linux experience.

<details>
<summary><h3>Windows + Intel (Native)</h3></summary>

Install the latest version of:
- [Microsoft Visual Studio Community](https://visualstudio.microsoft.com/)
- [Intel® oneAPI Base Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html)
- [Intel® oneAPI HPC Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/hpc-toolkit-download.html)
<summary><h3>Windows + WSL (Recommended)</h3></summary>

Then, in order to initialize your development environment, open a terminal window and run:
Install [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/) on Windows 11:
Either
1. Open a terminal with administrator privileges and run the following command:
```shell
"C:\Program Files (x86)\Intel\oneAPI\setvars.bat"
wsl --install
```
Or
1. Open the Start menu, search for "Windows Features", and select "Turn Windows features on or off". Enable "Windows Subsystem for Linux" by checking the corresponding box.
2. Open the Microsoft Store, search for "Linux", and install your preferred distribution (e.g., [Ubuntu](https://apps.microsoft.com/store/detail/ubuntu/9PDXGNCFSCZV))

To follow this guide, please replace `./mfc.sh` with `mfc.bat` when running any commands. `./mfc.sh` is intended Unix-like systems.
You will also have access to the `.sln` Microsoft Visual Studio solution files for an IDE (Integrated Development Environment).
Useful software to install for using WSL on Windows:
- [Windows Terminal](https://apps.microsoft.com/store/detail/windows-terminal/9N0DX20HK701)
- [Visual Studio Code](https://code.visualstudio.com/) and the [Remote - WSL](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl) extension

Once you have WSL installed, you can follow the instructions for *nix systems above (for Ubuntu, see [Via Aptitude](#via-aptitude)).

</details>

<details>
<summary><h3>Windows + WSL</h3></summary>

Install the latest version of the [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/) as well as a distribution such as Ubuntu which can be found [here](https://apps.microsoft.com/store/detail/ubuntu/9PDXGNCFSCZV). Acquiring an interactive session is as simple as typing `wsl` in your command prompt, or alternatively, selecting the distribution from the dropdown menu available in the [Microsoft Terminal](https://apps.microsoft.com/store/detail/windows-terminal/9N0DX20HK701).
<summary><h3>Native Windows (Intel)</h3></summary>

Install the latest version of:
- [Microsoft Visual Studio Community](https://visualstudio.microsoft.com/)
- [Intel® oneAPI Base Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html)
- [Intel® oneAPI HPC Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/hpc-toolkit-download.html)
- [Strawberry Perl](https://strawberryperl.com/) (Install and add `C:\strawberry\perl\bin\perl.exe` or your installation path to your [PATH](https://www.architectryan.com/2018/03/17/add-to-the-path-on-windows-10/))
Please note that Visual Studio must be installed first, and the oneAPI Toolkits need to be configured with the installed Visual Studio, even if you plan to use a different IDE.

Then, in order to initialize your development environment, run the following command (or your installation path) in command prompt:
```shell
"C:\Program Files (x86)\Intel\oneAPI\setvars.bat"
```
Alternatively, you can run the following command in Powershell:
```shell
cmd.exe "/K" '"C:\Program Files (x86)\Intel\oneAPI\setvars.bat" && powershell'
```
You could verify the initialization by typing `where mpiexec` in the command prompt terminal (does not work in Powershell), which should return the path to the Intel MPI executable.
To continue following this guide, please stay in the initialized terminal window. Replace `./mfc.sh` with `.\mfc.bat` for all commands.

If `.\mfc.bat build` produces errors, please run the command again. Repeating this process three times should resolve all errors (once each for pre_process, simulation, and post_process). If the same error persists after each attempt, please verify that you have installed all required software and properly initialized the development environment. If uncertain, you could try deleting the build directory and starting over.

You can now follow the appropriate instructions for your distribution.
You will also have access to the `.sln` Microsoft Visual Studio solution files for an IDE (Integrated Development Environment).

</details>

Expand Down
6 changes: 3 additions & 3 deletions mfc.bat
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ goto label_windows

:label_windows

if not exist "%cd%\toolchain\mfc.py" (
if not exist "%cd%\toolchain\main.py" (
echo.
echo ^[mfc.bat^] You must call this script from within MFC's root folder
echo.
Expand All @@ -16,7 +16,7 @@ if not exist "%cd%\toolchain\mfc.py" (
mkdir "%cd%\build" 2> NUL

if not exist "%cd%\build\venv" (
python3 -m venv "%cd%\build\venv"
python -m venv "%cd%\build\venv"
if %errorlevel% neq 0 (
echo.
echo ^[mfc.bat^] Failed to create the Python virtual environment. Delete the build/venv folder and try again.
Expand All @@ -40,7 +40,7 @@ if %errorlevel% neq 0 (
copy "%cd%\toolchain\requirements.txt" "%cd%\build" 2> NUL
)

python3 "%cd%\toolchain\mfc.py" %*
python "%cd%\toolchain\main.py" %*
set main_py_err=%errorlevel%

call "%cd%\build\venv\Scripts\deactivate.bat"
Expand Down
2 changes: 1 addition & 1 deletion mfc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ code=$?
echo

if [ $code -ne 0 ]; then
error "mfc.py finished with a $code exit code."
error "main.py finished with a $code exit code."
fi

# Deactivate the Python virtualenv in case the user "source"'d this script
Expand Down
5 changes: 4 additions & 1 deletion toolchain/mfc/run/queues.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import typing, dataclasses
import os, typing, dataclasses

from mfc import common
from ..state import ARG
Expand Down Expand Up @@ -26,6 +26,9 @@ def is_active(self) -> bool:
return True

def gen_submit_cmd(self, filepath: str) -> typing.List[str]:
if os.name == 'nt':
return [filepath]

return ["/bin/bash", filepath]


Expand Down
2 changes: 1 addition & 1 deletion toolchain/mfc/run/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ def get_baked_templates() -> dict:
def __job_script_filepath() -> str:
return os.path.abspath(os.sep.join([
os.path.dirname(ARG("input")),
f"{ARG('name')}.sh"
f"{ARG('name')}.{'bat' if os.name == 'nt' else 'sh'}"
]))


Expand Down
5 changes: 4 additions & 1 deletion toolchain/mfc/test/case.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,10 @@ def get_dirpath(self):
return os.path.join(common.MFC_TESTDIR, self.get_uuid())

def get_filepath(self):
return os.path.join(self.get_dirpath(), "case.py")
filepath = os.path.join(self.get_dirpath(), "case.py")
if os.name == 'nt':
return filepath.replace('\\', '\\\\')
return filepath

def delete_output(self):
dirpath = self.get_dirpath()
Expand Down
120 changes: 72 additions & 48 deletions toolchain/templates/default.mako
Original file line number Diff line number Diff line change
@@ -1,65 +1,89 @@
<%!
import os
from mako.exceptions import RuntimeException
%>
% if os.name == 'nt':
@echo off
% else:
#!/usr/bin/env bash
% endif

<%namespace name="helpers" file="helpers.mako"/>

${helpers.template_prologue()}

% if engine == 'batch':
error "The$MAGENTA default$COLOR_RESET template does not support batch jobs. Please use a different template via the $MAGENTA--computer$COLOR_RESET option.\n"
exit 1
% endif
<%
if engine == 'batch':
raise RuntimeException("The default template does not support batch jobs. Please use a different template via the --computer option.")
%>

warn "This is the$MAGENTA default$COLOR_RESET template."
warn "It is not intended to support all systems and execution engines."
warn "Consider using a different template via the $MAGENTA--computer$COLOR_RESET option if you encounter problems."
% if os.name != 'nt':
warn "This is the$MAGENTA default$COLOR_RESET template."
warn "It is not intended to support all systems and execution engines."
warn "Consider using a different template via the $MAGENTA--computer$COLOR_RESET option if you encounter problems."

% if mpi:
# Find a suitable MPI launcher and store it in the variable "binary".
for binary in ${binary or ''} jsrun srun mpirun mpiexec; do
if command -v $binary > /dev/null; then
break
fi
done
% if mpi:
# Find a suitable MPI launcher and store it in the variable "binary".
for binary in ${binary or ''} jsrun srun mpirun mpiexec; do
if command -v $binary > /dev/null; then
break
fi
done

if ! command -v $binary > /dev/null; then
error ":( Could not find a suitable MPI launcher.\n"
exit 1
else
ok ":) Selected MPI launcher $MAGENTA$binary$COLOR_RESET. Use$MAGENTA --binary$COLOR_RESET to override."
fi
% endif
if ! command -v $binary > /dev/null; then
error ":( Could not find a suitable MPI launcher.\n"
exit 1
else
ok ":) Selected MPI launcher $MAGENTA$binary$COLOR_RESET. Use$MAGENTA --binary$COLOR_RESET to override."
fi
% endif

% for target in targets:
${helpers.run_prologue(target)}
% for target in targets:
${helpers.run_prologue(target)}

% if not mpi:
(set -x; ${profiler} "${target.get_install_binpath(case)}")
% else:
if [ "$binary" == "jsrun" ]; then
(set -x; ${profiler} \
jsrun --nrs ${tasks_per_node*nodes} \
--cpu_per_rs 1 \
--gpu_per_rs ${1 if gpu else 0} \
--tasks_per_rs 1 \
"${target.get_install_binpath(case)}")
elif [ "$binary" == "srun" ]; then
(set -x; ${profiler} \
srun --ntasks ${nodes*tasks_per_node} \
"${target.get_install_binpath(case)}")
elif [ "$binary" == "mpirun" ]; then
(set -x; ${profiler} \
$binary -np ${nodes*tasks_per_node} \
% if not mpi:
(set -x; ${profiler} "${target.get_install_binpath(case)}")
% else:
if [ "$binary" == "jsrun" ]; then
(set -x; ${profiler} \
jsrun --nrs ${tasks_per_node*nodes} \
--cpu_per_rs 1 \
--gpu_per_rs ${1 if gpu else 0} \
--tasks_per_rs 1 \
"${target.get_install_binpath(case)}")
elif [ "$binary" == "mpiexec" ]; then
(set -x; ${profiler} \
$binary --ntasks ${nodes*tasks_per_node} \
elif [ "$binary" == "srun" ]; then
(set -x; ${profiler} \
srun --ntasks ${nodes*tasks_per_node} \
"${target.get_install_binpath(case)}")
fi
% endif
elif [ "$binary" == "mpirun" ]; then
(set -x; ${profiler} \
$binary -np ${nodes*tasks_per_node} \
"${target.get_install_binpath(case)}")
elif [ "$binary" == "mpiexec" ]; then
(set -x; ${profiler} \
$binary --ntasks ${nodes*tasks_per_node} \
"${target.get_install_binpath(case)}")
fi
% endif

${helpers.run_epilogue(target)}

${helpers.run_epilogue(target)}
echo
% endfor
% else:
% for target in targets:
${helpers.run_prologue(target)}

echo
% endfor
% if not mpi:
${profiler} "${target.get_install_binpath(case)}.exe"
% else:
${profiler} \
mpiexec -n ${nodes*tasks_per_node} \
"${target.get_install_binpath(case)}.exe"
% endif

${helpers.run_epilogue(target)}
% endfor
% endif

${helpers.template_epilogue()}
Loading

0 comments on commit b4dc8ad

Please sign in to comment.