[project] WASM compatibility for QuantEcon Lectures

[mmcky]

This project is to review and recommend changes that need to be made to:

• QuantEcon lectures
• QuantEcon.py package

that will help us better use and support the upcoming WASM environments such as those provided by JupyterLite.

Aim: To focus on https://github.com/quantecon/lecture-python-intro and identify solutions with the aim to enable the entire lecture set to run in the browser via WASM. If this aim is not fully achievable due technical restrictions -- we need to document all those cases and identify solutions.

TimeFrame: This lecture set will be used to support the QuantEcon Africa workshops that will be run in the first two weeks of July 2023

Date	Milestone
15th May 2023	Complete review of lecture series dependencies and report compatibility
01st June 2023	QuantEcon.py compatibility implemented and/or a sub-package available for the lecture series (in the short term)

Tasks include:

• install https://jupyterlite.readthedocs.io/en/latest/ and setup some instructions to build a common environment so those working on this project use the same baseline.
• review https://github.com/quantecon/lecture-python-intro and build a full dependency list to support lectures execution (not the build environment).
• review dependencies (i.e. numba) to see what is compatible
• for any incompatible dependencies we need to identify a solution to support execution in the browser (first best) or alternatives for running that lecture in colab (second best).
• can we implement a decorator in QuantEcon.py that is context driven, such that in a WASM environment numba jit is not used (if not supported)

Resources

The jupyterlite project

• setup custom jupyterlite on gh-pages for testing and construct a test bed that we can all work on together.
• see if jupyterlab-myst is runnable to render full myst notebooks
• load test notebooks (myst or download notebook set)

[mmcky]

It looks like graphviz is not currently supported in jupyterlite

xflr6/graphviz#197

[Smit-create]

@mmcky Thanks for the detailed setup. This looks promising.

can we implement a decorator in QuantEcon.py that is context driven, such that in a WASM environment numba jit is not used (if not supported)

I think numba has a flag(I guess DISABLE_JIT) which will just return python functions and should work for our case.

[jstac]

Thanks @Smit-create , sounds promising.

Please keep @oyamad included in any decisions that involve QuantEcon.py

[mmcky]

Please keep @oyamad included in any decisions that involve QuantEcon.py

Thanks @jstac -- for sure.

@Smit-create I have now create a project repository where we can deploy jupyterlite into gh-pages and share a common test environment. We can also document our findings in Issues and/or Wiki pages.

https://github.com/QuantEcon/project.lecture-wasm

[Smit-create]

pyodide currently has the supported list of packages given here: https://pyodide.org/en/stable/usage/packages-in-pyodide.html

Now, since this doesn't support numba, we won't be able to use quantecon (not even install it using piplite) as it's dependent on it.

So, here's one thing that comes to my mind with minimal changes in Quantecon.py:

1. Remove numba from its dependency list.
2. Instead of raising an error for numba not present, we can add a fallback by creating a wrapper module with a function, let's say qe_jit which will return numba jitted function in case numba is present, or else, just return pure python function. Also, we will raise a slight warning while loading quantecon that "Numba is not present, some computations may take longer time or may fail in same cases.". So something like this:

# We have function in qe.utils

def qe_jit(func, ...):
    if NUMBA_PRESENT:
        return numba.njit(func,...)
   else:
       return func

And remove this error message completely: https://github.com/QuantEcon/QuantEcon.py/blob/67fe7044148f95dbad42f85c82e288c7c1584650/quantecon/__init__.py#L8-L14

That is replacing all the numba decorators with wrappers.

This will open the doors for our lectures to be dependent on quantecon and some piplite-installable libraries.

[jstac]

Thanks @Smit-create , sounds promising.

Once @mmcky has reviewed this idea, let's run it by @oyamad .

[Smit-create]

Thanks @jstac.

There's one more way to handle this, but that might be kind of hack-ish or repetitive. We can create a new fork Quantecon.py focusing just on the methods/classes that are required to run the introductory series. The only drawback of this approach is that it limits the scope of moving other lecture series to jupyterlite. But for getting started, we can go by this route if we don't need much effort to get the introductory series running.

[mmcky]

@Smit-create that is exactly the sort of approach I was thinking as well use a decorator that could either return a jitted, jax, or plain python version of the function based on either a context flag or some other trigger.

[oyamad]

I am not very enthusiastic about the idea of allowing quantecon to run without Numba.

• Some functions in quantecon are useless without Numba, e.g., those for which functions of the same functionality exist in SciPy, such as linprog_simplex. Users who are not careful enough may install quantecon without Numba and conclude that some functions are too slow, which could deteriorate the reputation of quantecon.
• Adding a custom decorator would complicate the code and lead to a barrier to new contributors.
• I rather like the idea of creating a new fork, say quantecon-lite.