Skip to content

Commit

Permalink
Update + add C++ example
Browse files Browse the repository at this point in the history
  • Loading branch information
pcafrica committed Feb 17, 2025
1 parent cb90dc0 commit e9ed254
Show file tree
Hide file tree
Showing 4 changed files with 100 additions and 27 deletions.
55 changes: 28 additions & 27 deletions lectures/3/3.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ _class: titlepage

# Modules: reusable code in Python

In Python, the ability to reuse code is facilitated by modules. A module is a file with a `.py` extension that contains functions and variables. There are various methods to write modules, including using languages like C to create compiled modules.
In Python, the ability to reuse code is facilitated by modules. A module is a file with a `.py` extension that contains functions and variables. There are various methods to write modules, including using languages like C or C++ to create compiled modules.

When importing a module, to enhance import performance, Python creates byte-compiled files (`__pycache__/filename.pyc`). These files, platform-independent and located in the same directory as the corresponding `.py` files, speed up subsequent imports by storing preprocessed code.

Expand Down Expand Up @@ -476,14 +476,6 @@ In addition to the standard library, there is an active collection of hundreds o
_class: titlepage
-->
# **Zen of Python**:<br>"Explicit is better than implicit."<br>Run `import this` in Python to learn more.
---
<!--
_class: titlepage
-->
# Error handling
---
Expand All @@ -500,7 +492,7 @@ print("Another line") # Code fails before getting to this line.
---
# `try-except`
# `try`-`except`
```python
try:
Expand Down Expand Up @@ -553,7 +545,7 @@ my_list[5]
> IndexError: list index out of range
```python
my_tuple = (1,2,3)
my_tuple = (1, 2, 3)
my_tuple[0] = 0
```
> TypeError: 'tuple' object does not support item assignment
Expand Down Expand Up @@ -585,6 +577,14 @@ Finally, we can even define our own exception types by inheriting from the `Exce
_class: titlepage
-->
# **Zen of Python**:<br>"Explicit is better than implicit."<br>Run `import this` in Python to learn more.
---
<!--
_class: titlepage
-->
# Dependency management
---
Expand All @@ -593,7 +593,7 @@ _class: titlepage
1. **Code longevity:** How do you ensure that your code will still function as expected in one year, or even five? Consider the implications of using libraries such as Numpy, TensorFlow, or packages from sources like GitHub.
2. **Consistent results:** How can you guarantee that both your current and future collaborators are able to achieve the same computational results as you?
2. **Consistent results:** How can you guarantee that both your current and future collaborators are able to achieve the same numerical results as you?
3. **Easy installation:** What steps can you take to simplify the process for collaborators to set up your code with all required dependencies?
Expand All @@ -618,7 +618,7 @@ _class: titlepage
# PyPI (The Python Package Index)
- **Installation tool**: pip
- **Usage summary**: PyPI is primarily used for Python-only packages or Python interfaces to external libraries. It also hosts packages that include bundled external libraries, such as numpy.
- **Usage summary**: PyPI is primarily used for Python-only packages or Python interfaces to external libraries. It also hosts packages that include bundled external libraries, such as NumPy.
- **Number of packages**: Extensive, with long-term support for older versions.
- **Handling libraries**: Dependencies on external libraries require either inclusion in the package or installation through other means (e.g., OS installers or manual setup).
- **Pros**:
Expand Down Expand Up @@ -682,7 +682,7 @@ For effective management and replication of Python environments, recording depen
#### `requirements.txt`
This is a straightforward text file used by pip with virtual environments. It lists the packages your project depends on. Here’s a basic example:
This is a straightforward text file used by pip with virtual environments. It lists the packages your project depends on. Here’s a basic example, obtained by running `python -m pip freeze > requirements.txt`:
```
numpy
Expand All @@ -697,7 +697,7 @@ scipy
#### `environment.yml`
Used by Conda, this YAML file provides a structured format to specify the name of the environment, the channels from which packages should be sourced, and the dependencies themselves. Example:
Used by Conda, this YAML file provides a structured format to specify the name of the environment, the channels from which packages should be sourced, and the dependencies themselves. Example (`conda env export --name my_env > environment.yml`):
```yaml
name: my-environment
Expand Down Expand Up @@ -787,7 +787,7 @@ Docker is a platform for developing, shipping, and running applications inside c
# Pull the image.
docker pull python:latest
# Create a container.
# Create a container with shared folder.
docker run --name my_container -v /path/to/host/folder:/shared-folder -it -d python:latest
# Enable the container.
Expand All @@ -813,7 +813,7 @@ If the status of the container is `Up`, you can stop it with
docker stop my_container
```
Once you have created your container remember to **do not** use again the commad `run` but just `start`. Otherwise you will create every time a new container. If you want to remove a container you can run:
Once you have created your container remember to **not** use again the commad `run` but just `start`. Otherwise you will create a new container every time. If you want to remove a container you can run:
```bash
docker rm <name-of-the-container>
Expand Down Expand Up @@ -849,7 +849,7 @@ A Dockerfile is a text document that contains all the commands a user could call
---
# Building a Docker image: example
# Building a Docker image: example of `Dockerfile`
```Dockerfile
# Start from a Python base image.
Expand Down Expand Up @@ -891,7 +891,7 @@ docker push my_image:version # If needed.
_class: titlepage
-->
# Continuous Integration/Continuous Deployment
# Continuous Integration/<br>Continuous Deployment
---
Expand Down Expand Up @@ -976,18 +976,19 @@ jobs:
test-and-document:
runs-on: ubuntu-latest
container:
image: python:3.8-slim
options: --user 1001:1001
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.8"
- name: Set up Python environment
run: |
python -m venv venv
. venv/bin/activate
source venv/bin/activate
```
Expand All @@ -1003,12 +1004,12 @@ jobs:
- name: Run tests
run: |
. venv/bin/activate
source venv/bin/activate
pytest
- name: Generate documentation
run: |
. venv/bin/activate
source venv/bin/activate
cd docs
sphinx-build -b html . _build/html
Expand All @@ -1022,7 +1023,7 @@ jobs:
---
# Explanation of key components
1. **Container image**: The workflow runs in a container based on the `python:3.8-slim` image. This ensures that Python and any necessary system dependencies are already installed.
1. **Container image**: The workflow runs in a container based on the `python:3.8` image. This ensures that Python and any necessary system dependencies are already installed.
2. **Setup Python environment**: A virtual environment is set up within the container to isolate our project dependencies.
Expand Down
7 changes: 7 additions & 0 deletions lectures/3/c++_vs_py/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
cmake_minimum_required(VERSION 3.5)
project(example)

find_package(pybind11 REQUIRED)
include_directories(SYSTEM ${pybind11_INCLUDE_DIRS})

pybind11_add_module(matrix_ops matrix_multiplication.cpp)
34 changes: 34 additions & 0 deletions lectures/3/c++_vs_py/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import matrix_ops
import time
import numpy as np

# Create random matrices.
n = 500
mat1 = np.random.rand(n, n)
mat2 = np.random.rand(n, n)

# Benchmark C++ implementation (using pybind11).
start_time = time.time()
result_cpp = matrix_ops.matrix_multiply(mat1, mat2)
cpp_duration = time.time() - start_time
print(f"C++ implementation took {cpp_duration:.4f} seconds.")

def matrix_multiply_python(mat1, mat2):
rows = len(mat1)
cols = len(mat2[0])
inner_dim = len(mat2)

result = [[0 for _ in range(cols)] for _ in range(rows)]
for i in range(rows):
for j in range(cols):
for k in range(inner_dim):
result[i][j] += mat1[i][k] * mat2[k][j]
return result

# Benchmark Python implementation.
start_time = time.time()
result_python = matrix_multiply_python(mat1, mat2)
python_duration = time.time() - start_time
print(f"Python implementation took {python_duration:.4f} seconds.")

print(f"Speedup: {python_duration // cpp_duration}.")
31 changes: 31 additions & 0 deletions lectures/3/c++_vs_py/matrix_multiplication.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#include <pybind11/numpy.h>
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

#include <vector>

std::vector<std::vector<double>>
matrix_multiply(const std::vector<std::vector<double>> &mat1,
const std::vector<std::vector<double>> &mat2) {
const size_t rows = mat1.size();
const size_t cols = mat2[0].size();
const size_t inner_dim = mat2.size();

std::vector<std::vector<double>> result(rows, std::vector<double>(cols, 0));
for (size_t i = 0; i < rows; ++i) {
for (size_t j = 0; j < cols; ++j) {
for (size_t k = 0; k < inner_dim; ++k) {
result[i][j] += mat1[i][k] * mat2[k][j];
}
}
}

return result;
}

namespace py = pybind11;

PYBIND11_MODULE(matrix_ops, m) {
m.def("matrix_multiply", &matrix_multiply,
"A function which multiplies two NumPy matrices.");
}

0 comments on commit e9ed254

Please sign in to comment.