Performance between pure python and python/rust with pyo3

[mancioshell]

Hi,
I'm testing two microservices which exposes a rest api with fastapi and performs some heavy aggregation query on mongodb.
The first microservice is written in pure python, the second one has the same fastapi route, but the code which performs the query is written in rust and exposed to the asyncio loop through pyo3 and pyo3_async.

This is the pyo3 code which expose the function:

#![recursion_limit = "1024"]

use pyo3::{prelude::*, wrap_pyfunction};
mod models;
mod repositories;

use repositories::namespace_repository::NamespaceRepository;
use repositories::report_repository::ReportRepository;

use models::aggregation_model::{
    AggregationType, DayTimeWindow, MonthTimeWindow, QuarterTimeWindow, WeekTimeWindow,
};

use models::secret::Secret;

use futures::future::join_all;
use strum::IntoEnumIterator;

#[pyfunction]
fn update_report_view(
    py: Python,
    secret: Secret,
    account: String,
    aggregation_type: AggregationType,
) -> PyResult<Bound<PyAny>> {
    let aggregation_window_list: Vec<String>;

    aggregation_window_list = match aggregation_type {
        AggregationType::Day => DayTimeWindow::iter().map(|item| item.to_string()).collect(),
        AggregationType::Week => WeekTimeWindow::iter()
            .map(|item| item.to_string())
            .collect(),
        AggregationType::Month => MonthTimeWindow::iter()
            .map(|item| item.to_string())
            .collect(),
        AggregationType::Quarter => QuarterTimeWindow::iter()
            .map(|item| item.to_string())
            .collect(),
    };

    pyo3_asyncio_0_21::tokio::future_into_py(py, async move {
        let namespace_repository = NamespaceRepository::init(&secret).await;       
        let report_repository = ReportRepository::init(&secret).await;

        let namespaces: Vec<String> = namespace_repository.get_namespaces(&account).await.unwrap();

        let mut tasks = Vec::new();

        for aggregation_window in aggregation_window_list {
            for namespace in namespaces.iter() {
                let task = report_repository.update_report(
                    &account,
                    namespace,
                    aggregation_type,
                    aggregation_window.to_string(),
                );
                tasks.push(task);
            }
        }

        let _results = join_all(tasks).await;

        Ok(())
    })
    
}

#[pymodule]
fn metrics_aggregator(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add_class::<Secret>()?;
    m.add_class::<AggregationType>()?;
    m.add_function(wrap_pyfunction!(update_report_view, m)?)?;
    Ok(())
}

This is the python code in common between the two microservices.

from metrics_aggregator import update_report_view, AggregationType, Secret

async def aggregate(
        self, secret: Secret, account: str, aggregation_type: AggregationType
    ) -> None:
        """
        Perform aggregation query

        :param client: An injected Database Session
        :raises PersistenceException: if there is any error during interaction with CustomerRepository
        :return: returns Nothing
        """

        ### Pure Python

        # namespaces = await get_namespaces_by_account(
        #     secret, account
        # )

       #async with anyio.create_task_group() as tg:
       #     for namespace in namespaces:
       #             tg.start_soon(
       #                 self.aggregation_repository.update_report_view,
       #                 client,
       #                 account,
       #                 namespace,
       #                 aggregation_type,
       #             )
       ### Pure Python

        self._logger.info(
            f"Aggregation for account {account} has been successfully started"
        )

        ### Hybrid Python/Rust

        await update_report_view(
            secret,
            account,
            aggregation_type
        )        

        ### Hybrid Python/Rust

        self._logger.info(
            f"Aggregation for account {account} has been successfully completed"
        )

        return None

The code of python Repository Class and Rust Repository struct is pretty the same. He only takes a pipeline and perform an aggregagation query which write in a materialized view.

I'm struggle to understand why i haven't any performance enhancement. I have benchmarked with jmeter the microservices and the performance is almost the same.

Is it pyo3 not very usefull for io bound tasks?

Any help is appreciated.

[mancioshell]

In addition, if i stress the hybrid python/rust microservice, i received an OOMKilled (exit code 137).

[davidhewitt]

Are you able to share this as a runnable repro in some form? Perhaps I can then investigate.

[mancioshell]

@davidhewitt thank you for your reply. I can reproduce the issue in 2 repos. One for python and one for python/rust, but unfortunately i can't share the real mongodb collection. Maybe i can give you the structure of the documents. Is It ok?

[mancioshell]

@davidhewitt
This is the repo with a minimal example of the 2 repo (pure python and python/rust with pyo3).
Obviously, I simplified the aggregation query and also prepared a very limited data set in the ingestion part, compared to the real use case.

[davidhewitt]

Thanks, I will try to profile soon. Are you "stress testing" by repeatedly making the curl command you list in the README?

[mancioshell]

Yes, i have used jmeter to stress test the microservices in a k8s cluster, where i have limited cpu request to 0.1 (0.3 to limit) and memory request to 0.2Gi (0.5Gi to limit).
I was expected to have a performance enhancement in the python/rust one, but instead i have received an OOM Killed to the python/rust one, and no error from the python one.

The main questions are:

1. have i made any mistakes in the usage of py03?
2. maybe an IO/Bound use case like this, is not the best use case for enhance performance / reduce resources using py03?

Feel free to ask me any questions / clarifications.

Thank you in advance.