Per Thread Tracking

[Bootstrap360]

I am attempting to use Scalene to profile a python process that contains a computation graph.

The graph consists of data generators that are reading images from a video followed by a complex set of connected computation nodes that are a mix of classical computer vision and deep learning networks.

As far as I can tell, Scalene groups all profiles by thread so it is not possible to separate which threads are consuming the most and where to optimize, especially if the threads are sharing common code.

I propose that an additional layer be added to the stats dictionary that include the thread id.

https://github.com/plasma-umass/scalene/blob/master/scalene/scalene_profiler.py#L1199

[Bootstrap360]

The risk of this added level of depth is that if a program creates hundreds of threads, then the tree will be very wide and sparse

[Bootstrap360]

I think this mostly applies to memory tracking.

See the discussion here: #659

I think the issue is coming from the memory profiler assuming it is the main thread that is currently running and allocating memory.

https://github.com/plasma-umass/scalene/blob/master/src/source/pywhere.cpp
https://github.com/plasma-umass/scalene/blob/master/src/include/sampleheap.hpp