Support interactive graphics in python scripts #17587

linev · 2025-01-31T15:09:32Z

Currently it is problematic to use simple canvas (web or not-web) in non-interactive python.
Either macro exit very fast or created canvas is not possible to use interactively.

I propose to provide pythonization of TCanvas.Update() method which can block macro execution and wait until
user press <space> key. While python waits for key press ROOT event loop continues to run and interactive canvas is fully functional.

There is additional block parameter of canvas.Update() now which is False by default.
Without extra arguments it behaves as normal ROOT. Only when block = True, method blocks until key pressed.

Simple example of usage:

import ROOT
ROOT.gROOT.SetWebDisplay("chrome")

c = ROOT.TCanvas()

h = ROOT.TH1I("h1", "h1", 100, -5, 5)
h.FillRandom("gaus", 10000)
h.Draw()

# block here until space is pressed
c.Update(True)

# continue macro execution

In batch mode or in python notebooks blocking will not be performed.

Tested on:

Linux, Mac and Windows,
interactive/plain python mode,
web/normal graphics

The only difference between Windows and other platforms - in interactive python mode one
cannot use special hook to perform periodic actions while python prompt is waiting for next user command.
Therefore on Windows to work with canvas one always have to invoke c.Update(True)

This PR resolves #14943 and #13744

github-actions · 2025-02-01T02:20:28Z

Test Results

18 files 18 suites 4d 3h 52m 28s ⏱️
2 690 tests 2 687 ✅ 0 💤 3 ❌
46 722 runs 46 719 ✅ 0 💤 3 ❌

For more details on these failures, see this check.

Results for commit 0c77f7c.

♻️ This comment has been updated with latest results.

bellenot · 2025-02-03T07:05:05Z

@linev and what about the browser?

linev · 2025-02-03T08:44:00Z

what about the browser?

If approach works with canvas, one can provide similar solution for RBrowser and other widgets.

linev · 2025-02-03T18:45:10Z

@vepadulano @dpiparo

Can you check PR?
It finally allows to use graphics in python.
One may apply same approach also to TCanvas::Draw() method.

vepadulano · 2025-02-06T14:07:30Z

@linev thank you for this PR! I believe it is improving a situation for a specific use case of our Python interface, that's good! Let me summarise my findings from a few quick tests I ran locally.

The modified canvas.Update function allows to avoid having to specify the -i flag when running python -i myscript.py. As a side-note, this is more similar to what a script using matplotilib would do by default, with the extra functionality that the application can resume after the space key has been pressed. matplotlib by default blocks execution and you can just exit the Python process by pressing CTRL-C.
Probably we want to have a similar functionality for canvas.Draw, because that's the first thing a user would do when writing their plotting script, it always starts with a Draw call, not with Update.
This is a backwards-incompatible change in general. I could come up at least with one simple use case that would be broken by this change. That is, a live plot being updated. Consider this simple example which fills the contents of a histogram in a loop and refreshes the plot at every iteration:

import ROOT

c = ROOT.TCanvas()
h = ROOT.TH1D("h","h",100,-10,10)
h.Draw()
c.Draw()
c.Update()
ROOT.gSystem.ProcessEvents()

for _ in range(1000):
    h.FillRandom("gaus",100)
    h.Draw()
    c.Update()
    ROOT.gSystem.ProcessEvents()

which will be broken with the current changes. As discussed offline, this can be accommodated by adding a Python-only
optional argument to Update, Draw and anything else that will be modified, which switches to the blocking behaviour,
leaving by default the current behaviour which does not block.

Let me note that this example I came up with is just one, there could be many more. I wish there was a better way we could expedite this kind of testing. Notably, I have tried these changes at a Python prompt, at a IPython prompt and in a Jupyter notebook. Everything seems to work as intended there, but having to do this manually is unfortunate.

linev · 2025-02-06T15:39:18Z

@vepadulano

I add extra block argument to canvas.Update(block = False) pythonization.
By default it behaves as normal ROOT - means your macro will work as before.
Only when called canvas.Update(True) script execution will be blocked until <space> button is pressed

vepadulano

Thanks for the update, I left some comments. I still believe the Draw method sees more usage than Update.

vepadulano · 2025-02-07T08:50:24Z

bindings/pyroot/pythonizations/python/ROOT/_pythonization/_tcanvas.py

+   First invoke normal canvas Update.
+   If block == True specified, run ROOT event loop until <space> key is pressed
+   Event loop does not processed in batch mode or in ipython


Suggested change

First invoke normal canvas Update.

If block == True specified, run ROOT event loop until <space> key is pressed

Event loop does not processed in batch mode or in ipython

Updates the canvas. Also blocks script execution and runs the ROOT graphics event loop until the <space> keyword is pressed, but only if the following conditions are met:

* The `block` optional argument is set to `True`.

* ROOT graphics are enabled, i.e. `ROOT.gROOT.IsBatch() == False`.

* The script is being run in non-interactive mode, i.e. `python myscript.py`.

vepadulano · 2025-02-07T08:52:16Z

bindings/pyroot/pythonizations/python/ROOT/__init__.py

+# import atexit

+# def cleanup():
+#    # If spawned, stop thread which processes ROOT events
+#    facade = sys.modules[__name__]
+#    if "app" in facade.__dict__ and hasattr(facade.__dict__["app"], "process_root_events"):
+#        facade.__dict__["app"].keep_polling = False
+#        facade.__dict__["app"].process_root_events.join()

-def cleanup():
-    # If spawned, stop thread which processes ROOT events
-    facade = sys.modules[__name__]
-    if "app" in facade.__dict__ and hasattr(facade.__dict__["app"], "process_root_events"):
-        facade.__dict__["app"].keep_polling = False
-        facade.__dict__["app"].process_root_events.join()
-
-atexit.register(cleanup)
+# atexit.register(cleanup)


Please remove all the commented code

vepadulano · 2025-02-07T08:52:25Z

bindings/pyroot/pythonizations/python/ROOT/_application.py

-            def _process_root_events(self):
-                while self.keep_polling:
-                    gSystem.ProcessEvents()
-                    time.sleep(0.01)
-            import threading
-            self.keep_polling = True # Used to shut down the thread safely at teardown time
-            update_thread = threading.Thread(None, _process_root_events, None, (self,))
-            self.process_root_events = update_thread # The thread is joined at teardown time
-            update_thread.daemon = True
-            update_thread.start()
+            # def _process_root_events(self):
+            #     while self.keep_polling:
+            #         gSystem.ProcessEvents()
+            #         time.sleep(0.01)
+            # import threading
+            # self.keep_polling = True # Used to shut down the thread safely at teardown time
+            # update_thread = threading.Thread(None, _process_root_events, None, (self,))
+            # self.process_root_events = update_thread # The thread is joined at teardown time
+            # update_thread.daemon = True
+            # update_thread.start()


Also here please remove commented code

vepadulano · 2025-02-07T08:54:36Z

bindings/pyroot/pythonizations/python/ROOT/_pythonization/_tcanvas.py

+from . import pythonization
+


We need the usual copyright header here and I would also add some documentation that will end up in the Python Interface section of the docs https://root.cern.ch/doc/master/group__Pythonizations.html. Take for example https://github.com/root-project/root/blob/master/bindings/pyroot/pythonizations/python/ROOT/_pythonization/_tfile.py

@vepadulano

I apply your changes and rebase commits

We are still missing the documentation as I suggested, see the \pythondoc block in _tfile.py. I think it should describe the fact that the Update method has been pythonized and a simple example usage.

If extra `block` flag specified, python script execution is blocked until <space> key is pressed. At the same time ROOT events loop continue running. Solves problem of ROOT graphics usage in non-interactive python

While now events processing has to be done in the canvas.Update method, additional processing is not necessary. Such approach was not thread safe and did not work on Windows at all

It is interrupt flag, which can be triggered from canvas context menu. Has similar effect as prssing space button

vepadulano · 2025-02-07T09:08:46Z

bindings/pyroot/pythonizations/python/ROOT/_pythonization/_tcanvas.py

+from . import pythonization
+


We are still missing the documentation as I suggested, see the \pythondoc block in _tfile.py. I think it should describe the fact that the Update method has been pythonized and a simple example usage.

linev · 2025-02-07T09:15:46Z

Yes, I add small docu and example macro

linev added affects:master in:Graphics in:Python Interface labels Jan 31, 2025

linev self-assigned this Jan 31, 2025

linev requested review from bellenot, dpiparo and vepadulano as code owners January 31, 2025 15:09

ferdymercury linked an issue Jan 31, 2025 that may be closed by this pull request

[PyROOT, Graphics] Cannot pause pyROOT script with input() and see canvases #13744

Open

1 task

linev force-pushed the python_draw branch from 459c60a to fb66cc8 Compare February 6, 2025 15:33

vepadulano requested changes Feb 7, 2025

View reviewed changes

linev added 4 commits February 7, 2025 10:00

[pyroot] introduce TCanvas.Update pythonization

2da34c2

If extra `block` flag specified, python script execution is blocked until <space> key is pressed. At the same time ROOT events loop continue running. Solves problem of ROOT graphics usage in non-interactive python

[pyroot] do not create extra thread for events processing

ffadc03

While now events processing has to be done in the canvas.Update method, additional processing is not necessary. Such approach was not thread safe and did not work on Windows at all

Do not use PyOS_InputHook on windows

fc5e61b

[pyroot] handle ProcessEvents() return value

798d7da

It is interrupt flag, which can be triggered from canvas context menu. Has similar effect as prssing space button

linev force-pushed the python_draw branch from fb66cc8 to 798d7da Compare February 7, 2025 09:05

linev requested a review from vepadulano February 7, 2025 09:06

vepadulano requested changes Feb 7, 2025

View reviewed changes

Add documentation for TCanvas.Update pythonization

0c77f7c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support interactive graphics in python scripts #17587

Support interactive graphics in python scripts #17587

linev commented Jan 31, 2025 •

edited

Loading

github-actions bot commented Feb 1, 2025 •

edited

Loading

bellenot commented Feb 3, 2025

linev commented Feb 3, 2025

linev commented Feb 3, 2025

vepadulano commented Feb 6, 2025

linev commented Feb 6, 2025

vepadulano left a comment

vepadulano Feb 7, 2025

vepadulano Feb 7, 2025

vepadulano Feb 7, 2025

vepadulano Feb 7, 2025

linev Feb 7, 2025

vepadulano Feb 7, 2025

vepadulano Feb 7, 2025

linev commented Feb 7, 2025

-   First invoke normal canvas Update.
-   If block == True specified, run ROOT event loop until <space> key is pressed
-   Event loop does not processed in batch mode or in ipython
+   Updates the canvas. Also blocks script execution and runs the ROOT graphics event loop until the <space> keyword is pressed, but only if the following conditions are met:
+   * The `block` optional argument is set to `True`.
+   * ROOT graphics are enabled, i.e. `ROOT.gROOT.IsBatch() == False`.
+   * The script is being run in non-interactive mode, i.e. `python myscript.py`.

Support interactive graphics in python scripts #17587

Are you sure you want to change the base?

Support interactive graphics in python scripts #17587

Conversation

linev commented Jan 31, 2025 • edited Loading

github-actions bot commented Feb 1, 2025 • edited Loading

Test Results

bellenot commented Feb 3, 2025

linev commented Feb 3, 2025

linev commented Feb 3, 2025

vepadulano commented Feb 6, 2025

linev commented Feb 6, 2025

vepadulano left a comment

Choose a reason for hiding this comment

vepadulano Feb 7, 2025

Choose a reason for hiding this comment

vepadulano Feb 7, 2025

Choose a reason for hiding this comment

vepadulano Feb 7, 2025

Choose a reason for hiding this comment

vepadulano Feb 7, 2025

Choose a reason for hiding this comment

linev Feb 7, 2025

Choose a reason for hiding this comment

vepadulano Feb 7, 2025

Choose a reason for hiding this comment

vepadulano Feb 7, 2025

Choose a reason for hiding this comment

linev commented Feb 7, 2025

linev commented Jan 31, 2025 •

edited

Loading

github-actions bot commented Feb 1, 2025 •

edited

Loading