clean up design

[JeffBezanson]

We need to rethink our GUI API.

As a small example, the Tk package currently has types TkWidget, Tk_Widget, TTk_Widget, Canvas, TkCanvas, and Tk_CairoCanvas. I know there are reasons for these, but this is not going to work from a usability perspective.

In the work I started in tkwidget.jl, I was not really trying to provide "Tk bindings". This goes against the usual way scripting languages wrap libraries, which is to provide the same API as the original library, just callable from the scripting language. The usual approach does have advantages: (1) all functionality is exposed, (2) you can refer to existing documentation for the library, (3) people who already know the library have a shallow learning curve. This is the direction that @jverzani took things, and I greatly appreciate that work since it brought us a lot of functionality, and we will probably keep most of it.

But I would also like something else, which is a set of simple julian abstractions that would be directly meaningful to our users. Examples might include Window, Canvas, DrawingWindow, PlotWindow, Canvas3D, etc. (not all of those exist yet). It is possible this should be a new package, e.g. "GUI.jl", which can be implemented on top of Tk.jl and/or Gtk.jl. Then all my non-idiomatic-tk stuff in tkwidget.jl can be removed and the design will be less split. Or we could change Tk.jl and Gtk.jl to provide the desired API directly.

The first approach is probably more typical. But personally, I value simplicity more than features and low-level control, and the higher-level "toy" API is the only one I'd want to use. It might be nice to preserve the ability to write Tk- or Gtk- specific code, but that also risks fragmentation, e.g. people writing GUI widgets that only work with one library. So I want to start talking about how we should move forward. cc @timholy

[ViralBShah]

I prefer to have a low-level API, ideally auto generated,with one to one wrappers and a higher level API that uses those. Random and LinAlg in base take this approach and it works rather well. Shortcomings in the abstraction layer do not become showstoppers then.

[ViralBShah]

Tk.jl should ideally be only TK wrappers and the abstraction is in GUI.jl, which can have different backends including an html backend.

Pythoncard is a good example to look at. It is simple enough for non computer scientists and we use it in circuitscape.

[JeffBezanson]

I don't think much of auto-generated bindings. You get a huge pile of code that just leaves you where you started; you might as well call the c library yourself.

Would you rather write:

win = Window("main", w, h, toplevel=true)

or

win = gtk_window_new(GtkWindowType.GTK_WINDOW_TOPLEVEL)
gtk_window_set_title(win, "main")
gtk_window_set_default_size(win, w, h)
gtk_widget_show_all(win)

Personally I don't need both versions.

[ViralBShah]

I would implement the first using the second rather than sprinkling ccalls all over. I prefer to organize code that way and it makes it easier for newcomers to contribute too.

[timholy]

I don't think there's a fundamental conflict between these two visions. There's value in centralizing ccalls if for no other reason than julia's nice auto-documentation of function arguments. On the other hand, there are occasional problems that are best solved by writing a higher-level wrap directly over the C library. With regards to auto-generation, perhaps the biggest practical problem is that such bindings can be painful to load when they programmatically generate their code (which is slow) or wrap a whole pile of C code that no one is yet using. So I am something of a fan of having a low-level wrap with higher-level functionality written on top (when it doesn't lead to awkward conflicts), but I'm of two minds about auto-generation. However, I think this is closer to "details" than a first-order design issue.

On the bigger point, I heartily agree with Jeff that a GUI cleanup is needed. Tk is surely not going to be the last GUI library used in Julia; Jameson's GTK is a great example, and if that's maturing it provides a great way to find out whether we can design something that can be fairly general. Even within Tk, the split between TkWidget and Tk_Widget is particularly unfortunate, although I'm well aware of the historical reasons for it.

Jeff, to your list of targets for abstraction I would add widgets (at least the core items), events, and layout management. For me, the last causes the most concern. I attempted to start a toolkit-independent layout manager with my LP-based Layout package, but that never really got off the ground and now I suspect we should start by wrapping toolkit layout managers rather than trying to handle everything on our own. I think layout is a first-order issue---the lack of decent layout management in Matlab is one of the principal things that makes Matlab GUIs seem so crude---and that if we don't consider it from the beginning we'll get ourselves in big trouble. What makes this quite complicated is that it also interacts closely with plotting (see this mailing-list thread), and to support it properly there might be some pretty deep changes that will need to be made to Winston. I did just take a gander at GTK and Qt, and both of them implement grid layouts that seem at least superficially similar to Tk's. I suspect we should standardize on that, rather than on pack.

On a related point: I couldn't help noticing that many of the "abstractions" in base/graphics.jl seem rather tightly-coupled to Cairo's way of thinking. Cairo is currently the only low-level drawing API I know, but motivated by this conversation I took a quick glance at QPainter. By-and-large it seems pretty compatible with Cairo's drawing model, so I suspect we're in pretty good shape. Some of the names and functionality in QPainter look like useful inspiration (e.g., setPen, setRenderHint, setBackground, etc.).

[jverzani]

I wrote a package for R (gWidgets) that is meant to be an abstract layer
for writing GUIs. That package just provides an interface, the details then
are provided by other packages (Tk, Gtk, Qt, even web). In theory it works
out OK. It makes simple GUIs really easy, and moderate-sized GUIs pretty
easy. Some people have found it pretty useful.

The goal of complete portability across platforms is not that simple
though. Even with the limited vocabulary of gWidgets, simple things like
getting the layouts to be similar is difficult (the layout managers all
differ somehow). As well, it seems everyone wants to put in one more thing
which inevitably leads to using one of the toolkits lower-level commands.

Unless someone wants to spend a lot of time on this, I'd say a better
approach if a julia standard is wanted, is to pick a toolkit that can be
installed easily and stick with that. Then provide some light-weight
wrappers for basic things. It has the risk of splitting the user base, but
if the top-100 R packages is any indicator, this is going to be a
relatively small part of the julia user base. R chose tcltk for historical
reasons, and bundles in the library with its window's binaries. I would
think Gtk is a better choice. The library has much more to offer over Tk
(models in addition to better widgets, events and layout, not to mention
the canvas/Cairo plotting issues on the mac that still aren't resolved), is
more active in new developments, and as easy to install.

However, writing wrappers for the current Gtk.jl package is really a pain
(at least for mortals like me). I looked into writing something for that
package like I did for Tk and quickly got turned off by the ccall bit. In
the R world, the RGtk2 package provides two things: it autogenerates the
bindings (so gtk_window_set_title doesn't need to be done by hand) and
more importantly has a means to dispatch methods within Gtk (so rather than
call gtk_widget_show_all(win), one calls win$showAll() and RGtk2
searches over the Gtk class hierarchy to find the gtk_widget part from a
GtkWindow object. That makes things much easier to start with. (The
PyCall package does this for python modules, and this makes using Qt
through PySide, https://github.com/jverzani/PySide.jl, pretty easy.)

On Sun, Jun 16, 2013 at 7:52 AM, Tim Holy notifications@github.com wrote:

I don't think there's a fundamental conflict between these two visions.
There's value in centralizing ccalls if for no other reason than julia's
nice auto-documentation of function arguments. On the other hand, there are
occasional problems that are best solved by writing a higher-level wrap
directly over the C library. With regards to auto-generation, perhaps the
biggest practical problem is that such bindings can be painful to load when
they programmatically generate their code (which is slow) or wrap a whole
pile of C code that no one is yet using. So I am something of a fan of
having a low-level wrap with higher-level functionality written on top
(when it doesn't lead to awkward conflicts), but I'm of two minds about
auto-generation. However, I think this is closer to "details" than a
first-order design issue.

On the bigger point, I heartily agree with Jeff that a GUI cleanup is
needed. Tk is surely not going to be the last GUI library used in Julia;
Jameson's GTK is a great example, and if that's maturing it provides a
great way to find out whether we can design something that can be fairly
general. Even within Tk, the split between TkWidget and Tk_Widget is
particularly unfortunate, although I'm well aware of the historical reasons
for it.

Jeff, to your list of targets for abstraction I would add widgets (at
least the core items), events, and layout management. For me, the last
causes the most concern. I attempted to start a toolkit-independent layout
manager with my LP-based Layout package, but that never really got off
the ground and now I suspect we should start by wrapping toolkit layout
managers rather than trying to handle everything on our own. I think layout
is a first-order issue---the lack of decent layout management in Matlab is
one of the principal things that makes Matlab GUIs seem so crude---and that
if we don't consider it from the beginning we'll get ourselves in big
trouble. What makes this quite complicated is that it also interacts
closely with plotting (see this mailing-list threadhttps://groups.google.com/d/msg/julia-dev/zkZoi6y1pM8/sJgC9NIzFQAJ),
and to support it properly there might be some pretty deep changes that
will need to be made to Winston. I did jus t take a gander at GTK and Qt,
and both of them implement grid layouts that seem at least superficially
similar to Tk's. I suspect we should standardize on that, rather than on
pack.

On a related point: I couldn't help noticing that many of the
"abstractions" in base/graphics.jl seem rather tightly-coupled to Cairo's
way of thinking. Cairo is currently the only low-level drawing API I know,
but motivated by this conversation I took a quick glance at QPainterhttp://qt-project.org/doc/qt-5.0/qtgui/qpainter.html.
By-and-large it seems pretty compatible with Cairo's drawing model, so I
suspect we're in pretty good shape. Some of the names and functionality in
QPainter look like useful inspiration (e.g., setPen, setRenderHint,
setBackground, etc.).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/35#issuecomment-19511339
.

John Verzani
Chair, Department of Mathematics
College of Staten Island, CUNY
verzani@math.csi.cuny.edu

[ViralBShah]

@jverzani You should probably register PySide into METADATA.

[timholy]

@jverzani, that's valuable perspective; I didn't fully appreciate the depth of your experience with supporting graphics toolkits in various languages.

I agree that there is a lot of merit in picking one and just using that, rather than trying to support many toolkits. Gtk, while being a step above Tk, worries me with regards to mobile (I have little doubt that someone will port julia to Android within a couple of years), and the momentum behind Gtk does not seem to be what it once was. So if we're going to pick just one and hope it's a great long-term solution, to me it seems to be basically a no-brainer that the choice should be between Qt and HTML5/web. What's held me back is that I'm put off by the barrier of generating a Qt wrapper (I definitely don't have time to start one myself), and I'm worried about the performance/interactivity of web. That, and the small matter that I don't know either one of them (as a programmer) myself.

Thanks for the link to your PySide, until then I hadn't taken the time even to understand what PySide referred to.

[jverzani]

Qt is much preferable to Gtk, assuming someone pushes past that barrier of
writing a wrapper and can get it to work under Windows (the qtbase package
for R still isn't working there last I checked). There is also the issue of
needing to subclass to handle some events.

On Sun, Jun 16, 2013 at 11:40 AM, Tim Holy notifications@github.com wrote:

@jverzani https://github.com/jverzani, that's valuable perspective; I
didn't fully appreciate the depth of your experience with supporting
graphics toolkits in various languages.

I agree that there is a lot of merit in picking one and just using that,
rather than trying to support many toolkits. Gtk, while being a step above
Tk, worries me with regards to mobile (I have little doubt that someone
will port julia to Android within a couple of years), and the momentum
behind Gtk does not seem to be what it once was. So if we're going to pick
just one and hope it's a great long-term solution, to me it seems to be
basically a no-brainer that the choice should be between Qt and HTML5/web.
What's held me back is that I'm put off by the barrier of generating a Qt
wrapper (I definitely don't have time to start one myself), and I'm worried
about the performance/interactivity of web. That, and the small matter that
I don't know either one of them (as a programmer) myself.

Thanks for the link to your PySide, until then I hadn't taken the time
even to understand what PySide referred to.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/35#issuecomment-19513936
.

John Verzani
Chair, Department of Mathematics
College of Staten Island, CUNY
verzani@math.csi.cuny.edu

[timholy]

Can you elaborate on these difficulties?

[timholy]

This conversation sent niggles off in my head that a little googling helped resolve; it turns out you're not the first to urge us to pick one and then run with it. At the time I was a main proponent of an abstraction layer, so this was basically an argument against my point of view :-).

[jverzani]

Unless Viral figured it out, there is the issue of upside down plots or oddities when resizing plots.

John Verzani
Chair, Dept. of Mathematics
CUNY/CSI
verzani@math.csi.cuny.edu

On Jun 16, 2013, at 1:01 PM, Tim Holy notifications@github.com wrote:

Can you elaborate on these difficulties?

—
Reply to this email directly or view it on GitHub.

[timholy]

Sorry, I meant the obstacles facing R for Qt support:

• getting qtbase working for Windows (I thought cross-platform was supposedly a great strength of Qt??). I found this link, it doesn't sound as if there is expected to be any fundamental issue however
• event-handling subclassing issues (big problem or little one?)

[JeffBezanson]

Tim, it's true that Graphics is tied to Cairo. I decided this was ok for a few reasons: Cairo's rendering model is basically the SVG model, and so is very generic already (moveto, lineto, etc.) Cairo also has good support for every backend out there, so it's pretty unlikely you'd need something else for vector graphics. If you did, most of its API could probably be implemented. I also selected a pretty small subset.

Between Jameson and me we'll be able to wrangle the needed ccalls for Gtk. Winston I think should be pulled apart into pieces and slowly put back together, so that will not be a barrier.

Thanks to that link to Peter Wang's post, I'm reminded to focus on use cases. For example, if somebody wants to do serious Android development in julia they will need to create Android.jl, and we probably can't worry about all such possibilities up front. I think my primary concern is exploratory (repl-style) use and one-off applications (small GUIs that help you solve a particular problem). On the implementation side my concern is simple generic APIs like moveto(gc,x,y), so that you have something to feel "safe" writing code against. For example people can write print(io,"text") without a care in the world about what it can or can't print to, and graphics should be the same to the extent possible.

canvas3d.jl has been my working example. It uses only Canvas, Window, MouseHandler with 3 events, moveto, lineto, fill, and stroke, and yet is pretty cool. If, as Tim says, we added a few widgets, layout, and better winston integration to that set we'd be in good shape. I'd like to incorporate things like Tim's ImageCanvas as well.

If Gtk solves our install and display problems I think that will become the preferred back end for this layer in the short term. If we really want Qt, I think it makes sense just to write the layer we need in C++. If somebody really wants full Qt bindings, I consider that a different problem.

As for this issue I think we should get rid of tkwidget.jl so Tk.jl can be the best Tk bindings it can be. The time may have come to admit defeat on tk_wrapper.c and get rid of it as well.

[timholy]

If we're first aiming for "simple" GUIs, then certainly Tk or Gtk will suffice, as long as the cross-platform issues work out satisfactorily. (Heck, on Linux Tk+Cairo is already much better than Matlab, so I don't think our needs here are particularly demanding.) And I love the idea of having a "safe" drawing API. So no complaints there.

If you want to merge anything from ImageView into a general graphics layer, I'm all for it. I've long known that rubberband.jl needs to go somewhere more general, but I was torn between adding it to Tk and making the necessary generalizations to base/graphics.jl to incorporate it there.

With regards to choosing an underlying toolkit, I do think it may be more than a bit shortsighted not to take "eventual support for mobile" into fairly serious consideration. Of course, since I'm reluctant to be the pioneer on an HTML5/Qt wrapper, this is largely academic.

On the issue of Tk, yes, I agree that at this point tkwidget.jl should be unified into the rest of the Tk wrapper.

[JeffBezanson]

I think the hard part is really the generic API design. For example on another thread Jameson and I are trying to hash out the meanings of draw, reveal, configure, and (possibly) redraw. Once we figure out the "right thing" and have detailed specs, I think it is relatively easy to port to different backends.

Your original concept of a generic layout manager would be nice to have. We could use it for graphical elements in a pure vector-rendering context with no UI, and also for widgets by using a toolkit's absolute placement as a backend. This also reduces the requirements on new backends; you just need x, y, width, height. Winston has a small amount of this kind of functionality, and it would be nice to abstract it out.

[timholy]

I think the hard part is really the generic API design.

Agreed. There have now been two people experienced with multiple toolkits in high-level languages who have said this may be sufficiently hard that it may not be worth it, and that we should just adopt one toolkit. That said, there's still a big part of me that's sympathetic to the notion of a generic layer for simple things, and thinks (perhaps foolishly) that for simple things an abstraction layer is achievable and worthwhile.

Your original concept of a generic layout manager would be nice to have.

You have it already :-). I got the Layout package to the point where it seemed to work at least for simple tasks, with Julia that was current at the time. The part that was missing was integrating it into Winston etc. See https://github.com/timholy/Layout.jl/blob/master/test/simple.jl for an API demo that should take you about 2 minutes to understand, if you reference the image example linked above (and also from that file).

It depends on SymbolicLP, which has recently been deleted from METADATA in preference for MathProg. However, the repository still exists in my github account, and if you decide you like Layout but would rather base on MathProg that should (I imagine) be quite straightforward.

[ViralBShah]

@jverzani The upside down plot oddity is only a mac issue, I believe, because the co-ordinates have to be transformed, and not all code-paths end up transforming. At casual glance, I was not able to fix it. The code is a bit all over the place, and I am not familiar with Tk - so I need to find some time to go through it patiently.

[ViralBShah]

@rsrock just fixed some of the display issues, and the OS X inversion is described by @loladiro in JuliaGraphics/Winston.jl#55 - just need to figure out how to fix it. GTK may still be better in other aspects, but the current Tk visual artefacts definitely seem addressable. Also, the fact that Tk is small is nice to have it for minimal systems.

[jverzani]

@timholy On qt issues with windows: I was premature on this. R's package is not available as a binary, but there are instructions for installation that would look familiar to any command-line user (http://cran.r-project.org/web/packages/qtbase/INSTALL). As for subclassing, the qtbase program does it here https://github.com/ggobi/qtbase/blob/master/R/class.R . I can't really comment if that is a big problem or a little one except that it would be huge for me.