Processing 4

Some of these items have been started, or are covered in more detail on the Project List page.

Changes under the hood

• Move to Java 11
  • Some of this work is in progress
  • If we move to Java 11, we will drop support for Java 8. We have no resources to do backwards compatibility, and need to make things less complicated.
  • We're now using Java 11, and in September 2021, we'll use Java 17.
• Drop 32-bit support
  • People who need 32-bit support can use Processing 3.
  • This is a bummer for the ARM side of things, where most devices are still 32-bit.
  • The build process is currently 64-bit only, and there are no plans to change that.
• Change base platforms that are supported/tested
  • Windows 10 and later – support for Windows 7 ended in 2018
  • macOS 10.13 or 10.14 and later – Apple is upgrading once a year, we're forced to chase them. Major OS releases are supported just 3 years, we shouldn't (can't, for lack of time) be extending beyond that.
  • Ubuntu 18.04 and later

Renderers

• Switch OpenGL from JOGL to LWJGL
  • The JOGL project hasn't been receiving updates lately, but LWJGL is quite active.
  • This work has been started in a branch, but needs to be completed.
  • It may also make sense to move OpenGL back out of core as a result.
• Move JavaFX out of core and into a Library
  • Oracle has removed it from the JDK, so it's now a heavyweight addition.
  • This would simply mean using Import Library > JavaFX in sketches.
  • Our intent was to eventually replace the “default” (Java2D) renderer with JavaFX as the default. However, given the quirks of JavaFX and lack of time, we've never gotten there. With Oracle kicking JavaFX out of the main distribution, its future is also muddy, so we probably won't be able to rely upon it this way.

New API

• Fixed aspect ratio option/“designed” width/height
  • Something like aspect(1920, 1080), which would maintain that aspect ratio, no matter how the window might be resized. Mouse coordinates will be scaled to the original code, while aspect ratio is maintained even as the sketch window size is changed. I use this on a regular basis in my own work, but don't have a good idea for how to name it properly, otherwise we'd include it now.
• Touch Events
  • High-level events for swipes, taps, etc
• Better handling of surface
  • The surface variable was introduced in 3.x, however we had no idea quite how many people were using those features in their sketches. Because it's so common, we need a better API that doesn't require use of the surface object. (And for more testing across renderers to ensure that those methods work, which is the major reason they live in surface as an advanced feature.)

Rebuilding the PDE

• Moving to a Language Server implementation for the preprocessor, compiler, etc.
  • This would allow Processing to work with other editors, reducing the stress on the PDE needing to support every last detail that advanced users seek.
  • This requires a full refactor of Java Mode, to separate the UI and code handling functions.
  • More discussion can be seen here.
• Rebuild the preprocessor
  • It's fragile and newer Java language features are not supported.
• Use a new Editor text area component
  • Or perhaps more likely, switch to HTML/CSS/JS (see below) for the PDE.
• Remove requirement for the strict sketch folder layout
  • Make the sketch.properties file ever-present, and don't require the main .pde file to share the sketch name.
  • This comes from a shift in priorities since we first started. When we began, we didn't even have multiple tabs, and it was common to have single .pde files for entire sketches. We didn't want to make things needlessly complex by adding another file that just pointed to the first file. Nowadays, many/most sketches have multiple tabs, and folks don't mind project files all that much.

Other nuts and bolts

• Redo JSON implementation
  • Working with JSON in Java is a huge headache. We can do better.
• New vector class
  • PVector is optimized in ways that people don’t really use, making it more confusing than necessary
• Better handling of pixel vs. non-pixel surfaces
  • We started at a time when Processing was about high-performance ways to work with pixel data from surfaces. Nowadays, those features make Processing slower than necessary.
  • We have a hodgepodge of workarounds like loadPixels() and hidden API calls in OpenGL for sending video to textures, but these need to be ironed out better.
• Mutable/immutable versions of the “data” classes
  • Table was built to be mostly immutable, but that's not the way a lot of people use it. Splitting into separate versions would help performance and make behavior clearer.
• Improve or replace PShape
  • PShape has grown very complex and should probably be handled differently.
  • When started, we were looking for a lightweight (a few Kb) way to do SVG. Nowadays, size footprint isn't much of an issue, so it might be better to get improved support by using something like Batik underneath.
• Move more of the utility functions in PApplet into a PUtil class that doesn't require a PApplet instance. I often wind up building quasi versions of this in professional work, which undermines the point of reusable code.
• Reworking parts of the processing.data classes for concurrency (one of the primary reasons to use Java for data handling work).

More fun, but even more work

• HTML/CSS/JavaScript version of the PDE
  • Having refactored to a Language Server setup, it would be nice to get rid of the Swing-based PDE and rewrite it as a web UI (i.e. an Electron app) that speaks the language server protocol. It's been incredibly difficult to find people to contribute to fixing the Editor, and this would open it up to more contributors.
• Automatic version control of sketches
  • Some sort of semi-automated way of running Git behind the scenes for sketches, but without requiring people to learn Git. This would harken back to the “History” feature we started with in 2001, but never finally implemented.
• Sketchbook sync to Box or S3 or Dropbox or…
• Improving the startup and examples experience for the PDE
  • Have an easier way of getting started—a useful welcome page that also covers recent updates
• Implement a better way to browse examples
  • We have hundreds of examples and they're central to how people learn. They need to be easier to introduce, navigate, and encourage modification.
• Focus on large-scale, full screen work
  • With many smaller projects using p5.js, the desktop version could instead focus on more advanced use.
  • This might also involve redoing the examples to use a larger area, or at least making them more adjustable.
  • Working from 4K and larger displays
• Generally speaking, covering the split between what can't be done with JS (performance or implementation-wise) and what's still difficult to do with C++ and others.
• Better than 8-bit color… This is a huge shift, but things are moving quickly in this area, and you can make some really, really beautiful things with HDR, 10-bit color, etc.
• Create paid app store versions of Processing to support the Foundation
  • Build a nice version of the software that's available from the Mac App Store or Windows Store. Using these releases would be optional, but provide a subscription-based way for people to support the project.
  • Multiple projects have already released paid, closed-source software that use our API and approach (even to the point of copying our documentation), usually without attribution. It'd be nice to at least have a venue for paid support of the project outside of asking for donations.
  • This would automate update handling.
  • We could also add additional features (i.e. the sketchbook sync features mentioned earlier) to this version to encourage folks to use it.

Goodbye Java?

Sun has never been supportive of our project in spite of the large number of users we bring to the platform. And they've never made up their mind about how much they do/do not care about Java on the desktop. In recent years, it's been even more frustrating watching Oracle make a mess of Java in general.

Unfortunately, there's no obvious language to move to: JavaScript gives us the web, but isn't nearly as performant (e.g. working with a lot of data) or mature (e.g. working with threads, etc) as Java in many places. Python is terrific, but there aren't a lot of readily available pieces in place for doing interactive graphics (i.e. a well-supported, complete OpenGL implementation). When we started in 2001, Java was the best of both worlds (well sorta) because we had decent performance (a lot better than JS or ActionScript) and could easily distribute over the web (with applets).

A few approaches, sometimes with pros and cons:

• A full port of Processing to JavaScript/Node/etc
  • p5.js is not a port—nor should it be
  • …though done the wrong way, this could be a lot of duplicate effort
  • Would be nice to be able to port over existing work in a more line-by-line way
  • Sharing via the web is hugely important
• Porting the entire library to Python
  • The Jython implementation is great, but it'd be awesome to have available NumPy and all those other things that make Python wonderful.
• Implement things on Apple's Swift
  • Excellent performance, works on iOS, macOS, watchOS tvOS… but Apple's rules are antithetical to many of our core values. App store approval is a gateway to distribution, everyone must use Xcode, etc.
• Rust seems excellent, providing C/C++ level performance without… making everyone learn C++.