Mockup and resource registry redone

[thet]

PLIP (Plone Improvement Proposal)

Responsible Persons

Proposer: Johannes Raggam

Seconder: Maik Derstappen

Abstract

This PLIP is about reworking Mockup to be based on up-to-date JavaScript standards and to simplify the resource registry.

Mockup will be based on Patternslib version 4 and integrate many concepts from there:

• ES6 module imports instead of RequireJS.
• ES6+ compliant code base.
• Dynamic module loading for 3rd party dependencies.
• Webpack as build system instead of Grunt.
• Jest as test framework.
• Markdown based documentation.

The simplification of the resource registry is largely based on the removal of RequireJS:

• No through-the-web compilation via RequireJS and LESS.
• Bundle registration is kept but resources will be removed as there is no need for them anymore.
• Removing unnecessary attributes from the bundle registration.
• Improvements to the caching URLs.
• Reduction of number of default bundles.

Mockup and the resource registry should get much easier to understand and to work with, more up-to-date and match developer expectations better.

Additionally we want to advertise the development of new JavaScript functionality as web components but any JavaScript framework should be compatible with Plone.

Motivation

Mockup was once and avant garde answer to the growing problem of managing JavaScript in Plone. Based on Patternslib it offered clear and defined dependencies via RequireJS, a JavaScript testing framework and a flexible build system incorporating the resource registry with through the web compilation and the possiblity to overwrite specific resources with a custom implementation. Other technologies like web components or reactive JavaScript frameworks were not available or not an option back then.

Now, Mockup lags years behind current JavaScript development. It's no big fun to develop for Mockup (although there are nice parts, like live reloading in Plone without having to build bundles). Mockup depends on some long outdated and inactive libraries - the most important here is RequireJS. RequireJS was the corner stone to solve the dependency problem with JavaScript in Plone 4. But meanwhile it prevents us from using modern JavaScript features. There are also other outdated canditates: the complex integration of the Grunt build framework, the use of React 0.10 for our documentation framework, the integration of the unmaintained jquery.drag/drop which holds us on jQuery 1.x (there is a seperate PR for fixing this) - and so on.

The resource registry is also complex and hard to understand. The through the web compilation (actully client side compilation on the web browser) works like magic but creates a hard dependency on RequireJS and a specific version of the LESS compiler. This bounds us to a complex development workflow and prevents developers to make their own choices regarding JavaScript libraries, ECMAScript versions and build frameworks.

For good reasons we Plone community failed get developers on board helping out in the JavaScript area of Plone.

This PLIP aims to fix this situation by a number of actions outlined below.

Assumptions

1. In Plone 6, we still use Mockup and the resource registry.

Proposal & Implementation

Mockup will be completely restructured and based on the concepts of Patternslib 4:

• We base the whole Mockup code on modern JavaScript ES6+.

• We completly remove the dependency on RequireJS and use the ES6 module exports/import exclusively (this also removes the through-the-web compilation feature in CMFPlone).

• We use dynamic imports (lazy loading) for 3rd party dependencies like TinyMCE or Select2. This dramatically reduces the size of generated bundles as no external dependencies other than the required minimum (Mockup, Patternslib core, jQuery, underscore) are loaded. All dynamically imported dependencies are compiled by Webpack to a chunks folder.

• We put all patterns (except some, like the resource registry or model editor) into one bundle. Due to the lazy loading approach of 3rd party dependencies we can include all useful patterns into the bundle without significantly increasing the bundle size.

• We remove the customized Grunt build system.

• For building JavaScript bundles we use Webpack.

• We use BabelJS for transpiling ES6+ code to target older browsers.

• We use a seperate polyfill package for IE11, which is loaded optionally via browser detection (and remove this as soons as IE11 is not relevant any more).

• For documentation we use Markdown files and compile them with the 11ty static site generator.

• We remove our self written documentation framework which is based on the outdated React 0.10 library.

• For testing we use Jest, which itself uses jsdom instead of a browser backend. jsdom is a JavaScript implementation of the document object model and allows for much faster test runs than testing against PhantomJS.

• PhantomJS, Karma, Mocha, Chai will be removed.

• We apply consistent code formating rules throughout the whole Mockup code base, based on prettier and eslint, using a indentation of 4 spaces.

• Optional: Where possible we want to use Patterns from Patternslib instead of Mockup.
  Patternslib is better maintained and each pattern follows a strict specification defined in a design phase.
  Patternslib patterns do not use a JSON configuration syntax but a simpler CSS like syntax.
  Wether we subclass Patternslib patterns, adapt the configuration and change the parser or we start using also the Patternslib configuration syntax in Plone.
  However, this would probably also include some markup changes (an example would be the switch from the date/time picker pat-pickadate to pat-datetime-picker).
  This is an optional task and would only be done after all existing patterns from Mockup are migrated and the test suite is green.
  This would also raise the version number from 4 (version with ES6 migration) to 5 (optimized Patterns).

For the resource registry the changes will be:

• We keep the bundle registration mechanism. JavaScript and CSS bundles (or resources) are registered as bundles similar to Plone 5.x.

• A bundle can contain one JS and/or one CSS file but not multiple JS files like it's currently the case.

• We remove the concept of resources which was only necessary for RequireJS to find it's named dependencies and to compile TTW or via plone-compile-resources.

• All resource definitions can be removed from the Plone registry.

• add_bundle_on_request and remove_bundle_on_request will still be available.

• add_resource_on_request will be a deprecated alias for add_bundle_on_request.

• Removals:

  • TTW bundle compilation functionality.

  • plone-compile-resources script. The Mockup bundle will be created with Webpack, other bundles as developers like to.

  • Removal of the concept of stub modules.

  • Removal of the conditional comment setting (not supported since IE11)

  • Removal of unnecessary Plone core bundles.

• A upgrade path should be provided to ensure a migration where it is possible. Bundles with compile set to on depend on RequireJS and need to be rewritten, though.

• jQuery will still be globally available by default as a separate bundle.

Caching URL improvements:

• A restart of Plone should refresh all resources.
• A press of a button should refresh individual bundles or all resources.
• A API endpoint should exist to update caching URLs.
• It should still be possible to change caching URLs via profile import (last_compiled).
• (Optional idea) Bundles should be delivered individually and not merged together into production bundles. Due to HTTP/2 this indirection should not be necessary or helpful anymore. For some scripts this can even be a precondition to not change the bundle URL after compilation - e.g. to find the chunks folder.

For integrators we encourage to write new JavaScript functionality wether as Patternslib Pattern or as a "web component" or "custom element". Bobtemplates.plone shows an example how to create a web component based on "Svelte", which has a low footprint and uses a standards-compliant web component approach. However, the framework is not important as anything can be used.

Deliverables

• PR for restructured resource registry
• PR for restructured Mockup
• Updated documentation
• Upgrade documentation
• bobtemplates.plone patterns-templates

Risks

• Through-the-web compilation will not be possible anymore which can be a problem for integrators.
• JavaScript code in Plone core (mostly Mockup) and add ons using RequireJS (bundle setting compile set to on) need to be slightly rewritten.

Participants

• Johannes Raggam
• Maik Derstappen

Resources

Mockup ES6 rewrite PR (WIP): plone/mockup#1025
Patternslib ES6 (production ready): https://github.com/patternslib/Patterns
Simple example pattern using Webpack, ES6, dynamic imports, Jest: https://github.com/Patternslib/pat-sortable-table/blob/master/src/pat-sortable-table.js

/cc @plone/framework-team

[frapell]

Having to do a Javascript bugfix with the existing resource registry is a complete nightmare... Huge +1 from me here

[tisto]

I am generally +1 on the intention of this PLIP. Though, we have to take into account that this completely breaks the add-on architecture of Plone as we know it. We were always reluctant to do this in the past because it is a huge step and a complete paradigm change.

How do you plan to keep the add-ons control panel with this new approach? You can not go to the add-ons control panel in Plone and install an add-on that has JS/CSS. You will always have to do a separate compile step and you need all the JS tooling properly installed for this. How do you plan to handle this? The add-ons control panel would give the user a false promise of being able to install an add-on product TTW, which won't be possible any longer. Are we going to remove TTW add-ons? Do we build a new JS package infrastructure? How do you plan to make JS packages depend on each other? How do you plan to do optimize the bundle when you have lots of add-ons installed?

I am playing devil's advocate here (mainly because the questions above are the challenges we had to solve in Volto). Though, this loss of one of Plone's core features was one of the main reasons we started with Plone-Angular/Plone-React/etc. We could not imagine doing such a fundamental break in Plone itself. And if the user has to install the complete JS toolchain and run webpack for each new add-on anyways, you can as well go all the way

If we talk about going in what I believe is the right direction, we should also talk about the long-term implications of this PLIP. Say we do this painful first step, the next step that makes sense is to make patternslib rely on Plone REST API, then make patternslib use a new modern JS framework (Rok wanted to rewrite mockup in React even before Plone 5 was released) and replace to old cruft. React is already in core and it basically won the JS framework war. Using something else at this point and not re-using Volto/React components would be just nuts (sorry for being blunt here). At this point we will have a wild mix of backend and frontend templates which is hellish (I saw many companies and people trying this and they failed 100% of the time). Then you want a clean separation between frontend and backend (which Plone REST API provides) at some point...

We went through this entire learning and decisions making process 3-5 years ago. We did all the mistakes, we tried way too many frameworks to end up with Volto. If this PLIP gets into core and we continue to work successfully on this, we either end up with a second Volto or with Plone classic becoming more and more what Volto already is. In any case, we will split the limited resources that we have working towards the same goal, to have a well maintained JS stack for Plone.

As said, I am not opposed to this PLIP. I am strongly against bringing yet another JS framework into core. Though, I hate to see that we put effort into solving a problem that we had not too long time ago and that we already successfully solved with Volto.

One other thing to consider: we had to come up with a complete new add-ons architecture for Volto and we still have to figure out how to make this work in Plone 6. If we will have two JS add-on architectures and JS build systems for Plone 6 this will confuse the hell out of people.

[jensens]

How do you plan to keep the add-ons control panel with this new approach? You can not go to the add-ons control panel in Plone and install an add-on that has JS/CSS.

As I understand @thet this is still perfectly possible, and much easier than before.

[tisto]

Do you plan to use @datakurre's approach to querying the Plone resources and bundle them externally?

[thet]

@tisto thanks for raising these questions!

I think most of add on JavaScript might just be installable as simple, separately packaged bundles. That should not make much problems as long as they do not have dependencies to Mockup's dependencies (except jQuery which is still available in the global namespace). If so, we might end up delivering multiple versions of these dependencies. To solve that, integrators would need to create a custom JavaScript bundle with all the necessary dependencies - something I see a neccessity for any mid-sized project anyways, also with our current infrastructure in Plone 5.x.
So, quick add-on installations should just work because they can register their compiled bundles with all their dependencies - the drawback here will be at worst non-optimized resource delivery with multiple versions of the same libraries.

Regarding moving to plone.restapi: a big +1 for all of Mockup's API calls. For Patternslib this is not so relevant - currently we only use the Patternslib core to register patterns. At a recent talk with @MrTango we agreed that we might not move to plone.restapi right away to keep up the backwards compatibility - there might be some customized endpoints used in projects. However, using plone.restapi instead of custom endpoints is on my wishlist too.

Regarding a rewrite to use a modern JS framework: Also +1 but not right away. The first goal would be to modernize the whole codebase. Then we can explore possibilities to integrate other frameworks - even much easier when we switched to ES6+ and removed RequireJS.

Reusing Volto components: definetly a good idea, especially since Volto has a lot of components which Mockup also offers but in a much better quality. I'd love to have Volto more modularized and be able to just import individual components without having to rewrite Mockup in React. The other way around would also be a very interesting option: moving towards Web Components and reusing thosre in Mockup and Volto at the same time (Mockup then might even be obsolete).

And no, I don't see the resource registry involved in compiling bundles, as @datakurre did for his Webpack projects. Actually all resources should be cleared from the resource registry - IMO we do and should not need them anymore. If one wants to create a optimized bundle it would be necessary to set up a JavaScript project, add all dependencies to package.json and create a bundle there. We need a template to make it easy to bootstrap such a project.

[sneridagh]

I have good news for (all of) you. This PLIP is already implemented, it's called Volto:
https://github.com/plone/volto/

There is a whole track (Track 1) at the Plone conference about it, so you all can get updated with all the new things the Volto community has implemented there, and the plans for the future.

https://2020.ploneconf.org/schedule

Oh, one more thing, as far as I know Volto is Plone 6.

[jensens]

@sneridagh that does not help, even if you put many blood and tears into Volto. We need Plone 6 Core still, we can not all switch immediately to Plone 6 Volto. Do not get me wrong, I really like Volto. But Plone 6 Core with server side template based rendering will exist for a while.

Oh, one more thing, as far as I know Volto is Plone 6.

So no, Plone 6 is both: Core and Volto.

On install there must be a choice what to install. I know the marketing team hates that, the Volto team either. But we can not ignore the reality of Plone users, companies, organisations, developler, ...

[sneridagh]

That's not the issue, there can't be two modern JS stories in Plone. Full stop.

Svelte? Add yet another different web technology to the stack? Web components what? Really? No, thanks. We worked our asses off in Volto to improve the approachability to Plone for now do this.

When you start to work on it, you'll get into the same pitfalls and will solve the same issues we already solved. You'll end up doing exactly the same we did for Volto.

By doing this, you will confuse people, and valuable resources will be spent in re-writing what it's already done. That also hurts.

Sorry, overlook them and ignore (and continue doing so) them hurts me, and all the Volto community (and yes, there is one, a healthy one).

[thet]

@sneridagh Plone 6 will still ship with the page templates rendered on the server. There is also the barceloneta-lts project for modernizing the UI. And for that we need (IMO) an updated JavaScript stack.

This PLIP is just about modernizing the existing JavaScript stack and is not meant to compete with Volto.

Regarding Svelte, Web Components etc: Just ignore that. Whatever developers use to add JS functionality to "classic" Plone is up to them.

Still, I'd really love to see Volto modularized in a way that individual components like folder content browser can be reused outside of Volto. I did not have time to explore that, but this is on my list.

[MrTango]

@sneridagh I understand your point, but I don't think it's all black or white.
And these discussion of everybody should work on the same project, in the same way is as old as open source is. The reality is, we have to worlds here and the gab is not small. Not everyone can just jump on the Volto train right now.
My vision of this is, to update and fix the existing Mockup/patterns and where needed make it possible to be creative and create new better once.

The goal is, what ever get's in there has to be really reusable. Svelte components and especially web components (custom elements) are ideal for this job. They work pretty much everywhere (Plone, static HTML, Patternslib, JS Frameworks) and also be able to use Redux, like Volto.

The way I see it, this will bring the core Plone and Volto closer together and make it easier for people to switch at some point.

It might not be obvious right away, but Volto can benefit from sharing components with existing Plone UI.

But back to the main points of this PLIP:

• we need to get rid of the broken Resource Registry and the TTW compiling
• we need to lower the barrier for addon developers. It is way to hard to use some existing JS tools in Plone 5, due to requireJS.
• we need to update and fix the existing UI components, we have in Plone
• we need to cleanup and even remove some of the existing patterns, some are unused and undocumented or sort of broken
• we would like to make it possible for everybody how is working with this stack, to use modern components by reducing unnecessary barriers from the stack

I know, nothing in this list is concerning, when one is using the Volto frontend, but this is not an option for everybody and not for every project at this time.

I feel comfortable with backend and frontend development, but many existing Plone developer are not and for them the existing Plone core does a better job. Let's please not forget them and force people to abandon the existing Stack. They both can exist and hopefully start sharing more together than the backend. Volto will grow and so will the developer base on the frontend side, but this is not going over night.

[fulv]

LTS is one of the suggested names for the old-style Plone stack with server-side rendered templates, aka Plone 6 Classic / Plone 6 Core / Plone 6 LTS.

Anyway, in the spirit of LTS I think it would make little sense to force people to rewrite any add-on code whatsoever. Any backwards incompatibility means we are forcing someone to decide whether to spend their dev money on an upgrade vs a switch to a different platform. We simply cannot afford to lose a single Plone installation. So don't give them a reason to leave.

I appreciate all of the ideas in this PLIP, and I know how painful the resource registry and the JS experience is. So I welcome any and all improvements.

But, again, in the spirit of LTS, I suggest making only the improvements that don't break backwards compatibility. The resulting PLIP might be much uglier and possibly even harder when done with this goal in mind, but a bird in the hand is worth two in the bush.

[jensens]

• LTS as term is fine since it reflects the idea. To have a point to start to get Plone Core ready for the next ~five years (to not change much in future) is a solid foundation. With the current JS stack at end of life I doubt core's JS can be maintained in a sane way in the 20ies. IMO we already can not.
• Same is valid for Bootstrap 3, which is already very outdated (not part of this PLIP).
• Also, I doubt a rewrite of add-ons is needed. Minor changes in registration may be needed. No need to rewrite to ES6 for everything, unless one is happy to do so. Thus said +1 for not breaking backward compatibility of existing JS code, except if it comes to loading/RequireJS/AMD issues and updating long deprecated JQuery calls.
• TTW compiling was error-prone (read: broken) all the time since 5.0. Removing it is a relief.

[MrTango]

+1 for not breaking with add-on's. There is no intention of that, we will do what we can to make it as seamless as possible for add-on developers.
The whole idea is to keep as much existing devs in Plone land and make it easy enough for new people to contribute. The ES6 changes are for Mockup and are the way to go to replace RequireJS.

[MrTango]

And to make it clear, there will be no other JS framework comming into core. custom-elements are not a framework but an existing web standard which is supported by all modern browsers. How you create such custom elements is up to you. One way is from cratch, one is to use libraries like LIT-Element and another is to use compiler like Stencil and Svelte, which produces JS at the end. So there is no framework at the end and also there is no runtime of a framework.

[thet]

Removing RequireJS will break backwards compatibility for add-ons which depend on RequireJS.
I'd like to know if there are many. I have the impression that there are only a handful of add-ons which have a pattern defined. Those can be easily rewritten to ES6 module imports as part of this PLIP. However, this ES6 module import rewrite is easy.

[jensens]

@thet and add-ons with JavaScript not implemented as pattern are not affected. Except if they are using RequireJS?

[thet]

@jensens yes, exactly. "legacy" bundles as we called them (no compile flag set, no requirejs) should still work as before.

[bloodbare]

@sneridagh you can always be Guillotina 7 Super power UI ;)

[thet]

Added a note about using Patterns from Patternslib instead of Mockup where possible as an optional task. First step would of course to be to bring Mockup to ES6 and the test suite green.

[datakurre]

@thet @MrTango You are such heros for working on this! Just a few thoughts from my experience on building all of our Plone 5 themes with webpack to possibly keep in mind:

• CDN support could come for free; we have at least 20 sites with the same theme and thanks to Webpack's publicPath we serve the same theme version for all of them directly from Apache with gzip, caching headers, etc... (hopefully soon also HTTP/2)

• Hot Module Replacement support in patterns / patternslib is something to think about. Obviously not a priority nor trivial (unless Patternslib 4 already has this), but something you miss when returning from other JS projects and see pattern code update requiring page refresh.

• Style updates could be instant. In practice, if an update to a single style requires compiling all styles (including styles from patterns and theme, possibly mixing LESS ans SCSS), style update could take longer than page refresh for a JS pattern update. I assume that this could be avoided with good design, so that it is easy to separate customized/developed styles into a separate top level import (to limit the amount of recompilation required after every change).

[MrTango]

Since we had some confusions in todays FrameWorkTeam meeting, about what the change with the Resource Registry and the removal of requireJS means for Add-on's.

Add-on's will be installable as before, no change there. There will be no big Webpack to rule them all. We will use webpack to build the plone-core bundle which includes things like Mockup/Patterns and some core resources. This bundle will be registered in the Resource Registry as it is now.

Resouces for add-on's will be manged by the individual add-on. There is no dependency injection happening. Add-on's should build decent bundle sizes and thing about if they schould be loaded all the time or just in some situations. If an add-on has a bigger library it depends on, this should be dynamically (lazy) loaded when the code is used. This keeps the bundle sizes down and resourcs are only loaded when they are needed.

[alecpm]

Hi all, I hope I'm not speaking out of turn here. This seems to me like a very good approach, but I'd like to stress the importance of high quality comprehensive documentation for major infrastructural changes like this.

In my experience, one of the biggest issues with the Plone 5 Resource Registry was a lack of documentation of how to use it either as an add-on developer or a site integrator. The initial documentation was largely inadequate until a while after release, and it's still far from comprehensive. For example, there's still no documentation indicating best practices for writing/updating an add-on to support multiple versions of Plone with different RR schemes. I believe that sort of thing is very important if we want people to e.g. update add-ons to properly support newer versions of Plone.

[thet]

From the discussion at the open space at the Plone conference 2020:

Adding JavaScript code via addons.

Adding JavaScript code via addons will be easier in a way that we allow to add bundles by providing them as static resources and adding a registry entry for that. The simplification to the current situation will be:

• Adding bundles with compiled = False will be the only bundle option. No more compiled = True as the TTW compilation option will be gone. No more mismatched anonymous define errors as we do not have RequireJS at our core.

• A browser:resource will do, plone:resource is not necessary as we do not need a way to override aleady compiled bundles from the filesystem with a TTW compiled version in the ZODB.

As far as the optimization of extra bundles added via addons goes, we have these options:

1. Not optimizing bundles at all. If an integrator doesn't know how to optimize it might not be that of an important site anyways and optimization not worth the effort.

2. Running webpack after buildout. Doing some basic automatic optimization as part of the Plone build step.

3. Webpack Module Federation. Allowing bundles to depend on Code from other bundles with this new Webpack 5 feature. Jupyter Notebooks is doing the same. This is an intersting option where we want to invest some time in to get some experience.

Notheworthy mention: CalmJS (https://pypi.org/project/calmjs/) allows for dependencies managed in Python code (setup.py/cfg). As this is much like Funstatic a solution fun off the standard we probably do not follow it.

Whatever we do, we need to make it simpler. We want to reduce complexity and probide a well documented, painless, easy, sane and reproducable way of managing JavaScript in Plone.

[datakurre]

💯

IMO, an after all the discussions, considering the community response to sophisticated Plone 5 resource management, returning back to "just bundles" is the only right way to do. Hopefully there is sane way add-ons to support this in backwards compatible way, when required. Conditional records should help https://pypi.org/project/plone.app.registry/#conditional-records

I'm looking forward for Webpack module federation to solve most of the issues with add-on families similar to eea-packages and allow them to avoid duplicating dependencies in individual bundles.

The complete Webpack approach presented in plonetheme.webpacktemplate will become a bit harder now that the source files are now longer in the registry (or packed with the add-on packages), but must most probably be fetched from their original sources or npmjs. That was my first approach years ago, but back then it was a nightmare to maintain compared to fetching everything from Plone registry. But I'm sure that this will become much easier now that Mockup itself is modernized. So, once I'd updated plonetheme.webpacktemplate for Plone 6, also this use case is covered.

[Rudd-O]

PLIP looks good. Only thing I would love to see is documentation for addon writers to port their addons to the new tech.

[MrTango]

Resource Registry

We have a new much simpler resource registry (RG) which is basically a form not a JS based UI as before, so it will not break when you change things in the RG. You only register compiled bundles, this is very close to as it was possible before. There will be no more individual resources, as we don't support bundling in the browser. What you use to create your resource bundles is up to you. It can be plain CSS/JS or compiled by Rollup or Webpack & co.

Mockup / Patterslib Patterns

All resources are standard ES-modules and can be use in you own project Webpack & co and be further optimized.
Patternslib as already available as an npm package, this could be done with Mockup too, to make it easier to reuse it in projects.
In general the components are more lightweight and easier to customize via CSS overrides, since they are no longer scoped.
Most big resources are fractured out and loaded dynamically when they actually needed.

Better Add-on's support

Besides that the registration of resources will be simpler in general, you will have some advanced options to use JS-libraries, which you can use if you want. Things like Mockup, Patternslib and potentially others can be shared by add-on's via Webpack Module Federation. For an add-on this means, with a bit extra config, you can share a library with other participants in Plone and avoid double loading and conflicts. But all this is opt-in, so if you don't care, nobody will bother you.

The migration of add-on's will be fairly simple. If it uses so called legacy (externally compiled) bundles, there is nothing to do. If you where so brave and wrapped the resources with requireJS, this can be removed. All you need is a simple bundle registration of whatever CSS/JS you have. For add-on's which where using Mockup/Patternslib via RequireJS, this needs to be changed to a Webpack Module Federation style, which will be documented soon.

Status

We only have some small bugs in pat-relateditems and some unit tests to fix, the rest in already in a good shape.
The Webpack-Module-Federation will be tested with some add-on's in the coming weeks and the whole story documented.

[ale-rt]

This was discussed in the @plone/framework-team mailing list and nobody spoke against.
Personally I would consider this one approved and also not really @plone/framework-team material given that is geared towards the frontend and the framework team decided to limit its activity to the backend code.

Anyway it is still good to have a PLIP, I think.

[mauritsvanrees]

I have tested it, from the point of view of a site admin or editor, and a bit as add-on writer. I did not look at how things are implemented in the code. Tested on Mac in Firefox.

Some review notes:

• The extra session refresh support from plone.session breaks. Should probably be solved there. Can also point to a wider problem, when add-ons do similar things.
• On a site migrated from Plone 5.2 Python 3, I get a javascript console error Uncaught (in promise) ReferenceError: jQuery is not defined. This causes most javascript to not work. For example click on "Add New" and you do not get a drop down, but you go to the folder_factories page. TinyMCE does not show up. On a new site, this works fine. Comparing the Resource Registry control panels, the migrated site only has the bootstrap-js and plone bundles, so it is missing jquery, plone-legacy and plone-logged-in.
• In a new site, go to Theme Settings and then Custom Styles. The custom css textarea is black and you cannot type in it. On a migrated site, this part actually works. [Update after merge: fixed.]
• In folder contents, dragging items does not work. I see a status message "Successfully moved item", but the moved item is immediately moved back. The 'Rearrange' button does work. The actions to move to the top or bottom also work. [Update after merge: fixed.]
• Some icon lookups fail. That might not really be part of the ES6 story though. Icon resolver lookup of '' failed, fallback to Plone icon. and Icon resolver lookup of 'contenttype' failed, fallback to Plone icon. [Update after merge: fixed.]

I have created an issue for the first point. I can create separate issues for the others if wanted. Where? In the Products.CMFPlone issue tracker?

The migration problem strikes me as the main one that would need to be tackled soon, otherwise a migrated site basically does not work. Being able to move items in folder contents would be welcome too. The other items seem minor and could be fixed later.

I did not notice problems in the Site Setup or TinyMCE. Everything works the same or better. I did not do in depth testing though.

I did not try adding an add-on with some css and javascript. But it looks really easy. You point to one javascript and one css file of the add-on. Maybe specify one or more other bundles like jquery as depends, which I guess simply means that those are loaded first. And that is it. No bundling or other combining. I think you get more css and js files than in Plone 5.2, exactly because nothing is combined anymore. Or is some bundling still happening, simply by concatenating files?
Anyway, this seems a small price to pay for sanity restored with an up to date javascript story. Marvelous work!

Actually, since it sounds easy, let me create an add-on with plonecli create addon collective.foo. I manually created a foo.js and foo.css, where the javascript file uses jQuery ($). I added this resource.xml:

  <records prefix="plone.bundles/collective.foo" interface='Products.CMFPlone.interfaces.IBundleRegistry'>
    <value key="enabled">True</value>
    <value key="jscompilation">++plone++collective.foo/foo.js</value>
    <value key="csscompilation">++plone++collective.foo/foo.css</value>
    <depends>jquery</depends>
    <value key="load_async">False</value>
    <value key="load_defer">False</value>
  </records>

I activated the add-on, and both my javascript and css worked. Without jquery in the depends it fails because $ is not defined. So it works as I expect it to, and is easy. Great! Two minor things:

• I wonder if depends should be a list, where you can depend on for example jquery and plone. Or maybe a package with an extra widget for easyform could depend on jquery and easyform, if that bundle exists.
• When I look in portal_registry for collective.foo, I see various keys that are not in my registry.xml file, and that are deprecated, for example plone.bundles/collective.foo.compile. These should not get created. But I can imagine that these are needed for backwards compatibility.

I added collective.easyform 3.0.5, which does not yet have any changes for Plone 6. The registry.xml only tries to load the css, and this works unchanged. The javascript is not handled via the resource registries, but is only loaded on the pages for editing fields and actions. This fails with AttributeError: ++resource++schemaeditor.js. So easyform is a special package which needs some updates for Plone 6. There is a PR with fixes so I try it. Then it works. I now see that easyform does not even have own javascript, except a small bit inline. And the most important thing that makes it work, is that it uses pat-schemaeditor.

On the network tab I see some javascript chunks getting loaded. I don't know if that is the module federation or something else. But it seems to work.

Summary: it is looking pretty good!

[yurj]

• On a new site, this works fine. Comparing the Resource Registry control panels, the migrated site only has the bootstrap-js and plone bundles, so it is missing jquery, plone-legacy and plone-logged-in.

It happened to me too, If you re-run the last staticresources import step and cmfplone one, jquery bundle pops up. Seems like some upgrade steps add/erase things, and something runs not in the right order.

[jensens]

No bundling or other combining. I think you get more css and js files than in Plone 5.2, exactly because nothing is combined anymore. Or is some bundling still happening, simply by concatenating files?

This is by intend.

• In past we had no HTTP/2.0.
• Map files are working w.o concat
• For JS we anyway have module federation, this loads lots of small files on demand, no large bundles anymore
• Simplification/no magic

[mauritsvanrees]

@thet has merged everything. Thank you Johannes, @MrTango, @petschki, @jensens, @agitator, @pbauer and lots more people!
[Feel free to edit for adding more people, or add comments below. Listing names is dangerous...]

[mauritsvanrees]

Some remaining issues I saw:

• plone.session install breaks on ES6 plone.session#23
• Some bundles are missing after upgrade to Plone 6 with ES6 plone.app.upgrade#275
• Still uses plone-legacy bundle plone.app.event#350

[ale-rt]

Also I would like to see some documentation on this one and if possible a demo add on, something similar to https://github.com/collective/example.p4p5. It does not need top work on both p5 and p6 (though it would be a nice plus) but at least it should be really clear how to wire up custom css + javascript in Plone 6 add on.

[mauritsvanrees]

This can be closed, right?
A documentation issue is still open: plone/documentation#1142