Current architecture

Current architecture

Using eXist as the central service and adding others as needed

With the current version of eXist dropping .war support eXist itself cannot be part of another servlet container.

But despite the eXist project dropping .war builds eXist is still running in a servlet container: Jetty.

Recently Jetty became a servlet container that is as capable as tomcat in all aspects used in this project.

Running orbeon together within the eXist servlet

jetty started by eXist can run any servlet. It is not configured to run JSP by default because there is no use for JSPs when running eXist based web applications. Nevertheless, the missing functionality can be enabled if necessary by adding some jar files and some configuration instructions.

In an eXist 2-4 distribution the servlets and there configuration are in tools/jetty/webapps. In eXist 5 this directory is etc/jetty/webapps.

Two servlets are always configured:

• portal, which forwards to the dashboard if someone accesses eXist only by hostname without any path (or the eXist path)
• eXist itself which is a servlet but is spaced out over the entire distribution directory tree. It is registered with jetty here using a configuration file exist-webapp-context.xml

Configuring eXist for coexistence and orbeon/MerMEId specific file types

eXist as root servlet

Because orbeon tries to "helpfully" expand /exist/apps/absolute/paths with prepending the scheme, host and servlet context path (http://localhost:8080/exist/exist/apps/absolute/path) it is not possible to run eXist within its default context path exist/. However, eXist db optionally can run as the "root context" servlet (context path /). So this needs to be changed in exist-webapp-context.xml. It seems the portal servlet that is usually bound to the "root context" needs to be removed in some setups (Windows but not linux) so it es best to just delete the tools/jetty/webapps/portal or etc/jetty/webapps/portal subdirectory.

Applying the orbeon-xforms-filter to a particular path

For other servlets (including eXist) to interact with orbeon on XForms there is a filter in the orbeon distribution that can be put into the classpath of any other servlet and then used to filter using a servlet container's filter and filter-mapping function.

• eXist 2-4: From the extracted orbeon.war copy tools/jetty/webapps/orbeon/WEB-INF/lib/orbeon-xforms-filter.jar to lib/user

• eXist 5:

  • From the extracted orbeon.war copy etc/jetty/webapps/orbeon/WEB-INF/lib/orbeon-xforms-filter.jar to some directory (e.g. lib/user which you have to create)

  • For a standard distribution running on a full operating system: Add lib/user/orbeon-xforms-filter.jar to the jars loaded on startup like this: Edit etc/startup.xml within the dependencies tag add:

      <dependency>
        <groupId>org.orbeon.orbeon-xforms-filter</groupId>
        <artifactId>orbeon-xforms-filter</artifactId>
        <version>4.9.0</version>
        <relativePath>user/orbeon-xforms-filter.jar</relativePath>
      </dependency>

    You can change groupId, artefactId and version to your liking if you use a more recent orbeon distribution for example, it does not seem to matter for eXist-5's jar loader.

  • For a standard Docker image of eXist 5 pulled from Docker hub: There is no configuration file like startup.xml. You have to override the ENV CLASSPATH= in the Dockerfile to also contain your custom jar, e.g. ENV CLASSPATH=/exist/lib/exist.uber.jar:/exist/lib/user/orbeon-xforms-filter.jar, and make sure orbeon-xforms-filter.jar is available at /exist/lib/user/orbeon-xforms-filter.jar by further customizing the official Dockerimage and COPYing it there. See here.

• Edit web.xml in webapp/WEB-INF/web.xml or etc/webapp/WEB-INF/web.xml

  • Remove betterform (another XForms processor) if it is enabled as a filter in web.xml:

     <!--====================== betterFORM filter mapping ======================-->
   <!-- filter-mapping>
      <filter-name>XFormsFilter</filter-name>
      <url-pattern>/apps/*</url-pattern>
   </filter-mapping>
   <filter-mapping>
      <filter-name>XFormsFilter</filter-name>
      <servlet-name>XFormsPostServlet</servlet-name>
   </filter-mapping>
   <listener>
      <listener-class>de.betterform.agent.web.servlet.BfServletContextListener</listener-class>
   </listener -->
   <!--====================== betterFORM filter and servlets ======================-->

  This is configured to be used on everything in apps/ so it will interfere with orbeon.

  • Configure in which subdirectory orbeon xforms filter is to be used:

  <filter>
    <filter-name>orbeon-xforms-filter</filter-name>
    <filter-class>org.orbeon.oxf.servlet.OrbeonXFormsFilter</filter-class>
    <init-param>
        <param-name>oxf.xforms.renderer.context</param-name>
        <param-value>/orbeon</param-value>
    </init-param>
    <init-param>
        <param-name>oxf.xforms.renderer.default-encoding</param-name>
        <param-value>UTF-8</param-value>
    </init-param>
  </filter>
  <!-- Any resource fetched using http under /apps/mermeid/forms/* is processed by the XForms engine -->
  <filter-mapping>
    <filter-name>orbeon-xforms-filter</filter-name>
    <url-pattern>/apps/mermeid/forms/*</url-pattern>
    <!--Servlet 2.4 configuration allowing the filter to run upon forward in addition to request-->
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
  </filter-mapping>

  Please note that the orbeon xforms filter expects any file in this subdirectory and subdirectories thereof to be parseable as XML. So you can not put binary resources like pictures there for example.

  There is no way to disable that filter at runtime. So if you want to get the XML contents in that subdirectory via http you have to use eXist's rest API or use a controller.xql that fetches the XML content within this subdirectory in one with a different name.

  You can use the filter for other subdirectories as well e.g.:

  <!-- Any web resource under /apps/orbeon-forms/* is processed by the XForms engine -->
  <filter-mapping>
    <filter-name>orbeon-xforms-filter</filter-name>
    <url-pattern>/apps/orbeon-forms/xforms/*</url-pattern>
    <!--Servlet 2.4 configuration allowing the filter to run upon forward in addition to request-->
    <dispatcher>REQUEST</dispatcher>
    <dispatcher>FORWARD</dispatcher>
  </filter-mapping>

Mime type of xbl files

Orbeon and MerMEId use a specific extension for some parts of the forms included by the mein XML file. They end in .xbl. By default, this is no file which is associated by eXist with the mime-type application/xml. Because of that these files are not considered when resolving XInclude.

In eXist 2-4 there is a file mime-types.xml (and mime-types.xml.tmpl) in the root directory of the distribution where .xbl can be added to the list of application/xml type files:

    <!-- Mime types stored as XML -->
    <mime-type name="application/xml" type="xml">
        <description>XML document</description>
        <extensions>.xml,.xbl,.xsd,.rng,.mods,.xmp,.xmi,.xconf,.xmap,.xsp,.wsdl,.x3d,.owl,.dbx,.tei,.xces,.ead,.xqx,.xform,.gml,.fo,.nvdl,.sch,.imdi,.cmdi,.odd,.jcmconnect,.ditaval</extensions>
    </mime-type>

In eXist 5 this file no longer is considered. There is a mime-types.xml as a resource in lib/exist-core-5.x.y.jar/org/exist/util/. It can be edited and replaced there to achieve the same effect. You would also need to edit that JAR file within the exist.uber.jar distributed in the docker container.

There is also the xmldb:store#4 function, which has a $mime-type parameter, that might be able to do the same at runtime by reading the .xbl files and storing them again this time as "real" XML files.

Adding other servlets to eXist's Jetty servlet container

To add more servlets the easiest way is to put them into this webapps either as a .war file if the servlet does not have to be changed or unpacked into a subdirectory here (or elsewhere for that matter). It is quite easy to change unpacked .war files like deleting unused parts as is recommended for orbeon.war for example.

In addition to putting the .war or its contents into the webapps directory a small configuration file is needed. All filenames and directory names are suggestions as jetty will read any .xml file in the webapps directory on startup.

orbeon.xml

<Configure class="org.eclipse.jetty.webapp.WebAppContext">
 <Set name="war"><Property name="jetty.webapps" default="."/>/orbeon</Set>
 <Set name="contextPath">/orbeon</Set>
 <Get name="securityHandler">
    <Set name="loginService">
      <New class="org.eclipse.jetty.security.HashLoginService">
        <Set name="name">Orbeon</Set>
        <Set name="config"><Property name="jetty.webapps" default="."/>/orbeon.properties</Set>
        <Set name="hotReload">true</Set>
        <Call name="start"></Call>
      </New>
    </Set>
 </Get>
</Configure>

The only additional service configured here is a login service. This is for test purpose only. The login service should be shared with eXist.

If the configuration is correct you will see some messages from orbeon during startup of eXist:

2020-06-08 14:00:03,561 INFO  [org.orbeon.oxf.webapp.Orbeon$] : Starting Orbeon Forms 4.9.0.201505052329 CE
 Starting Orbeon Forms 4.9.0.201505052329 CE
2020-06-08 14:00:03,571 INFO  [org.orbeon.oxf.webapp.Orbeon$] : Initializing Resource Manager with: {oxf.resources.priority.5=org.orbeon.oxf.resources.ClassLoaderResourceManagerFactory, org.orbeon.oxf.resources.WebAppResourceManagerImplWebAppContext=org.orbeon.oxf.webapp.ServletWebAppContext@401926df, oxf.resources.factory=org.orbeon.oxf.resources.PriorityResourceManagerFactory, oxf.resources.priority.3=org.orbeon.oxf.resources.WebAppResourceManagerFactory, oxf.resources.priority.3.oxf.resources.webapp.rootdir=/WEB-INF/resources}
 Initializing Resource Manager with: {oxf.resources.priority.5=org.orbeon.oxf.resources.ClassLoaderResourceManagerFactory, org.orbeon.oxf.resources.WebAppResourceManagerImplWebAppContext=org.orbeon.oxf.webapp.ServletWebAppContext@401926df, oxf.resources.factory=org.orbeon.oxf.resources.PriorityResourceManagerFactory, oxf.resources.priority.3=org.orbeon.oxf.resources.WebAppResourceManagerFactory, oxf.resources.priority.3.oxf.resources.webapp.rootdir=/WEB-INF/resources}
2020-06-08 14:00:03,576 INFO  [org.orbeon.oxf.webapp.Orbeon$] : Using run mode: prod
 Using run mode: prod
2020-06-08 14:00:03,578 INFO  [org.orbeon.oxf.webapp.Orbeon$] : Using properties file: oxf:/config/properties-prod.xml
 Using properties file: oxf:/config/properties-prod.xml

A simple check if XForms actually work is browsing to the following URL: http://localhost:8080/orbeon/xforms-hello/ (this does not use eXist in any way, the context is orbeon)

Possible downsides of this setup

• The exit JVM needs to use some additional resources for the other services
• The setup is more complex than just using some standard containers and connecting them using docker.