Skip to content

Commit e685507

Browse files
rjrudinrjrudin
committed
Added example project for generating a custom REST rewriter and REST modules
1 parent 4730db0 commit e685507

File tree

16 files changed

+1322
-0
lines changed

16 files changed

+1322
-0
lines changed

‎examples/custom-rest-rewriter-project/.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
.gradle
2+
build
3+
.idea
4+
*.iml
5+
src/main/ml-modules/root/custom-rest-rewriter
6+
src/main/ml-modules/root/noauth-rewriter.xml

‎examples/custom-rest-rewriter-project/README.md‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
This project shows one approach for when you'd like to use the MarkLogic REST API for an app server, but you need
2+
to do some pre-processing on every request. A common need is to use application layer security for the app server
3+
with a default user that has minimum privileges, and then perform an xdmp:login call based on e.g. the value of an
4+
HTTP header. That approach is shown in this example project, but you can apply this approach for generating a custom
5+
rewriter and REST modules to perform any kind of pre-processing that you need.
6+
7+
## Generating the custom rewriter modules
8+
9+
To try this out locally, first run the custom Gradle task for generating a copy of every REST API dispatch modules and
10+
a custom REST XML rewriter that invokes each of the dispatch modules:
11+
12+
./gradlew generateCustomRewriterModules
13+
14+
This will write several dozen modules to src/main/ml-modules/root/custom-rest-rewriter (which is gitignore'd so that you
15+
can generate these yourself). If you look at one of the modules, you'll see the following code:
16+
17+
xquery version '1.0-ml';
18+
import module namespace login = 'org:example' at '/login-lib.xqy';
19+
login:login-noauth-user(),
20+
xdmp:invoke('/MarkLogic/rest-api/default.xqy')
21+
22+
In this example, every module has a call to "login:login-noauth-user", which handles making an xdmp:login call based
23+
on the value of an HTTP header, and then invokes the appropriate MarkLogic REST module to handle the request.
24+
25+
These modules are generated via 3 files in the root of this directory:
26+
27+
- rewriter-original.xml = a copy of the default REST XML rewriter, as of MarkLogic 9.0-7
28+
- rewriter-modify-dispatches.xsl = an XSL transform that modifies the value of each dispatch element in the rewriter
29+
- rewriter-extract-dispatches.xsl = an XSL transform that returns the value of every dispatch element in a simple XML
30+
structure that is easy to manipulate within Groovy
31+
32+
You are free to store these files wherever you want, they don't have to be in the root directory. Just be sure to
33+
update the Gradle task to account for any changes you make.
34+
35+
36+
## Deploy the application
37+
38+
Next, deploy the application:
39+
40+
./gradlew -i mlDeploy
41+
42+
The application deploys two REST servers - a standard one on port 8228 that uses digest authentication, and then a
43+
"no authentication" one on port 8229 that uses application layer authentication and the custom rewriter.
44+
45+
## Test the application
46+
47+
To test the difference between the two servers, try a simple request to the search endpoint on 8228 using your admin user
48+
(change the admin password as needed):
49+
50+
curl --anyauth --user admin:admin http://localhost:8228/v1/search
51+
52+
This will work fine, you'll get an XML search response back. But if you try the same request on port 8229:
53+
54+
curl --anyauth --user admin:admin http://localhost:8229/v1/search
55+
56+
You'll get an error stating "You do not have permission to this method and URL.". That's because the admin user/password
57+
are ignored with application level security, and the default user - "custom-rest-rewriter-user" - isn't getting any
58+
roles via the xdmp:login that give it permission.
59+
60+
We can fix this by specifying the header that login-lib.xqy is looking for (and there's no point in specifying a user
61+
because it'll be ignored in favor of the default user):
62+
63+
curl -H "X-my-marklogic-role: rest-admin" http://localhost:8229/v1/search
64+
65+
The value of the header - "rest-admin" - will be added as a role to the request, thus allowing the search request to
66+
succeed.
67+
68+
Note that this is just a demonstration of how to generate a custom XML rewriter with a set of custom endpoints that
69+
call a function performing any logic that you wish. For this particular example, you'd want to ensure that no system
70+
could access the "noauth" REST server except those known to have already authenticated a user. But your function wouldn't
71+
have to call xdmp:login - it could perform some custom logging, or do any sort of pre-processing that you wish. The key
72+
is that this project shows an easy to insert that pre-processing into an XML rewriter.
73+
74+
After you finish testing this out, you can get rid of the app:
75+
76+
./gradlew -Pconfirm=true -i mlUndeploy

‎examples/custom-rest-rewriter-project/build.gradle‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
plugins {
2+
id "com.marklogic.ml-gradle" version "3.9.0"
3+
}
4+
5+
task generateCustomRewriterModules {
6+
description = "Uses XSL transforms to modify the default REST rewriter so that it uses custom endpoints that perform an xdmp.login. " +
7+
"This should only be run at development time to generate code; it is not run as part of a deployment."
8+
doLast {
9+
def modifiedRewriterPath = "src/main/ml-modules/root/noauth-rewriter.xml"
10+
def transformerFactory = javax.xml.transform.TransformerFactory.newInstance()
11+
12+
def transform = file("rewriter-modify-dispatches.xsl").text
13+
def originalRewriter = file("rewriter-original.xml").text
14+
def transformer = transformerFactory.newTransformer(new javax.xml.transform.stream.StreamSource(new StringReader(transform)))
15+
transformer.setParameter("restModulePrefix", restModulePrefix)
16+
def outputStream = new FileOutputStream(modifiedRewriterPath)
17+
transformer.transform(new javax.xml.transform.stream.StreamSource(new StringReader(originalRewriter)), new javax.xml.transform.stream.StreamResult(outputStream))
18+
outputStream.close()
19+
20+
transform = file("rewriter-extract-dispatches.xsl").text
21+
transformer = transformerFactory.newTransformer(new javax.xml.transform.stream.StreamSource(new StringReader(transform)))
22+
outputStream = new ByteArrayOutputStream()
23+
def modifiedRewriter = file(modifiedRewriterPath).text
24+
transformer.transform(new javax.xml.transform.stream.StreamSource(new StringReader(modifiedRewriter)), new javax.xml.transform.stream.StreamResult(outputStream))
25+
26+
def xml = new XmlSlurper().parseText(new String(outputStream.toByteArray()))
27+
xml.dispatch.each {
28+
def modifiedModule = it.text()
29+
def originalModule = modifiedModule.replace(restModulePrefix, "")
30+
31+
/**
32+
* When adapting this for your own project, modify the generated code here to invoke whatever function you want.
33+
* The login-lib.xqy module is just an example for a common use case.
34+
*/
35+
def moduleFile = new File("src/main/ml-modules/root" + modifiedModule)
36+
def content = "xquery version '1.0-ml';\nimport module namespace login = 'org:example' at '/login-lib.xqy';\nlogin:login-noauth-user(),\nxdmp:invoke('" + originalModule + "')"
37+
if (modifiedModule.endsWith(".sjs")) {
38+
content = "const login = require('/login-lib.xqy');\nlogin.loginNoauthUser();\nxdmp.invoke('" + originalModule + "')"
39+
}
40+
def file = file(moduleFile)
41+
file.getParentFile().mkdirs()
42+
println "Writing modified endpoint: " + file
43+
file.write content
44+
}
45+
}
46+
}

‎examples/custom-rest-rewriter-project/gradle.properties‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
mlAppName=custom-rest-rewriter
2+
mlRestPort=8228
3+
mlNoauthRestPort=8229
4+
mlUsername=admin
5+
mlPassword=admin
6+
7+
# Defines the prefix added to every custom REST module that's based on an OOTB ML REST module
8+
restModulePrefix=/custom-rest-rewriter
9+
10+
# Defines username/password for the default user for the application layer ("noauth") REST server
11+
noauthUsername=custom-rest-rewriter-user
12+
noauthPassword=changeme
13+

‎examples/custom-rest-rewriter-project/gradle/wrapper/gradle-wrapper.jar‎

54.9 KB
Binary file not shown.

‎examples/custom-rest-rewriter-project/gradle/wrapper/gradle-wrapper.properties‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
distributionBase=GRADLE_USER_HOME
2+
distributionPath=wrapper/dists
3+
distributionUrl=https\://services.gradle.org/distributions/gradle-4.10.2-bin.zip
4+
zipStoreBase=GRADLE_USER_HOME
5+
zipStorePath=wrapper/dists

‎examples/custom-rest-rewriter-project/gradlew‎

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
#!/usr/bin/env sh
2+
3+
##############################################################################
4+
##
5+
## Gradle start up script for UN*X
6+
##
7+
##############################################################################
8+
9+
# Attempt to set APP_HOME
10+
# Resolve links: $0 may be a link
11+
PRG="$0"
12+
# Need this for relative symlinks.
13+
while [ -h "$PRG" ] ; do
14+
ls=`ls -ld "$PRG"`
15+
link=`expr "$ls" : '.*-> \(.*\)$'`
16+
if expr "$link" : '/.*' > /dev/null; then
17+
PRG="$link"
18+
else
19+
PRG=`dirname "$PRG"`"/$link"
20+
fi
21+
done
22+
SAVED="`pwd`"
23+
cd "`dirname \"$PRG\"`/" >/dev/null
24+
APP_HOME="`pwd -P`"
25+
cd "$SAVED" >/dev/null
26+
27+
APP_NAME="Gradle"
28+
APP_BASE_NAME=`basename "$0"`
29+
30+
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
31+
DEFAULT_JVM_OPTS=""
32+
33+
# Use the maximum available, or set MAX_FD != -1 to use that value.
34+
MAX_FD="maximum"
35+
36+
warn () {
37+
echo "$*"
38+
}
39+
40+
die () {
41+
echo
42+
echo "$*"
43+
echo
44+
exit 1
45+
}
46+
47+
# OS specific support (must be 'true' or 'false').
48+
cygwin=false
49+
msys=false
50+
darwin=false
51+
nonstop=false
52+
case "`uname`" in
53+
CYGWIN* )
54+
cygwin=true
55+
;;
56+
Darwin* )
57+
darwin=true
58+
;;
59+
MINGW* )
60+
msys=true
61+
;;
62+
NONSTOP* )
63+
nonstop=true
64+
;;
65+
esac
66+
67+
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
68+
69+
# Determine the Java command to use to start the JVM.
70+
if [ -n "$JAVA_HOME" ] ; then
71+
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
72+
# IBM's JDK on AIX uses strange locations for the executables
73+
JAVACMD="$JAVA_HOME/jre/sh/java"
74+
else
75+
JAVACMD="$JAVA_HOME/bin/java"
76+
fi
77+
if [ ! -x "$JAVACMD" ] ; then
78+
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
79+
80+
Please set the JAVA_HOME variable in your environment to match the
81+
location of your Java installation."
82+
fi
83+
else
84+
JAVACMD="java"
85+
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
86+
87+
Please set the JAVA_HOME variable in your environment to match the
88+
location of your Java installation."
89+
fi
90+
91+
# Increase the maximum file descriptors if we can.
92+
if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
93+
MAX_FD_LIMIT=`ulimit -H -n`
94+
if [ $? -eq 0 ] ; then
95+
if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
96+
MAX_FD="$MAX_FD_LIMIT"
97+
fi
98+
ulimit -n $MAX_FD
99+
if [ $? -ne 0 ] ; then
100+
warn "Could not set maximum file descriptor limit: $MAX_FD"
101+
fi
102+
else
103+
warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
104+
fi
105+
fi
106+
107+
# For Darwin, add options to specify how the application appears in the dock
108+
if $darwin; then
109+
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
110+
fi
111+
112+
# For Cygwin, switch paths to Windows format before running java
113+
if $cygwin ; then
114+
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
115+
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
116+
JAVACMD=`cygpath --unix "$JAVACMD"`
117+
118+
# We build the pattern for arguments to be converted via cygpath
119+
ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
120+
SEP=""
121+
for dir in $ROOTDIRSRAW ; do
122+
ROOTDIRS="$ROOTDIRS$SEP$dir"
123+
SEP="|"
124+
done
125+
OURCYGPATTERN="(^($ROOTDIRS))"
126+
# Add a user-defined pattern to the cygpath arguments
127+
if [ "$GRADLE_CYGPATTERN" != "" ] ; then
128+
OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
129+
fi
130+
# Now convert the arguments - kludge to limit ourselves to /bin/sh
131+
i=0
132+
for arg in "$@" ; do
133+
CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
134+
CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
135+
136+
if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
137+
eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
138+
else
139+
eval `echo args$i`="\"$arg\""
140+
fi
141+
i=$((i+1))
142+
done
143+
case $i in
144+
(0) set -- ;;
145+
(1) set -- "$args0" ;;
146+
(2) set -- "$args0" "$args1" ;;
147+
(3) set -- "$args0" "$args1" "$args2" ;;
148+
(4) set -- "$args0" "$args1" "$args2" "$args3" ;;
149+
(5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
150+
(6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
151+
(7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
152+
(8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
153+
(9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
154+
esac
155+
fi
156+
157+
# Escape application args
158+
save () {
159+
for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
160+
echo " "
161+
}
162+
APP_ARGS=$(save "$@")
163+
164+
# Collect all arguments for the java command, following the shell quoting and substitution rules
165+
eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"
166+
167+
# by default we should be in the correct project dir, but when run from Finder on Mac, the cwd is wrong
168+
if [ "$(uname)" = "Darwin" ] && [ "$HOME" = "$PWD" ]; then
169+
cd "$(dirname "$0")"
170+
fi
171+
172+
exec "$JAVACMD" "$@"

0 commit comments

Comments
 (0)