Sort out why i18n doesn't work from Jar file

[simon-brooke]

When you start Smeagol as an executable Jar file (i.e. running from Jetty within a Jar file), a number of things go wrong. But the oddest is that it fails to access its i18n resources - which are in the jar file, so should be available.

I'm missing something here.

[desolate-planet]

Didn't realise you already fixed this on dev (as it appears to me working now in dev), but I got it working using a different approach. Updating utils to (clojure.java.io/resource "filename")

(defn raw-get-messages
"Return the most acceptable messages collection we have given the
Accept-Language header in this request."
[request]
(merge
(i18n/get-messages
((:headers request) "accept-language")
(str (clojure.java.io/resource "i18n/"))
"en-GB")
config))

This ensures the classpath location is being used to locate the file: https://stackoverflow.com/questions/8009829/resources-in-clojure-applications

[simon-brooke]

H'mmm... my solution doesn't work for me, so I'll try yours ;-)

More to the point I'm facing an issue where I need to supply configuration parameters in one way if Smeagol is launched as an executable Jar file, and in another if Smeagol is run as a War file in a servlet container.

I've submitted a pull request on lein-ring which allows abitrary config-params to be specified in project.clj and written into the web.xml, but actually that's not the right solution either because you need to be able to deploy the build artefact and the configuration separately - see the principles of the twelve factor app, which seems to me good architectural thinking.

One thing is I could create a form editable by the admin user which allows editing of the contents of the config.edn file.

I haven't reached a solution I'm entirely comfortable with; but at the same time I want to have a finished 1.0.0 release sometime this month, so I need to make up my mind.

[desolate-planet]

Odd, I put that change in (mines) and tried to push, but realised I was still on master and switched to dev minus my changes and it still worked. Possibly I didn't clear my browser cache properly, but that change certainly worked when I tried it. And it was just a tweak to the utils file.

[desolate-planet]

Not sure if I have permission to push/create pull requests to your project.

[desolate-planet]

One idea is to use something simple like couchdb or sqlite as a backing store and having an initial setup page to specify configuration parameters/overrides. I'm sure you have already considered this in the past, which is why I was reluctant to suggest it :).

[desolate-planet]

I've now read up on Twelve-Factor App, this is something I've been ignorant of, until now..

[simon-brooke]

Ha! The whole motivation for building Smeagol was I wanted to get away from having an SQL back end. Yes, that would work.

To submit a pull request, fork the project (there's a button to do that top right of the project page), check out from your repository, make your change, commit it to your repository, and then create a pull request from yours to mine. You'll find the github workflow runs really smoothly.

If you download the Zenhub plugin for Chrome, you'll be able to see my kanban board (and then we can work out how to give you permission to edit it).

[desolate-planet]

Pull request is in. Hopefully that works for you.

[desolate-planet]

Setup zenhub and can now see your kanban board.

[simon-brooke]

Damn. Still doesn't work.

I'm getting:

17-09-02 08:46:04 fletcher ERROR [smeagol.middleware:48] - {:error #error {
 :cause "/home/simon/workspace/smeagol/jar:file:/home/simon/workspace/smeagol/target/smeagol-0.99.8-SNAPSHOT-standalone.jar!/i18n/en-GB.edn (No such file or directory)"
 :via
 [{:type java.io.FileNotFoundException
   :message "/home/simon/workspace/smeagol/jar:file:/home/simon/workspace/smeagol/target/smeagol-0.99.8-SNAPSHOT-standalone.jar!/i18n/en-GB.edn (No such file or directory)"
   :at [java.io.FileInputStream open0 "FileInputStream.java" -2]}]
 :trace
 [[java.io.FileInputStream open0 "FileInputStream.java" -2]
  [java.io.FileInputStream open "FileInputStream.java" 195]
  [java.io.FileInputStream <init> "FileInputStream.java" 138]
  [clojure.java.io$fn__9520 invokeStatic "io.clj" 229]
  [clojure.java.io$fn__9520 invoke "io.clj" 229]
  [clojure.java.io$fn__9433$G__9426__9440 invoke "io.clj" 69]
  [clojure.java.io$fn__9532 invokeStatic "io.clj" 258]
  [clojure.java.io$fn__9532 invoke "io.clj" 254]
  [clojure.java.io$fn__9433$G__9426__9440 invoke "io.clj" 69]
  [clojure.java.io$fn__9494 invokeStatic "io.clj" 165]
  [clojure.java.io$fn__9494 invoke "io.clj" 165]
  [clojure.java.io$fn__9446$G__9422__9453 invoke "io.clj" 69]
  [clojure.java.io$reader invokeStatic "io.clj" 102]
  [clojure.java.io$reader doInvoke "io.clj" 86]
  [clojure.lang.RestFn invoke "RestFn.java" 410]
  [clojure.lang.AFn applyToHelper "AFn.java" 154]
  [clojure.lang.RestFn applyTo "RestFn.java" 132]
  [clojure.core$apply invokeStatic "core.clj" 648]
  [clojure.core$slurp invokeStatic "core.clj" 6657]
  [clojure.core$slurp doInvoke "core.clj" 6657]
  [clojure.lang.RestFn invoke "RestFn.java" 410]
  [scot.weft.i18n.core$raw_get_messages invokeStatic "core.clj" 148]
  [scot.weft.i18n.core$raw_get_messages invoke "core.clj" 126]
  [clojure.lang.AFn applyToHelper "AFn.java" 160]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 646]
  [clojure.core$memoize$fn__5708 doInvoke "core.clj" 6107]
  [clojure.lang.RestFn invoke "RestFn.java" 436]
  [smeagol.util$raw_get_messages invokeStatic "util.clj" 59]

The path is correct:

simon@fletcher:~/workspace/smeagol$ jar tvf target/smeagol-0.99.8-SNAPSHOT-standalone.jar | grep i18n/en-GB.edn
  7337 Tue Aug 08 23:52:24 BST 2017 i18n/en-GB.edn

Don't know. Not fixed.

[desolate-planet]

Very strange, I duplicated the problem yesterday, updated that line of code and the main smeagol page was displayed. I'll nuke my local copy and checkout development again to verify.

[desolate-planet]

I did as I said above and the main landing page/login page is displayed, which was not happening previously.

I'm developing this on Windows at the moment, which I doubt is a factor. Is this another section of the site your getting this error?

[desolate-planet]

Can you verify with just my latest change? I'm thinking you might have other stuff lingering around that might be causing this i.e another issue from the I18n?

[simon-brooke]

OK, where you running from an ubrerjar file? The recipe is

lein ring uberjar
export SMEAGOL_CONFIG=`pwd`/resources/config.edn#
export SMEAGOL_CONTENT_DIR=~/tmp/test-content
java -jar target/smeagol-0.99.8-SNAPSHOT-standalone.jar 

Obvs, point SMEAGOL_CONTENT_DIR at some directory which exists and is writable (but doesn't contain anything you value!)

That has to work.

[desolate-planet]

I've not done any of that. I just did

`lein bower install
 lein ring server

Originally, I got the error about resolving the I18n file, but after adding my fix, I can now see the main page.

I will retry with your recipe above.`

[simon-brooke]

It could be in the i18n library, but I don't think so. I realise I haven't pushed the i18n library up to Github - will do that.

[simon-brooke]

Yes I have - thought I had - just forgotten the name.

[desolate-planet]

I'm getting something different:
java.io.FileNotFoundException: C:\development\clojure-work\smeagol\temp_output_side-bar.md (The system cannot find the path specified)

The path is accessible. I assume I've missed a step to generate content to that folder as it's looking for _side-bar.md on startup.

[desolate-planet]

Ok, I definitely must be missing something as when I start up smeagol following your guide ,I get the above error. What I did to correct this was copy the content of C:\development\clojure-work\smeagol\resources\public\content (which has _side-bar.md) to C: C:\development\clojure-work\smeagol\temp_output and smeagols auth page rendered correctly.

[desolate-planet]

Never mind, I misunderstood what you said before. The name of the variable says it all SMEAGOL_CONTENT_DIR !!! I've just pointed it at that and as I said before, the smealgol auth & home page displays fine for me.

[desolate-planet]

In addition to it working with uberjar, I've tried uberwar (with my latest pull request) and it deploys fine to the latest tomcat. I am able to view the landing page and login as admin/admin.

[desolate-planet]

Have you had a chance to check my latest push? I've tested on both tomcat and ring/jetty. Seems to work fine for me.

[simon-brooke]

Sorry, been unwell. On it now.

[desolate-planet]

no worries, hope your on the mend. Let me know if that latest pull request sorts things.

[simon-brooke]

OK, this still isn't working, but I think it's better.

The underlying problem is that I'm confusing files with resources and vice-versa, and this problem exists in botth the Smeagol code base and in the internationalisation code base. I think it's fixed now in internationalisation, but the resources are not being picked up in unit tests which I think is a unit test misconfiguration rather than a code error (but might not be).

Smeagol is BETTER, in that it will now run from a Jar file without internationalisation breaking, but it's not fixed.

[simon-brooke]

Fix.