key not found: new-reg

[fellipetenorio]

Hi, I'm trying to use this project but in the sample (and in my project with the dependency) throws this exception:

java.util.NoSuchElementException: key not found: new-reg at scala.collection.MapLike$class.default(MapLike.scala:228) at scala.collection.AbstractMap.default(Map.scala:59) at scala.collection.MapLike$class.apply(MapLike.scala:141) at scala.collection.AbstractMap.apply(Map.scala:59) at io.github.valters.acme.AcmeProtocol.package$Directory.get(AcmeProtocol.scala:746) at io.github.valters.acme.AcmeProtocol.package$AcmeServer.<init>(AcmeProtocol.scala:755) at io.github.valters.acme.AcmeController$$anonfun$io$github$valters$acme$AcmeController$$cer $1.apply(AcmeController.scala:166) at io.github.valters.acme.AcmeController$$anonfun$io$github$valters$acme$AcmeController$$cer $1.apply(AcmeController.scala:164) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl. a:121) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.pollAndExecAll(ForkJoinPool.java:1253) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1346) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) 2017-12-19 18:57:22,739 [error] - Failure on all levels: java.util.concurrent.TimeoutException: Futures timed out after [200 seconds] java.util.concurrent.TimeoutException: Futures timed out after [200 seconds] at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223) at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190) at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$2$$anon$4.block(ExecutionContextImpl.scala:48) at scala.concurrent.forkjoin.ForkJoinPool.managedBlock(ForkJoinPool.java:3640) at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$2.blockOn(ExecutionContextImpl.scala:45) at scala.concurrent.Await$.result(package.scala:190) at io.github.valters.acme.AcmeController.finishChallenge(AcmeController.scala:325) at io.github.valters.acme.AcmeController.io$github$valters$acme$AcmeController$$certify(AcmeController.scala:182) at io.github.valters.acme.AcmeController$$anonfun$certify$1$$anonfun$apply$1.apply$mcV$sp(AcmeController.scala:135)

[valters]

It means we tried to get the server "directory" JSON, but it does not contain "new-reg" expected key. I wonder if the GET actually worked at all.
See
https://acme-staging.api.letsencrypt.org/directory
https://acme-v01.api.letsencrypt.org/directory

What JSON payload did you receive?

[valters]

A good idea (for me) would be to catch the exception and then wrap it into something else, to also print received JSON payload that something is confusing the client. Or possibly there is no error check earlier because we did not get the JSON payload at all (some error occurred during GET or similar issue).

[fellipetenorio]

From what I saw the Request was 200 and the JSON was well formatted.
Here is the log:

[info] play.api.Play - Application started (Dev) [info] i.g.v.a.AcmeHttpClient - GET https://acme-staging.api.letsencrypt.org/directory [debug] i.g.v.a.AcmeHttpClient - GET https://acme-staging.api.letsencrypt.org/directory receives response OK [info] i.g.v.a.AcmeHttpClient - body= { "key-change": "https://acme-staging.api.letsencrypt.org/acme/key-change", "meta": { "terms-of-service": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf" }, "new-authz": "https://acme-staging.api.letsencrypt.org/acme/new-authz", "new-cert": "https://acme-staging.api.letsencrypt.org/acme/new-cert", "new-reg": "https://acme-staging.api.letsencrypt.org/acme/new-reg", "revoke-cert": "https://acme-staging.api.letsencrypt.org/acme/revoke-cert", "tPr7Voz9A9Q": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417" }, nonce= Some(Vg6mh7X8AxbgW-vQgIQEB_vS511fKhmcmIpxZO8gM94) java.util.NoSuchElementException: key not found: new-reg at scala.collection.MapLike$class.default(MapLike.scala:228) at scala.collection.AbstractMap.default(Map.scala:59) at scala.collection.MapLike$class.apply(MapLike.scala:141) at scala.collection.AbstractMap.apply(Map.scala:59) at io.github.valters.acme.AcmeProtocol.package$Directory.get(AcmeProtocol.scala:746) at io.github.valters.acme.AcmeProtocol.package$AcmeServer.<init>(AcmeProtocol.scala:755) at io.github.valters.acme.AcmeController$$anonfun$io$github$valters$acme$AcmeController$$certify$1.apply(AcmeController.scala:166) at io.github.valters.acme.AcmeController$$anonfun$io$github$valters$acme$AcmeController$$certify$1.apply(AcmeController.scala:164) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

[the-pavels]

So basically the issue is that format of "directory" seems to be changed and now "meta" is not a string, but an object. The solution is to drop this field (as we dont use it for parsing).
Something like:

val directory = Json.parse( jsonBody ).as[JsObject] - "meta" directory.validate[AcmeProtocol.Directory] match {

But there are more issues, seems that Challenge format is also changed.