Real world demo app with jsoo-react

[jchavarri]

Hey 👋

I migrated @jihchi Real World demo app to jsoo-react:

https://github.com/jchavarri/jsoo-react-realworld-example-app

Initially the main purpose was to measure bundle size (which I will be doing and publishing results next), but in the end it turned out to be a great way to test jsoo-react in some close to real scenario, and see the kind of things that were missing.

In general I'd say the experiment is quite successful, here are some highlights:

• Usage of data-xxx attributes can be improved, but it's possible (see with_test_id.ml)
• Bindings to fetch were missing, implemented from bs-fetch (thanks @glennsl and contributors), see fetch.mli
• Usage of ppx_jsobject_conv to avoid having to implement serializers manually, see shape.ml.
  • Ideally I'd like to have an atd runtime for js_of_ocaml, but first we'd need some library in the same spirit than bs-json.
• Bindings with gen_js_api, probably some optimizations can be done to produce require("foo") calls, see markdown.mli

The readme contains installation instructions. It uses very simple approach to bundling with Webpack.

Let me know what you think, or if there are any improvements or questions. Maybe this can serve as baseline to measure bundle size over time, test API proposals etc. :)

[glennsl]

Nice! This will be very useful for both testing and showcasing I think.

• Usage of ppx_jsobject_conv to avoid having to implement serializers manually, see shape.ml.
  • Ideally I'd like to have an atd runtime for js_of_ocaml, but first we'd need some library in the same spirit than bs-json.

Any reason you're not also considering using yojson directly? I believe that should already work with JSOO.

[jchavarri]

Main reason is that ppx_jsobject_conv expects a JS object as input, which means I can use response.json() when getting data from server. This defers to the browser the parsing from string to json, with some benefits:

• runtime performance: I doubt Yojson parser from string to JSON will run faster than any browser implementation of response.json or JSON.parse
• bundle size: prob yojson requires more code to run, as it needs to perform this string to JSON parsing stage, which ppx_jsobject_conv avoids altogether
• simplicity :)

This is not only applicable to network requests with fetch, but any JSON can be parsed (fast) using JSON.parse and then proceed to pass it to ppx_jsobject_conv decoder.

There was an interesting conversation in https://discuss.ocaml.org/t/json-encode-decode-in-js-of-ocaml/8539/12.

Still, even with all these advantages, I still have the impression that bundle size increases way too much than what it should with the encoders/decoders generated by ppx_jsobject_conv. 🤔

[glennsl]

Thanks! Great explanation.

ppx_jsobject_conv seems to make sense when consuming a JSON API that is fairly close to your application needs, which I think is the case here. Decoders such as those from bs-json are very useful when consuming a JSON API that does not conform to your application needs. And while ATD can build on that, I don't think it would be able to offer much of the flexibility of decoders, and therefore not add much value over just using ppx_object_jsconv.

For my own use I'm more interested in sharing type definitions and functionality between the client and server, however. This is the niche I think both yojson and ATD fits best, with some important differences between them:

• ATD requires using a separate mini-language for type definitions, while yojson uses ordinary OCaml type definitions.
• ATD requires a separate build step to do code generation, while yojson uses a deriving PPX.
• ATD seems very limited in its ability to have shared code associated with its type definitions.
• yojson should work with jsoo already. But then so should ATD as it can generate yojson codecs.
• ATD is more flexible in that it can generate code for different languages and runtimes. It's not entirely clear to me if yojson can be optimized for jsoo.

Ideally, since I'm going to use OCaml on both client and server, I'd like to skip the added complexity of ATD with its separate definition language and code generator, and just have ordinary OCaml type definitions with deriving annotations and associated functions defined in the same module. yojson should already work for this, and compared to ATD, the downside is "merely" an optimization problem. I'm not sure it's possible to optimize yojson specifically however, as its JSON type is not opaque, but in theory it should be possible to derive an optimized implementation without needing the complexity of ATD.

So I'm thinking perhaps it would be better to fork ppx_deriving_yojson and make it runtime-independent? Unless there is something else desirable about ATD which we can't get through a simpler deriving mechanism?

[jchavarri]

ppx_jsobject_conv seems to make sense when consuming a JSON API that is fairly close to your application needs, which I think is the case here.

Why do you think this? I see it equally powerful as ppx_deriving_yojson (except it's lacking the backend / native side).

This is the niche I think both yojson and ATD fits best, with some important differences between them

I believe yojson and atd main difference is that one is based on OCaml language and the other is not, with everything that entails. yojson depends in ppx infra (which is its own can of worms, especially with a ppx with such large scope as yojson), while atd skips this issue altogether. The main downside in my experience with atdis that tooling like ocamlformat does not work with it. The fact that it requires special rules to be built is not ideal, but in the end is not that much of a big deal (type modules like those in atd change less over time that modules with actual functions).

Being its own syntax gives atd a lot of room on the design space to solve problems that are specific to the API interface space. For example, one nice feature of atd is that it allows to update the protocol without breaking existing clients in a very expressive way: https://atd.readthedocs.io/en/latest/tutorial.html#smooth-protocol-upgrades.

I also think adding a jsoo specific runtime to atd —so that jsoo can have more optimized generated code— might be slightly easier because there's already the precedent of https://github.com/ahrefs/bs-atdgen-codec-runtime/. It is also very well integrated with OCaml, providing escape hatches like falling back to custom wrapping/unwrapping functions, or validating data on the edge, which is very useful.

---

It's not entirely clear to me if yojson can be optimized for jsoo.

Have you checked https://github.com/janestreet/ppx_yojson_conv? It seems this is the version of the ppx that is more modern, and it uses the same foundation as ppx_jsobject_conv I think. Maybe it's possible to use both but only pick the jsobject functions on JS side? e.g.

type 'a user_body = { user : 'a } [@@deriving yojson, jsobject ]

I'm not sure how this should work in practice, maybe some wrapper is needed.

If you don't have a lot of bundle size constraints, I would probably try atd as is, with some real use cases (including API migrations, etc). atd has been built precisely for the use case that you seem to be interested in, and this pays off in the long term, in my opinion.

[glennsl]

Thanks for the detailed response!

Why do you think this? I see it equally powerful as ppx_deriving_yojson (except it's lacking the backend / native side).

Why I think ppx_object_jsconv is a better choice than ppx_deriving_json here? Just because it's lighter and faster, which is probably precisely because it doesn't have to support native.

The main downside in my experience with atd is that tooling like ocamlformat does not work with it.

Lacking ocamlformat is a very minor issue for me, but the fact that it's a different language with different syntax (although very similar to OCaml) and, more importantly, different semantics is a much bigger problem. It's not entirely obvious how its concepts map to OCaml, or how OCaml concepts map to it, which means there's more to learn, to remember, and a much bigger need for documentation. The documentation is fortunately both comprehensive and pretty good, but several of the sites associated with ATD are already dead. I fear the documentation will eventually go as well, making it even less accessible.

The second major problem is the conceptual complexity of it. Not so much the existence of a special build rule in itself, but how the inputs connect to the outputs and how that in turn needs to be connected other pieces of shared code (e.g. validation, computed properties etc.) that should be associated with it. It might be that my ignorance is exaggerating this problem, but the only way I can imagine this working is to have common code files alongside the .atds, then have additional rules to copy those files as well, so they can depend on the atd-generated type definitions. Not unbearable, but quite a bit more complex than using a PPX.

For example, one nice feature of atd is that it allows to update the protocol without breaking existing clients in a very expressive way

I'm not sure what this does that couldn't also be accomplished with yojson derivers (using [@default] and such). There doesn't seem to be explicit support for versioning, which would be nice. Perhaps there's some conceptual difference I don't understand that somehow distinguishes consumers and producers for example, making it easier manage upgrades? Ultimately it ends up as plain OCaml either way though, and potentially as yojson specifically, so it seems limited by what's possible to express directly using yojson anyway, though perhaps more inconveniently.

I also think adding a jsoo specific runtime to atd —so that jsoo can have more optimized generated code— might be slightly easier

That's probably true, possibly a lot easier even, but it's also a problem that only needs to be solved once. Integrating and using it is a problem that needs to be solved for every project.

Have you checked https://github.com/janestreet/ppx_yojson_conv? It seems this is the version of the ppx that is more modern, and it uses the same foundation as ppx_jsobject_conv I think. Maybe it's possible to use both but only pick the jsobject functions on JS side?

That's an interesting idea! I'll need to dive into dunes documentation for this. Wouldn't surprise me if they have something for this exact use case buried somehwere. Worst case it's probably possible to write a simple PPX that picks one or the other depending on an environment variable or something. Or that one such already exists.

If you don't have a lot of bundle size constraints, I would probably try atd as is, with some real use cases

Unfortunately we'll start off with client-side only, so I will be very limited in what I can test. I'll try to make a simple server to do some real use case testing though, but will still have some more time to study the options. Thanks again for helping me understand all this!