-
Notifications
You must be signed in to change notification settings - Fork 7
do we need a server
TL;DR: It is highly unlikely that we can get away with not having a server, as we have not yet found a technology stack that works without a server. It's probably better to assume we need a server to set up connections between clients, and further explore a true peer-to-peer system later.
In order to sync between different devices in the field, we need some mechanism for devices to find and communicate between each other. A Server (in this case) is some system that does not run inside the web app (in the case of Android/iOS) or web browser (other OSes), in order to allow devices to find each other. Ideally, we want to have no server, but we may have to have one (or bundle one inside the apps), if browsers do not provide sufficient tools to talk between devices without a server.
In addition, we should consider how stable and reliable a particular setup/technology stack is, as this has implications for usability. Such stability/reliability information should be provided to users in the documentation in a understandable manner, so users are aware of and can make meaningful judgements around what they take to the field.
Websockets do not work for peer-to-peer: in the browser, websockets are client only (we cannot create a server - i.e. we can connect to but cannot bind to a port).
WebRTC seems like the best option: wide support, somewhat standard (there are polyfills which cover systems which don't match the standard), peer-to-peer. However, it's not entirely clear how to do the initial setup without a server (as most examples are trying to handle communicating over NAT, which hopefully we don't need to worry about). WebRTC is the most promising option.
-
Stack Overflow asking about exactly what we want to do w.r.t. LAN discovery
-
WebRTC helper lib, has server linked (PeerJS)
Lots of discussion around doing this in browsers online, however there's no current implemented standard (that I've found), so likely lots of issues if we tried to do this. If WebRTC fails, this could be a fallback option, but likely a time sink. On mobile devices where we have more access to networking, this is probably a viable option.
- https://www.w3.org/TR/discovery-api/ - discontinued standard which would do what we need.
Let us take the following as starting requirements for our system---we will relax these, and note when we do, but we're going to start with the most restrictive set of requirements:
-
There is no server which stores field data, syncs data between clients, or provides discovery of peers (i.e. entirely peer-to-peer).
-
There is no distinction between peers: neither OS nor browser causes a client to have more/less (sync) features than a client using a different browser/OS---this implies not installing additional software on clients.
-
Networking between peers can be assumed.
-
Clients have some way to check the validity of synced data (which is covered by Authentication & User management).
We will also assume that in terms of weighting what we do not want a server to do, storage > syncing > discovery (as the former depends on the latter).
First, we note that unless we do get WebRTC working without a server on all our target systems, we are either going to need a server for discovery (checking MDN and https://caniuse.com/, James cannot see any other standards that are on our target systems) - which invalidates our first requirement, or we will need to write custom Android and iOS support for discovering peers, and have this peer discovery work in laptop browsers. The latter is going to be both harder to maintain, and very likely less reliable (in effect, we are writing two implementations of a server, and trying to run it on a mobile platform). Therefore, we need a server which will perform (at a minimum) peer discovery.
How then should our peer discovery work with the server(s)? Either we hardcode a DNS name and/or IP into the app, or let the user enter the details. The latter does require the user to have some technical knowledge, but possibly we can reduce the barriers by providing some default options (i.e. "Android hotspot", "iPhone/iPad hotspot", "Mac hotspot"). We could let mobile devices do peer-to-peer discovery independently of laptop browsers (e.g. using mDNS, which is widely supported on all OSes apart from Windows), but this seems to be a value-add, rather than a way to avoid having a server.
If we do have a custom device (e.g. RPi), we can both set up the network with reasonable network defaults, have a DNS/IP specifically for the discovery server, and avoid the need for platform-specific code (for syncing). This is probably also the most reliable option (we shouldn't need to worry about the device standing by/low power mode, we have full permissions to install/update software etc.). In effect, unless WebRTC can be done completely peer-to-peer, every alternative must be weighed against this. A custom device will be the most reliable setup to take into the field, given we can thoroughly test it, and be able to provide more complete documentation and troubleshooting information
The next step down from the custom device is to either:
-
have a server (written in Node.js) that runs on a laptop
-
write platform specific code to run a server on Android and iOS (this will likely be quite large in terms of the codebase)
Given the preference to have a single language (as much as possible) for the whole system, the laptop server seems like the better option (and then can be reused for the custom device). Given we'd want this to run on as many different systems as possible, we'd likely look at running the system in docker. However, this assumes users have permission to install docker/software in general. Referencing Networking setup for FAIMS, the preference for systems would be Unix/Linux > MacOS > Windows (based on the fact that setup would be easier if the system running the server was also providing the WiFi networking). There are definite questions about reliability here (given the diversity of devices available, and their configuration), so while we may be able to provide some troubleshooting information, its completeness in covering all possible issues that users may encounter vs that of the custom device will be much reduced.
Summary: server provides discovery, syncing and data transfer is peer-to-peer.
The above assumes we can sync data peer-to-peer (either with pouch natively supporting this, or via WebRTC). If we cannot get this to work (either we have issues with pouch or we cannot get WebRTC and pouch to work together), then we are going to need the server to also perform the syncing between devices. Exactly how to do this best (e.g. do we run pouch or couch) needs thought, but ideally syncing via the server isn't needed.
The above assumes that peers can validate the data that it sent to them via a peer. If they cannot do this, then this is in effect the same case as the one above (cannot sync peer-to-peer), but the server must also be able to validate the data---this rules out anything on Android/iOS, and makes the case for a custom device even stronger.