Distributing the market price data in the P2P network

[HenrikJannsen]

Currently we load the market price data via http request from the price nodes. We have clearnet and tor nodes but no I2P nodes (could be set up).
I saw in dev testing that quite often the requests fail when using Tor. The application is heavily dependent on a price feed (see #1160).

One solution for not requiring I2P based price nodes and for higher reliability for price data would be to broadcast the price data from the oracle node (or a dedicated node). We could broadcast those data with a lower frequency to avoid to add too much load to the network (e.g. 5 min. instead of 1 min. as used for http requests). The oracle node would just do normal http requests but as the oracle node runs with clearnet and tor, there is no need for I2P.

The user will receive then at startup immediately price data and start themself to request every minute by the http request. They take the most recent data, so if the http request succeed the distributed data are ignored.

It adds a bit of additional trust into the oracle node (which is a bonded role and used for reputation and verifying other bonded roles). But also the price nodes need to be trusted, so there is not really much difference here, and as the more recent data win it has even less relevance.

It adds a bit of complexity to the code but reduces the risk when http requests fail (which is not uncommon) and reduce the burden of recovery strategies (we try other nodes if one fails).

For dev testing it is not required to set up an oracle node as the http requests still are in use (only for I2P it would fail).

Any feedback?

[namloan]

What's the availability % of the price nodes and what can be done to improve it? We'd need accurate metrics of this and scoping the potential action items to properly assess whether increasing the complexity here is worth it.

Answers to the questions above would also help resolve one of the main points from #1160 - are we going to let users create or take offers when no price is available at all?

No doubt this proposal increases resilience - I think we'd need some numbers to determine whether we should prioritize it vs something else.

[HenrikJannsen]

I don't have metrics, but at dev testing I get sometimes (using clear net) no response. Might be rate limiting on the server side and there is only one clearnet server. But Tor comes with its own instability risks and delays (an aspect not mentioned above).
UX would be increased if the price data is available at the first get inventory response, rather to depend on an extra tor http call which might in the worst case time out and then the next is used. That would result in > 2 min. no price data and required handling at the domain/UI level.
If we can reduce the probability that no price data is available once the application is usable (initial data has arrived), we can deactivate all use cases where price data is required (e.g. creating/taking an offer). That would be easier than to figure our an alternative way if price data is not available (not sure if that is feasible).

[namloan]

Thanks @HenrikJannsen that was helpful.

If we can reduce the probability that no price data is available once the application is usable (initial data has arrived), we can deactivate all use cases where price data is required (e.g. creating/taking an offer)

Let's take this direction then.

We could use #1160 to stop allowing creating and taking offers when there's no market price and create a new issue for distributing the market price data from the node.

[HenrikJannsen]

Yes, I guess blocking price data dependent trade operations might be the easiest way. We might find a better solution later, but for now we could go for that.

[namloan]

do we want to allow fixed price offers operations when there's no market price?

[HenrikJannsen]

yes that might be an option, but i am not sure how much effort on the dev side that creates. maybe easier to first block it and look into that later.