[Bug]: Candles stream does not handle empty/zero-volume candles very well

[wwestgarth]

Problem encountered

On discord a user pointed out that the candles stream is a bit odd on markets where no trades occur, and this seems to be due to the handling of empty candles.

On testnet the market 89cc0f6d1616a14c79e7e9a310ef47bdbee0a6874654581ee1f4999c4fcdcd96 is inactive where the last trade occurred on the 20th Jun. If you stream any candles for this market you receive nothing apart from the old candle from months ago where you would expect updates with empty candles:

curl https://api.n00.testnet.vega.rocks/api/v2/stream/candle/data?candleId=trades_candle_15_minutes_89cc0f6d1616a14c79e7e9a310ef47bdbee0a6874654581ee1f4999c4fcdcd96

The reason for this seems to be because when you query candles with no end-timestamp you do not get trailing empty candles. This can bee seen here:

https://api.n00.testnet.vega.rocks/api/v2/candle?candleId=trades_candle_5_minutes_89cc0f6d1616a14c79e7e9a310ef47bdbee0a6874654581ee1f4999c4fcdcd96

but if you do supply an end-timestamp you do get trailing empty candles:

https://api.n00.testnet.vega.rocks/api/v2/candle?candleId=trades_candle_5_minutes_89cc0f6d1616a14c79e7e9a310ef47bdbee0a6874654581ee1f4999c4fcdcd96&toTimestamp=1723112638634349000

The way the candle stream service asks for candles updates is without an end-timestamp:

candles, _, err = s.candleSource.GetCandleDataForTimeSpan(ctx, s.candleID, &start, nil, entities.CursorPagination{})

which in times with no trades will result in no empty candles coming through, and the odd effect where once a trade does occur we will suddenly get all of the empty candles through to the stream at the same time and then the non-empty candle. If a trade never comes through then candle stream remains stuck on the last candle with volume.

Observed behaviour

The stream-candle endpoint does not handle empty-candles very well.

Expected behaviour

The stream-candles endpoint works as expected when empty candles exist.

Steps to reproduce

1. Submit `xyz` transaction
2. With this payload
3. Query `abc` API
4. Observe the error

Software version

develop

Failing test

No response

Jenkins run

No response

Configuration used

No response

Relevant log output

No response