WIP: Add query autocomplete in code builder

[sbordeyne]

Related to issue #338

I started implementation of syntax highlighting and autocompletion in the victorialogs datasource plugin.

I took the implementation from Loki's datasource as the base, and am working on gradually modifying the implementation to fit VictoriaLog's LogsQL instead of Loki's LogQL, the two are still quite similar, thus it's a good starting point.

This PR is currently a work in progress, and not ready to merge

Currently supported:

• Stream labels and value autocomplete
• Basic unpack_$parser pipe

---

Note: I added a few services in compose.yaml to add logs to VictoriaLogs for manual testing during development, if needed I can easily revert the changes from that commit.

[arturminchukov]

@sbordeyne thanks for pr. Currently, there is a plan to implement the autocompletion feature on the part of VictoriaLogs. This will enable us to have the same auto-completion mechanism in both the plugin and the Victoria Logs UI, as well as support the newly added LogsQL functions. We should wait for this feature to avoid unnecessary effort costs. Will duplicate this message to the related issue to update the status.

[sbordeyne]

@sbordeyne thanks for pr. Currently, there is a plan to implement the autocompletion feature on the part of VictoriaLogs. This will enable us to have the same auto-completion mechanism in both the plugin and the Victoria Logs UI, as well as support the newly added LogsQL functions. We should wait for this feature to avoid unnecessary effort costs. Will duplicate this message to the related issue to update the status.

@arturminchukov If I understand correctly, VictoriaMetrics means to implement autocompletion on the backend ? In my opinion it seems like a pretty bad idea, even if I understand the goal, which is to avoid duplicating code. This sort of feature has to be done purely on the frontend for performance reasons, making an API call at each keystroke to get completion suggestions would be incredibly wasteful.

I see that there already is some autocompletion available in VMUI. A better alternative might just be to separate the completion code from VMUI and into a JS package, that way it could be reused in both the grafana datasource and VMUI with minimal work. Both are using React, which would make it relatively easy to export just a component for autocomplete. It could also mean improved query parsing, since both the parser used in VMUI and the parser used in the grafana datasource do not seem to match the actual LogsQL grammar from what I could tell.

[arturminchukov]

@arturminchukov If I understand correctly, VictoriaMetrics means to implement autocompletion on the backend ?

yes, you're right.

which is to avoid duplicating code.

Not only because of the duplication of code, but also because the LogsQL parser has already been implemented on the backend, which is necessary to provide suggestions correctly. Parser could be changed in the future(e.g., adding new functions), so it will be easier and better just to maintain one place in code.

making an API call at each keystroke to get completion suggestions would be incredibly wasteful

Currently, you need to do api calls to get the fieldnames and fieldvalues

[sbordeyne]

Not only because of the duplication of code, but also because the LogsQL parser has already been implemented on the backend, which is necessary to provide suggestions correctly. Parser could be changed in the future(e.g., adding new functions), so it will be easier and better just to maintain one place in code.

A parser is also required to provide syntax highlighting, does that mean that syntax highlighting won't come to the datasource ? I can't imagine trying to write a query and have the colors come in after a 100ms delay.

Currently, you need to do api calls to get the fieldnames and fieldvalues

True but that can be cached on the frontend pretty easily, which is what the loki datasource does already.

Regardless, I won't challenge the decision to do autocompletion on the backend further, though what would be the ETA on that feature coming to the datasource ? Are there endpoints done already to provide completions for a partial query ? I could do the implementation on the datasource side if that would help