Account for FILTERs when considering greedy query planning

[joka921]

Since #1442, QLever switches to greedy query planning for large connected components. A connected component is considered large when the number of connected subgraphs is above the threshold determined by the runtime parameter query-planning-budget.

So far, FILTERs were simply ignored when counting the number of subgraphs. However, FILTERs can add significant complexity to the standard query planning because for each subplan, our query planner considers either adding all applicable FILTERs to it or none of them. As a result, for certain queries with a medium-sized component but a significant number of FILTERs, the query planning complexity was underestimated and the query was not planned greedily and the standard query planning took very long.

This is now fixed by replacing, for the purpose of query planning, each FILTER by a dummy VALUES clause which uses the set of distinct variables from the FILTER. A FILTER that has many variables in common with other triples will then increase the subgraph count substantially. If multiple FILTERs have the same set of distinct variables, the dummy VALUES clause is added only once (because our query planner either adds all applicable FILTERs at a certain point or none of them). Note that this trick overestimates the true query planning complexity. That is, the worst that can happen now is that with many FILTERs, we switch to greedy planning even though standard query planning would have still been feasible,

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.87%. Comparing base (acb6633) to head (982cff7).
Report is 19 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1705   +/-   ##
=======================================
  Coverage   89.86%   89.87%           
=======================================
  Files         389      389           
  Lines       37308    37329   +21     
  Branches     4204     4209    +5     
=======================================
+ Hits        33527    33549   +22     
+ Misses       2485     2484    -1     
  Partials     1296     1296           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hannahbast]

Looks great + nice trick with the filters. A further optimization would be to not have the dummy VALUES "plans" (one for each FILTER) have edges between them.

[hannahbast]

After discussion with Johannes: only add one dummy VALUES clause for each distinct set of variables in a FILTEr. Then we can still overestimate, but the cases where that happens will be rare, and the only bad outcome then is that we compute a greedy query plan in a case, where non-greedy would have worked as well.

[sparql-conformance]

Conformance check passed ✅

No test result changes.

Details: https://qlever.cs.uni-freiburg.de/sparql-conformance-ui?cur=982cff756dc2d031e44813db62ea73aeb6ac33a7&prev=acb6633debc7341985341aff147b5038cc8d951b

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[hannahbast]

Thanks a lot + I wrote a proper description and will merge this now!