Skip to content

corto_select should return objects from store when MOUNT_QUERY is not specified #628

@hendrenja

Description

@hendrenja

When a mount specifies OWNER_LOCAL ownership, objects in the object store that are owned by that mount are ignored by corto_select. To ensure that corto_select does not return duplicate results and provides the full dataset, it will query the mount and use the resulting data instead.

This ensures that corto_select does not have to check for duplicate values- which would increase space complexity to O(n) (instead of the current O(1)) and algorithmic complexity to O(log n) (instead of the current O(1)).

However, when a mount does not implement onQuery, corto_select will take objects from the object store, as in this case the mount must be storing objects directly in the store. This behavior should be extended with also falling back on the object store when CORTO_MOUNT_QUERY is not provided in the mount mask (indicating that the mount onQuery method should be deactivated).

Currently, when not specifying CORTO_MOUNT_QUERY in combination with a CORTO_OWNER_LOCAL ownership policy, corto_select will not invoke onQuery and also ignore data from the object store, which results in returning an empty set.

This issue can be reproduced by configuring a mount (with onQuery or onHistoricalQuery implementations) to a mount point such as /data/weather. Assume there is data in the object store along the data/weather/x/y/z path. If the mount is configured without MOUNT_QUERY or MOUNT_HISTORICAL_QUERY masks, then Corto will not return data. Instead, the data from the object store should be returned.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions