Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OBSOLETE FEATURE: indexed objects for pgsphere #238

Open
kimakan opened this issue May 13, 2024 · 2 comments
Open

OBSOLETE FEATURE: indexed objects for pgsphere #238

kimakan opened this issue May 13, 2024 · 2 comments

Comments

@kimakan
Copy link
Contributor

kimakan commented May 13, 2024

I guess that at some point there were plans to use the extra column of type spoint(ra, dec) which is indexed and then, it can be used instead of ra, dec. I'm not quiet sure whether the implementation was completed. The code in question is

# first run to replace with get_indexed_objects
processor.set_query(query)
processor.process_query(indexed_objects=get_indexed_objects(), replace_schema_name={
'TAP_SCHEMA': settings.TAP_SCHEMA,
'tap_schema': settings.TAP_SCHEMA,
'TAP_UPLOAD': settings.TAP_UPLOAD,
'tap_upload': settings.TAP_UPLOAD,
})

def get_indexed_objects():
indexed_objects = {}
for column in Column.objects.exclude(index_for=''):
# TODO implement xtype 'spoint' properly
#if column.datatype not in indexed_objects:
# indexed_objects[column.datatype] = [column.indexed_columns]
#else:
# indexed_objects[column.datatype].append(column.indexed_columns)
if 'spoint' not in indexed_objects:
indexed_objects['spoint'] = [column.indexed_columns]
else:
indexed_objects['spoint'].append(column.indexed_columns)
return indexed_objects

IMHO, the feature is obsolete because the indexes can be created directly for the spoint() expression. Meaning that instead of creating a new column pos as spoint(ra, dec), creating an index for it and then trying in some hacky way to replace spoint(ra,dec) with the indexed column, we just create an index as

Create index schema_table_spoint_ra_dec on schema.table  using gist (spoint(RADIANS(ra), RADIANS(dec)));

No need to create an extra column in the database and no need to connect ra/dec with the spoint(ra,dec) in the translated query.

Removing this feature would simplify daiquiri and handling of the database.

@jochenklar
Copy link
Member

Maybe we just did not know enough about databases back then. Did you test if your solution is as fast as the one with the extra column?

@kimakan
Copy link
Contributor Author

kimakan commented May 14, 2024

Yes, we have already tested (and still using) the index spoint(RADIANS(ra), RADIANS(dec)) for gaia DR3. In the case of the cone search, the new solution is even faster (~3x).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants