e-Callisto NG - Rest API for e-Callisto data

[kenfus]

Provide a general description of the issue or problem.

Hello, I have developed a Rest API for E-Callisto and wrapped it as a python package:

https://pypi.org/project/ecallisto-ng/

it also contains some basic preprocessing and plotting. However, I would rather develop Radiospectra instead of inventing everything twice.

There are still some missing features, such as:

• no metadata from the fits file is in the database (what is exactly needed?).
• better error handling, for requests which return no data.
• error handling for crazy request (e.g. give me three years of full resolution data).
• instruments have different capitalization because of SQL limitations.

I am unsure on how to wrap this into Radiospectra, especially with the metadata. Any support would be of great help!

[samaloney]

Hi @kenfus that's really nice work, would be great to integrate as much as possible and I am keen to do so.

I think it should be possible to write a Fido client to query the API much like the current clients work and then transform the data and store the Spectrogram container. We already have some issues open to add in some features that were lost in recent rewrite but I haven't had the time to get around to them yet (e.g. background subtraction).

For the metadata try to pass the entire fits header to the Spectrogram container but currently only rely on having the instrument/detector, time info, frequency info and the data array.

One area that is a bit unclear is that so far radiospecta (and sunpy) has mainly file based for various reasons (reproducibly, long term stability, lack of alternatives, ...). The benefits of an API powered approach are clear but the file based approach also has it's place it so question of how both can be supported in a sensible way.

We could discuss this at the sunpy community meeting held on Wednesday's at 16:00 UTC or if that doesn't work we could arrange another meeting time? Might be easier to discuss over a call than on here.

I'd also be really interested in the backend code, looks like FastAPI from what I could see? In particular if the API could return a list of urls of files which match the query that would be really useful.

[kenfus]

Yes, Wednesday at 16:00 UTC works! See you tomorrow, then.

Thank you for your suggestions and feedback.

[samaloney]

The link for the meeting is http://sunpy.org/jitsi