Packet definition organization

[tloubrieu-jpl]

Hello,

In my project, a ccsds parser for Europa clipper, I would like to organize the packet definitions (VariableLength object instances) as plugins so that I don't need to declare in different places the packet structure and the apid it applies to. That would make the extension of this parser more smooth and elegant.

I wanted to retrieve my plugins/ccsds structure definitions using namespace packages, see https://packaging.python.org/en/latest/guides/creating-and-discovering-plugins/#using-namespace-packages

I was thinking of using 'ccsdspy.packets' or in my case ' ccsdspy.packets.europa_clipper'.
That could be a way a standardizing the publication of packet definitions which could be public (eg on pypi) and make them re-usable by anyone. A different mission could then publish a package namespace ' ccsdspy.packets.other_mission'.

I could start that on my side and tell you more when I am done, but I believe that would not work since the ccsdspy project has an init.py which would prevent the root package to be seen as a namespace package.

Any thoughts on that ? I can start a different namespace I guess but let me know if there was any similar plans on the publication of packet definitions or making ccsdspy a namespace package.

Thanks !

Thomas

[ddasilva]

This is an interesting idea, but I'm cautious of taking on the responsibility of curating/pushing the community's packet definitions with the library. I do agree that making the packet definitions more accessible and easy to find would be a benefit to all.

Would it be possible to publish your packet definitions under a PyPI, so people can do pip install europa_clipper_packetdefs?

If someone wants to lead the effort, I think it's within our scope to put up a section on our website called "Publishing Packet Definitions" which includes:

• instructions on how to publish packet definitions per our recommendations
• list of missions/instruments and the associated modules that followed our recommendations

This could function kind of like AstroPy affiliated packages (https://www.astropy.org/affiliated/) or the Python in Heliophysics community (https://heliopython.org/projects/). Your effort with Europa Clipper could prototype these recommendations.

Any thoughts @ehsteve ?

[tloubrieu-jpl]

Hi @ddasilva ,

I agree with your approach where the packet definition code would not be part of the core ccsdspy library/repo, but would be maintained in different repositories in GitHub, published as additional packages in pypi, according to guidelines provided on the ccsdspy web site.

Time permitting, I can try to demo that with our Europa clipper packets. I could propose the guidelines by documenting the demo.

We can use that thread to collect new ideas or feedback on that.

[tloubrieu-jpl]

Here is an early proposal to expose/share CCSDS packet definition, in a common python namespaced package.

For convenience it is hosted in a pre-existing open-source project, but the CCSDSpy packet definition code could be managed in standalone repositories and published on pypi. They would be then available as mission or instrument plugins which can be integrated in documentation generation software, test dataset generation and of course parsing code.

Here is the example for Europa-Clipper:

https://github.com/nasa/pyDownlinkParser/tree/main/src/ccsds/packets/europa_clipper

The packets are defined in package ccsds.packets.<mission>.<instrument>.<packets type>

We use these definitions to parse the downlinks for 3 instruments of the mission as part of the Science Data System.

If we can consolidate the representation of packet definition across missions that would ease the re-usability of the software around this packet definitions, as we already can do for parsing.

Your feedback is welcome on the structure of the namespace package in the perspective that it could be re-used across missions.

Thanks