Documentation compatible with pypi

[mn5hk]

No description provided.

[abhimhamane]

Sorry to bother you again, but I found some functions with missing documentation. There are no doc strings for functions like clm2cs, clm2sc, eigengrav, klm2sc, many more they only have script docstrings and acknowledgements.

[mn5hk]

Hi @abhimhamane ,

Thanks for pointing this out. I will follow up in a few days.

Best,
AS
c.c. @lsmvivek

[abhimhamane]

Hi @mn5hk @lsmvivek ,

This is related to the presentation and styling of the code. It is generally a best practice to make the code compliant with the PEP8 Python style guide. It will make the codebase more consistent and accessible. I saw some minor inconsistencies in the spacing and commenting.

Some examples:

1. Line no: 51, 59, 64 could have used # instead to ''' ... ''' for single line comments
2. In order to make code more readable we could use consistent spacing like in line no 58
3. For showing operator precedence we could use spacing properly.
4. degree_order=int((Lmax+1)*(Lmax+2)/2) reads better if it were degree_order = int((Lmax+1) * (Lmax+2)/2)

I'm not sure about this one, but what is the difference between importing a module within and outside the function? I have rarely seen modules being imported within a function.

I would like to know your thoughts on this. It would be a straightforward task for me to fix as a newbie.

Best,
Abhishek

[mn5hk]

Hi @abhimhamane,

Right now, our next task would be to build a quality documentation. Substantial work has taken place in the direction already, but it still needs further inputs to be finalized.

Before we further proceed, two different approaches to documentation can be done: my preference would be to get both of them done (although one could also be enough). These are:

• A github wiki page https://github.com/mn5hk/pyshbundle/wiki
• A github pages document https://mn5hk.github.io/pyshbundle/

In the github wiki page, we have provided documentation for most of the codes already. As you have rightly pointed out earlier in the thread, some of the codes are still missing some documentation (https://github.com/mn5hk/pyshbundle/issues/31#issuecomment-1500379188). For the codes missing documentation, I will need to support you closely. I will try to do it, rather slowly, over few weekends.

For the codes for which documentation already exists, these were first written (before ensuring pip compatibility) as independent program documentation. Could you edit it in a way to make it in sync with the new changes and pypi compatibility? Additionally, the formatting is rather rough on the document. Feel free to work around and make it look better.

I would recommend you to create a new github wiki page (https://github.com/mn5hk/pyshbundle/wiki), and start merging all the other wiki page information there. You can start editing the documentation for pypi compatibility as well as better formatting. Once this is ready, we will add the same content to the github pages document as well.

Once this step is completed, I expect the package to be near completion for the first draft release. After this, we can maybe discuss how to take the project even further at some point.

Thanks a lot for your contributions,
AS

[abhimhamane]

Hello @mn5hk ,

Apologies for the delayed response. It's actually the last week of this semester, so a bit busy with assignments, projects and other deadlines before the end-sem exam.

It's a good idea to develop both the GitHub Wiki and Documentation webpage using GitHub Pages. Sure, I'll first complete the documentation of functions with doc strings. I can work with you later, whenever you're free, for those function which do not have doc strings.

I'll review the PyPi Documentation guidelines and report with the major styling and formatting recommendations. After which, I can format the codebase to ensure compatibility.

P.S. Can you update the paper_abhimhamane.pdf with the latest one, I think the latest one should not have any formatting or affiliation related errors. Latest one after the build - paper_joss_latest.pdf

Regards,
Abhishek

[abhimhamane]

Hi @mn5hk ,

I was going through the best practices for documenting a Python project. I have tried to summarise what I learned. I also propose certain approaches to be incorporated into our project.

Documentation

Popularity of any module does not solely depend on its feature capability, but on the ease of usability for others. Ease in terms of usability, bug tracking, bug-fixing and new-features. A comprehensive documentation is essential for this to happen.

Key features of a well written documentation (based on organization of django documentation):

• Reference-guides - They describe how module works and how to use it but assume that you have a basic understanding of key concepts so basically the doc-strings make up this section.

• Tutorials - self-contained basic which is basically a walk through for explaining applicability of the module; one of the best example - GEE and Geemap tutorials by Prof.Qiusheng Wu. It can be videos or easy-to-follow Jupyter notebooks.

• Topic-Guides - A discuss key topics and concepts at a fairly high level and provide useful background information and explanation

• How-to-guides - These are more advanced that tutprials.

Similar approach have been used by the following scientific processing libraries like numpy, matplotlib, scipy, etc.

Code-the-Docs

The standard is to follow the Code-the-Docs approach. Python has special dunder function __doc__ which extracts the doc-strings. Packages like Sphinx or MkDocs provide automated way of extraction doc-strings, formatting and exporting as web pages, LaTeX or any other format. Markdown is generally a good choice but a more pythonic way is to use reStructuredText.

Intro to Python Enhancement Proposal PEP-8 guidelines

Provides certain recommendations and best practices to write more pythonic code and make it more accessible, managable, bug-free and human readable:

• Max. length of character in single line = 80
• Indent using spaces instead of Tab
• use consistent "" or '' for strings
• Variable naming convention - use snake_case instead of camelCase or PascalCase
• Naming convention for constants should be in all caps - INV_FLATTENING
• Naming convention for functions - similar to that of variables
• Naming convention for Class names - PascalCase
• Naming Exceptions - NameException since it acts as a class
• There are certain conventions regarding writing algebraic expressions
• There are some more major guidelines and many minor recommendations which are not that important. It's easily accessible at the site mentioned

There are some tools which automate the static formating checking process - popular ones are flake8 or black. This can easily be incorporated as a CI GitHub Action or setup onto code-editors like VS-Code using standard plugins.

Code-the-Docs approach also means we can use the automated Docs generating tools like sphinx or MkDocs, former being the standard. Python documentation itself was generated using sphinx.

Read the Docs simplifies software documentation by automating building, versioning, and hosting of your docs for you. It's actually an open-source project, many of the prominant libraries have been hosted here like

Proposals

1. I proporse to follow the PEP-8 conventions along with numpy style guide - example numpy docs. That is we use reStructuredText instead of Markdown for writing doc-strings.

   • Module summary
   • fuction summary
   • parameters - type and description
   • Returns - type and description
   • Other params / optional params - type and description
   • Raises - Exceptions and
   • See also - relavant other function
   • References - research paper or other references
   • Examples - A simple use-case with simplified inputs

2. I propose to use Sphinx as it is much more versatile than MkDocs. Sphinx has a steeper learning curve as compared to MkDocs but keeping long term in view, I guess it would be useful. Sphinx or MkDocs can export the docs directly as HTML or LaTeX

3. Using the open-source Read the Docs to organize and publish docs on our GitHub pages site. A lot of python projects use this.

4. After this we can focus on other components like tutorials and How-to-guides.

Some Good Documented Libraries

1. NumPy and SciPy - Standard scientific processing libraries of Python
2. Leafmap - Geospatial package

Important VS-Code Plugins

1. Markdown All in one
2. autoDocstring
3. pylint
4. flake8
5. reStructuredText

Important Resources - Website/ Blogs/ YouTube

• Building Docs like Code - sphinx, read the docs - PyCon presentation
• Comments v/s Doc-Strings - post with examples
• Documenting Your Code with Python - Simple Introduction

[mn5hk]

Sounds great

[abhimhamane]

Hi @mn5hk ,

I was thinking that do we need the license, acknowledgement and referred papers to be included in each of the file. I was thinking we can remove it and have this info in the README alone.

As I mentioned earlier, the convention in Python is to use snake_case for variable and function naming, I found that some sections have camelCase or PascalCase. Should we change it or let it be for the time being?

Let me know what do you think.

Regards,
Abhishek

[mn5hk]

Hi @abhimhamane ,

Thanks for pointing out the issues. Let them both be as it is for now..

Best,
AS

[mn5hk]

@abhimhamane the case mismatch is because we have tried to retain as much as possible from the original codes.

In case a user wants to compare the python and the matlab code versions, we hope the choice of variable nomenclature in our python implementation would assist their work.

[abhimhamane]

Hi @mn5hk ,

I have solved the math-markdown error in the docs/index.md. I guess you had setup MkDocs to build the docs. In order to interpret the LaTeX code, mkdocs.yml needed a MathJaxextension. I have added that and I build it on my local machine and its working.

I tried changing the theme to the popular Read-the-Docs. If its not useful I can revert back to the original material theme.

Regards,
Abhishek

[mn5hk]

@abhimhamane looks awesome, well done :)

[abhimhamane]

Thanks @mn5hk , I merged the changes but its not getting reflected in the github pages site. Can you tell me how did you setup the github actions for automated deployment. I do not find any .github/workflow file for the site deployment.

[mn5hk]

@abhimhamane I believe the doc file is made; when I implemented the following tutorials for publishing Python package with pypi:

• https://www.youtube.com/watch?v=DAPAv9KbYZ0&list=PLAxJ4-o7ZoPeUqGpMhvJoVk5G-TrvMAd-&index=12
• https://www.youtube.com/watch?v=7FcX9uWDuIQ&list=PLAxJ4-o7ZoPeUqGpMhvJoVk5G-TrvMAd-&index=12

Feel free to go through them in your free time.

Also, can you share the updated draft paper with me? I will share it with the team for review.

Best,
AS

[abhimhamane]

Hi @mn5hk , There is some issue with the mkdocs.yml and ci-docs-gh-pages.yml I edited those to test out the static docs website on my forked repo. But I guess I was not cautious enough while creating a pull request. I'll make the necessary changes and make sure this does not happen again. Like I said yesterday, I cannot figure out why the docs does not seem to work, it worked fine when I served it on my locahost:8000.

Regards,
Abhishek

[mn5hk]

@abhimhamane I have my suspisions that running the setup.py file should make it work; I will check it over some weekend.

Thanks,
AS

[abhimhamane]

Hi @mn5hk , I guess I found the issue. I hosted the docs on my gh-page (https://abhimhamane.github.io/pyshbundle/). Due to my previous commit there is a new branch gh-pages in the repo. Can you try changing the pages setting for this repo from main branch to gh-pages branch.

[mn5hk]

Hi @abhimhamane, thanks for the update. I just changed the branch to gh-pages as you have requested. However, it does not appear to affect the page. I will let it be for some time, just in case it takes time to refresh (highly unlikely imo).

[abhimhamane]

Hi @mn5hk , you have selected the /docs. I think it should be /root instead, cause the index.html is in the root folder of gh-pages.

[mn5hk]

Hi @abhimhamane thanks. The update has been made and is successful.

[abhimhamane]

Hi @mn5hk, Can you guide me on how I should test out the code for any unwanted changes I might have made. I see there are no unit tests, but there are some test .txt scripts. Till I test out my code you can follow the updated docs at - https://abhimhamane.github.io/pyshbundle/

Regards,
Abhishek