Merge pull request #111 from lxylxy123456/master

Release 0.5.0

Author: lxylxy123456

README.md

@@ -1,170 +1,24 @@
 # ngshare
 [nbgrader](https://github.com/jupyter/nbgrader) sharing service.
 
+<img src="ngshare/favicon.png" width="64px" />
+
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 [![Build Status](https://travis-ci.org/lxylxy123456/ngshare.svg?branch=master)](https://travis-ci.org/lxylxy123456/ngshare)
 [![codecov](https://codecov.io/gh/lxylxy123456/ngshare/branch/master/graph/badge.svg)](https://codecov.io/gh/lxylxy123456/ngshare)
 [![Documentation Status](https://readthedocs.org/projects/ngshare/badge/?version=latest)](https://ngshare.readthedocs.io/en/latest/?badge=latest)
 
-**This service is under development. Use this at your own risk.**
-
-Click [here](#installation-and-setup) for installation instructions.
-
-<img src="ngshare/favicon.png" width="64px" />
-
 ## What can I use it for?
 You can use ngshare if you
-* Need to run nbgrader on a distributed set up (probably using
- [lxylxy123456/nbgrader](https://github.com/lxylxy123456/nbgrader))
-* Have something similar to nbgrader that also needs an API to manage courses,
- homework submissions and feedbacks, etc.
-* Want to learn Flask, SQLAlchemy, or Tornado Web Server. 
+* Need to run nbgrader on a distributed set up (such as Zero to JupyterHub, using DockerSpawner, etc)
+* Need better course management without messing with filesystem permissions and user groups
 
 ## Documentation
-The API specifications and documentation for `ngshare` are in
- [Read the Docs](https://ngshare.readthedocs.io/en/latest/).
-
-## Installation and setup
-* See [/testing](/testing#testing-setup) for setting up `ngshare` and `JupyterHub` for simple testing.
-
-To install `ngshare` onto your cluster with some default values, simply do:
-
-`helm install ngshare helmchart/ngshare`
-
-We recommend using some configurations manually. Here's a sample `config.yaml` file:
-```yaml
-deployment:
-  resources:
-    limits:
-      cpu: 100m
-      memory: 128Mi
-
-ngshare:
-  # You may omit this, but the token will be randomly generated.
-  # It's recommended to specify an API token here.
-  hub_api_token: ENTER_TOP_SECRET_TOKEN_HERE
-
-pvc:
-  # Amount of storage for ngshare
-  storage: 1Gi
-```
-For a reference of all the options you can specify, check [here](/helmchart/ngshare/values.yaml).
-
-After you install, you should see a message like this:
-```
-Congrats, ngshare should be installed!
-To get started, add the following to your JupyterHub helm chart's values:
-
-hub:
-  extraConfig:
-    ngshare.py: |
-      c.JupyterHub.services.append({
-        'name': 'ngshare',
-        'url': 'http://ngshare:8080',
-        'api_token': 'a4IHeiHZuswZrmYbWxSGpLZs3x0pXVxa'})
-```
-You should:
-1. Follow the first part of the instructions, and add the `extraConfig` part into your JupyterHub's helm chart.
-2. Modify your singleuser image to install our fork of nbgrader, and add some configuration to your default `nbgrader_config.py` so it uses ngshare. You can add something like this to your singleuser Dockerfile:
-```dockerfile
-RUN pip install git+https://github.com/lxylxy123456/nbgrader@exchange_server && \
-jupyter nbextension install --symlink --sys-prefix --py nbgrader && \
-jupyter nbextension enable --sys-prefix --py nbgrader && \
-jupyter serverextension enable --sys-prefix --py nbgrader
-
-COPY nbgrader_config.py /etc/jupyter/
-```
-with an accompanying `nbgrader_config.py` like this:
-```python
-from nbgrader.exchange import ngshare
-c = get_config()
-c.ExchangeFactory.exchange = ngshare.Exchange
-c.ExchangeFactory.fetch_assignment = ngshare.ExchangeFetchAssignment
-c.ExchangeFactory.fetch_feedback = ngshare.ExchangeFetchFeedback
-c.ExchangeFactory.release_assignment = ngshare.ExchangeReleaseAssignment
-c.ExchangeFactory.release_feedback = ngshare.ExchangeReleaseFeedback
-c.ExchangeFactory.list = ngshare.ExchangeList
-c.ExchangeFactory.submit = ngshare.ExchangeSubmit
-c.ExchangeFactory.collect = ngshare.ExchangeCollect
-```
-Afterwards, the setup should be complete.
-
-## Note about users in JupyterHub and ngshare
-In ngshare, all users (instructors and students) are identified using their username in JupyterHub. They are authenticated using the API token inside their notebook server. Be careful when reusing usernames in JupyterHub, as users with the same name will be identified as the same. We haven't added functionality to rename or delete users in ngshare, so be sure not to delete a user and create a new one with the same name. If you do, you will have to manually edit the ngshare database to remove or rename that user.
-
-## Demo
-If you are configuring our project correctly, you should be able to run this demo.
-1. Setup a clean environment using JupyterHub + nbgrader + ngshare (debug mode). You can use the [minikube testing setup](/testing#testing-setup) to do it easily.
-2. Login as user "user". All usernames are login-able with any passwords.
-3. Go to "Control Panel" at upper right corner, then Services -> ngshare
-4. Click on "init with test data". You should see
-	`{"success": true, "message": "done"}`.
-5. Login as user "kevin".
-6. Create a new file with New -> Text File, name it `nbgrader_config.py` and
-	with the following content:
-```
-c = get_config()
-c.CourseDirectory.course_id = "course1"
-```
-7. Go to "Control Panel", click on "Stop My Server"
-8. Click on "Start My Server"
-9. Go to "Formgrader".
-10. Click "Add new assignment..."
-11. Click on the name of the assignment you just added
-12. New -> Notebook -> Python 3, and edit the notebook as in normal nbgrader
-	1. Add some code to the block
-	2. View -> Cell Toolbar -> Create Assignment
-	3. Select "Autograded answer"
-	4. ...
-	5. Save notebook
-13. Click "Generate" in Formgrader
-14. Click "Release" in Formgrader
-15. Login as user "lawrence" (you may want to use incognito mode).
-16. Go to "Assignments" tab
-17. Click "Fetch" for the new assignment (the one that is not "challenge")
-18. Do your homework.
-19. Click "Submit".
-20. Login as user "kevin".
-21. Click "Collect" in Formgrader. 
-22. You should see "1" under "# Submissions". Click on this number. 
-23. Click "Autograde"
-24. Click Student Name, and then the notebook name, then write some feedback and
-	click "Next".
-25. Go back to "Manage Assignments"
-26. Click "Generate Feedback", and "Release Feedback" in order.
-27. Login as user "lawrence".
-28. Under "Assignments", click "Fetch Feedback"
-29. You should see "(view feedback)" on the right of the time stamp, but do not
-	click on it. 
-30. Go to "Files" tab and go to `<assignment name>/feedback/<timestamp>`, then
-	you can view the html feedbacks. 
-
-### Youtube Video Demo
-http://www.youtube.com/watch?v=iiaVpKLj89c
-
-[![Youtube Video Demo](http://img.youtube.com/vi/iiaVpKLj89c/0.jpg)](http://www.youtube.com/watch?v=iiaVpKLj89c)
-
-## Database migrations
-ngshare uses [Alembic](https://alembic.sqlalchemy.org/) to manage database
- migrations.
-
-For development, first install ngshare as a repo using
- `pip3 install . --user --upgrade`, then initialize the database using
- `python3 dbutil.py upgrade head` (the path to database is defined in
- `dbutil.py`, which is `sqlite:////tmp/ngshare.db` by default).
+The API specifications, installation guide, and documentation for `ngshare` are
+ in [Read the Docs](https://ngshare.readthedocs.io/en/latest/).
 
-After changing database structure, use `pip3 install . --user --upgrade` and
- then `python3 dbutil.py revision --autogenerate -m "message"` to automatically
- detect changes, then `python3 dbutil.py upgrade head` to upgrade database
- structures.
+## Youtube Video Demo
+http://www.youtube.com/watch?v=SEJCaqD7xXQ
 
-Ngshare / vngshare automatically checks database upgrade each time it starts up.
- You are expected to have a good way to backup your database before running
- ngshare / vngshare in case alembic corrupts the database. If you want to stop
- this behavior you can use `--no-upgrade-db`. If you do this, you must manually
- upgrade the database when upgrading ngshare. Only use this option if you know
- EXACTLY what you're doing, otherwise using incompatible versions will cause
- things to break catastrophically.
+[![Youtube Video Demo](http://img.youtube.com/vi/SEJCaqD7xXQ/0.jpg)](http://www.youtube.com/watch?v=SEJCaqD7xXQ)
 
-## Code formatting
-`black -S -l 80 ngshare`

docs/api/authentication.rst

@@ -4,9 +4,9 @@ Authentication
 ngshare Authentication
 ----------------------
 
-This section is under construction
+``ngshare`` uses JupyterHub authentication tokens to authenticate the user. This is usually in the ``JUPYTERHUB_API_TOKEN`` environment variable in each user's notebook servers. ``ngshare`` will use this token to fetch the username of the current user. The username is the only information used to identify the user.
 
-For ``ngshare``, use JupyterHub authentication token.
+To send the token to ``ngshare``, use the ``Authorization: token`` header in HTTP requests to ``ngshare``.
 
 GET Example
 ^^^^^^^^^^^
@@ -33,7 +33,7 @@ POST Example
 vngshare Authentication
 -----------------------
 
-For ``vngshare``, supply the username to the GET param or POST data ``user``.
+For ``vngshare``, there is no password authentication. The username is specified in the GET param or POST data field ``user``.
 
 GET Example
 ^^^^^^^^^^^
@@ -54,4 +54,3 @@ Post Example
     Content-Length: 38
 
     instructors=%5B%22eric%22%5D&user=root
-

docs/contributer_guide/decisions.rst

@@ -1,8 +1,6 @@
 Decisions
 =========
 
-This section is under construction
-
 Technologies Employed
 ---------------------
 

docs/contributer_guide/development.rst

@@ -40,7 +40,4 @@ We use `black <https://github.com/psf/black>`_ to format our code.
 Contributing
 ------------
 
-If you want to contribute to ``ngshare``, submit a pull request to `https://github.com/lxylxy123456/ngshare/ <https://github.com/lxylxy123456/ngshare/>`_.
-
-This section is under construction.
-
+If you want to contribute to ``ngshare``, submit a pull request to `https://github.com/lxylxy123456/ngshare/pulls <https://github.com/lxylxy123456/ngshare/pulls>`_.

docs/contributer_guide/project_structure.rst

@@ -53,6 +53,20 @@ Deployment
 
 Testing
 -------
-``testing/`` is for installing ``ngshare`` in a Kubernetes setup.
+``testing/`` contains setups used for testing ngshare, ngshare_exchange, nbgrader, and Z2JH.
 
-This section is under construction
+``testing/docker`` contains a Docker environment for initial testing. It is slightly out of date and still uses our fork of ngshare rather than ngshare_exchange.
+
+``testing/minikube`` contains a minikube environment. This is the main testing setup for local development, and it uses ngshare and ngshare_exchange on the local filesystem.
+
+``testing/install_jhmanaged`` contains a Docker environment that demonstrates how a regular user would install ngshare and ngshare_exchange.
+
+``testing/install_z2jh`` contains a minikube environment that demonstrates how a regular user would install ngshare and ngshare_exchange on a standard Kubernetes cluster.
+
+ngshare_exchange
+----------------
+The client side of ngshare is packaged into a `separate repo <https://github.com/lxylxy123456/ngshare_exchange>`_.
+
+``ngshare_exchange/*.py`` implement a nbgrader pluggable exchange that uses ngshare to release, fetch, and submit assignments.
+
+``ngshare_exchange/course_management.py`` will be installed as the ``ngshare-course-management`` command. It is used for admins and instructors to manage course rosters.

docs/index.rst

@@ -23,6 +23,13 @@ ngshare
 
 To solve this problem, we are letting exchange to gather all information it needs from a set of REST APIs, which is implemented by ``ngshare``.
 
+Youtube Video Demo
+------------------
+
+.. raw:: html
+
+    <p><iframe width="640" height="360" src="https://www.youtube.com/embed/SEJCaqD7xXQ" frameborder="0" allowfullscreen></iframe></p>
+
 Table of Contents
 -----------------
 

docs/user_guide/install_z2jh.rst

@@ -1,7 +1,7 @@
 Installing on a Z2JH Cluster
 ============================
 
-This guide assumes you are already familiar with installing Z2JH. You should also be familiar with using Helm.
+This guide assumes you already have a Kubernetes cluster with a persistent volume provisioner (which should be the case if you run Z2JH). You should also be familiar with installing Z2JH and using Helm.
 
 If you prefer looking at examples instead, `here's <https://github.com/lxylxy123456/ngshare/tree/master/testing/install_z2jh>`_ a sample installation setup. It doesn't demonstrate all the configurable options, though.
 

docs/user_guide/notes_admin.rst

@@ -6,10 +6,20 @@ Admin Users
 -----------
 Admin users are the only users who can create courses and assign instructors to them. This is to prevent unauthorized users from creating courses. All admins have full access to every course on ``ngshare``, so keep this in mind when assigning admins. Courses can be created and managed using the `ngshare-course-management <course_management.html>`_ tool that comes with ``ngshare_exchange``.
 
+User Name Reuse
+---------------
+In ngshare, all users (instructors and students) are identified using their username in JupyterHub. They are authenticated using the API token inside their notebook server. Be careful when reusing usernames in JupyterHub, as users with the same name will be identified as the same. We haven't added functionality to rename or delete users in ngshare, so be sure not to delete a user and create a new one with the same name. If you do, you will have to manually edit the ngshare database to remove or rename that user.
+
 Race Condition
 --------------
 ngshare should NOT be run concurrently, or there may be race conditions and data may be corrupted. For example, do not create multiple ngshare instances that share the same underlying database.
 
+Storage
+-------
+If you're using the Helm chart, only 1GiB of storage is allocated by default. You may increase this limit by specifying `pvc.storage` in the Helm values. If ngshare returns 500 for requests, lack of storage space could be a reason.
+
+Also, when courses or assignments are deleted, their corresponding files are not automatically deleted. You may want to delete these files to clean up storage. See the Removing Semantics section below for more info.
+
 Database Upgrade
 ----------------
 ngshare checks the database version every time it starts up. If the database version is older than the ngshare version, it performs schema and data migration. 

ngshare/version.py

@@ -1 +1 @@
-__version__ = '0.4.0'
+__version__ = '0.5.0'