Skip to content

Commit

Permalink
re-factored and rewrote readme
Browse files Browse the repository at this point in the history 
some of the content went into various rst files in `docs/`
  • Loading branch information
@ofloveandhate
ofloveandhate committed Jul 11, 2024
1 parent 15645d8 commit f462b4c
Show file tree
Hide file tree
Showing 5 changed files with 65 additions and 157 deletions.
185 changes: 30 additions & 155 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
A library for containerizing local markdown content to be published into the Canvas learning management system.

:warning: This library is under active development and the interface is NOT stable.
[Documentation on Github Pages](https://ofloveandhate.github.io/markdown2canvas/)

---

# Why I wrote/maintain this library

The particular problem this library solves is that of putting Canvas content under version control, and also using Markdown for that content. Canvas pages are not well-suited to version control *per se*, because they live on the LMS. I wanted local files, with repos I can share with other designers and instrutors.

Further, I want to be able to find-and-replace across many pieces of Canvas content at once. Local files with my editor of choice is the way to do that; it's impossible on Canvas. Hence, this library.

Additionally, uniform appearance and ability to change much with little effort. I wrote a "style" system that puts headers and footers around my content, eliminating repetitive and error-prone work. Want emester-specific text at the top of all your content? Trivial with `markdown2canvas`: just change the header file, re-publish, and do something better with your time than wait for Canvas pages to load.

A secondary problem this library solves is that of images. Images on Canvas are bare, and it's easy to end up with duplicate versions, as well as not have alt text. By using markdown/html under version control, I can write my alt text directly into page source, instead of using the crappy click-heavy interface on Canvas.

---
Expand All @@ -20,138 +24,26 @@ Containerization is accomplished by making pages/assignments live in folders. T
- `meta.json` -- a json file containing attributes. keys should be compliant with the expectations of `canvasapi`.
- `source.md` -- a markdown file. Can contain latex math, html, references to local images, emoji using double-colon notation and shortcodes, and of course online images.



---

# Testing

This library comes with several test pages/assignments:
- pages:
- `plain_text` -- source is just plain old text. start simple, ya know?
- `uses_latex` -- a page that contains latex math
- `has_remote_images` -- a page that has remote images embedded
- `has_local_images` -- a page that uses local images
- `uses_droplets` -- a page using [the Droplets framework from UWEX](https://media.uwex.edu/app/droplets/index.html).
- `uses_droplets_via_style` -- a page using [the Droplets framework from UWEX](https://media.uwex.edu/app/droplets/index.html). The code enabling Droplets comes from a header/footer contained in a style folder. The main purpose of this test page is the header/footer style thing.
- assignments:
- `programming_assignment` -- an assignment that has a local image

The above list is not exhaustive.
You can also put needed files, images, etc in the folder for a piece of content. `markdown2canvas` aims to automate as much of the process as possible.

---

# Installation

1. Clone the repo / pull from the repo
2. Move to repo location in terminal
3. `pip install .` If you already had it installed, then use `pip install . --upgrade` to make sure you get the newer version.

## Critical setup step, do not skip this

You must also define an environment variable called `CANVAS_CREDENTIAL_FILE`, which is the location of a `.py` file containing two variables:
1. `API_URL` -- a string, the url of how to access your Canvas install.
- At UW Eau Claire, it's `https://uweau.instructure.com/`.
- I cannot possibly tell you your url, but your local Canvas admin can.
2. `API_KEY` -- a string, the key you can get from Canvas. Here's [a link to a guide on how to generate yours](https://community.canvaslms.com/t5/Admin-Guide/How-do-I-obtain-an-API-access-token-in-the-Canvas-Data-Portal/ta-p/157). Do not share it with anyone -- having only this one piece of data, anyone can act as you. Protect it at least as much as you would any other password or sensitive information.


# Alternate-ish Installation for Windows Users

These instructions are tested on Windows 11 on February 26, 2024.

## Get Canvas Credentials and Make Canvas Credential File - this is the same step as above.

Your first step will need to be to get a Canvas API Key.

1. On Canvas, navigate Accounts -> Settings
2. Scroll to the button labeled `+ New Acces Token`
3. Add a description for yourself to know, later, what the access token is for and optionally add an expiration date. (I like to make a new one every semester, for safety.)
4. Copy the text of the token (you won't get to see this again) to a file that we will name `canvas_credential_file.py`.
5. Create a variable in `canvas_credential_file.py` named `API_KEY`, whose value is the string that we just copied from canvas.

Additionally, add a second variable `API_URL` whose value is the string that is the general Canvas URL you use. For UWEC, this is `'https://uweau.instructure.com/'`.

Ultimately, your `canvas_credential_file.py` will contain the lines:
Please see [the documentation](https://ofloveandhate.github.io/markdown2canvas/) -- we have two tutorials, one for Mac/Linux and one for Windows.

```
API_KEY = "stringofrandomcharacters"
API_URL = "https://uweau.instructure.com/"
```

## Initial Setup - Using VS Code

1. Install python via the Microsoft Store
2. Install VS code and GitBash - at UW Eau Claire this is done via the software center, you might use the Microsoft Store for this step as well.
3. Clone the markdown2canvas repo from github
4. Open VS code, open GitBash terminal and run the command

```
pip install /path/to/markdown2canvas
````
Then also run the command
```
pip install lxml beautifulsoup4
```
Note that the default terminal that VSCode opens will be the Windows powershell, don't use that.
## Generate necessary global variables
1. Run the following command to make a file called `.bashrc` and save the location of your canvas credential file in your home directory.
```
echo 'CANVAS_CREDENTIAL_FILE=h:\\path\\to\\canvas_credential_file.py' >> ~/.bashrc
```
Note that you should be using `\\` here as directory separators because you are using Windows. If you use `/` you run the risk of the operating system not understanding the path.
2. Open a new git bash terminal and see if the following works:
```
echo $CANVAS_CREDENTIAL_FILE
```
if not, you might need to run the following command in your bash terminal:
```
source ~/.bashrc
```

---

# Some quick examples
This library is under active development. I suggest checking out the `test_*.py` files in the `test` folder for example code.
Assuming you did my setup step, defining the environment variable and creating that file. Do that first.
### Download all pages, with a filter on the name of the pages
```
import markdown2canvas as mc
course_id = 127210000000003099 # silviana's sandbox for development

canvas = mc.make_canvas_api_obj() # gets link and api key via environment variable
course = canvas.get_course(course_id)

destination = 'downloaded_pages'
# Some things you can do with this library

my_filter = lambda title: '📖' in title # pages about readings have an open book in their names.
mc.download_pages(destination, course, even_if_exists=True, name_filter=my_filter)
```
---
# Things you can do with this library
See the [the documentation](https://ofloveandhate.github.io/markdown2canvas/). This is just highlights in a root readme.

## Replacements during translation

The purpose of this library is to increase modularity and flexibility, while reducing duplication in source code and allowing version control. I implemented a simple text replacement feature as part of this, so that I can create uniform appearances in my content without duplicate code.

That is, you can specify a set of string replacements using a .json file, and during translation from markdown to html, before uploading, each substitution happens.
That is, you can specify a set of string replacements using a `.json` file, and during translation from markdown to html, before uploading, each substitution happens.

For example, you can create a `replacements.json` file in a folder at root level (relative to the folder for the course) called `_course_metadata`, and in this file put the content:

Expand All @@ -164,13 +56,6 @@ For example, you can create a `replacements.json` file in a folder at root level
}
```

I'm using simple python `.replace` to do the replacements. There are consequences:
* It will replace strings exactly, there are no implemented efforts to allow patterns or functions.
* It's case sensitive, and includes exact spacing.
* The dollar signs above are NOT special. They're just a nice way to indicate the text will be replaced.
⚠️ Furthermore, I'm not sure what order the replacements will be done in, so if the target of one replacement includes the source of another, I can't guarantee you at this time that it will actually happen in a deterministic order. If you want this feature, please add it and submit a PR to this repo.
You can specify a default set of replacements to happen for every file (except those with overridden replacements). To do this, make a file `_course_metadata/defaults.json`, and create a record `"replacements": "relative/path/to/replacements_filename.json"`. The name of the replacements file is arbitrary, and it's relative to root of the course folder.

To override the default replacements, put a record in the `meta.json` file for the content (page / assignment) of the form `"replacements": "relative/path/to/replacements_filename.json"`.
Expand All @@ -180,8 +65,19 @@ Examples of content using replacements can be found in the `test/` folder of thi
If a replacements file doesn't exist where you say it should, an exception will be raised at `publish` time for the `CanvasObject` (`Page` or `Assignment`). (You can construct a thing with a bad replacements file and not know it until you try to publish!)


## Referencing existing Canvas assignments, pages, and files
ℹ️ I'm using simple Python `str.replace` to do the replacements. There are consequences:

* It will replace strings exactly, there are no implemented efforts to allow patterns or functions.
* It's case sensitive, and includes exact spacing.
* The dollar signs above are NOT special. They're just a nice way to indicate the text will be replaced.

⚠️ Furthermore, I'm not sure what order the replacements will be done in, so if the target of one replacement includes the source of another, I can't guarantee you at this time that it will actually happen in a deterministic order. If you want this feature, please add it and submit a PR to this repo.



## Reference existing Canvas assignments, pages, and files

Whereas I find it to be a pain to link to other content on Canvas using their editor, it's easy using `markdown2canvas`.

To link to an existing Canvas assignment, use a link of the form

Expand All @@ -197,17 +93,22 @@ To link to an existing Canvas file, use a link of the form
<a href="file:DavidenkoDiffEqn.pdf">Link to file called DavidenkoDiffEqn.pdf</a>
```

If the "existing" content doesn't yet exist when the content is published, a broken link will be made. This is ok. Think of the publishing process using Markdown2Canvas similar to the compilation of a TeX document, which is done in multiple passes. Once the page / assignment / file exists, the link will resolve correctly to it. Publish all content about twice to get links to resolve.
ℹ️ If the "existing" content doesn't yet exist when the content is published, a broken link will be made. This is ok. Think of the publishing process using Markdown2Canvas similar to the compilation of a TeX document, which is done in multiple passes. Once the page / assignment / file exists, the link will resolve correctly to it. Publish all content about twice to get links to resolve.


## Emoji conversion from shortcodes

This library supports the automatic conversion of shortcodes to emoji. For example, `:open_book:` goes to 📖. I use the [`emoji`](https://pypi.org/project/emoji/) library to do this. [Shortcodes can be found here](https://carpedm20.github.io/emoji/).

Right now, emoji shortcodes can only be used in content, not in names of things -- shortcodes in names will not be emojized.



## Automatic uploading and warehousing of images and embedded content

List your images relative the folder containing the `source.md` for the content.
List your images relative the folder containing the `source.md` for the content and they'll automatically be uploaded when you publish. If the image is already uploaded, a link to the existing image will be generated instead of uploading.



## "Styling" -- Automatic inclusion of uniform headers and footers

Expand All @@ -216,37 +117,11 @@ This library attempts to provide a way to uniformly style pages across sections
* I have content in my course in four blocks, and want a different header for each block. But, copypasta for that header content sucks (avoid repitition is a key tenet of programming). So, I'd rather specify a "style" for the four blocks, and make the pages refer to the styles.
* I use Droplets from UWEX, and don't want to have to put that code in *every single page*. I'd rather put it one place (or, at least, only a few places). So the html code that brings in Droplets lives in a header/footer html code file.

### Style basics
Put your "style" folders in a folder in your course. In my DS150 course, I have the following structure:
* `_styles/`
* `/generic.style`
* `/assignments.style`
And in the `meta.json` file for the pages / assignments, I simply have to put the record `"style":"_styles/generic.style"` or whatever.
As of July 2022, there is no default style -- if a page doesn't list a style, it gets no style.
### Additional notes about styles:
The folder for each style should have the following four files:
* `header.html`
* `header.md`
* `footer.md`
* `footer.html`
They'll get concatenated around `source.md` in that order. HTML around markdown, and header/footer around source.
If you want to use images in your header/footer, put them in the markdown part (even if they appear in html tags), and use the text `$PATHTOMD2CANVASSTYLEFILE` before typing the name of the file, so that its filepath gets listed correctly. (This happens via a simple string replacement)

## Assignments

### Possible Upload Types

In the `meta.json` file for an assignment, the submission type is encoded by a line that looks like the following.
Reduce your mental load by specify possible upload types in the `meta.json` file for an assignment. The submission type is encoded by a line that looks like the following.

```
"submission_types":['online_text_entry', 'online_url', 'media_recording', 'online_upload']
Expand Down
1 change: 1 addition & 0 deletions docs/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ Notes
on_meta_dot_json
making_links_to_existing_content
wishlist
unit_tests


Useful links
Expand Down
10 changes: 10 additions & 0 deletions docs/on_meta_dot_json.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,16 @@ Valid properties
Assignments
*****************

In the `meta.json` file for an assignment, the submission type is encoded by a line that looks like the following.


"submission_types":['online_text_entry', 'online_url', 'media_recording', 'online_upload']


These are four of the five upload types available with Canvas. The other is an annotation. You may omit this line or include any sublist of this list. If you choose to allow online upload, you may also specify the allowable file types by including an allowed extensions list in your `meta.json` file for the assignment.


"allowed_extensions": ["pdf","docx"]


Pages
Expand Down
8 changes: 6 additions & 2 deletions docs/tutorials/text_replacements.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ Note that `_course_metadata/replacements.json` is just a regular old JSON file.




Usage
--------

Expand All @@ -32,5 +31,10 @@ Custom text replacements per-content

Content can override which replacements file is used, say in case you want all your assignments of a certain type to use one style, and the pages of another type to use their own. It's easy.

In the `meta.json` for the content you want to use the non-default replacements file, specify the key-value pair `'replacements':'path/to/custom_replacements.json'`
In the `meta.json` for the content you want to use the non-default replacements file, specify the key-value pair

.. code-block::
"replacements":"path/to/custom_replacements.json"
The path is relative to the root of the course folder.
18 changes: 18 additions & 0 deletions docs/unit_tests.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
Unit tests
==============

This library comes with several test pages/assignments:

* pages:

* `plain_text` -- source is just plain old text. start simple, ya know?
* `uses_latex` -- a page that contains latex math
* `has_remote_images` -- a page that has remote images embedded
* `has_local_images` -- a page that uses local images
* `uses_droplets` -- a page using [the Droplets framework from UWEX](https://media.uwex.edu/app/droplets/index.html).
* `uses_droplets_via_style` -- a page using [the Droplets framework from UWEX](https://media.uwex.edu/app/droplets/index.html). The code enabling Droplets comes from a header/footer contained in a style folder. The main purpose of this test page is the header/footer style thing.

* assignments:
* `programming_assignment` -- an assignment that has a local image

The above list is not exhaustive.

0 comments on commit f462b4c

Please sign in to comment.