Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alpha 1 #2

Open
wants to merge 42 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
5a09f0a
Update README with badges.
haydenjackson01 Sep 6, 2017
378d6a1
Add sphinx documentation.
haydenjackson01 Sep 6, 2017
ed008bf
Add queue service.
haydenjackson01 Sep 6, 2017
4019f4f
Add queue service.
haydenjackson01 Sep 6, 2017
aa46d48
Add docker build scripts and requirements file.
haydenjackson01 Sep 6, 2017
f0439d1
Configuration and render script.
haydenjackson01 Sep 6, 2017
87f7acb
Fix travis commands to start render service.
haydenjackson01 Sep 6, 2017
c36aafd
Configure and setup flake8 and coverage for travis.
haydenjackson01 Sep 7, 2017
88b0b3d
Add resources that will be mounted externally.
haydenjackson01 Sep 7, 2017
e56d301
Fix missing image and adjust pointer.
haydenjackson01 Sep 10, 2017
330fdc2
Merge in new resources.
haydenjackson01 Sep 10, 2017
3dbe6ec
Update and merge in unittests from csu.
haydenjackson01 Sep 11, 2017
f4971b1
Merge in updates to resources from csu.
haydenjackson01 Sep 12, 2017
a3a9cae
Remove 2 copy testing requirement.
haydenjackson01 Sep 12, 2017
0ea8e33
Fix daemon tests by extending timeout timer.
haydenjackson01 Sep 12, 2017
95d7abd
Update documentation with some summary information.
haydenjackson01 Sep 12, 2017
5b3d41b
Fix errors in consumption pipeline.
haydenjackson01 Sep 12, 2017
44250e1
Remove testing entrypoint.
haydenjackson01 Sep 12, 2017
3ec0138
Add missing resources, and update treasure hunt.
haydenjackson01 Sep 12, 2017
a150b89
Structure and bug fixes.
haydenjackson01 Sep 13, 2017
4e99d57
Add some documentation around render tasks.
haydenjackson01 Sep 14, 2017
995a09d
Use tags to separate tasks and store results.
haydenjackson01 Sep 15, 2017
60e5414
Fix if statement.
haydenjackson01 Sep 15, 2017
6bbf5f0
Setup dev test file and host networking.
haydenjackson01 Sep 15, 2017
c326322
Fix incorrect queue name and lengthen generation time.
haydenjackson01 Sep 15, 2017
f1b116e
Rename dev script to a sensible name.
haydenjackson01 Sep 18, 2017
35ec627
Remove incorrect instruction from queue_client header.
haydenjackson01 Sep 18, 2017
28bc02e
Style fixes.
haydenjackson01 Sep 18, 2017
cdc5954
Style fix.
haydenjackson01 Sep 18, 2017
0d8ee94
Add inline documentation to queue_client script.
haydenjackson01 Sep 18, 2017
fe7cd05
Documentation fixes from talk with Hayley.
haydenjackson01 Sep 24, 2017
47292bb
Basic tidy based on PR.
haydenjackson01 Sep 25, 2017
9bcf529
Fix incorrect scaling of the images in resources.
haydenjackson01 Sep 25, 2017
79d6c01
Add argument and return types to docstrings.
haydenjackson01 Sep 26, 2017
1e6903e
Enable deployment of images to gcloud service.
haydenjackson01 Sep 26, 2017
f5bb667
Clarify error message in queue service.
haydenjackson01 Sep 26, 2017
6330a94
Python documentation style fix.
haydenjackson01 Sep 26, 2017
13424f5
Fix bug from find and replace error.
haydenjackson01 Sep 26, 2017
0237999
Remove ports from render as it is running in host mode.
haydenjackson01 Sep 26, 2017
855a2f1
Configure docker to not use host mode for network.
haydenjackson01 Sep 27, 2017
936dc00
Fixes issues with mounting bucket in cloud.
haydenjackson01 Sep 29, 2017
07371c6
Final fixes for mounting buckets in cloud.
haydenjackson01 Sep 29, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[run]
branch = True
source =
render
omit =

[report]
fail_under=50
27 changes: 27 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
sudo: required
services:
- docker
env:
- DOCKER_COMPOSE_VERSION=1.11.2
language: python
python:
- '3.6'
before_install:
# See https://github.com/travis-ci/travis-ci/issues/7940
- sudo rm -f /etc/boto.cfg
install:
- sudo docker --version
- sudo docker-compose --version
- pip install -r requirements.txt
jobs:
include:
- stage: test
script: ./render ci test_with_coverage
- script: ./render ci style
- script: ./render ci docs
notifications:
email: false
slack:
rooms: deptfunstuff:abJKvzApk5SKtcEyAgtswXAv
on_success: change
on_failure: change
21 changes: 19 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,19 @@
# render
Render Service for generating PDFs for CSUnplugged and CS Field Guide
# Render Service

[![Build Status](https://travis-ci.org/uccser/render.svg?branch=master)](https://travis-ci.org/uccser/render)[![Documentation Status](https://readthedocs.org/projects/uccser-render/badge/?version=latest)](http://uccser-render.readthedocs.io/en/latest/?badge=latest)

Render Service for generating PDFs for CS Unplugged and CS Field Guide.

## Documentation

Documentation for this project can be found on
[ReadTheDocs](http://uccser-render.readthedocs.io/en/latest/),
and can also
be built from the documentation source within the `docs/` directory.

## Contributing

We would love your help to make this guide the best it can be!
Please read our
[contribution guide](http://uccser-render.readthedocs.io/en/latest/getting_started/contributing_guide.html)
to get started.
111 changes: 111 additions & 0 deletions dev/queue_client.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
"""Allows access to queue in order to create tasks and get results.

You will need to install from the renderservice
pip install -r requirements.txt
"""

import base64
import importlib.machinery
import sys
import os


PROJECT_NAME = "cs-unplugged-develop"
QUEUE_NAME = "render-queue"
DISCOVERY_URL = "http://localhost:5052/api/{api}/{apiVersion}"


def action_add(queue):
"""Add a render task to the given queue.

Please modify this as needed.

Args:
queue: QueueHandler to interact with.
"""
queue.create_task({
"kind": "task#render",
"resource_slug": "binary-cards",
"resource_name": "Binary Cards",
"resource_view": "binary_cards",
"url": "resources/binary-cards.html",
"display_numbers": False,
"black_back": True,
"paper_size": "a4",
"header_text": "",
"copies": 1,
}, tag="task")


def action_list(queue):
"""List up to 100 tasks in the queue.

Args:
queue: QueueHandler to interact with.
"""
tasks = queue.list_tasks()
print("Number of tasks: {}".format(len(tasks)))
for task in tasks:
print(task)


def action_lease(queue):
"""Lease tasks tagged with task from the queue for 60 seconds.

Args:
queue: QueueHandler to interact with.
"""
tasks = queue.lease_tasks(tasks_to_fetch=100, lease_secs=60, tag="task")
print("Number of tasks: {}".format(len(tasks)))
for task in tasks:
print(task)


def action_document(queue):
"""Save out document results from the queue.

Args:
queue: QueueHandler to interact with.
"""
tasks = queue.list_tasks()
for task in tasks:
if task["payload"]["kind"] == "result#document":
data = task["payload"]["document"].encode("ascii")
document = base64.b64decode(data)
filename = task["payload"]["filename"]
with open(filename, "wb") as f:
f.write(document)


def action_flush(queue):
"""Attempt to delete all tasks from the queue.

Args:
queue: QueueHandler to interact with.
"""
tasks = queue.list_tasks()
while len(tasks) > 0:
for task in tasks:
queue.delete_task(task["id"])

tasks = queue.list_tasks()


if __name__ == "__main__":
directory = os.path.abspath(os.path.join(os.getcwd(), os.pardir, 'renderservice/render/daemon/QueueHandler.py'))
loader = importlib.machinery.SourceFileLoader('render.daemon', directory)
handle = loader.load_module('render.daemon')

queue = handle.QueueHandler(project_name=PROJECT_NAME, taskqueue_name=QUEUE_NAME, discovery_url=DISCOVERY_URL)
action = sys.argv[1]

if action == "add":
action_add(queue)
elif action == "list":
action_list(queue)
elif action == "lease":
action_lease(queue)
elif action == "document":
action_document(queue)
elif action == "flush":
action_flush(queue)
41 changes: 41 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
version: "3.2"
services:

redis:
restart: always
image: redis
ports:
- "6379:6379"

render:
build:
context: .
dockerfile: ./renderservice/Dockerfile-local
args:
PROCESS_PORT: 5051
FLASK_PRODUCTION: 0
PROJECT_NAME: cs-unplugged-develop
QUEUE_NAME: render-queue
API_DISCOVERY_URL: http://localhost:5052/api/{api}/{apiVersion}
STATIC_DIRECTORY: /renderservice/static_mnt
volumes:
- ./static:/renderservice/static_mnt
ports:
- "5051:5051"
depends_on:
- queue
network_mode: "host"

queue:
build:
context: .
dockerfile: ./queueservice/Dockerfile-local
args:
PROCESS_PORT: 5052
ports:
- "5052:5052"
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
depends_on:
- redis
20 changes: 20 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = CSUnplugged
SOURCEDIR = source
BUILDDIR = build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
36 changes: 36 additions & 0 deletions docs/make.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
@ECHO OFF

pushd %~dp0

REM Command file for Sphinx documentation

if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=source
set BUILDDIR=build
set SPHINXPROJ=CSUnplugged

if "%1" == "" goto help

%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)

%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
goto end

:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%

:end
popd
10 changes: 10 additions & 0 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Check Python style
flake8==3.3.0
pydocstyle==2.0.0

#Coverage Tools
coverage==4.4.1

# Documentation
sphinx==1.6.2
sphinx-rtd-theme==0.2.4
46 changes: 46 additions & 0 deletions docs/source/changelog.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
Changelog
##############################################################################

This page lists updates to the render service.
All notable changes to this project will be documented in this file.

.. note ::

We base our numbering system from the guidelines at `Semantic Versioning 2.0.0`_.

Given a version number MAJOR.MINOR.HOTFIX:

- MAJOR version change when major backend or text modifications are made
(for example: new topic).
- MINOR version change when content or functionality is added or updated (for
example: new videos, new activities, large number of text (typo/grammar) fixes).
- HOTFIX version change when bug hotfixes are made (for example: fixing a typo).
- A pre-release version is denoted by appending a hyphen and the alpha label
followed by the pre-release version.

1.0.0-alpha.1
==============================================================================

- **Release date:** 15th September 2017
- **Downloads:** `Source downloads are available on GitHub`_

**Notable changes:**

The first major step in releasing a open source version of CS Unplugged.
While some existing content from the classic version of CS Unplugged have yet
to be adapted into the new format and system, we are releasing this version as
a sneak peek for teachers.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy/paste leftovers :P

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmmm copy pasta.


Features:

-

Fixes:

-

.. _Semantic Versioning 2.0.0: http://semver.org/spec/v2.0.0.html
.. _Source downloads are available on GitHub: https://github.com/uccser/render/releases
.. _Hayley van Waas: https://github.com/hayleyavw
.. _Hayden Jackson: https://github.com/ravenmaster001
.. _Jack Morgan: https://github.com/JackMorganNZ
Loading