-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Alpha 1 #2
Open
haydenjackson01
wants to merge
42
commits into
master
Choose a base branch
from
develop
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Alpha 1 #2
Changes from 30 commits
Commits
Show all changes
42 commits
Select commit
Hold shift + click to select a range
5a09f0a
Update README with badges.
haydenjackson01 378d6a1
Add sphinx documentation.
haydenjackson01 ed008bf
Add queue service.
haydenjackson01 4019f4f
Add queue service.
haydenjackson01 aa46d48
Add docker build scripts and requirements file.
haydenjackson01 f0439d1
Configuration and render script.
haydenjackson01 87f7acb
Fix travis commands to start render service.
haydenjackson01 c36aafd
Configure and setup flake8 and coverage for travis.
haydenjackson01 88b0b3d
Add resources that will be mounted externally.
haydenjackson01 e56d301
Fix missing image and adjust pointer.
haydenjackson01 330fdc2
Merge in new resources.
haydenjackson01 3dbe6ec
Update and merge in unittests from csu.
haydenjackson01 f4971b1
Merge in updates to resources from csu.
haydenjackson01 a3a9cae
Remove 2 copy testing requirement.
haydenjackson01 0ea8e33
Fix daemon tests by extending timeout timer.
haydenjackson01 95d7abd
Update documentation with some summary information.
haydenjackson01 5b3d41b
Fix errors in consumption pipeline.
haydenjackson01 44250e1
Remove testing entrypoint.
haydenjackson01 3ec0138
Add missing resources, and update treasure hunt.
haydenjackson01 a150b89
Structure and bug fixes.
haydenjackson01 4e99d57
Add some documentation around render tasks.
haydenjackson01 995a09d
Use tags to separate tasks and store results.
haydenjackson01 60e5414
Fix if statement.
haydenjackson01 6bbf5f0
Setup dev test file and host networking.
haydenjackson01 c326322
Fix incorrect queue name and lengthen generation time.
haydenjackson01 f1b116e
Rename dev script to a sensible name.
haydenjackson01 35ec627
Remove incorrect instruction from queue_client header.
haydenjackson01 28bc02e
Style fixes.
haydenjackson01 cdc5954
Style fix.
haydenjackson01 0d8ee94
Add inline documentation to queue_client script.
haydenjackson01 fe7cd05
Documentation fixes from talk with Hayley.
haydenjackson01 47292bb
Basic tidy based on PR.
haydenjackson01 9bcf529
Fix incorrect scaling of the images in resources.
haydenjackson01 79d6c01
Add argument and return types to docstrings.
haydenjackson01 1e6903e
Enable deployment of images to gcloud service.
haydenjackson01 f5bb667
Clarify error message in queue service.
haydenjackson01 6330a94
Python documentation style fix.
haydenjackson01 13424f5
Fix bug from find and replace error.
haydenjackson01 0237999
Remove ports from render as it is running in host mode.
haydenjackson01 855a2f1
Configure docker to not use host mode for network.
haydenjackson01 936dc00
Fixes issues with mounting bucket in cloud.
haydenjackson01 07371c6
Final fixes for mounting buckets in cloud.
haydenjackson01 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
[run] | ||
branch = True | ||
source = | ||
render | ||
omit = | ||
|
||
[report] | ||
fail_under=50 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
sudo: required | ||
services: | ||
- docker | ||
env: | ||
- DOCKER_COMPOSE_VERSION=1.11.2 | ||
language: python | ||
python: | ||
- '3.6' | ||
before_install: | ||
# See https://github.com/travis-ci/travis-ci/issues/7940 | ||
- sudo rm -f /etc/boto.cfg | ||
install: | ||
- sudo docker --version | ||
- sudo docker-compose --version | ||
- pip install -r requirements.txt | ||
jobs: | ||
include: | ||
- stage: test | ||
script: ./render ci test_with_coverage | ||
- script: ./render ci style | ||
- script: ./render ci docs | ||
notifications: | ||
email: false | ||
slack: | ||
rooms: deptfunstuff:abJKvzApk5SKtcEyAgtswXAv | ||
on_success: change | ||
on_failure: change |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,19 @@ | ||
# render | ||
Render Service for generating PDFs for CSUnplugged and CS Field Guide | ||
# Render Service | ||
|
||
[![Build Status](https://travis-ci.org/uccser/render.svg?branch=master)](https://travis-ci.org/uccser/render)[![Documentation Status](https://readthedocs.org/projects/uccser-render/badge/?version=latest)](http://uccser-render.readthedocs.io/en/latest/?badge=latest) | ||
|
||
Render Service for generating PDFs for CS Unplugged and CS Field Guide. | ||
|
||
## Documentation | ||
|
||
Documentation for this project can be found on | ||
[ReadTheDocs](http://uccser-render.readthedocs.io/en/latest/), | ||
and can also | ||
be built from the documentation source within the `docs/` directory. | ||
|
||
## Contributing | ||
|
||
We would love your help to make this guide the best it can be! | ||
Please read our | ||
[contribution guide](http://uccser-render.readthedocs.io/en/latest/getting_started/contributing_guide.html) | ||
to get started. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,111 @@ | ||
"""Allows access to queue in order to create tasks and get results. | ||
|
||
You will need to install from the renderservice | ||
pip install -r requirements.txt | ||
""" | ||
|
||
import base64 | ||
import importlib.machinery | ||
import sys | ||
import os | ||
|
||
|
||
PROJECT_NAME = "cs-unplugged-develop" | ||
QUEUE_NAME = "render-queue" | ||
DISCOVERY_URL = "http://localhost:5052/api/{api}/{apiVersion}" | ||
|
||
|
||
def action_add(queue): | ||
"""Add a render task to the given queue. | ||
|
||
Please modify this as needed. | ||
|
||
Args: | ||
queue: QueueHandler to interact with. | ||
""" | ||
queue.create_task({ | ||
"kind": "task#render", | ||
"resource_slug": "binary-cards", | ||
"resource_name": "Binary Cards", | ||
"resource_view": "binary_cards", | ||
"url": "resources/binary-cards.html", | ||
"display_numbers": False, | ||
"black_back": True, | ||
"paper_size": "a4", | ||
"header_text": "", | ||
"copies": 1, | ||
}, tag="task") | ||
|
||
|
||
def action_list(queue): | ||
"""List up to 100 tasks in the queue. | ||
|
||
Args: | ||
queue: QueueHandler to interact with. | ||
""" | ||
tasks = queue.list_tasks() | ||
print("Number of tasks: {}".format(len(tasks))) | ||
for task in tasks: | ||
print(task) | ||
|
||
|
||
def action_lease(queue): | ||
"""Lease tasks tagged with task from the queue for 60 seconds. | ||
|
||
Args: | ||
queue: QueueHandler to interact with. | ||
""" | ||
tasks = queue.lease_tasks(tasks_to_fetch=100, lease_secs=60, tag="task") | ||
print("Number of tasks: {}".format(len(tasks))) | ||
for task in tasks: | ||
print(task) | ||
|
||
|
||
def action_document(queue): | ||
"""Save out document results from the queue. | ||
|
||
Args: | ||
queue: QueueHandler to interact with. | ||
""" | ||
tasks = queue.list_tasks() | ||
for task in tasks: | ||
if task["payload"]["kind"] == "result#document": | ||
data = task["payload"]["document"].encode("ascii") | ||
document = base64.b64decode(data) | ||
filename = task["payload"]["filename"] | ||
with open(filename, "wb") as f: | ||
f.write(document) | ||
|
||
|
||
def action_flush(queue): | ||
"""Attempt to delete all tasks from the queue. | ||
|
||
Args: | ||
queue: QueueHandler to interact with. | ||
""" | ||
tasks = queue.list_tasks() | ||
while len(tasks) > 0: | ||
for task in tasks: | ||
queue.delete_task(task["id"]) | ||
|
||
tasks = queue.list_tasks() | ||
|
||
|
||
if __name__ == "__main__": | ||
directory = os.path.abspath(os.path.join(os.getcwd(), os.pardir, 'renderservice/render/daemon/QueueHandler.py')) | ||
loader = importlib.machinery.SourceFileLoader('render.daemon', directory) | ||
handle = loader.load_module('render.daemon') | ||
|
||
queue = handle.QueueHandler(project_name=PROJECT_NAME, taskqueue_name=QUEUE_NAME, discovery_url=DISCOVERY_URL) | ||
action = sys.argv[1] | ||
|
||
if action == "add": | ||
action_add(queue) | ||
elif action == "list": | ||
action_list(queue) | ||
elif action == "lease": | ||
action_lease(queue) | ||
elif action == "document": | ||
action_document(queue) | ||
elif action == "flush": | ||
action_flush(queue) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
version: "3.2" | ||
services: | ||
|
||
redis: | ||
restart: always | ||
image: redis | ||
ports: | ||
- "6379:6379" | ||
|
||
render: | ||
build: | ||
context: . | ||
dockerfile: ./renderservice/Dockerfile-local | ||
args: | ||
PROCESS_PORT: 5051 | ||
FLASK_PRODUCTION: 0 | ||
PROJECT_NAME: cs-unplugged-develop | ||
QUEUE_NAME: render-queue | ||
API_DISCOVERY_URL: http://localhost:5052/api/{api}/{apiVersion} | ||
STATIC_DIRECTORY: /renderservice/static_mnt | ||
volumes: | ||
- ./static:/renderservice/static_mnt | ||
ports: | ||
- "5051:5051" | ||
depends_on: | ||
- queue | ||
network_mode: "host" | ||
|
||
queue: | ||
build: | ||
context: . | ||
dockerfile: ./queueservice/Dockerfile-local | ||
args: | ||
PROCESS_PORT: 5052 | ||
ports: | ||
- "5052:5052" | ||
environment: | ||
- REDIS_HOST=redis | ||
- REDIS_PORT=6379 | ||
depends_on: | ||
- redis |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Minimal makefile for Sphinx documentation | ||
# | ||
|
||
# You can set these variables from the command line. | ||
SPHINXOPTS = | ||
SPHINXBUILD = sphinx-build | ||
SPHINXPROJ = CSUnplugged | ||
SOURCEDIR = source | ||
BUILDDIR = build | ||
|
||
# Put it first so that "make" without argument is like "make help". | ||
help: | ||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) | ||
|
||
.PHONY: help Makefile | ||
|
||
# Catch-all target: route all unknown targets to Sphinx using the new | ||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). | ||
%: Makefile | ||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
@ECHO OFF | ||
|
||
pushd %~dp0 | ||
|
||
REM Command file for Sphinx documentation | ||
|
||
if "%SPHINXBUILD%" == "" ( | ||
set SPHINXBUILD=sphinx-build | ||
) | ||
set SOURCEDIR=source | ||
set BUILDDIR=build | ||
set SPHINXPROJ=CSUnplugged | ||
|
||
if "%1" == "" goto help | ||
|
||
%SPHINXBUILD% >NUL 2>NUL | ||
if errorlevel 9009 ( | ||
echo. | ||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx | ||
echo.installed, then set the SPHINXBUILD environment variable to point | ||
echo.to the full path of the 'sphinx-build' executable. Alternatively you | ||
echo.may add the Sphinx directory to PATH. | ||
echo. | ||
echo.If you don't have Sphinx installed, grab it from | ||
echo.http://sphinx-doc.org/ | ||
exit /b 1 | ||
) | ||
|
||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% | ||
goto end | ||
|
||
:help | ||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% | ||
|
||
:end | ||
popd |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
# Check Python style | ||
flake8==3.3.0 | ||
pydocstyle==2.0.0 | ||
|
||
#Coverage Tools | ||
coverage==4.4.1 | ||
|
||
# Documentation | ||
sphinx==1.6.2 | ||
sphinx-rtd-theme==0.2.4 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
Changelog | ||
############################################################################## | ||
|
||
This page lists updates to the render service. | ||
All notable changes to this project will be documented in this file. | ||
|
||
.. note :: | ||
|
||
We base our numbering system from the guidelines at `Semantic Versioning 2.0.0`_. | ||
|
||
Given a version number MAJOR.MINOR.HOTFIX: | ||
|
||
- MAJOR version change when major backend or text modifications are made | ||
(for example: new topic). | ||
- MINOR version change when content or functionality is added or updated (for | ||
example: new videos, new activities, large number of text (typo/grammar) fixes). | ||
- HOTFIX version change when bug hotfixes are made (for example: fixing a typo). | ||
- A pre-release version is denoted by appending a hyphen and the alpha label | ||
followed by the pre-release version. | ||
|
||
1.0.0-alpha.1 | ||
============================================================================== | ||
|
||
- **Release date:** 15th September 2017 | ||
- **Downloads:** `Source downloads are available on GitHub`_ | ||
|
||
**Notable changes:** | ||
|
||
The first major step in releasing a open source version of CS Unplugged. | ||
While some existing content from the classic version of CS Unplugged have yet | ||
to be adapted into the new format and system, we are releasing this version as | ||
a sneak peek for teachers. | ||
|
||
Features: | ||
|
||
- | ||
|
||
Fixes: | ||
|
||
- | ||
|
||
.. _Semantic Versioning 2.0.0: http://semver.org/spec/v2.0.0.html | ||
.. _Source downloads are available on GitHub: https://github.com/uccser/render/releases | ||
.. _Hayley van Waas: https://github.com/hayleyavw | ||
.. _Hayden Jackson: https://github.com/ravenmaster001 | ||
.. _Jack Morgan: https://github.com/JackMorganNZ |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
copy/paste leftovers :P
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mmmm copy pasta.