Merge pull request #1 from pcrespov/feature/report_review

Feature/report review

.gitignore

@@ -1 +1,2 @@
 /node_modules/
+.DS_Store

docs/d1.1-techreview/conclusion.md

@@ -0,0 +1,2 @@
+
+TODO Conclusions

docs/d1.1-techreview/index.md

@@ -0,0 +1,14 @@
+## Executive Summary
+
+TODO : ...
+
+### Table of Contents
+  1. [Objectives](objectives.md)
+  1. [Methodology](methodology.md)
+  1. [Results](results.md)
+    - [Front-end](parts/client-side.md)
+    - [Web-server](parts/server-side.md)
+    - [Communication models](parts/communication.md)
+    - [Computational services](parts/comp-services.md)
+    - [Main Demonstrators](parts/demos.md)
+  1. [Conclusions](conclusion.md)

docs/d1.1-techreview/methodology.md

@@ -0,0 +1,14 @@
+## Pre-selection of technologies
+
+This evaluation started with a pre-selection of technologies (i.e. libraries, framework, toolkits, models, ...) for different components of the oSPARC platform, namely the front-end (client-side), the web back-end (server-side), the communication models and the computational services.
+
+The criteria used for the pre-selection consisted of mainly three factors:
+
+- **Completeness**: the selected framework must include functions or accept extensions that support key features for its purpose. For instance, a suitable web-framework must include features for routing, database models, web-security, etc, which cover the essential functionality expected for the server-side.
+- **Sustainability**: the selected technology should be long-term maintainable as an open-source project. Two key measures of sustainability considered here are its popularity, the support of the developers community (i.e documentation, fixes, releases, ...) and the license terms.  
+- **Productivity**: we believe that productivity can be achieved with the combination of a suitable framework (e.g. complete, easy-to-use, and sustainable), the effectiveness reusing existing functionality and the previous experience of the team on a specific technology. For that reason, the pre-selection should incorporate options that combine both novel as well as already established programming paradigms (e.g. languages, design-patterns or even specific frameworks).
+
+
+## Review process
+
+After the pre-selection of technologies, the review adopts a more practical approach. Every group of technologies are tested within a given context defined in a user story. This approach is adopted gradually from more simple to more realistic scenarios, resulting collection of demos that are [published the project repository](https://github.com/ITISFoundation/osparc-lab/tree/master/demos). Among all demos, there are three [prototypes](parts/demos.md) that combines different technologies within the full-stack and covers realistic scenarios expected in the final platform. The specifics of each review are documented in place, next to its correspond demo. Nonetheless, for the sake of clarity we provide a summarize version together with the final recommendations in the [results](results.md) section.

docs/d1.1-techreview/objectives.md

@@ -0,0 +1,2 @@
+
+TODO: what are the objectives of this report??

docs/d1.1-techreview/parts/client-side.md

@@ -0,0 +1,45 @@
+## Frameworks
+
+The front-end part corresponds to the software running on the client side. The server-side transmits the program/instructions to the web-browser, on the client-side, which interprets and executes them. The front-end interacts directly with the user and for that reason, one of its main features of front-end frameworks is the Graphical User Interface (GUI). There is a wide variety of frameworks that supports the development of front-ends among which we have pre-selected: [vue], [react], [qooxdoo] and [wt].
+
+TODO: user-stories + map w/ demos
+TODO: 3D with three.js
+
+## [vue]
+![vue-screenshot](../img/vue.png)
+TODO: screenshot
+TODO: short description review, pros, cons
+
+## [react]
+![react-screenshot](../img/react.jpg)
+TODO: screenshot
+TODO: short description review, pros, cons
+
+## [qooxdoo]
+![qx-screenshot](../img/qx.png)
+TODO: short description review, pros, cons
+
+## [wt]
+![wt-screenshot](../img/wt.png)
+TODO: short description review, pros, cons
+
+## Conclusions
+This is a comparative table with the pre-selected framework and some key-points
+
+|                | vue          | react        |      qooxdoo |
+|----------------|--------------|--------------|--------------|
+|  **License**   | MIT licensed | MIT licensed | MIT licensed |
+| **Popularity** | <ul><li>Released in 2014</li><li>75000 stars in Github</li><li>12000 questions in Stack overflow</li></ul> | <ul><li>Released in 2013</li><li>83000 stars in Github</li><li>65000 questions in Stack overflow</li></ul> | <ul><li>Released in 2009</li><li>600 stars in Github</li><li>1000 questions in Stack overflow</li></ul> |
+|    **Pros**    | <ul><li>Mix between Angular and React</li><li>HTML, CSS and logic nicely separated</li></ul> | <ul><li>Facebook supports it. One of the most popular Front-End JavaScript framework</li><li>Lot of dedicated and easy to install 3rd party libraries | <ul><li>Desktop-like web application</li><li>Object Oriented programming model</li><li>Lots of native 3rd party libraries available</li><li>Three.js used directly</li><li>Complete control over dependencies |
+|    **Cons**    | <ul><li>No specific company supporting it</li><li>Not many dedicated 3rd party libraries available</li><li>Lack of control over dependencies</li></ul> | <ul><li>HTML, CSS and logic not very well separated</li><li>Lack of control over dependencies</li></ul> | <ul><li>Not a big community behind</li><li>Some 3rd party libraries do not integrate very well</li></ul> |
+
+
+TODO: recommendations
+
+
+
+
+[vue]: https://vuejs.org
+[react]: https://reactjs.org
+[qooxdoo]: http://www.qooxdoo.org
+[wt]: https://www.webtoolkit.eu/wt

docs/d1.1-techreview/parts/communication.md

@@ -0,0 +1,20 @@
+### Communication and Interoperability
+
+In order to achieve interoperability between different services, both inside (e.g. among computational services) or outside (e.g. with other SPARC-cores services) the platform is paramount for a uservices based framework. We focus our review on two different communication models, namely
+
+- Remote Procedure Call or RPC: TODO short desc
+- Representational State Transfer or RESTful: TODO short desc
+
+
+  TODO: justify C+S+P
+
+#### Review
+
+  TODO: review thrift and swagger??
+  TODO: user-stories + map w/ demos
+  TODO: Interoperability with S4L as a service
+
+#### Summary
+
+
+#### Recommendations

docs/d1.1-techreview/parts/comp-services.md

@@ -0,0 +1,20 @@
+## Computational services
+
+Computational services
+  - Containers Technology
+  - Orchestration framework
+
+
+  TODO: Preselection. Justify C+S+P
+
+#### Review
+
+  TODO: user-stories + map w/ demos
+
+  TODO: different models analyzed for 3rd party services. a) i/o files, b) SDK layer?  c) SDK w/ UI override?
+
+
+#### Summary
+
+
+#### Recommendations

docs/d1.1-techreview/parts/demos.md

@@ -0,0 +1,19 @@
+
+## Prototypes
+All prototypes and proof-of-concepts created during the review process can be found under the [demos folder](https://github.com/ITISFoundation/osparc-lab/tree/master/demos) in the GitHub repository. Every subfolder
+contains a sample program that demonstrates a technology and/or a programming concept. More details about each case is documented in place.
+
+## Demonstrators
+In order to demonstrate the different technologies in a wider context, we have designed three user stories that covers front to back the entire platform.
+
+![image-demo1](../img/demo1.jpeg)
+The first prototype demonstrates a web-application with dynamic content. The client-side implements GUIs with similar features in three different technologies and the server-side demonstrates different frameworks to serve these apps. All three front-ends demonstrate a 3D viewer, a workbench to schedule task pipelines, a dynamic display of the settings for a given computational service (embedded in service meta-data) and several viewers for results (e.g. 2D plots, tables, etc).
+
+![image-demo2](../img/demo2.jpeg)
+This prototype demonstrates the orchestration and execution of computational services in the back-end. The demonstration starts with a pipeline of tasks that gets scheduled in a queueing system. Every task in the queue is then executed by a worker service that pulls and runs the actual computational service from a docker registry. The worker is implemented as a [side-car](https://docs.microsoft.com/en-us/azure/architecture/patterns/sidecar) in order to leverage the computational service from any direct interaction with the platform.
+
+![image-demo3](../img/demo3.jpeg)
+This demonstration build a web application that offers the entire [Sim4Life](https://www.zurichmedtech.com/sim4life/) application as a service. The user can create multiple sessions and get a full anatomical model from the Virtual Population [ViP3](https://www.itis.ethz.ch/virtual-population/virtual-population/vip3/) rendered in the browser. To achieve this, the web server communicates with the [Sim4Life](https://www.zurichmedtech.com/sim4life/) service via RPC and requests to load, process and transmit the model in chunks. When the information reaches the client-side it is rendered using ```three.js```.
+
+
+**NOTE**: These three demonstrations were presented live during the review meeting on Dec 13, 2017.

docs/d1.1-techreview/parts/server-side.md

@@ -0,0 +1,77 @@
+## Frameworks
+
+At the back-end, the server-side of the web application while for the back-end (web server-side) the following web-frameworks were pre-selected: [express] in node.js, [flask]/[sanic] in python and [wt] in C++
+
+All these web-frameworks provide standard features (included or within packages) for:
+
+- routing URLs to handlers
+- interacting with databases
+- support user sessions and authorization
+- templating system to render data (e.g HTML, JSON, XML, ...)
+- security against web attacks
+
+Most of the points below are highlights useful for the discussion in the context of this project.
+
+
+## [express]
+  - **Language**: node.js, js
+  - **Popularity**: Github score 92, Stack-overflow score 83
+  - **License**: MIT license
+
+**Pros**
+  - *Productivity*:
+    - Very high (if no issues with npm or configurations)
+    - Same language as in client side
+  - *Functionality*:
+    - Large amount of packages for virtually everything!
+    - npm proofs to be state-of-the-art managing dependencies but its complexity makes it very difficult (for non-experts) to track and solve errors.
+    - ... (review ongoing)
+
+**Cons**
+  - Difficult to integrate python/C++ code (only via other processes)
+  - ... (review ongoing)
+
+## [flask]/[sanic]
+  - **Language**: python 3
+  - **Popularity**: Github score 91, Stack-overflow score 77
+  - **License**: BSD/MIT license
+
+**Pros**
+- *Productivity*: light-weight complete web-framework.
+- *Functionality*: standard web-framework
+  - Using asynchronous APIs (sanic)
+  - Allows mixed python/C++
+  - ... (review ongoing)
+
+**Cons**
+- ... (review ongoing)
+
+## [wt]
+  - **Language** C++
+  - **Popularity**: Github score 56, Stack-overflow score 40
+  - **License**: Dual: GNU General Public + commercial
+
+**Pros**
+- *Productivity*: UI programed in a similar way as in desktop app (analogous to Qt).
+- *Functionality*: All power of C++: concurrency, courroutines, multi-threading ... you name it
+    - Has the potential of replacing one-to-one MFC but difficult to proof
+    - ... (review ongoing)
+
+**Cons**
+  - First trials shows a high level of communication with the server
+  - Needs some heavy-lifting to get it up & running in container (c++ libs, build, ...)
+  - ... (review ongoing)
+
+## Conclusions
+Currently these are our recommendation of a web-framework (in order):
+
+1. [flask]/[sanic]: preferred to develop the framework
+1. [express]: as prototype/mockup servers
+1. [wt] : keep an eye
+
+
+
+[express]: http://expressjs.com/
+[flask]: http://flask.pocoo.org/
+[sanic]: https://github.com/channelcat/sanic
+[wt]: http://www.webtoolkit.eu/wt

docs/d1.1-techreview/results.md

@@ -0,0 +1,9 @@
+
+TODO: Why and what is this section? This is the review
+TODO: summary of pre-selected groups. inline to enumeration items
+
+- [Front-end](parts/client-side.md)
+- [Web-server](parts/server-side.md)
+- [Communication models](parts/communication.md)
+- [Computational services](parts/comp-services.md)
+- [Main Demonstrators](parts/demos.md)

mkdocs.yml

@@ -0,0 +1,14 @@
+site_name: D1.1 Technology Review & Technical Specs.
+pages:
+  - Home: 'd1.1-techreview/index.md'
+  - Objectives: 'd1.1-techreview/objectives.md'
+  - Methodology: 'd1.1-techreview/methodology.md'
+  - Results:
+    - 'Overview': 'd1.1-techreview/results.md'
+    - 'Front-end': 'd1.1-techreview/parts/client-side.md'
+    - 'Web-server': 'd1.1-techreview/parts/server-side.md'
+    - 'Communication models': 'd1.1-techreview/parts/communication.md'
+    - 'Computational services': 'd1.1-techreview/parts/comp-services.md'
+    - 'Demonstrations': 'd1.1-techreview/parts/demos.md'
+  - Conclusions: 'd1.1-techreview/conclusion.md'
+theme: readthedocs

tools/doc-generator/Dockerfile

@@ -0,0 +1,34 @@
+FROM python:alpine3.6
+
+WORKDIR /usr/app
+
+COPY requirements.txt ./
+RUN pip install --no-cache-dir -r requirements.txt
+
+#COPY . .
+
+VOLUME /usr/app
+EXPOSE 8000
+
+CMD ["python", "-m", "mkdocs", "serve", "--dev-addr", "0.0.0.0:8000" ]
+
+# Usage
+# cd osparc-lab
+# docker build -t doc-generator tools/doc-generator
+# docker run -it -v $(pwd):/usr/src/app -p 8000:8000 doc-generator
+#
+
+# http://www.mkdocs.org/
+#  MkDocs - Project documentation with Markdown.
+
+#Options:
+#  -V, --version  Show the version and exit.
+#  -q, --quiet    Silence warnings
+#  -v, --verbose  Enable verbose output
+#  -h, --help     Show this message and exit.
+
+#Commands:
+#  build      Build the MkDocs documentation
+#  gh-deploy  Deploy your documentation to GitHub Pages
+#  new        Create a new MkDocs project
+#  serve      Run the builtin development server

tools/doc-generator/README.md

@@ -0,0 +1,29 @@
+## doc-generator
+
+Builds a docker with tools (currently [mkdocs](http://www.mkdocs.org/)) that parse docs/ and publish documentation in a website
+
+#### Usage
+
+[mkdocs](http://www.mkdocs.org/) reads [mkdocs.yml](../../mkdocs.yml) configuration and serves a website with live-reload (i.e. changes in doc will be automatically reflected in the web live):
+```
+  cd osparc-lab
+
+  # builds tools
+  docker build -t doc-generator tools/doc-generator
+
+  # or can simply pull it from itisfoundation registry
+  docker pull itisfoundation/doc-generator:latest
+
+  # serves doc website (dev)
+  docker run -it -v $(pwd):/usr/app -p 8000:8000 doc-generator
+
+
+  # builds website
+  docker run -it -v $(pwd):/usr/app -p 8000:8000 doc-generator python -m mkdocs build
+```
+
+To [generate a static web](http://www.mkdocs.org/#building-the-site) then run
+```
+docker run -it -v $(pwd):/usr/app -p 8000:8000 doc-generator python -m mkdocs build
+```
+and will create all the code in a ```site``` folder.

tools/doc-generator/requirements.txt

@@ -0,0 +1,14 @@
+certifi==2017.11.5
+click==6.7
+CommonMark==0.5.4
+docutils==0.14
+Jinja2==2.10
+livereload==2.5.1
+Markdown==2.6.10
+MarkupSafe==1.0
+mkdocs==0.17.2
+PyYAML==3.12
+recommonmark==0.4.0
+six==1.11.0
+tornado==4.5.2
+wincertstore==0.2

tools/doc-generator/serve.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+# run from repo root folder
+docker run -it -v $(pwd):/usr/app -p 8000:8000 itisfoundation/doc-generator:latest