-
Notifications
You must be signed in to change notification settings - Fork 8
Initial PR (complete layout tool) #14
Changes from 181 commits
a6057fe
87377e5
5fc57ba
37b02e4
de866f8
ec0298b
d8ebd83
bcb456b
febd8b5
1d02fc6
65e2065
f3669c4
1aaecca
d66839d
72b6235
08a18c4
50937ca
7013cc9
29a141b
45e7820
bb2ab0f
64a2c02
bd4c193
46bbcb0
e45f6da
c9d3e71
25cae24
1c333a2
917e967
70f15f0
bef27ff
2bfc697
803ac9b
c0f4b48
d688b4d
03feedc
266c289
865a65c
ef615ed
742c727
6e3166f
b23fbfa
a9a304e
c6ad00c
48971b9
b7ed0d2
8dc69e6
78d8b0a
45a0a52
d347318
d598ebe
be13453
86873f5
88028c1
c4242fd
7f7a0d4
17685f2
52be68d
3b5c1a0
52f1f7d
a68d0a1
2aa2c48
4059e7d
c62b9f4
215f79e
6696426
0e79420
e3db4bf
e4e454c
ecda9b7
d0137d1
325e530
b391579
a9ec1a2
792a87c
5fec449
fa78346
732f114
a7cefb1
2a06c9d
6767bbc
a3b583a
ed06bdf
bfa78a8
d12b1e8
373173b
de40e6a
a669dc3
fc25cdd
6ef6791
1ec9a14
e865c8f
9025d91
c5d3a04
819073e
5470ace
c99b715
96c42fb
8c2c709
09ad936
4699209
86f04e2
05e5e81
224571e
9992e91
45fa2e3
f04df37
6c1507c
11eebb6
2f9eb47
2b9b2bb
08bb201
7cadd29
c3fdd36
95c68c6
f4c5329
a5ca4c6
381d8c9
4fdabfd
efaf8df
c2a3769
145c15d
b0b912f
3d4ef3f
8a8f946
47e0563
8ce3a1e
82dcf84
d4ef8ef
b78d1b8
aa51870
f80c7a7
b5dd182
c6714b4
b24374c
ffbe92b
dc41ea2
5e0ac7f
1c91076
bcdabcb
cb612f3
84de860
abde3bc
1cc956d
cc919d1
4e66cc6
771bfac
62df4eb
1f1baf8
6f6a23d
05a95df
762e16b
04b2147
7a10de2
8f6c3c2
75c27f7
0927546
3883da0
27d5321
9e050b4
05dd627
6a9ea9a
e3ed3c3
d2867c4
5e24927
2f437a9
a265bb8
a56bfdb
fa6a967
6a60ff9
5ab4f28
75f2b59
88c0bd6
a27f46f
8ea6621
711faec
4f7a895
f2104a3
1bbbc53
6f8e13d
ebf524d
7e82633
415b866
2091bd3
1310274
bc31725
855e5cc
9f311ff
4d25719
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# Byte-compiled / optimized / DLL files | ||
__pycache__/ | ||
*.py[cod] | ||
*$py.class | ||
|
||
in-toto | ||
node_modules | ||
|
||
# Exclude vendor JS file, copy them freshly using: | ||
# `gulp` | ||
static/vendor/* | ||
!static/vendor/.keep | ||
|
||
files/* | ||
!files/.keep | ||
|
||
# Exclude *.css files, compile them freshly using: | ||
# `sass static/scss/main.scss:static/css/main.scss.css` | ||
.sass-cache | ||
static/css/* | ||
!static/css/.keep |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
The MIT License (MIT) | ||
|
||
Copyright (c) 2017 New York University | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👍 |
||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all | ||
copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
# in-toto Layout Creation Wizard | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider saying somewhere in the README that an instance of the layout tool is available and/or link to it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||
|
||
A Flask based web app to guide project owners through creating an | ||
[in-toto layout](https://in-toto.io). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The linked text is Maybe linking to https://in-toto.engineering.nyu.edu/, where an instance of the layout tool exists, makes more sense? This repository is also more about the layout tool rather than in-toto. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||
|
||
Mockups can be found at [`editor-and-wizard-wip/mockups`](https://github.com/in-toto/layout-web-tool/blob/editor-and-wizard-wip/mockups/layout-wizard.pdf). | ||
|
||
|
||
### Installation | ||
|
||
**Requirements** | ||
- [Python 2.7](https://www.python.org/download/releases/2.7/) -- backend | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do you have to install any Python development headers? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
||
- [npm](https://www.npmjs.com/) -- frontend dependencies | ||
- [Ruby](https://www.ruby-lang.org/en/documentation/installation/) and [SASS](http://sass-lang.com/install) -- CSS preprocessor | ||
- [MongoDB](https://docs.mongodb.com/manual/installation/) -- to persist | ||
user session data (for usage analysis) | ||
|
||
|
||
```shell | ||
# Start `mongod` (if not already running) | ||
# Note: `service` won't be available in the future | ||
sudo service mongod start | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. service is an outdated debian/ubuntu construct. If you're using systemd compliant, this command should be sudo systemctl start mongod Can you check this out for me please? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The official mongodb installation/usage guide suggests There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see, I guess they need to update their docs too. Could you at least leave a note for it? (serivce won't be available in the future) |
||
|
||
# Install backend (c.f. requirements.txt) | ||
pip install -r requirements.txt | ||
|
||
# Install and vendorize frontend dependencies and compile scss | ||
# c.f. dependencies and scripts in package.json | ||
npm install | ||
``` | ||
|
||
### Deployment | ||
- Add an [instance folder](http://flask.pocoo.org/docs/0.12/config/#instance-folders) with your | ||
deployment configuration, e.g.: | ||
```python | ||
# Example configuration in FLASK_APP_ROOT/instance/config.py | ||
DEBUG = False | ||
SECRET_KEY = '?\xbf,\xb4\x8d\xa3"<\x9c\xb0@\x0f5\xab,w\xee\x8d$0\x13\x8b83' #CHANGE THIS!!!!! | ||
|
||
``` | ||
|
||
- Take a look at `wizard.wsgi` and [these`mod_wsgi` instructions](http://flask.pocoo.org/docs/0.12/deploying/mod_wsgi/) | ||
for further guidance. | ||
|
||
### Development Tips | ||
- Run the development server like this: | ||
```shell | ||
python wizard.py | ||
``` | ||
- Run a `sass` watcher during development to automatically compile css on file change: | ||
```shell | ||
sass --watch static/scss/main.scss:static/css/main.scss.css | ||
``` | ||
- Make extensive use of (e.g. chrome's) browser developer tools, e.g. [map | ||
DevTool files to your local workspace](https://developers.google.com/web/tools/setup/setup-workflow) to live edit `*.scss` and `*.js` files. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does this project need an acknowledgement section? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,154 @@ | ||
# -*- coding: utf-8 -*- | ||
#!/usr/bin/env python | ||
|
||
""" | ||
<Program Name> | ||
create_layout.py | ||
<Author> | ||
Lukas Puehringer <lukas.puehringer@nyu.edu> | ||
<Started> | ||
March 23, 2017 | ||
<Copyright> | ||
See LICENSE for licensing information. | ||
<Purpose> | ||
Creates a basic in-toto layout by reading an ordered list of step link files. | ||
** Infer layout fields: ** | ||
expires: | ||
default value | ||
keys: | ||
FIXME: Keys are currently ignored in this module | ||
steps: | ||
add steps in the order of passed link files | ||
name: | ||
link.name | ||
expected_command: | ||
link.command | ||
threshold: | ||
default value | ||
material_matchrules/product_matchrules: | ||
currently uses simple approach (see below) | ||
FIXME: Should use more complex approach (see ideas below) | ||
inspections: | ||
FIXME Inspections are currently ignored in this module | ||
signatures: | ||
empty (use `in-toto-sign` command line utility) | ||
** Infer step artifact rules (simple approach) ** | ||
** material_matchrules ** | ||
IF no materials were recorded | ||
material_matchrules: [["DISALLOW", "*"]] | ||
ELSE IF materials were recorded and it is the first step | ||
material_matchrules: [["ALLOW", "*"]] | ||
ELSE | ||
material_matchrules: [["MATCH", "*", "WITH", "PRODUCTS", "FROM", <PREVIOUS STEP>] | ||
** product_matchrules ** | ||
IF no products were recorded | ||
product_matchrules: [["DISALLOW", "*"]] | ||
ELSE products were recorded: | ||
product_matchrules: [["ALLOW", "*"]] | ||
** Ideas for more complexity: ** | ||
- explicitly, ALLOW or MATCH files by name instead of "*", e.g.: | ||
material_matchrules = \ | ||
[["ALLOW", material] for material in links[index].materials.keys()] | ||
- for MATCH rules | ||
match only those that already were in the previous step | ||
allow the rest by name | ||
<Usage> | ||
``` | ||
# Create a layout given an ordered list of link file paths | ||
links = [] | ||
for LINK_PATH in LINK_PATHS: | ||
link = in_toto.models.link.Link.read_from_file(LINK_PATH) | ||
links.append(link) | ||
layout = create_layout_from_ordered_links(links) | ||
layout.dump() | ||
``` | ||
""" | ||
import os | ||
import in_toto.models.link | ||
import in_toto.models.layout | ||
|
||
def create_material_matchrules(links, index): | ||
"""Create generic material rules (3 variants) | ||
* No materials recorded -> disallow any artifact | ||
* Materials recorded (first step) -> allow artifacts that existed beforehand | ||
* Materials recorded (latter step) -> match from previous products | ||
Returns a list of material rules | ||
NOTE: Read header docstring for ideas for more complexity. """ | ||
|
||
material_matchrules = [] | ||
|
||
if not links[index].materials: | ||
material_matchrules = [["DISALLOW", "*"]] | ||
|
||
elif index == 0 and links[index].materials: | ||
material_matchrules = [["ALLOW", "*"]] | ||
|
||
else: | ||
material_matchrules = [ | ||
["MATCH", "*", "WITH", "PRODUCTS", "FROM", links[index - 1].name]] | ||
|
||
return material_matchrules | ||
|
||
|
||
def create_product_matchrules(links, index): | ||
"""Create generic material rules (2 variants) | ||
* No products recorded -> disallow any artifact | ||
* Products recorded -> allow all artifacts | ||
Returns a list of product rules | ||
NOTE: Read header docstring for ideas for more complexity. """ | ||
|
||
if not links[index].products: | ||
product_matchrules = [["DISALLOW", "*"]] | ||
|
||
else: | ||
product_matchrules = [["ALLOW", "*"]] | ||
|
||
return product_matchrules | ||
|
||
|
||
def create_layout_from_ordered_links(links): | ||
"""Creates basic in-toto layout from an ordered list of in-toto link objects, | ||
inferring material and product rules from the materials and products of the | ||
passed links. """ | ||
# Create an empty layout | ||
layout = in_toto.models.layout.Layout() | ||
layout.keys = {} | ||
|
||
for index, link in enumerate(links): | ||
step_name = link.name | ||
step = in_toto.models.layout.Step(name=step_name, | ||
material_matchrules=create_material_matchrules(links, index), | ||
product_matchrules=create_product_matchrules(links, index), | ||
expected_command=link.command) | ||
|
||
layout.steps.append(step) | ||
|
||
return layout |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
/***************************************************************** | ||
<File Name> | ||
gulpfile.js | ||
|
||
<Author> | ||
Lukas Puehringer <lukas.puehringer@nyu.edu> | ||
|
||
<Started> | ||
May 05, 2017 | ||
|
||
<Copyright> | ||
See LICENSE for licensing information. | ||
|
||
<Purpose> | ||
Front-end build tool used to copy third-party JS scripts to | ||
static/vendor from where the app serves them. | ||
|
||
TODO: | ||
Add gulp task for scss (styles) compilation (on change) | ||
Currently this is done with a separate command, i.e. | ||
``` | ||
sass --watch static/scss/main.scss:static/css/main.scss.css | ||
``` | ||
but it would be nice to have all in one place. | ||
|
||
<Usage> | ||
``` | ||
# Install front-end dependencies (in same directory) | ||
npm install | ||
# Run default gulp task | ||
gulp | ||
``` | ||
|
||
*****************************************************************/ | ||
var gulp = require("gulp"); | ||
|
||
gulp.task("default", function() { | ||
var js = [ | ||
"node_modules/jquery/dist/jquery.js", | ||
"node_modules/bootstrap/dist/js/bootstrap.js", | ||
"node_modules/tether/dist/js/tether.js", | ||
"node_modules/html5sortable/dist/html.sortable.js", | ||
"node_modules/d3/d3.js", | ||
"node_modules/dagre-d3/dist/dagre-d3.js", | ||
"node_modules/dropzone/dist/dropzone.js", | ||
"node_modules/select2/dist/js/select2.js" | ||
]; | ||
js.forEach(function() { | ||
gulp.src(js).pipe(gulp.dest("./static/vendor/")); | ||
}); | ||
}); |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
{ | ||
"name": "in-toto-layout-tool", | ||
"version": "1.0.0-beta.0", | ||
"scripts": { | ||
"postinstall": "gulp && sass static/scss/main.scss:static/css/main.scss.css" | ||
}, | ||
"dependencies": { | ||
"bootstrap": "4.0.0-alpha.6", | ||
"d3": "^3.5.17", | ||
"dagre-d3": "^0.4.17", | ||
"dropzone": "^4.3.0", | ||
"gulp": "^3.9.1", | ||
"html5sortable": "^0.5.1", | ||
"select2": "^4.0.3" | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
Flask | ||
Flask-PyMongo | ||
Flask-WTF | ||
-e git://github.com/in-toto/in-toto.git@c39b04cec329bead34232a39742ebda5947633fd#egg=in-toto |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this is a Flask app, what about ignoring Flask-specific files?
Example:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added
instance/
.Adding
.webassets-cache
won't be necessary as we don't usewebassets
/Flask-Assets