Skip to content

Commit

Permalink
Merged PR 156215: add legal info; update use and contributor guidance
Browse files Browse the repository at this point in the history
* Update README.md and CONTRIBUTING.md with usage and contribution info.
* Update legal and other metadata per open source publishing guidance.

Related work items: #752891
  • Loading branch information
Josh Gavant committed Jan 2, 2019
1 parent 7c4a2f5 commit 28912b7
Show file tree
Hide file tree
Showing 3 changed files with 185 additions and 59 deletions.
112 changes: 87 additions & 25 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,98 @@
We welcome all contributions to the Oryx project: documentation, bug reports,
feature requests and source code. Please submit these using GitHub issues and
pull requests.
# Contents

To build and test the project, install the following prerequisites and run the
following scripts.
1. [Contributing](#contributing)
1. [Architecture](#architecture)

## Prerequisites
# Contributing

- bash v4.4
- dotnet v2.1
- docker v18.06.1-ce. This is the version used in CI too. We want a consistent
version across development and CI to avoid surprises.
- Golang 1.11 (for the startup command generators only)
We welcome all contributions to the Oryx project: questions and answers, bug
reports, feature requests and source code. Please submit these using GitHub
issues and pull requests.

Most contributions require you to agree to a Contributor License Agreement
(CLA) declaring that you have the right to, and actually do, grant us the
rights to use your contribution. For details, visit
<https://cla.microsoft.com>.

When you submit a pull request, a bot will determine whether you need to
provide a CLA and decorate the PR appropriately (e.g., label, comment).
Simply follow the instructions provided by the bot. You will only need to do
this once across all repositories using our CLA.

## Conduct

Run these scripts from the repo root:
This project has adopted the [Microsoft Open Source Code of
Conduct](https://opensource.microsoft.com/codeofconduct/). For more
information see the [Code of Conduct
FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact
[opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional
questions or comments.

- Build build image: `./build/build-buildimages.sh`
- Build runtime images: `./build/build-runtimeimages.sh`
- Build and test build image: `./build/test-buildimages.sh`
- Build and test runtime images: `./build/test-runtimeimages.sh`
- Build and test build and runtime images and other tests: `./build.sh`
# Architecture

## Repo Contents
The system comprises *Build* and *Run* images. **Build** images include
compilers, libraries, headers and other tools necessary to prepare artifacts;
**Run** images are smaller and contain only components required to run
programs.

* `build`: scripts for building the script generator and build and runtime
images
After the generated build script runs, artifacts are exported as files from
the Build image to be mounted into the Run image or added to a derived
container in a Dockerfile.

Another script generator in Run images generates a script to start the app
correctly.

## Repo structure

* `build`: scripts for building the script generator and build and runtime images
* `images`: Dockerfiles for the build and runtime images
* `src`: source code for the build script generator and startup command generators
* `src`: source code for the build and startup script generators
* `tests`: tests.
* `vsts`: CI/CD configuration.

## Script generators

A key element of the Build image is the
[BuildScriptGenerator](./src/BuildScriptGenerator), which analyzes the
codebase and prepares an appropriate build script. A key element of the Run
image is the [StartupScriptGenerator](./src/startupscriptgenerator), which
analyzes the codebase and prepares an appropriate start script.

## Images

The `build` image manifest is at
[./images/build/Dockerfile](./images/build/Dockerfile). It is built and
published via the Microsoft Container Registry (MCR) ([info][]) as
`mcr.microsoft.com/oryx/build` and syndicated to Docker Hub as
[index.docker.io/oryxprod/build:latest][]. Pull with `docker pull
mcr.microsoft.com/oryx/build:latest`.

[info]: https://azure.microsoft.com/en-us/blog/microsoft-syndicates-container-catalog/
[index.docker.io/oryxprod/build:latest]: https://hub.docker.com/r/oryxprod/build
[index.docker.io/microsoft/oryx-build]: https://hub.docker.com/r/microsoft/oryx-build

The *Run* images are defined in [`./images/runtime`](./images/runtime) and
published to MCR and Docker Hub as well at
<https://hub.docker.com/u/oryxprod>.

**TODO**: include a couple examples of pulling run images once repo location
is final.

## Prerequisites

The following are required to run and test this project locally.

- bash v4.4
- dotnet v2.1
- go 1.11+ (for startup script generator)
- docker v18.06.1-ce

## Golang
# Go (Golang)

The startup command generators are written in Go. If you're going to work with them, it will be helpful
to have the Go SDK installed. You won't need it to build the runtime images, since the tools are build inside the docker containers using the golang base image.
The startup script generator is written in Go to reduce storage space
required. Nevertheless you don't have to install Go to build this project
since it's available in the build containers.

Set the `GOPATH` variable to include the Oryx repo, e.g. `GOPATH=$GOPATH:c:\src\oryx`. Since the applications are inside the `src` folder there, Go should be able to find the packages and produce builds.
Set the `GOPATH` variable to include the Oryx repo, e.g.
`GOPATH=$GOPATH:c:\src\oryx`. Since the applications are inside the `src`
folder there, Go should be able to find the packages and produce builds.
8 changes: 6 additions & 2 deletions LICENSE.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
Copyright Microsoft Corporation
Oryx

Copyright (c) Microsoft Corporation. All rights reserved.

MIT license

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
Expand All @@ -16,4 +20,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOFTWARE.
124 changes: 92 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,114 @@
# Oryx Build System
# Oryx

The Oryx build system compiles a code repo into runnable artifacts. It
generates and runs an opinionated build script based on analysis of the repo's
contents; for example if it discovers `package.json` it includes `npm run
build` in the build script. Currently supported runtimes are listed [below](#supportedRuntimes).
Oryx is a build system which automatically compiles source code repos into
runnable artifacts.

The system is divided in two sets of images: build, which includes the SDKs for all platforms,
and runtime, which are much smaller in size and are specific to a given language and version.
Both sets of images are made to interact through a file sharing mechanism, either using a network volume,
or by copying the output of the command ran in the build image into the runtime-base image using a Dockerfile.
Oryx generates and runs an opinionated build script based on analysis of the
repo's contents; for example, if `package.json` is discovered in the repo
Oryx includes `npm run build` in the build script; if `requirements.txt` is
found it includes `pip install -r requirements.txt`. Oryx also analyzes and
selects a run-time entry point for the app such as `npm run start` for
Node.js or a WSGI host for Python.

The "build" image is defined in `./images/build`, and its development bits are pushed to Docker repository `oryxdevms/build`; it contains many build-time tools like compilers and header files.

The runtime images are defined in `./images/runtime`, and their development images are listed in
[https://hub.docker.com/r/oryxdevms/]. They also contain a tool that detects how the app should be started by
analyzing the build output directory, and that tool can be found under `/opt/startupcmdgen/startupcmdgen` inside
each image. This tool can be accessed by a link called 'oryx' too. They output a startup script to a file that then
will run the app when executed.

## <a name="supportedRuntimes">Supported runtimes
# Supported runtimes

Runtime | Version
--------|--------
Python | 2.7<br />3.6,3.7
Node.js | 4.4,4.5,4.8<br />6.2,6.6,6.9,6.10,6.11<br />8.0,8.1,8.2,8.8,8.9,8.11,8.12<br />9.4<br />10.1,10.12,10.14.1
Node.js | 4.4,4.5,4.8<br />6.2,6.6,6.9,6.10,6.11<br />8.0,8.1,8.2,8.8,8.9,8.11,8.12<br />9.4<br />10.1,10.12,10.14

Patches (0.0.**x**) are applied as soon as possible after they are released upstream.

# Use the system yourself

Though primarily intended for use within Azure services, you can also use the
Oryx build system yourself for troubleshooting and tests.

Oryx includes two command-line applications; the first is included in the
build image and generates a build script. The second is included in run
images and generates a run script. Both are aliased and accessible as `oryx`
in their respective environments.

## Build (`oryx build`)

* `build`: Generate and run build scripts.
* `script`: Generate build script and print to stdout.
* `languages`: Show the list of supported languages and versions.

For all options, specify `oryx --help`.

The build command takes the parameter `--output` to specify where prepared
artifacts will be placed; if not specified the source directory is used for
output as well.

# Using the system
## Run (`oryx -appPath`)

To build an app, mount it as a volume inside the build container, and run our build tool using the `oryx` command. For details further details, run `docker run oryx --help`.
The Oryx application in the run image generates a start script named run.sh, by
default in the same folder as the compiled artifact.

Currently supported commands include:
## Build and run an app

* build: Generate and run build scripts.
* languages: Show the list of supported languages and their versions.
* script: Generate build script and print to stdout.
To build and run an app from a repo, follow these steps. An example script
follows.

The build command accepts an optional output directory where the compiled bits will be placed, and if none is provided they will be added in the source directory itself. This directory can then be volume mounted in the runtime
image corresponding to the language and version being used by the app. Using the startup detection tool, the app can
be started from there, for example using `docker run -v <path to source>:/app oryxdevms/python-3.7 bash -c "oryx -appPath /app -output /app/start.sh && /app/start.sh"`.
1. Mount the repo's directory as a volume in Oryx's "build" container.
1. Run `oryx build ...` within the container to build a runnable artifact.
1. Mount the output directory from build in one of Oryx's "run" containers.
1. Run `oryx --appPath ...` within the container to write a start script.
1. Run the generated started script, by default `./run.sh`.

From your locally-cloned repo, run the following. Be sure to add
`-p/--publish` and `-e/--env` flags to the "run" docker command as necessary.

```bash
# build
docker run --volume $(pwd):/repo \
'mcr.microsoft.com/oryx/build:latest' \
'oryx build /repo --output /repo/out'

# run
docker run --detach --rm \
    --volume $(pwd)/out:/app \
# use --publish to expose ports
# --publish 8080:8080 \
# use --env to add env vars
# --env MYKEY=value \
'mcr.microsoft.com/oryx/node-10.12:latest' \
# or use the Python image
# 'mcr.microsoft.com/oryx/python-3.7:latest \
    sh -c 'oryx -appPath /app && /app/run.sh'
```

# Contributing

See [CONTRIBUTING.md](./CONTRIBUTING.md).

# License

MIT, see [LICENSE.md](./LICENSE.md).

# Contributing
# Security

Security issues and bugs should be reported privately, via email, to the
Microsoft Security Response Center (MSRC) at
[secure@microsoft.com](mailto:secure@microsoft.com). You should receive a
response within 24 hours. If for some reason you do not, please follow up via
email to ensure we received your original message. Further information,
including the [MSRC
PGP](https://technet.microsoft.com/en-us/security/dn606155) key, can be found
in the [Security
TechCenter](https://technet.microsoft.com/en-us/security/default).

# Data/Telemetry

When utilized within Azure services, this project collects usage data and
sends it to Microsoft to help improve our products and services. Read
[Microsoft's privacy statement][] to learn more.

See [CONTRIBUTING.md](./contributing.md).
[Microsoft's privacy statement]: http://go.microsoft.com/fwlink/?LinkId=521839

This project follows the [Microsoft Open Source Code of Conduct][coc]. For more
information see the [Code of Conduct FAQ][cocfaq]. Contact
This project follows the [Microsoft Open Source Code of Conduct][coc]. For
more information see the [Code of Conduct FAQ][cocfaq]. Contact
[opencode@microsoft.com][cocmail] with questions and comments.

[coc]: https://opensource.microsoft.com/codeofconduct/
Expand Down

0 comments on commit 28912b7

Please sign in to comment.