-
Notifications
You must be signed in to change notification settings - Fork 44
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #1 from northwesternmutual/pull_request_damien
Initial Release
- Loading branch information
Showing
414 changed files
with
31,025 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# Hidden | ||
.gopath~ | ||
.DS_Store | ||
.vscode | ||
|
||
# Directories | ||
vendor | ||
bin | ||
n1data | ||
n2data | ||
n3data | ||
portainer-data | ||
scripts/db | ||
scripts/log |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
sudo: false | ||
|
||
language: go | ||
|
||
env: | ||
- GO111MODULE=on | ||
|
||
go: | ||
- 1.11.x | ||
|
||
git: | ||
depth: 1 | ||
|
||
before_script: | ||
- go mod vendor | ||
|
||
script: | ||
- go test -v -parallel 3 --race ./... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
# How to Contribute to Grammes | ||
|
||
## Getting Started | ||
|
||
This project uses Go modules to manage dependencies, and is built on Go version `1.11`. | ||
|
||
```sh | ||
==> go get github.com/northwesternmutual/grammes | ||
``` | ||
|
||
## Testing | ||
|
||
Grammes unit tests can be run by directing yourself to the root directory for the project and running this command in a terminal: | ||
|
||
```sh | ||
==> go test -parallel 3 --race ./... | ||
``` | ||
|
||
## Project Structure | ||
|
||
Below is an overview of this project's structure (files are not listed) | ||
|
||
```flat | ||
github.com/northwesternmutual/grammes | ||
assets/ - Contains images and shell scripts for customizing JanusGraph | ||
docs/ - Miscellaneous documents and charts to help explain Grammes | ||
examples/ - Examples of all resources on how to use Grammes | ||
gremconnect/ - The package for handling connections and dialers | ||
gremerror/ - Contains custom error types for Grammes | ||
logging/ - Has the custom loggers for the Grammes client | ||
manager/ - Contains the GraphManager and all graph related functions | ||
model/ - Holds all of the structs and methods for Go counterpart objects | ||
query/ - Contains query related packages and the Query interface | ||
cardinality/ - Describes the number of relationship occurrences | ||
column/ - Used to reference particular parts of a column in Gremlin | ||
consumer/ - Controls how barriers emit their values | ||
datatype/ - String counterparts to reference Gremlin types | ||
direction/ - Denotes the direction of edges | ||
graph/ - The verbose graph traversal object (Query) | ||
multiplicity/ - Controls property sets for edges | ||
operator/ - Used to apply mathematical operations in a graph traversal | ||
pop/ - Determines the gathered values | ||
predicate/ - Controls the search conditions of a query | ||
scope/ - Controls the relations of a graph traversal | ||
token/ - Defines the parts of a vertex | ||
traversal/ - The preferred graph traversal (Query) | ||
quick/ - Used when executing queries without a Grammes client | ||
testing/ - Has the mock for the GraphManager when testing Grammes in your project | ||
``` | ||
|
||
## Imports Grouping | ||
|
||
This projects adheres to the following pattern when grouping imports in Go files: | ||
|
||
* imports from standard library | ||
* imports from other projects | ||
* imports from internal project | ||
|
||
In addition, imports in each group must be sorted by length. For example: | ||
|
||
```go | ||
import ( | ||
"time" | ||
"errors" | ||
|
||
"go.uber.org/zap" | ||
|
||
"github.com/northwesternmutual/grammes/query/token" | ||
"github.com/northwesternmutual/grammes" | ||
) | ||
``` | ||
|
||
## Making a Change | ||
|
||
Before making any significant changes, please [open an issue](https://github.com/northwesternmutual/grammes/issues). Discussing your proposed changes ahead of time will make the contribution process smooth for everyone. | ||
|
||
Once we've discussed your changes and you've got your code ready, make sure that tests are passing (go test) and open your PR. Your pull request is most likely to be accepted if it: | ||
|
||
* Includes tests for new functionality. | ||
* Follows the guidelines in [Effective Go](https://golang.org/doc/effective_go.html) and the [Go team's common code review comments](https://github.com/golang/go/wiki/CodeReviewComments). | ||
* Has a good commit message. | ||
|
||
## License | ||
|
||
By contributing your code, you agree to license your contribution under the terms of the [Apache License](https://github.com/northwesternmutual/grammes/blob/master/LICENSE). | ||
|
||
If you are adding a new file it should have a header like below. | ||
|
||
```flat | ||
// Copyright (c) 2018 Northwestern Mutual. | ||
// | ||
// Permission is hereby granted, free of charge, to any person obtaining a copy | ||
// of this software and associated documentation files (the "Software"), to deal | ||
// in the Software without restriction, including without limitation the rights | ||
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
// copies of the Software, and to permit persons to whom the Software is | ||
// furnished to do so, subject to the following conditions: | ||
// | ||
// The above copyright notice and this permission notice shall be included in | ||
// all copies or substantial portions of the Software. | ||
// | ||
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN | ||
// THE SOFTWARE. | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,228 @@ | ||
<p align="center"> | ||
<img src="assets/img/grammes-gopher-v4.png" alt="Grammes" title="Grammes" width="50%"> | ||
</p> | ||
|
||
# Grammes | ||
|
||
[![Build Status](https://travis-ci.com/northwesternmutual/grammes.svg?branch=master)](https://travis-ci.com/northwesternmutual/grammes) [![Coverage Status](https://coveralls.io/repos/github/northwesternmutual/grammes/badge.svg?branch=master)](https://coveralls.io/github/northwesternmutual/grammes?branch=master) [![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://godoc.org/github.com/northwesternmutual/grammes) [![Go Report Card](https://goreportcard.com/badge/github.com/northwesternmutual/grammes)](https://goreportcard.com/report/github.com/northwesternmutual/grammes) | ||
|
||
Grammes is an API/Wrapper for Gremlin and Janusgraph. It's written purely in Golang and allows for easy use of Gremlin without touching the Gremlin terminal. | ||
|
||
## Table of Contents | ||
|
||
- [Grammes](#grammes) | ||
- [Table of Contents](#table-of-contents) | ||
- [Local Setup](#local-setup) | ||
- [Cloning Grammes](#cloning-grammes) | ||
- [Setting up JanusGraph](#setting-up-janusgraph) | ||
- [Using Grammes](#using-grammes) | ||
- [Testing Grammes](#testing-grammes) | ||
- [Additional Resources](#additional-resources) | ||
- [Documentation on Gremlin](#documentation-on-gremlin) | ||
- [Examples](#examples) | ||
- [Troubleshooting](#troubleshooting) | ||
- [Fixing time outs when starting Janusgraph](#fixing-time-outs-when-starting-janusgraph) | ||
|
||
## Local Setup | ||
|
||
You need to setup all of the following tools to run the service locally | ||
|
||
- Go 1.12 | ||
- Git | ||
- Elastic Search | ||
- Cassandra | ||
- Java 8 | ||
|
||
--- | ||
|
||
### Cloning Grammes | ||
|
||
Begin by opening up a terminal or command prompt and clone the grammes repository. | ||
|
||
```sh | ||
go get -u github.com/northwesternmutual/grammes | ||
``` | ||
|
||
--- | ||
|
||
### Setting up JanusGraph | ||
|
||
*if you have decided to use another graph database then you may move on to [project setup](#using-grammes)* | ||
|
||
First off, direct your terminal to the Grammes' `scripts` directory. | ||
|
||
```sh | ||
cd $GOPATH/src/github.com/northwesternmutual/grammes/scripts | ||
``` | ||
|
||
In here you can find the `gremlin.sh` and `janusgraph.sh` scripts. To set up JanusGraph just run the `janusgraph.sh` script. | ||
|
||
```sh | ||
./janusgraph.sh | ||
``` | ||
|
||
This should download and/or begin the graph and TinkerPop server. | ||
|
||
To make sure that everything is running try running `gremlin.sh`. | ||
|
||
```sh | ||
$ ./gremlin.sh | ||
SLF4J: Class path contains multiple SLF4J bindings. | ||
SLF4J: Found binding in [jar:file:/Users/<username>/projects/nm/gocode/src/github.com/northwesternmutual/grammes/bin/janusgraph-0.3.1-hadoop2/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class] | ||
SLF4J: Found binding in [jar:file:/Users/<username>/projects/nm/gocode/src/github.com/northwesternmutual/grammes/bin/janusgraph-0.3.1-hadoop2/lib/logback-classic-1.1.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] | ||
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. | ||
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] | ||
15:05:59 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | ||
gremlin> | ||
``` | ||
|
||
Once Gremlin starts then you may begin by running this command. | ||
|
||
```sh | ||
gremlin> :remote connect tinkerpop.server conf/remote.yaml | ||
===>Configured localhost/127.0.0.1:8182 | ||
``` | ||
|
||
If you see the message that Gremlin was configured to the localhost then quit Gremlin. | ||
|
||
```sh | ||
gremlin> :exit | ||
``` | ||
|
||
Finally, run the `janusgraph.sh` script again, but this time with the `status` flag. | ||
|
||
```sh | ||
./janusgraph.sh status | ||
``` | ||
|
||
--- | ||
|
||
### Using Grammes | ||
|
||
Once you have cloned the repository then you may begin implementing it into a project. Let's begin with creating a place for your code in the `$GOPATH`, i.e., | ||
|
||
```sh | ||
$GOPATH/src/github.com/<username-here>/<project-name-here> | ||
``` | ||
|
||
Next, you'll want to create a `main.go` file. For this example I will be using [MS Code](https://code.visualstudio.com/download), but you may use any editor you prefer. | ||
|
||
```sh | ||
code main.go | ||
``` | ||
|
||
In this file we can begin by making it a typical empty `main.go` file like this: | ||
|
||
``` go | ||
package main | ||
|
||
func main() { | ||
} | ||
``` | ||
|
||
Next, import the grammes package and begin using it by connecting your client to a gremlin server. | ||
|
||
``` go | ||
package main | ||
|
||
import ( | ||
"log" | ||
|
||
"github.com/northwesternmutual/grammes" | ||
) | ||
|
||
func main() { | ||
// Creates a new client with the localhost IP. | ||
client, err := grammes.DialWithWebSocket("ws://127.0.0.1:8182") | ||
if err != nil { | ||
log.Fatalf("Error while creating client: %s\n", err.Error()) | ||
} | ||
|
||
// Executing a basic query to assure that the client is working. | ||
res, err := client.ExecuteStringQuery("1+3") | ||
if err != nil { | ||
log.Fatalf("Querying error: %s\n", err.Error()) | ||
} | ||
|
||
// Print out the result as a string | ||
log.Println(string(res)) | ||
} | ||
``` | ||
|
||
Once the client is created then you can begin querying the gremlin server via the `.ExecuteQuery` method in the client. To use this function you must put in a `Query` which is an interface for any kind of Query-able type in the package. These include: `graph.String` and `traversal.String`. For an example of querying the gremlin server for all of the Vertex labels: | ||
|
||
``` go | ||
package main | ||
|
||
import ( | ||
"log" | ||
|
||
"github.com/northwesternmutual/grammes" | ||
) | ||
|
||
func main() { | ||
// Creates a new client with the localhost IP. | ||
client, err := grammes.DialWithWebSocket("ws://127.0.0.1:8182") | ||
if err != nil { | ||
log.Fatalf("Error while creating client: %s\n", err.Error()) | ||
} | ||
|
||
// Executing a query to add a vertex to the graph. | ||
client.AddVertex("testingvertex") | ||
|
||
// Create a new traversal string to build your traverser. | ||
g := grammes.Traversal() | ||
|
||
// Executing a query to fetch all of the labels from the vertices. | ||
data, err := client.ExecuteQuery(g.V().Label()) | ||
if err != nil { | ||
log.Fatalf("Querying error: %s\n", err.Error()) | ||
} | ||
|
||
// Log out the response. | ||
log.Println(string(data)) | ||
} | ||
``` | ||
|
||
After this is all written you may run this file by saving it and hopping back on to your terminal. After starting your Gremlin Server and graph database in the terminal let's run this command to run the file: | ||
|
||
```sh | ||
go run main.go | ||
``` | ||
|
||
For more examples look in the `examples/` directory of the project. In there you'll find multiple examples on how to use the Grammes package. | ||
|
||
## Testing Grammes | ||
|
||
Grammes uses [goconvey](https://www.github.com/smartystreets/goconvey/) by [smartystreets](https://www.github.com/smartystreets/) for its tests. Before trying to run the unit tests in Grammes you should update your version of this repository using this command. | ||
|
||
```sh | ||
go get -u github.com/smartystreets/goconvey/convey | ||
``` | ||
|
||
Once you have this downloaded you may run the tests in Grammes how you normally would in Golang. | ||
|
||
```sh | ||
go test ./... | ||
``` | ||
|
||
## Additional Resources | ||
|
||
### Documentation on Gremlin | ||
|
||
To learn more about how to use Gremlin I highly recommend looking through their [Tinkerpop3 documentation](http://tinkerpop.apache.org/docs/current/reference/). It's full of examples and documentation on every traversal step available. | ||
|
||
### Examples | ||
|
||
To find examples look in the `examples/` directory of Grammes. In there you'll find plenty of examples related to how to use this package. Make sure you're running Janusgraph before you begin running the examples. | ||
|
||
## Troubleshooting | ||
|
||
### Fixing time outs when starting Janusgraph | ||
|
||
If Nodetool times out or any other part of the setup times out then the most common issue is that Cassandra is already running on your machine. To fix this run this command. | ||
|
||
``` bash | ||
# only for Unix at the moment. | ||
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.cassandra.plist | ||
``` |
Oops, something went wrong.