From aff2ad724f84e4ebeba2093d172c3144e785016c Mon Sep 17 00:00:00 2001 From: Akshay Shah Date: Fri, 8 Jan 2016 13:02:38 -0800 Subject: [PATCH 1/8] Remove TODO.md in favor of GitHub issues --- TODO.md | 106 -------------------------------------------------------- 1 file changed, 106 deletions(-) delete mode 100644 TODO.md diff --git a/TODO.md b/TODO.md deleted file mode 100644 index 46da3760..00000000 --- a/TODO.md +++ /dev/null @@ -1,106 +0,0 @@ -h1. Functionality - -- [ ] Cancel outbound requests - - Sends cancel message to server - - If server has made requests, should cancel those requests as well - - Should return from blocking calls with context.Cancelled - -- [ ] Cancel on start and cancel on finish -- [X] Pass Zipkin Tracing info as Context value, propagate to outbound Context -- [ ] Implement Farm32 -- [ ] Outbound connection pool management -- [ ] Track connections and do graceful shutdown when the channel is closed -- [ ] Provide a way to throttle max # of incoming requests -- [ ] Retry on retryable errors -- [ ] Allow setting the content type headers -- [X] Add JSONInput/JSONOutput argument types -- [ ] Add ThriftInput/ThriftOutput argument types -- [ ] Implement ping/pong handling - -h1. Cleanup - -- [X] Get rid of custom IOError type - not needed and misleading -- [X] Refactor bodyReader to work like bodyWriter and operate over the entire body not just an argument -- [X] Move message dumping into a separate hexer, make configurable -- [X] Remove hard-dependency on go-logging and replace with Logger interface -- [ ] Make sure we don't leak frames anywhere (I know we do) -- [ ] Implement real FramePool based on sync.Pool -- [X] Pass remotePeerInfo into inbound and outbound pipelines, once handshaking is complete. Alternately don't create those pipelines until after -- [ ] Benchmark -- [X] Review and edit godocs - -h1. Tests - -h2. Handshake and general connection tests - -1. Attempt to handshake after active -2. Attempt to handshake on closed connection -3. Attempt to handshake on inbound connetion -4. Server sends bad protocol version -5. Client sends bad protocol version -5. Client sends non init-req message as first in stream -6. Timeout waiting for init-res -7. Server immediately sends call-res after init-res -8. Receive unknown frame type -9. Server hangs up during read of request -10. Client hangs up during read of response - -h2. Fragmentation error tests - -1. Attempt to read from argument after argument has been completed -2. Attempt to write to stream after last fragment sent -3. Bad checksum value on read -4. Second received fragment has different checksum type than first fragment -5. Error retrieving fragment -6. Error beginning fragment on write -7. Chunk size does not fit into current fragment -8. Error flushing fragment on write -9. Error beginning new fragment for empty chunk -10. Error flushing final fragment on completion -11. Reader thinks they've finished with an argument, but there is more data in the chunk -12. Reader thinks they've finished with an argument, but there is more data for that argument in a subsequent fragment - - -h2. Outbound tests - -1. Attempt to make a call after the deadline has past results in a short circuit -2. Attempt to make an outbound call for a request that is still active -3. Make sure that expired response channels are proactively removed -4. Can't enqueue to response channel results in call failing -5. Receive CallRes for an expired request -6. Proactively remove response channels for expired and waiting calls -7. Error frame received for non-existent response -8. Attempt to make call from invalid state: - * connection not ready - * connection closed -9. Attempt to WriteArg2 twice -10. Attempt to WriteArg3 twice -11. Error allocating outbound fragment results in call terminating -12. Timeout / cancel while flushing fragment results in call terminating -13. Unable to send to send channel results in call terminating -14. Attempt to ReadArg2 twice -15. Attempt to ReadArg3 twice -16. Error reading arg2 (e.g. bad parse) -17. Error reading arg3 (e.g. pad parse) -18. Recv bad response frame for message -19. Die on protocol error - -h3. Inbound Tests - -1. Receive duplicate request id -2. Cannot parse first fragment for request -3. Receive continue request fragment for req that expired -4. Application not pulling request frames quickly enough causes request to die -5. Proactive remove request channels for expired and waiting calls -6. readMethod twice -7. Error reading method? -8. ReadArg2 twice -9. ReadArg3 twice -10. Timeout waiting for request fragment results in call being terminated -11. Error parsing request fragment results in call being terminated -12. Returning application error -13. Attempting to return application error after arg2 is sent results in error -14. Unable to send system error -15. Frame sender dead so response fragment cannot be flushed - - From 499727ca093dc08e03b0a73085f2c128f7b06db7 Mon Sep 17 00:00:00 2001 From: Akshay Shah Date: Fri, 8 Jan 2016 13:09:25 -0800 Subject: [PATCH 2/8] Add a separate license file --- LICENSE.md | 19 +++++++++++++++++++ README.md | 5 ++++- 2 files changed, 23 insertions(+), 1 deletion(-) create mode 100644 LICENSE.md diff --git a/LICENSE.md b/LICENSE.md new file mode 100644 index 00000000..b090ca4f --- /dev/null +++ b/LICENSE.md @@ -0,0 +1,19 @@ +Copyright (c) 2015 Uber Technologies, Inc. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. diff --git a/README.md b/README.md index 2db0b7ef..56a15697 100644 --- a/README.md +++ b/README.md @@ -121,4 +121,7 @@ change in the future. - mmihic - prashantv -## MIT Licenced +
+ +This project is released under the [MIT +License](https://github.com/uber/tchannel-go/blob/master/LICENSE.md). From 93aead8a07ab65508957be784e83055b869d3686 Mon Sep 17 00:00:00 2001 From: Akshay Shah Date: Fri, 8 Jan 2016 15:04:34 -0800 Subject: [PATCH 3/8] Add CONTRIBUTING.md When repos have contribution guidelines, GitHub automatically shows users a banner when they're opening PRs: https://github.com/blog/1184-contributing-guidelines. This is also the natural place for the nuts-and-bolts of getting the repo set up. --- CONTRIBUTING.md | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..64d61da1 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,34 @@ +Contributing +============ + +We'd love your help making tchannel-go great! + +## Getting Started + +TChannel uses [godep](https://github.com/tools/godep) to manage dependencies. +To get started: + +```bash +go get github.com/uber/tchannel-go +go get github.com/tools/godep +cd $GOPATH/src/github.com/uber/tchannel-go +godep restore +make # tests should pass +``` + +## Making A Change + +*Before making any significant changes, please [open an +issue](https://github.com/uber/tchannel-go/issues).* Discussing your proposed +changes ahead of time will make the contribution process smooth for everyone. + +Once we've discussed your changes and you've got your code ready, make sure +that tests are passing (`make test` or `make cover`) and open your PR! Your +pull request is most likely to be accepted if it: + +* Includes tests for new functionality. +* Follows the guidelines in [Effective + Go](https://golang.org/doc/effective_go.html) and the [Go team's common code + review comments](https://github.com/golang/go/wiki/CodeReviewComments). +* Has a [good commit + message](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html). From 829df411964319a886d358cb50df2622b0fda947 Mon Sep 17 00:00:00 2001 From: Akshay Shah Date: Fri, 8 Jan 2016 15:11:50 -0800 Subject: [PATCH 4/8] Bump version number --- version.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/version.go b/version.go index 18134194..98b5f2ad 100644 --- a/version.go +++ b/version.go @@ -23,4 +23,4 @@ package tchannel // VersionInfo identifies the version of the TChannel library. // Due to lack of proper package management, this version string will // be maintained manually. -const VersionInfo = "0.0.1-dev" +const VersionInfo = "1.0.0" From f6761c01090782b0040f392aea1a4a913606f067 Mon Sep 17 00:00:00 2001 From: Akshay Shah Date: Fri, 8 Jan 2016 15:40:54 -0800 Subject: [PATCH 5/8] Add READMEs for examples --- examples/keyvalue/README.md | 10 ++++++++++ examples/ping/README.md | 14 ++++++++++++++ 2 files changed, 24 insertions(+) create mode 100644 examples/keyvalue/README.md create mode 100644 examples/ping/README.md diff --git a/examples/keyvalue/README.md b/examples/keyvalue/README.md new file mode 100644 index 00000000..4776cb0a --- /dev/null +++ b/examples/keyvalue/README.md @@ -0,0 +1,10 @@ +# Key-Value Store + +```bash +./build/examples/keyvalue/server +./build/examples/keyvalue/client +``` + +This example exposes a simple key-value store over TChannel using the Thrift +protocol. The client has an interactive CLI that can be used to make calls to +the server. diff --git a/examples/ping/README.md b/examples/ping/README.md new file mode 100644 index 00000000..25f1e637 --- /dev/null +++ b/examples/ping/README.md @@ -0,0 +1,14 @@ +# Ping-Pong + +```bash +./build/examples/ping/pong +``` + +This example creates a client and server channel. The server channel registers +a `PingService` with a `ping` method, which takes request `Headers` and a `Ping` body +and returns the same `Headers` along with a `Pong` body. The client sends a ping +request to the server. + +Note that every instance is bidirectional, so the same channel can be used for +both sending and receiving requests to peers. New connections are initiated on +demand. From 2411edbafaa232c827f4b9d9da3bd393000d61c2 Mon Sep 17 00:00:00 2001 From: Akshay Shah Date: Fri, 8 Jan 2016 15:41:07 -0800 Subject: [PATCH 6/8] Make top-level README shorter --- README.md | 150 ++++++++++++++++-------------------------------------- 1 file changed, 44 insertions(+), 106 deletions(-) diff --git a/README.md b/README.md index 56a15697..50a61331 100644 --- a/README.md +++ b/README.md @@ -1,127 +1,65 @@ -# TChannel +# TChannel [![GoDoc][doc-img]][doc] [![Build Status][ci-img]][ci] -[![GoDoc](https://godoc.org/github.com/uber/tchannel-go?status.svg)](https://godoc.org/github.com/uber/tchannel-go) -[![Build Status](https://travis-ci.org/uber/tchannel-go.svg?branch=master)](https://travis-ci.org/uber/tchannel-go) +[TChannel][tchan-spec] is a multiplexing and framing protocol for RPC calls. +tchannel-go is a Go implementation of the protocol, including client libraries +for [Hyperbahn][hyperbahn]. -[TChannel](https://github.com/uber/tchannel/blob/master/docs/protocol.md) is a multiplexing and framing protocol for RPC calls. - -tchannel-go is a Go implementation of the TChannel protocol, and includes client libraries for [Hyperbahn](https://github.com/uber/hyperbahn). - -## Stability: experimental - -NOTE: `master:golang` is **not yet stable** - -## Thrift + TChannel Integration - -If you want to use Thrift+TChannel, you will want to read [this guide](guide/Thrift_Hyperbahn.md). - -## Getting Started - -Get Go from your package manager of choice or follow the [official installation instructions](https://golang.org/doc/install). - -```bash -brew install go - -# This will be your GOPATH where all Go code will live. -mkdir -p ~/gocode/src -``` - -Set up your environment for your shell of choice. - -```bash -export GOPATH="${HOME}/gocode" -export PATH="${PATH}":"${GOPATH}/bin" -``` - -TChannel uses [godep](https://github.com/tools/godep) to manage dependencies. To get started: - -```bash -go get github.com/uber/tchannel-go -go get github.com/tools/godep -cd $GOPATH/src/github.com/uber/tchannel-go -godep restore -make -``` -### Examples - -Simple examples are included which demonstrate the TChannel API and features. - - -#### PingPong -```bash -./build/examples/ping/pong -``` - -This example creates a client and server channel. The server channel registers a PingService -with a ping method, which takes request Headers and a Ping body and returns the -same Headers along with a Pong body. The client sends a ping request to the server - -Note that every instance is bidirectional, so the same channel can be used for both sending -and receiving requests to peers. New connections are initiated on demand. - - -#### KeyValue -```bash -./build/examples/keyvalue/server -./build/examples/keyvalue/client -``` - -This example exposes a simple keyvalue service over TChannel using the Thrift protocol. -The client has an interactive CLI that can be used to make calls to the server. +If you'd like to start by writing a small Thrift and TChannel service, check +out [this guide](guide/Thrift_Hyperbahn.md). For a less opinionated setup, see +the [contribution guidelines](CONTRIBUTING.md). ## Overview -TChannel is a network protocol with the following goals: +TChannel is a network protocol that supports: - * request / response model - * multiple requests multiplexed across the same TCP socket - * out of order responses - * streaming request and responses - * all frames checksummed - * transport arbitrary payloads - * easy to implement in multiple languages - * near-redis performance + * A request/response model, + * Multiplexing multiple requests across the same TCP socket, + * Out-of-order responses, + * Streaming requests and responses, + * Checksummed frames, + * Transport of arbitrary payloads, + * Easy implementation in many languages, and + * Redis-like performance. -This protocol is intended to run on datacenter networks for inter-process communication. +This protocol is intended to run on datacenter networks for inter-process +communication. ## Protocol -TChannel frames have a fixed length header and 3 variable length fields. The underlying protocol -does not assign meaning to these fields, but the included client/server implementation uses -the first field to represent a unique endpoint or function name in an RPC model. -The next two fields can be used for arbitrary data. Some suggested way to use the 3 fields are: +TChannel frames have a fixed-length header and 3 variable-length fields. The +underlying protocol does not assign meaning to these fields, but the included +client/server implementation uses the first field to represent a unique +endpoint or function name in an RPC model. The next two fields can be used for +arbitrary data. Some suggested way to use the 3 fields are: -* URI path, HTTP method and headers as JSON, body -* function name, headers, thrift / protobuf +* URI path + HTTP method and headers as JSON + body, or +* Function name + headers + thrift/protobuf. -Note however that the only encoding supported by TChannel is UTF-8. If you want JSON, you'll need -to stringify and parse outside of TChannel. +Note, however, that the only encoding supported by TChannel is UTF-8. If you +want JSON, you'll need to stringify and parse outside of TChannel. -This design supports efficient routing and forwarding of data where the routing information needs -to parse only the first or second field, but the 3rd field is forwarded without parsing. +This design supports efficient routing and forwarding: routers need to parse +the first or second field, but can forward the third field without parsing. -There is no notion of client and server in this system. Every TChannel instance is capable of -making or receiving requests, and thus requires a unique port on which to listen. This requirement may -change in the future. +There is no notion of client and server in this system. Every TChannel instance +is capable of making and receiving requests, and thus requires a unique port on +which to listen. This requirement may change in the future. - - See [protocol.md](https://github.com/uber/tchannel/blob/master/docs/protocol.md) for more details +See the [protocol specification][tchan-proto-spec] for more details. -## Further examples +## Examples - - [ping](examples/ping/main.go): A simple ping/pong example using raw TChannel. + - [ping](examples/ping): A simple ping/pong example using raw TChannel. - [thrift](examples/thrift): A Thrift server/client example. - [keyvalue](examples/keyvalue): A keyvalue Thrift service with separate server and client binaries. -## Tests - -`make test` or `make cover` - -## Contributors - - - mmihic - - prashantv -
- -This project is released under the [MIT -License](https://github.com/uber/tchannel-go/blob/master/LICENSE.md). +This project is released under the [MIT License](LICENSE.md). + +[doc-img]: https://godoc.org/github.com/uber/tchannel-go?status.svg +[doc]: https://godoc.org/github.com/uber/tchannel-go +[ci-img]: https://travis-ci.org/uber/tchannel-go.svg?branch=master +[ci]: https://travis-ci.org/uber/tchannel-go +[tchan-spec]: http://tchannel.readthedocs.org/en/latest/ +[tchan-proto-spec]: http://tchannel.readthedocs.org/en/latest/protocol/ +[hyperbahn]: https://github.com/uber/hyperbahn From e55a4dadd56ce92e270d13bc690f203538799aaf Mon Sep 17 00:00:00 2001 From: Akshay Shah Date: Fri, 8 Jan 2016 15:46:05 -0800 Subject: [PATCH 7/8] Add changelog --- CHANGELOG.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 CHANGELOG.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000..f853d781 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,15 @@ +Changelog +========= + +# 1.0.0 + +* First stable release. +* Supports making calls with JSON, Thrift or raw payloads. +* Services use thrift-gen, and implement handlers wiht a `func(ctx, arg) (res, + error)` signature. +* Supports retries. +* Peer selection (peer heap, prefer incoming strategy, for use with Hyperbahn). +* Graceful channel shutdown. +* TCollector trace reporter with sampling support. +* Metrics collection with StatsD. +* Thrift support, including includes. From 318322b1cdf7a9c50d699a87e9b95cb9a19e60bc Mon Sep 17 00:00:00 2001 From: Akshay Shah Date: Sat, 9 Jan 2016 18:19:44 -0800 Subject: [PATCH 8/8] Enable coveralls.io support --- .travis.yml | 3 ++- Makefile | 12 +++++++++--- README.md | 4 +++- 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/.travis.yml b/.travis.yml index 971d9b8b..57c42fcb 100644 --- a/.travis.yml +++ b/.travis.yml @@ -3,4 +3,5 @@ go: - 1.4 - 1.5 install: make install_ci - +script: make test_ci +after_success: make cover_ci diff --git a/Makefile b/Makefile index 80b7a453..ea2b1f17 100644 --- a/Makefile +++ b/Makefile @@ -46,6 +46,7 @@ install: GOPATH=$(GODEPS) godep restore install_ci: get_thrift install + go get -u github.com/mattn/goveralls help: @egrep "^# target:" [Mm]akefile | sort - @@ -78,12 +79,17 @@ benchmark: clean setup echo Running benchmarks: go test $(PKGS) -bench=. -parallel=4 -cover: clean setup - echo Testing packages: +cover_profile: clean setup + @echo Testing packages: mkdir -p $(BUILD) - go test ./ $(TEST_ARG) -coverprofile=$(BUILD)/coverage.out + go test ./ $(TEST_ARG) -coverprofile=$(BUILD)/coverage.out + +cover: cover_profile go tool cover -html=$(BUILD)/coverage.out +cover_ci: cover_profile + goveralls -coverprofile=$(BUILD)/coverage.out -service=travis-ci + vet: echo Vetting packages for potential issues... go tool vet $(PKGS) diff --git a/README.md b/README.md index 50a61331..cf613183 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# TChannel [![GoDoc][doc-img]][doc] [![Build Status][ci-img]][ci] +# TChannel [![GoDoc][doc-img]][doc] [![Build Status][ci-img]][ci] [![Coverage Status][cov-img]][cov] [TChannel][tchan-spec] is a multiplexing and framing protocol for RPC calls. tchannel-go is a Go implementation of the protocol, including client libraries @@ -60,6 +60,8 @@ This project is released under the [MIT License](LICENSE.md). [doc]: https://godoc.org/github.com/uber/tchannel-go [ci-img]: https://travis-ci.org/uber/tchannel-go.svg?branch=master [ci]: https://travis-ci.org/uber/tchannel-go +[cov-img]: https://coveralls.io/repos/uber/tchannel-go/badge.svg?branch=master&service=github +[cov]: https://coveralls.io/github/uber/tchannel-go?branch=master [tchan-spec]: http://tchannel.readthedocs.org/en/latest/ [tchan-proto-spec]: http://tchannel.readthedocs.org/en/latest/protocol/ [hyperbahn]: https://github.com/uber/hyperbahn