presentation.slide

What's new in Go 1.8
Go 1.8 Release Party

Borys Hulii
SE, SoftServe

* License and Materials

This presentation is licensed under the [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons Attribution-ShareAlike 4.0 International]] licence.

The materials for this presentation are available on GitHub:

.link https://github.com/davecheney/go-1.8-release-party

You are encouraged to remix, transform, or build upon the material, providing you distribute your contributions under the same license.

If you have suggestions or corrections to this presentation, please raise [[https://github.com/davecheney/go-1.8-release-party/issues][an issue on the GitHub project]].

# * What's changed

# So, what's happened in the last six months since Go 1.7?

# - Performance
# - Compiler changes
# - Tool changes
# - Runtime changes
# - Changes to the standard library

* Performance

* Performance

As always, the changes are so general and varied that precise statements about performance are difficult to make.

Most programs should run a bit faster, due to speedups in the garbage collector and optimizations in the standard library.

The new back end, based on static single assignment form (SSA), generates more compact, more efficient code and provides a better platform for optimizations such as bounds check elimination.

- The new back end reduces the CPU time required by our benchmark programs by 20-30% on 32-bit ARM systems.
- For 64-bit x86 systems, which already used the SSA back end in Go 1.7, the gains are a more modest 0-10%.
- Other architectures will likely see improvements closer to the 32-bit ARM numbers.

.link https://dave.cheney.net/2016/11/19/go-1-8-toolchain-improvements Go 1.8 Toolchain Improvements

* Garbage Collector

Garbage collection pauses should be significantly shorter than they were in Go 1.7, usually under 100 microseconds and often as low as 10 microseconds.

.link https://talks.golang.org/2017/state-of-go.slide#34 Garbage Collector Improvements Over Time (ht. @francesc)
.link https://twitter.com/brianhatfield/status/804355831080751104 Original post in twitter

# More work remains for Go 1.9.

# .link https://github.com/golang/proposal/blob/master/design/17503-eliminate-rescan.md Eliminating Stop-The-World Stack Re-scanning (Design doc)

* Garbage Collector (Go 1.4.3 vs Go 1.5)

From ~300 to ~40 milliseconds

.image img/GC_14vs15.png 500 _

* Garbage Collector (Go 1.5 vs Go 1.6)
 
From ~40 to ~10 milliseconds

.image img/GC_15vs16.jpg 500 _

* Garbage Collector (Go 1.6 vs Go 1.6.3)
 
From ~10 to ~3.5 milliseconds

.image img/GC_16vs163.jpg 500 _

* Garbage Collector (Go 1.6.3 vs Go 1.7)
 
From ~3.5 to ~3 milliseconds
.image img/GC_163vs17.jpg 500 _

* Garbage Collector (Go 1.7.3 vs Go 1.8 beta1)
 
From ~3 to less then 1 millisecond
.image img/GC_17vs18b.jpg 500 _

* Defer

The overhead of deferred function calls has been reduced by nearly half.

 name           old time/op  new time/op  delta
 Defer-4        75.1ns ± 1%  43.3ns ± 1%  -42.31%   (p=0.000 n=8+10)
 
 In reality, this speeds up defer by about 2.2X. The two benchmarks
 below compare a Lock/defer Unlock pair (DeferLock) with a Lock/Unlock
 pair (NoDeferLock). NoDeferLock establishes a baseline cost, so these
 two benchmarks together show that this change reduces the overhead of
 defer from 61.4ns to 27.9ns.
 
 name           old time/op  new time/op  delta
 DeferLock-4    77.4ns ± 1%  43.9ns ± 1%  -43.31%  (p=0.000 n=10+10)
 NoDeferLock-4  16.0ns ± 0%  15.9ns ± 0%   -0.39%    (p=0.000 n=9+8)

.link https://golang.org/cl/29656 Source: CL 29656
.link https://dave.cheney.net/2016/11/19/go-1-8-toolchain-improvements#defer Defer and cgo improvements
.link https://github.com/golang/go/issues/14939#issuecomment-249015936 Benchmarks
 
* Cgo

The overhead of cgo calls is reduced by more than half:

 name       old time/op  new time/op  delta
 CgoNoop-8   146ns ± 1%    56ns ± 6%  -61.57%  (p=0.000 n=25+30)

This is the result of merging two defers on the standard cgo calling path.

Combined with Austin's improvements to defer's overall performance (CL 29656), the 146ns call to an empty C function in Go 1.7 is now a 56ns call in Go 1.8

.link https://crawshaw.io/blog/go1.8-cgo Less cgo overhead in Go 1.8

* Miscellaneous performance improvements

Lots of small updates to the standard library.

There have been optimizations to implementations in the `bytes`, `crypto/aes`, `crypto/cipher`, `crypto/elliptic`, `crypto/sha256`, `crypto/sha512`, `encoding/asn1`, `encoding/csv`, `encoding/hex`, `encoding/json`, `hash/crc32`, `image/color`, `image/draw`, `math`, `math/big`, `reflect`, `regexp`, `runtime`, `strconv`, `strings`, `syscall`, `text/template`, and `unicode/utf8` packages.

.link http://beta.golang.org/doc/go1.8#library Go 1.8 Standard Library Changes
.link https://www.reddit.com/r/golang/comments/58b4ep/go_18_vs_go_17_performance_improvements/ Reddit: bechmarks 1.7vs1.8, 1.6vs1.8, 1.5vs.1.8

* Go 1.7 vs Go 1.8

.image img/17vs18.png 570 _

* Go 1.6 vs Go 1.8

.image img/16vs18.png 570 _

* Go 1.5 vs Go 1.8

.image img/15vs18.png 570 _


Compiler changes

* SSA backend

- All backends updated to use the new SSA form. Old "plan9" style compiler backend deleted.
- Big performance improvements for 32-bit ARM.
- Easy to add new SSA backends (for values of easy compiler engineers work in). eg, adding MIPS backend was straight forward, SPARC and RISC-V are rumored to be next.

.link https://youtu.be/If8ompI0faY?t=224 Kiev Go MeetUp September 2016
.link https://go-talks.appspot.com/github.com/dAdAbird/presentations/kiev.meetup.sep.2016/ssa.slide#1 Slides (And slight dip into new backend for the Go compiler)

* New parser

Robert Griesemer and Matthew Dempsky replaced the parser.

The new parser removes many of the package level variables which came along for the ride from the previous YACC based parser.

Removing package level globals is a stepping stone to making the compiler more parallel. In fact this work has started in the development branch.

This new parser is considerably faster, but in 1.8 is hamstrung by having to convert between the new style AST and the old version the rest of the compiler expects.

Expect this AST conversion to be removed in Go 1.9.

* Compiler speed

About a 12-15% improvement compared to Go 1.7

.image https://dave.cheney.net/wp-content/uploads/2016/11/Screenshot-2016-11-19-at-08.57.10.png

* Ports

The number of platforms Go supports, on balance, continues to grow.

- Go now supports 32-bit MIPS on Linux for both big-endian (linux/mips) and little-endian machines (linux/mipsle) that implement the MIPS32r1 instruction set with FPU or kernel FPU emulation. Note that many common MIPS-based routers lack an FPU and have firmware that doesn't enable kernel FPU emulation; Go won't run on such machines.
- On DragonFly BSD, Go now requires DragonFly 4.4.4 or later.
- On OpenBSD, Go now requires OpenBSD 5.9 or later.
- The Plan 9 port's networking support is now much more complete and matches the behavior of Unix and Windows with respect to deadlines and cancelation.
- Go 1.8 now only supports OS X 10.8 or later. _This_is_likely_the_last_Go_release_to_support_10.8_. Compiling Go or running binaries on older OS X versions is untested.
- No ARMv5 builders means ARMv5 support will be dropped in Go 1.9.

* Tool changes

* go tool yacc

The yacc tool (previously available by running “go tool yacc”) has been removed. As of Go 1.7 it was no longer used by the Go compiler.

It has moved to the “tools” repository and is now available at golang.org/x/tools/cmd/goyacc.

* go tool asm

`go`tool`asm` now supports a flotilla of Intel and PPC vector instructions.

For 64-bit x86 systems, the following instructions have been added: `VBROADCASTSD`, `BROADCASTSS`, `MOVDDUP`, `MOVSHDUP`, `MOVSLDUP`, `VMOVDDUP`, `VMOVSHDUP`, and `VMOVSLDUP`.

For 64-bit PPC systems, the common vector scalar instructions have been added: `LXS`, `LXSDX`, `LXSI`, `LXSIWAX`, `LXSIWZX`, `LXV`, `LXVD2X`, `LXVDSX`, `LXVW4X`, `MFVSR`, `MFVSRD`, `MFVSRWZ`, `MTVSR`, `MTVSRD`, `MTVSRWA`, `MTVSRWZ`, `STXS`, `STXSDX`, `STXSI`, `STXSIWX`, `STXV`, `STXVD2X`, `STXVW4X`, `XSCV`, `XSCVDPSP`, `XSCVDPSPN`, `XSCVDPSXDS`, `XSCVDPSXWS`, `XSCVDPUXDS`, `XSCVDPUXWS`, `XSCVSPDP`, `XSCVSPDPN`, `XSCVSXDDP`, `XSCVSXDSP`, `XSCVUXDDP`, `XSCVUXDSP`, `XSCVX`, `XSCVXP`, `XVCV`, `XVCVDPSP`, `XVCVDPSXDS`, `XVCVDPSXWS`, `XVCVDPUXDS`, `XVCVDPUXWS`, `XVCVSPDP`, `XVCVSPSXDS`, `XVCVSPSXWS`, `XVCVSPUXDS`, `XVCVSPUXWS`, `XVCVSXDDP`, `XVCVSXDSP`, `XVCVSXWDP`, `XVCVSXWSP`, `XVCVUXDDP`, `XVCVUXDSP`, `XVCVUXWDP`, `XVCVUXWSP`, `XVCVX`, `XVCVXP`, `XXLAND`, `XXLANDC`, `XXLANDQ`, `XXLEQV`, `XXLNAND`, `XXLNOR`, `XXLOR`, `XXLORC`, `XXLORQ`, `XXLXOR`, `XXMRG`, `XXMRGHW`, `XXMRGLW`, `XXPERM`, `XXPERMDI`, `XXSEL`, `XXSI`, `XXSLDWI`, `XXSPLT`, and `XXSPLTW`.

* go tool trace

The `trace` tool has a new `-pprof` flag for producing pprof-compatible blocking and latency profiles from an execution trace.

Garbage collection events are now shown more clearly in the execution trace viewer.

Garbage collection activity is shown on its own row and GC helper goroutines are annotated with their roles.

.link https://www.youtube.com/watch?v=mmqDlbWk_XA Rhys Hiltner's execution tracer presentation (dotGo 2016)

* go tool vet

Vet is stricter in some ways and looser where it previously caused false positives. Vet now checks for:

- copying an array of locks
- duplicate JSON and XML struct field tags
- non-space-separated struct tags
- deferred calls to HTTP `Response.Body.Close` before checking errors
- indexed arguments in `Printf`.

.code examples/vet_repeated_json_tags.go

.play examples/govet.sh /go tool/

* go fix

`go`fix` now rewrites references to `golang.org/x/net/context` to the standard library provided `context` package.

* go bug

The new `go`bug` command starts a bug report on GitHub, prefilled with information about the current system.

Example:

.play examples/gobug.sh /go bug/

* Default $GOPATH

Lastly, it is no longer necessary to set `$GOPATH` before using the `go` command.

When `GOPATH` is not defined, the tool will use:

- `$HOME/go` on Unix
- `%USERPROFILE%\go` on Windows

If you don't like these defaults, just set `$GOPATH` to your preferred location.

* Runtime changes

* Mutex contention profile

Peter Wienberger has added a new pprof profile, mutex contention.

.code examples/mutex_test.go /testing.B/,/^}/


_note_: Due to an oversight this only works with `sync.Mutex`, not `sync.RWMutex`, this oversight will be addressed early in Go 1.9.

.link https://github.com/golang/go/issues/18496 runtime: contended mutex profiling doesn't work for sync.RWMutex

* Mutex contention profile

    % go test examples/mutex_test.go -bench=. -mutexprofile=mutex.out
    % go tool pprof mutex.test mutex.out
    (pprof) list
    Total: 290.81ms
    ROUTINE ======================== command-line-arguments.(*Map).Put in /Users/dfc/devel/go-1.8-release-party/examples/mutex_test.go
         0   290.81ms (flat, cum)   100% of Total
         .          .      9:}
         .          .     10:
         .          .     11:func (m *Map) Put(key string, val int) {
         .          .     12:   m.Lock()
         .          .     13:   m.m[key] = val
         .   290.81ms     14:   m.Unlock()
         .          .     15:}
         .          .     16:
         .          .     17:func BenchmarkMutex(b *testing.B) {
         .          .     18:   const N = 16
         .          .     19:   for n := 0; n < b.N; n++ {

* Plugins

Go now supports a “plugin” build mode for generating plugins written in Go, and a new plugin package for loading such plugins at run time.

Plugin support is currently only available on Linux.

* Plugin Example

.code examples/plugins/plugin.go

Then to build plugin.so

 $ go build -buildmode=plugin plugin.go

* Plugin Example - Using Plugin
.code examples/plugins/main.go  /START OMIT/,/END OMIT/

 $ go run main.go
 main started...
 Plugin loading
 Hello, number 7

* Plugins demo

Demo video: [[https://twitter.com/francesc/status/827851085943566336][twitter.com/francesc]]

Source code: [[https://github.com/campoy/golang-plugins][github.com/campoy/golang-plugins]]

* Struct conversion

When explicitly converting a value from one struct type to another, as of Go 1.8 the tags are ignored.

.play examples/struct_conversion.go /START OMIT/,/END OMIT/

* os.Executable

`os.Executable` returns an absolute path to the currently running program.

.play examples/os_executable.go /START OMIT/,/END OMIT/

Executable returns the path name for the executable that started the current process. There is no guarantee that the path is still pointing to the correct executable.

If a symlink was used to start the process, depending on the operating system, the result might be the symlink or the path it pointed to. If a stable result is needed, path/filepath.EvalSymlinks might help.

.link http://beta.golang.org/pkg/os/#Executable os.Executable (godoc.org)

* Detection of concurrent map accesses

In Go 1.6, the runtime added lightweight, [[http://talks.godoc.org/github.com/davecheney/gosyd/go1.6.slide#18][best-effort detection of concurrent misuse of maps]]. Go 1.8 improves that detector with support for detecting programs that concurrently write to and iterate over a map.

.play examples/maps.go /func main/,/^}/

If the runtime detects this condition, it prints a diagnosis and crashes the program.

* Changes to the standard library

* sort.Slice

The `sort` package has a new convenice method `sort.Slice`

.code examples/sort.go /switch.+{/,/^\t}/

* HTTP shutdown

A long requested feature, graceful shutdown of a `http.Server` was added in Go 1.8.

Call `Shutdown` when a signal is received:

.code examples/shutdown.go /main\(\) {/,/^}

* HTTP shutdown demo

Source code: [[https://github.com/8tomat8/ShutdownDemo][github.com/8tomat8/ShutdownDemo]]

# * Server.Shutdown

# .code examples/server.go

* HTTP/2

`http.Response` now satisfies the `http.Pusher` interface.

    type Pusher interface {
        Push(target string, opts *PushOptions) error
    }

This can be useful when the server knows the client will need to have those responses available in order to fully process the response to the original request.
.link https://tools.ietf.org/html/rfc7540#section-8.2 RFC7540 (HTTP/2)
.link https://habrahabr.ru/post/304422/ Habrahabr: Реализуем http/2 server push с помощью nghttp2

* Conclusion

.image img/party-gopher.png

Upgrade to Go 1.8, now!
Released February the 16th, 2017

It's literally the best version of Go, _ever_.

* Link on presentation

.image img/qr_presentation.png
.link https://github.com/GolangUA/go-1.8-release-party