Skip to content

Commit

Permalink
Add benchmarks with BuntDB.
Browse files Browse the repository at this point in the history
Adds benchmark results to README and docs as well.

Signed-off-by: Vaibhav <vrongmeal@gmail.com>
  • Loading branch information
vrongmeal authored and murex971 committed Nov 5, 2020
1 parent 05af4df commit c852242
Show file tree
Hide file tree
Showing 49 changed files with 9,988 additions and 41 deletions.
33 changes: 33 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,39 @@ fmt.Println(str) // Hello, World!
Check out the [tutorial](https://kiwi.sdslabs.co/docs/tutorial-store.html) to
learn how to use Kiwi.

## Benchmarks

Following are the benchmarks comparing Kiwi with BuntDB on a MacBook Pro (8th gen
Intel i5 2.4GHz processor, 8GB RAM).

```
❯ go test -bench=. -test.benchmem ./benchmark
goos: darwin
goarch: amd64
pkg: github.com/sdslabs/kiwi/benchmark
BenchmarkBuntDB_Update-8 11777931 96.6 ns/op 48 B/op 1 allocs/op
BenchmarkBuntDB_View-8 23310963 47.1 ns/op 48 B/op 1 allocs/op
BenchmarkKiwi_Update-8 10356004 115 ns/op 48 B/op 3 allocs/op
BenchmarkKiwi_Get-8 21910110 53.2 ns/op 0 B/op 0 allocs/op
PASS
ok github.com/sdslabs/kiwi/benchmark 6.216s
```

Following are the key differences due to which Kiwi is a little slow:

1. BuntDB supports transactions, i.e., it locks the database once to apply all
the operations (and this is what is tested).
1. Kiwi supports dynamic data-types, which means, allocation on heap at runtime
(`interface{}`) whereas BuntDB is statically typed.

The above two differences are what makes Kiwi unique and suitable to use on
many occasions. Due to the aforementioned reasons, Kiwi can support typed values
and not everything is just another "string".

There are places where we could improve more. Some performance issues also lie
in the implementation of values. For example, when updating a string, not returning
the updated string avoids an extra allocation.

## Contributing

We are always open for contributions. If you find any feature missing, or just
Expand Down
55 changes: 55 additions & 0 deletions benchmark/buntdb_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// Copyright (c) 2020 SDSLabs
// Use of this source code is governed by an MIT license
// details of which can be found in the LICENSE file.

package benchmark

import (
"testing"

"github.com/tidwall/buntdb"
)

const (
buntdbTestKey = "buntdbTestKey"
buntdbTestVal = "buntdbTestVal"
)

func BenchmarkBuntDB_Update(b *testing.B) {
db, err := buntdb.Open(":memory:")
if err != nil {
b.Fatalf("couldn't open db: %v", err)
}
defer db.Close()

if err := db.Update(func(tx *buntdb.Tx) error {
for i := 0; i < b.N; i++ {
if _, _, err := tx.Set(buntdbTestKey, buntdbTestVal, nil); err != nil {
return err
}
}
return nil
}); err != nil {
b.Fatalf("cannot update: %v", err)
}
}

func BenchmarkBuntDB_View(b *testing.B) {
db, err := buntdb.Open(":memory:")
if err != nil {
b.Fatalf("couldn't open db: %v", err)
}
defer db.Close()

if err := db.View(func(tx *buntdb.Tx) error {
for i := 0; i < b.N; i++ {
_, err := tx.Get(buntdbTestKey)
if err != nil && err != buntdb.ErrNotFound {
return err
}
}
return nil
}); err != nil {
b.Fatalf("cannot update: %v", err)
}
}
51 changes: 51 additions & 0 deletions benchmark/kiwi_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Copyright (c) 2020 SDSLabs
// Use of this source code is governed by an MIT license
// details of which can be found in the LICENSE file.

package benchmark

import (
"testing"

"github.com/sdslabs/kiwi/stdkiwi"

"github.com/sdslabs/kiwi"
"github.com/sdslabs/kiwi/values/str"
)

const (
kiwiTestKey = "kiwiTestKey"
kiwiTestVal = "kiwiTestVal"
)

func BenchmarkKiwi_Update(b *testing.B) {
store, err := stdkiwi.NewStoreFromSchema(kiwi.Schema{
kiwiTestKey: str.Type,
})
if err != nil {
b.Fatalf("cannot create store: %v", err)
}
testStr := store.Str(kiwiTestKey)

for i := 0; i < b.N; i++ {
if err := testStr.Update(kiwiTestVal); err != nil {
b.Fatalf("couldn't update data: %v", err)
}
}
}

func BenchmarkKiwi_Get(b *testing.B) {
store, err := stdkiwi.NewStoreFromSchema(kiwi.Schema{
kiwiTestKey: str.Type,
})
if err != nil {
b.Fatalf("cannot create store: %v", err)
}
testStr := store.Str(kiwiTestKey)

for i := 0; i < b.N; i++ {
if _, err := testStr.Get(); err != nil {
b.Fatalf("couldn't get data: %v", err)
}
}
}
1 change: 1 addition & 0 deletions docs/.vuepress/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ module.exports = {
children: [
'',
'get-started',
'benchmarks',
]
},
{
Expand Down
32 changes: 32 additions & 0 deletions docs/docs/benchmarks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Benchmarks

Following are the benchmarks comparing Kiwi with BuntDB on a MacBook Pro (8th gen
Intel i5 2.4GHz processor, 8GB RAM).

```
❯ go test -bench=. -test.benchmem ./benchmark
goos: darwin
goarch: amd64
pkg: github.com/sdslabs/kiwi/benchmark
BenchmarkBuntDB_Update-8 11777931 96.6 ns/op 48 B/op 1 allocs/op
BenchmarkBuntDB_View-8 23310963 47.1 ns/op 48 B/op 1 allocs/op
BenchmarkKiwi_Update-8 10356004 115 ns/op 48 B/op 3 allocs/op
BenchmarkKiwi_Get-8 21910110 53.2 ns/op 0 B/op 0 allocs/op
PASS
ok github.com/sdslabs/kiwi/benchmark 6.216s
```

Following are the key differences due to which Kiwi is a little slow:

1. BuntDB supports transactions, i.e., it locks the database once to apply all
the operations (and this is what is tested).
1. Kiwi supports dynamic data-types, which means, allocation on heap at runtime
(`interface{}`) whereas BuntDB is statically typed.

The above two differences are what makes Kiwi unique and suitable to use on
many occasions. Due to the aforementioned reasons, Kiwi can support typed values
and not everything is just another "string".

There are places where we could improve more. Some performance issues also lie
in the implementation of values. For example, when updating a string, not returning
the updated string avoids an extra allocation.
6 changes: 5 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,8 @@ module github.com/sdslabs/kiwi

go 1.14

require github.com/wangjia184/sortedset v0.0.0-20200422044937-080872f546ba
require (
github.com/stretchr/testify v1.6.1 // indirect
github.com/tidwall/buntdb v1.1.4
github.com/wangjia184/sortedset v0.0.0-20200422044937-080872f546ba
)
27 changes: 27 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,2 +1,29 @@
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/tidwall/btree v0.2.2 h1:VVo0JW/tdidNdQzNsDR4wMbL3heaxA1DGleyzQ3/niY=
github.com/tidwall/btree v0.2.2/go.mod h1:huei1BkDWJ3/sLXmO+bsCNELL+Bp2Kks9OLyQFkzvA8=
github.com/tidwall/buntdb v1.1.4 h1:W7y9+2dM3GOswU0t3pz6+BcwZXjj/tVOhPcO6EHufME=
github.com/tidwall/buntdb v1.1.4/go.mod h1:06+/n7EFf6uUaIG5r9xZcExYN3H0Lnc+g/Kqx0fZFkI=
github.com/tidwall/gjson v1.6.1 h1:LRbvNuNuvAiISWg6gxLEFuCe72UKy5hDqhxW/8183ws=
github.com/tidwall/gjson v1.6.1/go.mod h1:BaHyNc5bjzYkPqgLq7mdVzeiRtULKULXLgZFKsxEHI0=
github.com/tidwall/grect v0.0.0-20161006141115-ba9a043346eb h1:5NSYaAdrnblKByzd7XByQEJVT8+9v0W/tIY0Oo4OwrE=
github.com/tidwall/grect v0.0.0-20161006141115-ba9a043346eb/go.mod h1:lKYYLFIr9OIgdgrtgkZ9zgRxRdvPYsExnYBsEAd8W5M=
github.com/tidwall/match v1.0.1 h1:PnKP62LPNxHKTwvHHZZzdOAOCtsJTjo6dZLCwpKm5xc=
github.com/tidwall/match v1.0.1/go.mod h1:LujAq0jyVjBy028G1WhWfIzbpQfMO8bBZ6Tyb0+pL9E=
github.com/tidwall/pretty v1.0.2 h1:Z7S3cePv9Jwm1KwS0513MRaoUe3S01WPbLNV40pwWZU=
github.com/tidwall/pretty v1.0.2/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
github.com/tidwall/rtree v0.0.0-20201027154624-32188eeb08a8 h1:BsKSRhu0TDB6Snq8SutN9KQHc6vqHEXJTcAFwyGNius=
github.com/tidwall/rtree v0.0.0-20201027154624-32188eeb08a8/go.mod h1:/h+UnNGt0IhNNJLkGikcdcJqm66zGD/uJGMRxK/9+Ao=
github.com/tidwall/tinyqueue v0.0.0-20180302190814-1e39f5511563 h1:Otn9S136ELckZ3KKDyCkxapfufrqDqwmGjcHfAyXRrE=
github.com/tidwall/tinyqueue v0.0.0-20180302190814-1e39f5511563/go.mod h1:mLqSmt7Dv/CNneF2wfcChfN1rvapyQr01LGKnKex0DQ=
github.com/wangjia184/sortedset v0.0.0-20200422044937-080872f546ba h1:jrGJzVnPfTOo7yb0qXeXc6biQFlDjzp4evucTpZZFTc=
github.com/wangjia184/sortedset v0.0.0-20200422044937-080872f546ba/go.mod h1:YkocrP2K2tcw938x9gCOmT5G5eCD6jsTz0SZuyAqwIE=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
40 changes: 0 additions & 40 deletions stdkiwi/benchmark_test.go

This file was deleted.

1 change: 1 addition & 0 deletions vendor/github.com/tidwall/btree/.travis.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 18 additions & 0 deletions vendor/github.com/tidwall/btree/LICENSE

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions vendor/github.com/tidwall/btree/README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit c852242

Please sign in to comment.