Skip to content

Commit

Permalink
Merge pull request #9 from TheThingsNetwork/feature/ttn-source
Browse files Browse the repository at this point in the history
Add TheThingsNetwork source
  • Loading branch information
Aggelos Kolaitis authored Nov 12, 2020
2 parents ef04f35 + 8580286 commit 3634534
Show file tree
Hide file tree
Showing 8 changed files with 497 additions and 0 deletions.
60 changes: 60 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Docker images are available on [Docker Hub](https://hub.docker.com/r/TheThingsNe

## Support

- [X] The Things Network Stack V2
- [X] [ChirpStack Network Server](https://www.chirpstack.io/)
- [ ] [LORIOT Network Server](https://www.loriot.io/)

Expand All @@ -33,6 +34,63 @@ Navigate to your application, click **Import End Devices**, select **The Things
$ ttn-lw-cli end-devices create --application-id test-app < devices.json
```

## The Things Network Stack V2

### Configuration

Configure with environment variables, or command-line arguments. See `--help` for more details:

```bash
$ export TTNV2_APP_ID="my-ttn-app" # TTN App ID
$ export TTNV2_APP_ACCESS_KEY="ttn-account-v2.a..." # TTN App Access Key (needs `devices` permissions)
$ export FREQUENCY_PLAN_ID="EU_863_870_TTN" # Frequency Plan for exported devices
```

See [Frequency Plans](https://thethingsstack.io/reference/frequency-plans/) for the list of frequency plans available on The Things Stack. For example, to use `United States 902-928 MHz, FSB 1`, you need to specify the `US_902_928_FSB_1` frequency plan ID.

Private The Things Network Stack V2 deployments are also supported, and require extra configuration. See `ttn-lw-migrate device --help` for more details. For example, to override the discovery server address:

```bash
$ export TTNV2_DISCOVERY_SERVER_ADDRESS="discovery.thethings.network:1900"
```

### Export Devices

To export a single device using its Device ID (e.g. `mydevice`):

```bash
$ ttn-lw-migrate device --source ttnv2 "mydevice" > devices.json
```

In order to export a large number of devices, create a file named `device_ids.txt` with one device ID per line:

```
mydevice
otherdevice
device3
device4
device5
```

And then export with:

```bash
$ ttn-lw-migrate device --source ttnv2 < device_ids.txt > devices.json
```

### Export Applications

Similarly, to export all devices of application `my-app-id`:

```bash
$ ttn-lw-migrate application --source ttnv2 "my-app-id" > devices.json
```

### Notes

- Payload formatters are not exported. See [Payload Formatters](https://thethingsstack.io/integrations/payload-formatters/).
- Active device sessions are exported by default. You can disable this by using the `--ttnv2.with-session=false` flag. It is recommended that you do not export session keys for devices that can instead re-join on The Things Stack.

## ChirpStack

### Configuration
Expand All @@ -46,6 +104,8 @@ $ export JOIN_EUI="0101010102020203" # JoinEUI for exported devices
$ export FREQUENCY_PLAN_ID="EU_863_870" # Frequency Plan for exported devices
```

See [Frequency Plans](https://thethingsstack.io/reference/frequency-plans/) for the list of frequency plans available on The Things Stack. For example, to use `United States 902-928 MHz, FSB 1`, you need to specify the `US_902_928_FSB_1` frequency plan ID.

> *NOTE*: `JoinEUI` and `FrequencyPlanID` are required because ChirpStack does not store these fields.
### Export Devices
Expand Down
1 change: 1 addition & 0 deletions cmd/ttn-lw-migrate/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import (
"os"

_ "go.thethings.network/lorawan-stack-migrate/pkg/source/chirpstack" // ChirpStack source
_ "go.thethings.network/lorawan-stack-migrate/pkg/source/ttnv2" // TTNv2 source

"go.thethings.network/lorawan-stack-migrate/cmd"
)
Expand Down
3 changes: 3 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ require (
github.com/Azure/azure-pipeline-go v0.2.3 // indirect
github.com/Azure/azure-storage-blob-go v0.10.0 // indirect
github.com/Azure/go-autorest/autorest/adal v0.9.2 // indirect
github.com/TheThingsNetwork/go-app-sdk v0.0.0-20191121100818-5bae20ae2b27
github.com/TheThingsNetwork/go-utils v0.0.0-20190516083235-bdd4967fab4e
github.com/TheThingsNetwork/ttn/utils/errors v0.0.0-20190516081709-034d40b328bd
github.com/aws/aws-sdk-go v1.34.9 // indirect
github.com/brocaar/chirpstack-api/go/v3 v3.7.5
github.com/envoyproxy/protoc-gen-validate v0.4.1 // indirect
Expand Down
49 changes: 49 additions & 0 deletions go.sum

Large diffs are not rendered by default.

135 changes: 135 additions & 0 deletions pkg/source/ttnv2/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
// Copyright © 2020 The Things Network Foundation, The Things Industries B.V.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package ttnv2

import (
"context"
"crypto/tls"
"crypto/x509"
"io/ioutil"
"os"

ttnsdk "github.com/TheThingsNetwork/go-app-sdk"
ttnlog "github.com/TheThingsNetwork/go-utils/log"
"github.com/TheThingsNetwork/go-utils/log/apex"
"github.com/spf13/pflag"
)

const (
clientName = "ttn-lw-migrate"
)

type config struct {
sdkConfig ttnsdk.ClientConfig

appAccessKey string
appID string

frequencyPlanID string

withSession bool
}

func flagSet() *pflag.FlagSet {
flags := &pflag.FlagSet{}
flags.String("ttnv2.frequency-plan-id", os.Getenv("FREQUENCY_PLAN_ID"), "Frequency Plan ID of exported devices")
flags.String("ttnv2.app-id", os.Getenv("TTNV2_APP_ID"), "TTN Application ID")
flags.String("ttnv2.app-access-key", os.Getenv("TTNV2_APP_ACCESS_KEY"), "TTN Application Access Key (with 'devices' permissions)")
flags.String("ttnv2.ca-cert", os.Getenv("TTNV2_CA_CERT"), "(only for private networks) CA for TLS")
flags.String("ttnv2.handler-address", os.Getenv("TTNV2_HANDLER_ADDRESS"), "(only for private networks) Address for the Handler")
flags.String("ttnv2.account-server-address", os.Getenv("TTNV2_ACCOUNT_SERVER_ADDRESS"), "(only for private networks) Address for the Account Server")
flags.String("ttnv2.account-server-client-id", os.Getenv("TTNV2_ACCOUNT_SERVER_CLIENT_ID"), "(only for private networks) Client ID for the Account Server")
flags.String("ttnv2.account-server-client-secret", os.Getenv("TTNV2_ACCOUNT_SERVER_CLIENT_SECRET"), "(only for private networks) Client secret for the Account Server")
flags.String("ttnv2.discovery-server-address", os.Getenv("TTNV2_DISCOVERY_SERVER_ADDRESS"), "(only for private networks) Address for the Discovery Server")
flags.Bool("ttnv2.discovery-server-insecure", false, "(only for private networks) Not recommended")
flags.Bool("ttnv2.with-session", true, "Export device session keys and frame counters")
return flags
}

func getConfig(ctx context.Context, flags *pflag.FlagSet) (config, error) {
stringFlag := func(f string) string {
s, _ := flags.GetString(f)
return s
}
boolFlag := func(f string) bool {
s, _ := flags.GetBool(f)
return s
}

cfg := ttnsdk.NewCommunityConfig(clientName)
if f := stringFlag("ttnv2.account-server-address"); f != "" {
cfg.AccountServerAddress = f
}
if f := stringFlag("ttnv2.account-server-client-id"); f != "" {
cfg.AccountServerClientID = f
}
if f := stringFlag("ttnv2.account-server-client-secret"); f != "" {
cfg.AccountServerClientSecret = f
}
if f := stringFlag("ttnv2.handler-address"); f != "" {
cfg.HandlerAddress = f
}
if f := stringFlag("ttnv2.discovery-server-address"); f != "" {
cfg.DiscoveryServerAddress = f
}
cfg.DiscoveryServerInsecure = boolFlag("ttnv2.discovery-server-insecure")

if ca := stringFlag("ttnv2.ca-cert"); ca != "" {
if cfg.TLSConfig == nil {
cfg.TLSConfig = &tls.Config{}
}
rootCAs := cfg.TLSConfig.RootCAs
if rootCAs == nil {
var err error
if rootCAs, err = x509.SystemCertPool(); err != nil {
rootCAs = x509.NewCertPool()
}
}
pemBytes, err := ioutil.ReadFile(ca)
if err != nil {
return config{}, errRead.WithAttributes("file", ca)
}
rootCAs.AppendCertsFromPEM(pemBytes)
}

appAccessKey := stringFlag("ttnv2.app-access-key")
if appAccessKey == "" {
return config{}, errNoAppAccessKey.New()
}
appID := stringFlag("ttnv2.app-id")
if appID == "" {
return config{}, errNoAppID.New()
}
frequencyPlanID := stringFlag("ttnv2.frequency-plan-id")
if frequencyPlanID == "" {
return config{}, errNoFrequencyPlanID.New()
}

logger := apex.Stdout()
if boolFlag("verbose") {
logger.MustParseLevel("debug")
}
ttnlog.Set(logger)

return config{
sdkConfig: cfg,

appID: appID,
appAccessKey: appAccessKey,
frequencyPlanID: frequencyPlanID,

withSession: boolFlag("ttnv2.with-session"),
}, nil
}
25 changes: 25 additions & 0 deletions pkg/source/ttnv2/errors.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// Copyright © 2020 The Things Network Foundation, The Things Industries B.V.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package ttnv2

import "go.thethings.network/lorawan-stack/v3/pkg/errors"

var (
errRead = errors.DefinePermissionDenied("read", "failed to read `{file}`")

errNoAppID = errors.DefineInvalidArgument("no_app_id", "no app id")
errNoAppAccessKey = errors.DefineInvalidArgument("no_app_access_key", "no app access key")
errNoFrequencyPlanID = errors.DefineInvalidArgument("no_frequency_plan_id", "no frequency plan id")
)
Loading

0 comments on commit 3634534

Please sign in to comment.