Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: elcritch/nesper
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.4.0
Choose a base ref
...
head repository: elcritch/nesper
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref

Commits on Oct 28, 2020

  1. Copy the full SHA
    09182ac View commit details
  2. Copy the full SHA
    5dff27a View commit details
  3. Copy the full SHA
    7fa4a03 View commit details
  4. Copy the full SHA
    7ac8f59 View commit details
  5. Update README.md

    Cleaning up examples and changes
    elcritch authored Oct 28, 2020
    Copy the full SHA
    5d02116 View commit details
  6. Update README.md

    Adding GPIO short example
    elcritch authored Oct 28, 2020
    Copy the full SHA
    93bf8ce View commit details
  7. Update README.md

    Adding SPI examples.
    elcritch authored Oct 28, 2020
    Copy the full SHA
    ac62226 View commit details
  8. Update README.md

    elcritch authored Oct 28, 2020
    Copy the full SHA
    e68d94b View commit details
  9. Update README.md

    Add example code... everyone likes examples.
    elcritch authored Oct 28, 2020
    Copy the full SHA
    bbde014 View commit details
  10. Update README.md

    elcritch authored Oct 28, 2020
    Copy the full SHA
    9737f47 View commit details
  11. reworking time apis a bit

    elcritch committed Oct 28, 2020
    Copy the full SHA
    31dc89c View commit details
  12. reworking time apis a bit

    elcritch committed Oct 28, 2020
    Copy the full SHA
    46a1f39 View commit details
  13. reworking time apis a bit

    elcritch committed Oct 28, 2020
    Copy the full SHA
    f0e08b9 View commit details
  14. reworking time apis a bit

    elcritch committed Oct 28, 2020
    Copy the full SHA
    1f56dca View commit details
  15. reworking time apis a bit

    elcritch committed Oct 28, 2020
    Copy the full SHA
    d87c52d View commit details
  16. reworking time apis a bit

    elcritch committed Oct 28, 2020
    Copy the full SHA
    671612c View commit details

Commits on Oct 29, 2020

  1. adding BasicTimer's

    elcritch committed Oct 29, 2020
    Copy the full SHA
    53efb4c View commit details
  2. adding basic timer elapsed

    elcritch committed Oct 29, 2020
    Copy the full SHA
    0d4217b View commit details
  3. tweaking json rpc tests

    elcritch committed Oct 29, 2020
    Copy the full SHA
    73a6d32 View commit details

Commits on Nov 3, 2020

  1. Copy the full SHA
    7c97757 View commit details
  2. Copy the full SHA
    1749331 View commit details
  3. Copy the full SHA
    58382a9 View commit details
  4. Copy the full SHA
    f8bb9ed View commit details

Commits on Nov 4, 2020

  1. adding multi gpio set

    elcritch committed Nov 4, 2020
    Copy the full SHA
    c39187a View commit details
  2. adding multi gpio set

    elcritch committed Nov 4, 2020
    Copy the full SHA
    fd2bb31 View commit details
  3. adding multi gpio set

    elcritch committed Nov 4, 2020
    Copy the full SHA
    ddb2e2a View commit details
  4. adding multi gpio set

    elcritch committed Nov 4, 2020
    Copy the full SHA
    159ce82 View commit details
  5. Copy the full SHA
    a1dc01d View commit details
  6. Copy the full SHA
    9c14d8d View commit details
  7. Copy the full SHA
    981467f View commit details
  8. Copy the full SHA
    01dff7d View commit details
  9. Copy the full SHA
    2583153 View commit details
  10. Copy the full SHA
    17199be View commit details
  11. Copy the full SHA
    9584f2c View commit details
  12. Copy the full SHA
    d57c9bf View commit details
  13. Copy the full SHA
    be1d435 View commit details
  14. Copy the full SHA
    457222a View commit details
  15. fix reversion

    elcritch committed Nov 4, 2020
    Copy the full SHA
    ddecd38 View commit details
  16. Copy the full SHA
    aac1c36 View commit details
  17. Copy the full SHA
    ac1da89 View commit details
  18. Copy the full SHA
    c6ef260 View commit details
  19. Copy the full SHA
    0747fd1 View commit details
  20. Copy the full SHA
    22817d2 View commit details
  21. Copy the full SHA
    c2e6306 View commit details

Commits on Nov 5, 2020

  1. comment out buffer

    elcritch committed Nov 5, 2020
    Copy the full SHA
    f81550e View commit details
  2. Copy the full SHA
    2648af1 View commit details
  3. cleaning up and reorg nvs

    elcritch committed Nov 5, 2020
    Copy the full SHA
    9b49881 View commit details
  4. Copy the full SHA
    8741f6d View commit details
  5. Copy the full SHA
    ac4f450 View commit details
  6. Copy the full SHA
    e7d800b View commit details
Showing 316 changed files with 16,548 additions and 3,222 deletions.
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.PHONY: tests

tests:
nimble test
259 changes: 212 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,109 @@
# nesper
# Nesper

Nim wrappers for ESP-IDF (ESP32). This library builds on the new FreeRTOS/LwIP API support in Nim.
Program the ESP32 using Nim! This library builds on the `esp-idf`. Nim now has support for FreeRTOS & LwIP. Combined with the new ARC garbage collector makes Nim an excellent language for programming the ESP32.

## Updates
See [Releases](https://github.com/elcritch/nesper/releases) for updates.

## Status

**Version 0.2.0**
- Cleaned up the SPI api, implemented a GPIO api's.
- Tested GPIO on board, working; tested SPI init, still testing other SPI features
- Fixed compile issues in ethernet libraries
- Fixed c imports for task and queues requiring FreeRTOS.h to be included first
Note: This might require future follow ups as it essentially compiles to a absolute path
- Fixed issues with using second with xTaskCreatePinned and rpc queue mpack passing arguments
This project is fairly stable and even being used in shipping hardware project. The documentation is rough and primarily only includes this README. As such, it may still require understanding the underlying ESP-IDF SDK for various use cases. However, it is useable and several people unfamiliar with ESP-IDF and embedded programming have added wrappers for esp-idf modules.

**Version 0.2.0**
- Wrote compilation tests for several important modules
- My internal esp32 project has been switched over to using Nesper
- It's possible to write a full Nim only esp32 app
- There are now Nim wrappers for esp_wifi, tcpip_adapter, event_groups, tasks, and more esp-idf modules
- More Nim friendly API's have been added for NVS, SPI (untested), I2C (untested), wifi, & events
Note: It's recommended to use the ESP-IDF.py v4.0 branch (as of 2020-11-24). Branch v4.1 has multiple serious bugs in I2C.

## General Usage

1. [Install ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/index.html#get-started-get-esp-idf)
+ TLDR: `git clone -b release/v4.0 --recursive https://github.com/espressif/esp-idf.git`
+ esp-idf version 4.0 is recommended for now since its more stable
+ esp-idf version can be set using the defines: `-d:ESP_IDF_V4_0` or `-d:ESP_IDF_V4_1`
2. Install [Nim 1.4+](https://nim-lang.org/install.html)
3. Use [Nimble](https://github.com/nim-lang/nimble#nimble-usage) to install Nesper (`nimble install https://github.com/elcritch/nesper` or for the devel branch `nimble install 'https://github.com/elcritch/nesper@#devel' `)
4. Create a new Nimble project `nimble init --git esp32_nim_example`
5. In the new project directory edit the Nimble file and add the lines:
```nim
requires "nesper >= 0.6.1"
# includes nimble tasks for building Nim esp-idf projects
include nesper/build_utils/tasks
````
+ Make sure _not_ to include a `bin` option like `bin = @["src/esp32_nim_example"]` as this will override the `nimble esp_build` and result in a broken idf.py build.
6. Run `nimble esp_setup` to setup the correct files for building an esp32/esp-idf project
### Compiling and Building
1. Run `nimble esp_build` to build the esp-idf project
2. Flash and monitor the esp32 board using: `idf.py -p </dev/ttyUSB0> flash monitor`
Notes:
- Running `nimble esp_build` will both compile the Nim code and then build the esp-idf project
- During development it's often handy just to run `nimble esp_compile` to check your Nim code works
- Sometimes the Nim build cache gets out of sync, use `nimble esp_build --clean` to force a full Nim recompile
- Sometimes the esp-idf build cache gets out of sync, use `nimble esp_build --dist-clean` to force a full Nim recompile
## Example Code
This code shows a short example of setting up an http server to toggle a GPIO pin. It uses the default async HTTP server in Nim's standard library. It still requires the code to initialize the ESP32 and WiFi or ethernet.
```nim
import asynchttpserver, asyncdispatch, net
import nesper, nesper/consts, nesper/general, nesper/gpios
const
MY_PIN_A* = gpio_num_t(4)
MY_PIN_B* = gpio_num_t(5)
var
level = false
proc config_pins() =
MOTOR1_PIN.setLevel(true)
proc http_cb*(req: Request) {.async.} =
level = not level
echo "toggle my pin to: #", $level
MY_PIN_A.setLevel(level)
await req.respond(Http200, "Toggle MY_PIN_A: " & $level)
proc run_http_server*() {.exportc.} =
echo "Configure pins"
{MY_PIN_A, MY_PIN_B}.configure(MODE_OUTPUT)
MY_PIN_A.setLevel(lastLevel)
echo "Starting http server on port 8181"
var server = newAsyncHttpServer()
waitFor server.serve(Port(8181), http_cb)
**Version 0.1.0**
- Initial framework layout
```

## Status
## Why

TLDR; Real reason? It's a bit of fun in a sometimes tricky field.

This is a Work in Progress (tm). I'm using it for a few work projects as I generally dislike programming C/C++ (despite C's elegance in the small). When you just want a hash table in C it's tedious and error prone. C++ is about 5 different languages and I have to idea how to use half of them anymore. Rust doesn't work on half of the boards I want to program. MicroPython? ... Nope - I need speed and efficiency.
I generally dislike programming C/C++ (despite C's elegance in the small). When you just want a hash table in C it's tedious and error prone. C++ is about 5 different languages and I have to idea how to use half of them anymore. Rust doesn't work on half of the boards I want to program. MicroPython? ... Nope - I need speed and efficiency.

## Library

The library is currently a collection of random ESP-IDF libraries that I import using `c2nim` as needed. Sometimes there's a bit extra wrapping to provide a nicer Nim API.

Caveat: these features are tested as they're used for my use case. However, both Nim and the esp-idf seem designed well enough that they mostly "just work". PR's are welcome!

Supported ESP-IDF libraries:
Supported ESP-IDF drivers with Nim'ified interfaces:

- [x] Nim stdandard library support for most basic POSIX network api's!
- [x] Nim stdandard library support for most basic POSIX network API's!
- [x] Most of the basic `FreeRTOS.h` header
- [x] NVS Flash
- [x] NVS Flash
- [x] UART
- [x] SPI
- [x] SPI (don't mix half-duplex & duplex devices)
- [x] I2C

Things I'd like to get to:
Other things:

- [ ] Nim standard library wrapping of FreeRTOS semaphore's, mutexes, etc
- [ ] Nim support for `xqueue` and other "thread safe" data structures
- [ ] Nim standard library support for FreeRTOS tasks using thread api's
- [x] Nim standard library wrapping of FreeRTOS semaphore's, mutexes, etc
- include `pthread` in your CMakeLists.txt file and use Nim's POSIX lock API's
- [x] Nim support for `xqueue` and other "thread safe" data structures
- Raw C Wrappers exist, see `[rpcsocket_queue_mpack.nim](https://github.com/elcritch/nesper/blob/master/src/nesper/servers/rpc/rpcsocket_queue_mpack.nim) for proper usage. Nim Channel's appear to work as well.
- [x] Nim standard library support for FreeRTOS tasks using thread api's
- include `pthread` in your CMakeLists.txt file and use Nim's POSIX Pthread API's

Things I'm not planning on (PR's welcome!)

@@ -58,38 +112,149 @@ Things I'm not planning on (PR's welcome!)
- [ ] LCDs
- [ ] Built-in ADC

## General Usage
### Manual Setup

This is the more manual setup approach:

1. Install Nim 1.4+ (currently only the `devel` branch works) with `asdf` or `choosenim`
2. nimble install https://github.com/elcritch/nesper
3. Nesper wrapper API names generally match the C names directly, usually in snake case
3. It's recommend to copy `nesper/esp-idf-examples/simplewifi` example project initially, to get the proper build steps.
- `git clone https://github.com/elcritch/nesper`
- `cp -Rv nesper/esp-idf-examples/simplewifi/ ./nesper-simplewifi`
- `cd ./nesper-simplewifi/`
- `make build` (also `make esp_v40` or `make esp_v41` )
5. Nesper wrapper API names generally match the C names directly, usually in snake case
+ FreeRTOS functions usually are camel case and start with an `x`, e.g. `xTaskDelay`
+ These api's are found under `nesper/esp/*` or `nesper/esp/net/*`, e.g. `nesper/esp/nvs`
4. Nesper Nim friendly api, usually in camel case
6. Nesper Nim friendly api, usually in camel case
+ These api's are found under `nesper/*`, e.g. `nesper/nvs`


## Example Async server on a ESP32-CAM (or other Esp32 Wifi board)

Copy one the example wifi:
See [SimpleWifi Example](https://github.com/elcritch/nesper/blob/master/esp-idf-examples/simplewifi/README.md)

The async code really is simple Nim code:

```nim
import asynchttpserver, asyncdispatch, net
var count = 0
```shell
git clone https://github.com/elcritch/nesper
cd nesper/esp-idf-examples/simplewifi/
export WIFI_SSID=[ssid]
export WIFI_PASSWORD=[password]
nim prepare ./main/wifi_example_main.nim
idf.py reconfigure # sometimes required if new Nim C files are generated
proc cb*(req: Request) {.async.} =
inc count
echo "req #", count
await req.respond(Http200, "Hello World from nim on ESP32\n")
# GC_fullCollect()
proc run_http_server*() {.exportc.} =
echo "starting http server on port 8181"
var server = newAsyncHttpServer()
waitFor server.serve(Port(8181), cb)
when isMainModule:
echo "running server"
run_http_server()
```

Then do the standard idf build and flash steps:
## Nim-ified ESP32 APIs

### GPIOs

```nim
import nesper, nesper/consts, nesper/general, nesper/gpios
const
MOTOR1_PIN* = gpio_num_t(4)
MOTOR2_PIN* = gpio_num_t(5)
proc config_pins() =
# Inputs pins use Nim's set `{}` notation
configure({MOTOR1_PIN, MOTOR2_PIN}, GPIO_MODE_INPUT)
# or method call style:
{MOTOR1_PIN, MOTOR2_PIN}.configure(MODE_INPUT)
MOTOR1_PIN.setLevel(true)
MOTOR2_PIN.setLevel(false)
```

```shell
idf.py build
idf.py -p [port] flash
idf.py -p [port] monitor
### SPIs

```nim
import nesper, nesper/consts, nesper/general, nesper/spis
proc cs_adc_pre(trans: ptr spi_transaction_t) {.cdecl.} = ...
proc cs_unselect(trans: ptr spi_transaction_t) {.cdecl.} = ...
proc config_spis() =
# Setup SPI example using custom Chip select pins using pre/post callbacks
let
std_hz = 1_000_000.cint()
fast_hz = 8_000_000.cint()
var BUS1 = HSPI.newSpiBus(
mosi = gpio_num_t(32),
sclk = gpio_num_t(33),
miso = gpio_num_t(34),
dma_channel=0,
flags={MASTER})
logi(TAG, "cfg_spi: bus1: %s", repr(BUS1))
var ADC_SPI = BUS1.addDevice(commandlen = bits(8),
addresslen = bits(0),
mode = 0,
cs_io = gpio_num_t(-1),
clock_speed_hz = fast_hz,
queue_size = 1,
pre_cb=cs_adc_pre,
post_cb=cs_unselect,
flags={HALFDUPLEX})
```

Later these can be used like:

```nim
const
ADC_READ_MULTI_CMD = 0x80
ADC_REG_CONFIG0 = 0x03
proc read_regs*(reg: byte, n: range[1..16]): SpiTrans =
let read_cmd = reg or ADC_READ_MULTI_CMD # does bitwise or
return ADC_SPI.readTrans(cmd=read_cmd, rxlength=bytes(n), )
proc adc_read_config*(): seq[byte] =
var trn = read_regs(ADC_REG_CONFIG0, 2)
trn.transmit() # preforms SPI transaction using transaction queue
result = trn.getData()
```

See more in the test [SPI Test](https://github.com/elcritch/nesper/blob/master/tests/tspi.nim) or the read the wrapper (probably best docs for now): [spis.nim](https://github.com/elcritch/nesper/blob/master/src/nesper/spis.nim).

## Wear levelling / Virtual FAT filesystem

```nim
import nesper, nesper/esp_vfs_fat
var
base_path : cstring = "/spiflash"
s_wl_handle : wl_handle_t = WL_INVALID_HANDLE
mount_config = esp_vfs_fat_mount_config_t(format_if_mount_failed: true,
max_files: 10, allocation_unit_size: 4096)
err = esp_vfs_fat_spiflash_mount(base_path, "storage", mount_config.addr, s_wl_handle.addr)
if err != ESP_OK:
echo "Failed to mount FATFS."
else:
echo "FATFS mounted successfully!"
writeFile("/spiflash/hello.txt", "Hello world!")
echo readFile("/spiflash/hello.txt") # Hello world!
```

Notice: file extension of files on FAT filesystem is limited to maximum of 3 characters.

## Why Nim for Embedded?

Nim is a flexible language which compiles to a variety of backend "host" languages, including C and C++. Like many hosted languages, it has excellent facilities to interact with the host language natively. In the embedded world this means full compatability with pre-existing libraries and toolchains, which are often complex and difficult to interface with from an "external language" like Rust or even C++. They often also require oddball compilers, ruling out LLVM based lanugages for many projects (including the ESP32 which defaults to a variant of GCC).
4 changes: 4 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
from espressif/idf:release-v4.0
RUN wget https://nim-lang.org/download/nim-1.4.0-linux_x64.tar.xz
RUN tar xvf nim-1.4.0-linux_x64.tar.xz
ENV PATH="/nim-1.4.0/bin:${PATH}"
4 changes: 4 additions & 0 deletions docker/build_docker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash

docker image rm -f esp_nim
docker build -t esp_nim ./
4 changes: 4 additions & 0 deletions docker/run_shell.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash
pushd ../
docker run --rm -v $PWD:/project --device=/dev/ttyUSB0 -w /project -it esp_nim
popd
8 changes: 8 additions & 0 deletions esp-idf-examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@

The two examples `simplewifi` and `simplewifi-rpc` are currently the only examples that have been tested on hardware.

The `simplewifi` example is a simple method showing how to use async servers with Nim. However, as of Nim 1.4, there are still some challenges running async with ORC on small devices. This should get better over time and may work depending on your paramters.

For production usage, it's recommended to use one of the `simplewifi-rpc` methods. Using socket `select` allows efficiently servicing RPC calls without requiring async. However, this means only *one* RPC call can be handled at a time. This could be worked around by spawning more tasks or sockets, but is sufficient for many cases.


7 changes: 7 additions & 0 deletions esp-idf-examples/app_test/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
main/
*.txt
*.nim
*.cfg
*.nims
build/
sdkconfig
13 changes: 13 additions & 0 deletions esp-idf-examples/app_test/app_test.nimble
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Package

version = "0.1.0"
author = "Author Name"
description = "Nesper example"
license = "none"
srcDir = "src"

# Dependencies
requires "nim >= 1.4.0"
requires "nesper >= 0.5.0"
# includes nimble tasks for building Nim esp-idf projects
include nesper/build_utils/tasks
6 changes: 6 additions & 0 deletions esp-idf-examples/app_test/clean.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/sh

rm -Rf *.txt ./main/
rm -Rf ./src/

rm -Rf build sdkconfig
8 changes: 8 additions & 0 deletions esp-idf-examples/simple_rpc/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
build/
src/nimcache/
src/server
src/localnim/
.env
results.json
main/nimble/
rpc_cli
19 changes: 19 additions & 0 deletions esp-idf-examples/simple_rpc/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# The following lines of boilerplate have to be in your project's
# CMakeLists in this exact order for cmake to work correctly
cmake_minimum_required(VERSION 3.5)

# (Not part of the boilerplate)

set(EXTRA_COMPONENT_DIRS $ENV{IDF_PATH}/examples/common_components/protocol_examples_common)
# set(EXTRA_COMPONENT_DIRS src)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(simple_rpc)

idf_build_set_property(C_COMPILE_OPTIONS -Wno-error=unused-label APPEND)
idf_build_set_property(C_COMPILE_OPTIONS -Wno-error=parentheses APPEND)
idf_build_set_property(C_COMPILE_OPTIONS -Wno-error=implicit-function-declaration APPEND)
idf_build_set_property(C_COMPILE_OPTIONS -Wno-error=maybe-uninitialized APPEND)
idf_build_set_property(C_COMPILE_OPTIONS -Wno-error=nonnull APPEND)
idf_build_set_property(C_COMPILE_OPTIONS -Wno-error=address APPEND)
idf_build_set_property(C_COMPILE_OPTIONS -Wno-unused-but-set-variable APPEND)
Loading