v0.26.0

It's been 4 months since the last release, with major changes across all parts of the language - from syntax to runtime implementation, including new features, stdlib components, and bug fixes.

Changes in Existing Features

def Keyword

You are now must use def keyword to define components.

def Main(start any) (stop any) {
    :start -> :stop
}

The word def is short and common (used in Python, Ruby, Clojure, Scala, Elixir, Nim, Crystal, Groovy) and it means "define," which has a broader meaning. The word flow led to confusion because the abstraction is called "component." But thanks to @ajzaff for the suggestion anyway

Deferred Connections

Due to the addition of binary and ternary expression senders, we had to reserve () and select a different syntax for deferred connections.

Before

:start -> (42 -> println)

After

:start -> { 42 -> println }

Struct Selectors

Structs are now their own type of sender, rather than a modifier for other senders.

Before

someStruct.someField -> println

After

someStruct -> .someField -> println

Why?

This allows for implementing struct selectors on the receiver side in connections with multiple receivers.

s -> [
    .field1 -> r1
    .field2 -> r2
]

Note that you don't have to write foo -> .bar -> .baz -> ...; you can instead write foo -> .bar.baz -> ...

New Language Features

Range Expressions

You are now allowed to use syntax sugar for using Range component

:start -> 1..100 -> ...

Range is a type of sender, so it might be used anywhere, where any other sender is expected (with the respect of type-safety, ofcourse)

Will emit stream<int> of messages from 1 to 99 (range is exclusive)

Binary Operators

You can now write binary expressions as you would in other languages.

(1 + 2) -> println

Please note that you must use parentheses, as Nevalang currently lacks operator precedence. For now, nested binary expressions look like this:

((1 + 2) * 3) -> println

Supported Operators

14 binary operators are implemented: 6 arithmetic, 6 comparison, and 2 logical.

+ - * / % **
== != > < >= <=
&& ||

How It Works

Syntax is simple:

(sender_1 OPERATOR sender_1)

sender_1 and sender_2 can be any valid senders: port addresses, constant references, message literals, other binary expressions, etc.

The compiler understands these expressions and desugars them into regular connections using stdlib components. You can check builtin/operators.neva to see their API. They all follow the same pattern:

def Op(left T1, right T2) (res T3)

Both operands must be compatible with their operator, or the compiler will throw an error.

A note on Reduce

The higher-order component Reduce has been modified, and the interface for the reducer now looks like this

interface IReduceHandler<T, Y>(left T, right T) (res Y)

As you may have noticed, many binary operators are actually reducers. Thanks to @ajzaff for this idea.

Related issues: #742 and #721

Ternary Operator

Similar to binary expressions, ternary expressions use the ? : operator, as in most languages.

(condition ? ifValue : elseValue) -> ...

All 3 parts are senders, so all sender types are supported, including other ternary and binary expressions. The condition sender must emit bool, and the If and Else parts must both be compatible with the receiver, otherwise the compiler will throw an error.

Similar to binary operators, the ternary operator is just syntactic sugar for using the Ternary component:

def Ternary<T>(if bool, then T, else T) (res T)

For both binary and ternary expressions, you may explicitly use operators as components if the expression-based syntax doesn't cover your case.

Switch statement

Another type of sender was added to simplify the use of the Switch component. This is necessary when you need to trigger different branches of the network based on the value of an incoming connection. The syntax is as follows:

s -> switch {
    c1 -> r1
    c1 -> r2
    c1 -> r3
    _ -> r4
}

Here s means sender, which could be any sender (including binary and ternary expressions). c1, c2, c3 are "case senders" - they are also senders, and any senders will work as long as they are type-safe. Finally, _ is the default sender. The default branch is required, making each switch expression exhaustive. The compiler ensures that the incoming s -> and all c and _ senders are compatible with their corresponding receiver parts.

If one branch is triggered, other branches will not be (until the next message, if the corresponding pattern fires) - one way to think about this is that every branch has a "break" (and there's no way to "fallthrough").

I don't like this explanation because it's control-flow centric, while Nevalang's switch is pure dataflow, but it makes sense as an analogy.

The switch statement is syntactic sugar for the Switch component:

def Switch<T>(data T, [case] T) ([case] T, else T)

You are allowed to use Switch as a component, if you need to, but prefer statement syntax if possible

To better understand the switch statement, let's look at a few examples:

// simple
sender -> switch {
    true -> receiver1
    false -> receiver2
    _ -> receiver3
}

// multiple senders, multuple receivers
sender -> switch {
    [a, b] -> [receiver1, receiver2]
    c -> [receiver3, receiver4]
    _ -> receiver5
}

// with binary expression senders
sender -> switch {
    (a + b) -> receiver1
    (c * d) -> receiver2
    _ -> receiver3
}

// nested
sender -> switch {
    true -> switch {
        1 -> receiver1
        2 -> receiver2
        _ -> receiver3
    }
    false -> receiver4
    _ -> receiver5
}

// as chained connection
sender -> .field -> switch {
    true -> receiver1
    false -> receiver2
    _ -> receiver3
}

Related to #725

Note on multuple senders/receivers

In this example

sender -> switch {
    [a, b] -> [receiver1, receiver2]
    c -> [receiver3, receiver4]
    _ -> receiver5
}

If the sender message is equal to either a or b, it will be sent to both receiver1 and receiver2. You can also have multiple senders and one receiver, or one sender and multiple receivers.

Note on Pattern Matching (ROADMAP)

switch is a "router," not a "selector." It can only redirect incoming messages to a branch based on a condition (equality comparison). But what if we want to select one of the possible options instead of redirecting the incoming message? For this, another component called match will be implemented. It will work similarly to switch but act as a selector rather than a router.

num -> match {
    42: 'a'
    43: 'b'
    _: 'c'
} -> println

Note the outgoing -> that switch does not have.

WARNING: match is NOT implemented yet.

Related to #726 and #747

Changes in Standard Library

Tap component

Sometimes component needs to receive data, perform some action and pass that data further. However, due to impossibility to reuse same sender twice or more, it leads to need for explicit locks. Deferred connections do not cover this case. Explicit lock make network harder to reason about. Tap is a higher-order component, that implements this logic for you. All you need to do, is to provide dependency node, thar receives data and sends a signal when finishes. Signal could be of any type, so no need for dealing with locks manually.

def Tap<T>(data T) (res T)

For example here we need to pass data to FirstLine and then to SecondLine, but FirstLine only sends a signal, that it finished, so we need to lock data and send it further. This is tedious and error prone, so we can use Tap to handle this case:

def Next2Lines(data int) (sig any) {
	first Tap<int>{FirstLine}
	dec Dec<int>
	second SecondLine
	---
	:data -> first -> dec -> second -> :sig
}

New Runtime Design (race-free, simpler and much faster)

One of the biggest challenges for a long time was overcoming various race conditions and out-of-order delivery. Finally, a proper design has been found. As with all elegant solutions, its power lies in its simplicity, bringing not just predictability but also much better performance.

The new runtime design is "connectionless." After compilation, a Nevalang program consists of runtime functions passing messages through channels. There are no more intermediate message-passing goroutines that caused concurrency issues. This is achieved through graph reduction. After IR generation, we strip intermediate connections, leaving only runtime functions. The runtime also has a better API, making it easier than ever to work with runtime functions.

Issues solved with new design:

• #689
• #677
• #644
• #635

Please note that this doesn't mean your code will be free of race conditions. It means they are not related to the runtime implementation but rather to your program logic. There are still some issues at the stdlib level, such as #754.

Interpreter was removed

Generating Go and compiling it with the Go compiler is now the only way to run Nevalang programs. You don't have to do it manually; neva build will call go build under the hood, so you'll get a binary (unless you pass --target=go). The command neva run will also run the built executable and clean it up afterward. This change simplifies the project, as the interpreter required duplicated functiona...

v0.25.0

It was almost a month after we've released the last version of the language. All this time we've been doing some heavy-lifting. We still need to do it, but bright future is closed day by day. So, what was going on?

Behold, this is the biggest Nevalang release so far!

New Runtime Implementation

In order to fix out-of-order delivery issue with fan-in connections we've completely changed the way runtime operates.

New algorithm properly handles fan-in messages thanks to atomic counter that is used to mark each message when it's sent. With this approach we can serialize messages and deliver them in the same order they was sent. However, out-of-order might still happen in more complex cases. Please see #689 for more details.

New Runtime Functions API

With massive runtime rewrite we've also changed how runtime functions interact with the runtime. New API is much simpler and safe than the old one. You don't have to handle context cancelation manually or iterate over array port slots. No more interactions with the raw go channels. Checkout how runtime functions look now, it's about x1.5 less code!

Fan-Out Connections are now Syntax Sugar

With new design runtime can properly handle fan-in connections but can't handle fan-out. Thankfully this is solved by removing fan-out from low-level program representation completely. Fan-out connections now only exist as syntax sugar in high level representation. Compiler desugarer transforms fan-out connections to regular pipelines with new FanOut component in standard library. This component has the following signature:

pub flow FanOut(data any) ([data] any)

Graph Reduction Optimization (WIP)

Implemented algorithm that can turn IR (low-level program representation) graph into much smaller but functionally equal one. It is disabled now and needs a little bit more testing. We will enable it by default or under feature flag in next releases.

Performance Boost

Last, but not least - with new runtime Nevalang v0.25 is faster than v0.24 by approximately two orders of magnitude. I didn't do any benchmarks but examples/99_bottles runs x200 times faster that before... And this is without graph reduction enabled!

Crazy thing about this is that this improvement is accidental! Our goal was to me it correct, not fast. However, performance improvement can be seen by naked eye.

Another funny thing is that now, when runtime works so fast, it's possible to spot new issues that cannot occur when program works slowly. New issues are described in #689.

v0.24.0

What's Changed

• Upgrade command added by @dorian3343 in #608
• Group form entity declaration syntax removed by @emil14 in #614
• Product by @Catya3 in #641
• Zip by @Catya3 in #642
• sync.WaitGroup by @Catya3 in #628
• New std components by @dorian3343 in #649
• Replace "component" keyword with "flow" by @emil14 in #659
• For and Map components by @emil14 in #619

Full Changelog: v0.23.0...v0.24.0

v0.23.0

Removed net keyword

Before

component Main(start) (stop) {
    nodes { Println }
    net {
        :start -> println -> :stop
    }
}

After

component Main(start) (stop) {
    nodes { Println }
    :start -> println -> :stop
}

Motivation

While being less explicit and consistent with the rest of the language, it saves us one level of nesting which is about 2 lines per component. Also it looks more like a regular function with var block (it's nodes in our case) and "list of instructions in a body" (it's list of connections for us).

New image package in STDlib by @Catya3

New package with two public components: image.New and image.Encode. This package also have public types such as image.Image, image.Pixel and image.RGBA. This is stream-based API. Check out examples/image_png for a minimal example.

Also note that this is just the beginning. The goal is to make Neva language suitable for complex image processing. (It's still is and always will be general purpose language tho).

One-line multiple import

You are now allowed to import several packages in a single line by using ,, this is consistent with the nodes block syntax.

import { io, strconv }

However, old syntax (no ,, newline for each package) is supported and should be preferred when you have a lot of imports.

import {
    io
    lists
    strings
    strconv

    github.com/nevalang/x:foo
    github.com/nevalang/x:foo/bar
    github.com/nevalang/x:foo/bar/baz

    @:/utils
    @:/models/users
}

Rest

As usual little improvements and bug fixes in compiler here and there.

v0.22.0...v0.23.0

v0.22.0

What's Changed

• Error guard operator ? by @emil14 in #591
• Fix type arguments resolution by @emil14 in #589
• fixing some typos in the READMEs and the standard library by @lachsdachs in #590

Full Changelog: v0.21.0...v0.22.0

v0.21.0

New Components

• New http package with http.Get component by @Catya3
• New List component in builtin that turns stream into list

Backward Compatibility Changes

• x package was removed from std module
• New strconv package was added
• x.ParseNum was moved to strconv.ParseNum
• x.Scanln was moved to io.Scanln
• Reduce (decorator that allows stream processors like Add accept array-inport slots) was renamed to ReducePort
• New strings package
• Split and Join components were moved to strings package, they are now strings.Split and strings.Join respectfully
• Index signature was updated to work with both lists and strings. Usage now looks like Index<list<T>> or Index<string>

Other

• Bug fixes and improvements
• Refactoring of internals

---

Full Changelog: v0.20.0...v0.21.0

v0.20.0

What's Changed

• Stream rewrite by @emil14 in #578
• Update version.go by @emil14 in #576

Full Changelog: v0.19.1...v0.20.0

v0.19.1

What's Changed

• Update README.md by @emil14 in #572
• Fix selectors by @emil14 in #561

Full Changelog: v0.19.0...v0.19.1

v0.19.0

What's Changed

• Deps: Protect neva dir with .lock file by @Catya3 in #554
• Anon deps by @emil14 in #555
• DOT: Enhanced Graphviz with clearer in/out ports. by @Catya3 in #556
• Upd examples by @emil14 in #557
• reimplemented iter by @dorian3343 in #565
• feat(e2e): local imports by @emil14 in #569
• fix(cli): use module root to correctly identify main pkg name (path) by @emil14 in #570

Full Changelog: v0.17.0...v0.19.0

v0.17.0

What's Changed

• Runtime functions: get rid of select-hell and add few fixes by @emil14 in #548
• Port-less connections (new syntax sugar) by @emil14 in #549
• FileReader and FileWriter API by @Catya3 in #547
• Refactor examples by @emil14 in #551
• Parser: fix chained + deferred connection use-case by @emil14 in #550
• Refactor: remove unused code in desugarer by @emil14 in #552

Port-less connections syntax sugar

Now you can omit in/out port name if there's only one port per side. E.g. instead of

42 -> println:data
println:sig -> :stop

You can now write

42 -> println
println -> :stop

This can be combined with chaining:

42 -> println -> :stop

Could be infinitely chained:

p1 -> p2 -> p3 -> ... -> pN

Deferred connections backward compatibility broken

You can't have multiple deferred connections in a single () expression anymore. E.g. you can't write code like this

:start -> (
    $l -> push:lst,
    $f -> push:data
)

But you still can (and always will be able to) write it like this

:start -> [
    ($l -> push:lst),
    ($f -> push:data)
]

Full Changelog: v0.16.0...v0.17.0