Dynamic messages creation/building (builder, recorder, from record)

[emil14]

While it's clear how to use static messages (via IIPs) it's not clear how one should dynamically create messages, especially of composite types;

-. Scalars looks simple: you have operators that returns scalar values
-. What about composite types: arrays, records, vectors, maps?

---

There's at least 2 directions to investigate that are:

1. Introduce specific node types like "builder" for all needed data types
2. Introduce special operators for such cases

Solution 1 seems like an ad-hoc (like a generators from #106), but could add more compile-time safety. E.g. it could be possible to check that array builder can only have as much inport slots as size in corresponding array type. This is however must be investigated. On the other hand, maybe type system itself could be more complex to allow to program such things like "this component can have N inport slots for that inport, where N depends on some X"

[emil14]

Union messages probably should not be able to be created. This should only be a parameter. Even-though you can do so in TS

[emil14]

Naive implementation of the recorder (record/struct builder)

func recorder(m map[string]struct{}{}, io core.IO) error  {
	outport, err := io.Outort("v")
	if err != nil {
		return err
	}
	
	inports := make(map[string]chan core.Msg, m)
	
	for k := range m {
		port, err := io.Inport(k)
		if err != nil {
			return err
		}
		inports[k] = port
	}
	
	wg := sync.WaitGroup{}
	
	for {
		v := make(map[string]core.Msg, len(m)) // check
		for k, in := range inports {
			wg.Add(1)
			go func(k string) {
				v[k] = <- in 
				wg.Done()
			}(k)
		}
		wg.Wait()
		outport<-v
	}
}

Then having a recorder effect at runtime an special recorder node kind in compiler

The reason why it can't be a component

Is that type system doesn't have ability to say "this component will have these ports corresponding to this record"

[emil14]

https://stackoverflow.com/questions/45258055/is-it-safe-to-write-to-a-golang-map-concurrently-if-the-keys-are-always-unique

[emil14]

"From record" feature

Add a feature called "from record" that will allow to create inports based on given record field. It **should not be part of a type-system" because type system doesn't known anything about ports (and components) which is a good thing. So this should be implemented at compiler level

foo<t {}> {
  in = {...}
}

... in yaml:

constr: # {}
  kind: typeExpr
  typeExpr:
    type: struct
    structFields: []

foo:
  kind: component
  component:
    params: [{name: t, constraint: {}]
    io:
     in:
       fromRec: t # <-- here
     out:
       v: t

So it should be possible to say

f = foo<{a int, b str}>
ff = f.in(2 '3')
// ff == {a: 2, b: '3'}

IIPs (static ports)

Maybe this feature can be used to implement IIPs

foo:
  conf:
    staticPorts: {a: 42, b: 'x'} # <-- here
    di: ...

It's not clear whether this adds something good or not

Problem

1. What about array ports?
2. Isn't it an ad-hoc? Can we use it for anything else?

Arr ports

Is it possible to make array ports be converted into regular arrays?

---

Related #170

[emil14]

possible solutions are:

1. special node types
2. operator with "from record"
3. general meta (does not solve inports (header) typing problem)

[emil14]

Constructor and destructor

ctor and dtor are possible names for two operators:

• ctor<t {}> - constructs a record from inports
• dtor<t {}> destructor destructs the record to separate outports

typings via from rec feature

Problem

It's not clear if we need destructor since network manager can select fields

[emil14]

Let's think it solved with #149 (comment) for now

[emil14]

I have a bad feeling about this. Not sure if it's closed really