File tree Expand file tree Collapse file tree 1 file changed +2
-19
lines changed Expand file tree Collapse file tree 1 file changed +2
-19
lines changed Original file line number Diff line number Diff line change @@ -140,30 +140,13 @@ to specify a timeout, after which the batch is emitted even if it's not full. Th
140
140
when input stream is slow or sparse.
141
141
142
142
143
+
143
144
## Fan-In and Fan-Out
144
- Library provides a simple way to fan- in and fan- out data streams. Fan-in is done with the ** Merge** function,
145
+ The library offers mechanisms for fanning in and out data streams. Fan-in is done with the ** Merge** function,
145
146
which consolidates multiple data streams into a single unified channel.
146
147
Fan-out is done with the ** Split2** function, that divides a single input stream into two distinct output channels.
147
148
This division is based on a discriminator function, allowing parallel processing paths based on data characteristics.
148
149
149
- When splitting a stream, you create a scenario with a single producer and two consumers.
150
- It's important to note that if one consumer is blocked, it can lock the entire pipeline,
151
- affecting the producer and the other consumer.
152
- One way to mitigate this is to use a buffer on the output channels, but be aware that this will increase memory usage.
153
-
154
- ``` go
155
- out1 , out2 := rill.Split2 (input, 10 , func (item string ) bool {
156
- // Some splitting logic...
157
- })
158
-
159
- // out1 is slow. Buffer up to 100 items that go to it
160
- out1 = rill.Buffer (out1, 100 )
161
-
162
- // Now, work with out1 and out2 as usual
163
- ```
164
-
165
-
166
-
167
150
168
151
169
152
## Error handling
You can’t perform that action at this time.
0 commit comments