Skip to content

Commit

Permalink
add sparse noise section and minor content edits
Browse files Browse the repository at this point in the history
  • Loading branch information
banchan86 committed Jul 1, 2024
1 parent 87ca9f4 commit 89241af
Show file tree
Hide file tree
Showing 13 changed files with 434 additions and 17 deletions.
3 changes: 3 additions & 0 deletions docs/articles/define-display.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,6 @@ The `MeshMapping` operator is best used to correct distortions when projecting d

The `GammaCorrection` operator applies simple intensity mapping of the three colors, to make sure the stimuli are linear. It uses a simple LUT (Look-up-table), for Red, Green and Blue. Check out the [gamma calibration](./gamma-calibration.md) article for more information on how to measure and calibrate this operator.

## Other nodes to document (under construction)
`ViewMapping`

4 changes: 2 additions & 2 deletions docs/articles/display-environment-basics.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,10 @@ The disadvantage is that the arrangement of the planar mapping is not orthonorma

## Virtual Reality vs Augmented Reality

- Shawn's note - This needs more clarity as they both seem to be the same (fixed window, objects/environment),
- Shawn's note - This needs more clarity as they both seem to be the same (fixed window/eye, objects/environment move around).

### Virtual Reality (VR)
VR can be easily defined as a situation where the eye, and the screens (windows) are fixed positions, while the all the objects (or VR environment) moves across the eye.
VR can be easily defined as a situation where the eye, and the screens (windows) are fixed positions, while all the objects (or VR environment) moves across the eye.

![VR](~/images/DisplayLogic/VRcartoon.png){width=500}

Expand Down
11 changes: 8 additions & 3 deletions docs/articles/draw-stimuli.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@

`Bonvision` comes with a set of operators/primitives to draw basic shapes, display images and video, render common visual neuroscience stimuli, and display complex scenes and 3d models.

The starting point to test these out is a basic workflow (which was covered in the overview section)

The starting point to test these out is a basic workflow (which was covered in the [overview](basic-workflow.md) section).
:::workflow
![Basic Workflow](../workflows/overview-draw-circle.bonsai)
:::
Expand Down Expand Up @@ -54,7 +53,7 @@ In this example workflow, pressing the spacebar samples from a random number dis
![Neuroscience Primitives](../workflows/draw-stimuli-neuroscience-primitives.bonsai)
:::

While using these primitives are pretty straightforward, as gratings are commonly used in visual physiology experiments we have created a few additional nodes to help design grating experiments. These will be covered in more detail in the [Grating Stimuli](stimuli-grating.md) article.
These neuroscience primitives are covered in more detail in the [Neuroscience Stimuli](neuroscience-stimuli.md) article.


## 3D Model and Scene primitives
Expand All @@ -63,6 +62,8 @@ While using these primitives are pretty straightforward, as gratings are commonl
![3D Model and Scene Primitives](../workflows/draw-stimuli-scene-model-primitives.bonsai)
:::

### DrawModel

`DrawModel` is used to display 3D models without textures. `DrawModel` supports 3D models in the `.obj` file format.
The models have to be first loaded as a resource during the `Create Section` portion of the workflow using the `MeshResources` operator (which loads mesh or geometry resources).

Expand All @@ -74,6 +75,8 @@ In the `DrawModel` node, select the model `icosphere` under the `MeshName` prope
![Draw Model](../workflows/draw-stimuli-model.bonsai)
:::

### DrawTexturedModel

`DrawTexturedModel` is used to display 3D models with textures. To use `DrawTexturedModel`, during the `Create Section` portion of the workflow the model needs to be loaded using using the `MeshResources` operator and the texture needs to be loaded using the `TextureResources` operator (which loads image or video resources).

To try the example workflow out, download the 3D model `TextureSphere` and the `BlueMarble.jpg` from the [Bonvision Examples Repository](https://github.com/bonvision/examples/tree/master/Primitives).
Expand All @@ -93,6 +96,8 @@ Lastly add the `DrawTexturedModel` operator after the `BindTexture` operator, ch
> [!Note]
> A common use case for this node would be to create walls for VR environments with different textures and shapes. Bonvision comes with a inbuilt mesh called `Plane` that can be used for this purpose (it does not need to be loaded first with the `MeshResources` operator). Check out the VR corridor under the Demo section of the website for a walkthrough.
### DrawSceneModel

`DrawSceneModel` is used to display complex 3D scenes and models. To use the `DrawSceneModel`, during the `Create Section` portion of the workflow the scene needs to be loaded using using the `SceneResources` operator.The `SceneResources` operator uses the [Open Asset Import Library (assimp)](https://github.com/assimp/assimp) for loading scene files.

> [!Warning]
Expand Down
6 changes: 3 additions & 3 deletions docs/articles/drawing-region.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ All drawing views accept a `RenderFrame` as input and a `DrawX` stimuli as outpu
:::

### NormalizedView
`NormalizedView`is the default view for rapid prototyping of 2D stimuli.
`NormalizedView` is the default view for rapid prototyping of 2D stimuli.

This view scales the screen from -1 to 1 on both axes and has no properties to adjust.

Expand Down Expand Up @@ -81,7 +81,7 @@ Stimuli drawn will have sizes and locations defined in terms of metric units.
In this example workflow, we use `CubemapView` to simulate the walls of a virtual 3D room. This example workflow is a stripped down version of the virtual reality corridor in [Demos](demos/VR-demo.md) and requires downloading of the textures in that article.

:::workflow
![Perspective View](../workflows/drawing-region-cubemapview.bonsai)
![Perspective View](../workflows/drawing-region-cubemap-view.bonsai)
:::

> [!Note]
Expand All @@ -91,4 +91,4 @@ In this example workflow, we use `CubemapView` to simulate the walls of a virtua
> While the description of the operator suggests that you need to define six textures for each face of the cubemap, you can display an arbitary number of textures (including less than six).
### Other nodes to document (under construction)
`HMDview`
`HMDview` and `RenderHMD`
7 changes: 0 additions & 7 deletions docs/articles/stimuli-grating.md

This file was deleted.

29 changes: 29 additions & 0 deletions docs/articles/stimuli-neuroscience.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Neuroscience Stimuli

## Sparse Noise

The `SparseNoise` operator generates a non-overlapping discrete sparse grid of randomly activated quads.

Drawing the stimuli produced from the `SparseNoise` operator is a little different from the other `DrawX` stimuli.

1) Add a `Texture2D` resource called `DynamicVideo` using the `TextureResource` operator.
2) Add a `DrawImage` operator that renders the `DynamicVideo` texture.
3) Connect a `UpdateTexture` operator to the `SparseNoise` operator and choose the `DynamicVideo` texture to update.

:::workflow
![SparseNoise](../workflows/neuroscience-stimuli-sparse-noise.bonsai)
:::

- Add logging information?


## Gratings
As grating stimuli are commonly used in visual physiology experiments, we have created a few specialized nodes that make it convenient to design grating experiments very easily.

- Shawn's note - currently some of these nodes appear to be broken, awaiting feedback on https://github.com/bonsai-rx/bonsai/issues/1832

### Nodes to document
`GratingsSpecification`, `CreateGratingTrial`, `GratingSequence`, `DrawGrating`



2 changes: 1 addition & 1 deletion docs/articles/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@
- href: stimuli-multiple.md
- href: stimuli-timing.md
- href: stimuli-animating.md
- href: stimuli-grating.md
- href: stimuli-alternative.md
- href: stimuli-neuroscience.md
- name: Calibration
- href: gamma-calibration.md
- href: display-position-calibration.md
Expand Down
14 changes: 14 additions & 0 deletions docs/workflows/Events.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Frame,Timestamp,Value
10,0.1833524,{Width=10, Height=10, Depth=U8, Channels=1}
22,0.38393479999999996,{Width=10, Height=10, Depth=U8, Channels=1}
34,0.58393719999999993,{Width=10, Height=10, Depth=U8, Channels=1}
46,0.78393989999999991,{Width=10, Height=10, Depth=U8, Channels=1}
58,0.98394399999999993,{Width=10, Height=10, Depth=U8, Channels=1}
70,1.1839461999999998,{Width=10, Height=10, Depth=U8, Channels=1}
82,1.3839484,{Width=10, Height=10, Depth=U8, Channels=1}
90,1.5946148999999998,{Width=10, Height=10, Depth=U8, Channels=1}
102,1.7946171,{Width=10, Height=10, Depth=U8, Channels=1}
114,1.9946197999999999,{Width=10, Height=10, Depth=U8, Channels=1}
126,2.1946224000000001,{Width=10, Height=10, Depth=U8, Channels=1}
138,2.3946332999999997,{Width=10, Height=10, Depth=U8, Channels=1}
150,2.5995718999999999,{Width=10, Height=10, Depth=U8, Channels=1}
10 changes: 9 additions & 1 deletion docs/workflows/draw-stimuli-neuroscience-primitives.bonsai
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<WorkflowBuilder Version="2.8.1"
<WorkflowBuilder Version="2.8.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="https://bonsai-rx.org/2018/workflow">
<Workflow>
Expand Down Expand Up @@ -32,6 +32,14 @@
<Aperture>0</Aperture>
<Opacity>1</Opacity>
</Expression>
<Expression xsi:type="IncludeWorkflow" Path="BonVision:Collections.SparseNoise.bonsai">
<Duration>PT0.2S</Duration>
<ActiveQuads>5</ActiveQuads>
<GridSize>
<Width>10</Width>
<Height>10</Height>
</GridSize>
</Expression>
</Nodes>
<Edges />
</Workflow>
Expand Down
24 changes: 24 additions & 0 deletions docs/workflows/draw-stimuli-neuroscience-primitives.bonsai.layout
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,28 @@
</Size>
<WindowState>Normal</WindowState>
</DialogSettings>
<DialogSettings xsi:type="WorkflowEditorSettings">
<Visible>false</Visible>
<Location>
<X>0</X>
<Y>0</Y>
</Location>
<Size>
<Width>0</Width>
<Height>0</Height>
</Size>
<WindowState>Normal</WindowState>
<EditorDialogSettings>
<Visible>false</Visible>
<Location>
<X>4</X>
<Y>5</Y>
</Location>
<Size>
<Width>1294</Width>
<Height>762</Height>
</Size>
<WindowState>Normal</WindowState>
</EditorDialogSettings>
</DialogSettings>
</VisualizerLayout>
108 changes: 108 additions & 0 deletions docs/workflows/draw-stimuli-sparse-noise.bonsai
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
<?xml version="1.0" encoding="utf-8"?>
<WorkflowBuilder Version="2.8.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:gl="clr-namespace:Bonsai.Shaders;assembly=Bonsai.Shaders"
xmlns:res="clr-namespace:Bonsai.Resources;assembly=Bonsai.System"
xmlns="https://bonsai-rx.org/2018/workflow">
<Workflow>
<Nodes>
<Expression xsi:type="Combinator">
<Combinator xsi:type="gl:CreateWindow">
<gl:Width>640</gl:Width>
<gl:Height>480</gl:Height>
<gl:VSync>On</gl:VSync>
<gl:SwapSync>false</gl:SwapSync>
<gl:ClearColor>Black</gl:ClearColor>
<gl:ClearMask>DepthBufferBit ColorBufferBit</gl:ClearMask>
<gl:CursorVisible>true</gl:CursorVisible>
<gl:Location xsi:nil="true" />
<gl:WindowBorder>Resizable</gl:WindowBorder>
<gl:WindowState>Normal</gl:WindowState>
<gl:DisplayDevice>Primary</gl:DisplayDevice>
<gl:TargetRenderFrequency>60</gl:TargetRenderFrequency>
<gl:TargetUpdateFrequency xsi:nil="true" />
<gl:RenderState />
<gl:GraphicsMode>
<gl:ColorFormat>
<gl:Red>8</gl:Red>
<gl:Green>8</gl:Green>
<gl:Blue>8</gl:Blue>
<gl:Alpha>8</gl:Alpha>
</gl:ColorFormat>
<gl:Depth>16</gl:Depth>
<gl:Stencil>0</gl:Stencil>
<gl:Samples>0</gl:Samples>
<gl:AccumulatorFormat>
<gl:Red>0</gl:Red>
<gl:Green>0</gl:Green>
<gl:Blue>0</gl:Blue>
<gl:Alpha>0</gl:Alpha>
</gl:AccumulatorFormat>
<gl:Buffers>2</gl:Buffers>
<gl:Stereo>false</gl:Stereo>
</gl:GraphicsMode>
</Combinator>
</Expression>
<Expression xsi:type="IncludeWorkflow" Path="BonVision:Primitives.BonVisionResources.bonsai" />
<Expression xsi:type="Combinator">
<Combinator xsi:type="gl:TextureResources">
<gl:Textures>
<gl:TextureConfiguration xsi:type="gl:Texture2D">
<gl:Name>DynamicVideo</gl:Name>
<gl:Width xsi:nil="true" />
<gl:Height xsi:nil="true" />
<gl:InternalFormat>Rgba</gl:InternalFormat>
<gl:WrapS>Repeat</gl:WrapS>
<gl:WrapT>Repeat</gl:WrapT>
<gl:MinFilter>Nearest</gl:MinFilter>
<gl:MagFilter>Nearest</gl:MagFilter>
</gl:TextureConfiguration>
</gl:Textures>
</Combinator>
</Expression>
<Expression xsi:type="Combinator">
<Combinator xsi:type="res:LoadResources" />
</Expression>
<Expression xsi:type="Combinator">
<Combinator xsi:type="gl:RenderFrame" />
</Expression>
<Expression xsi:type="IncludeWorkflow" Path="BonVision:Environment.NormalizedView.bonsai" />
<Expression xsi:type="IncludeWorkflow" Path="BonVision:Primitives.DrawImage.bonsai">
<TextureName>DynamicVideo</TextureName>
<ExtentX>1</ExtentX>
<ExtentY>1</ExtentY>
<LocationX>0</LocationX>
<LocationY>0</LocationY>
<Layer>0</Layer>
<Angle>0</Angle>
<ScaleX>1</ScaleX>
<ScaleY>1</ScaleY>
<ShiftX>0</ShiftX>
<ShiftY>0</ShiftY>
</Expression>
<Expression xsi:type="IncludeWorkflow" Path="BonVision:Collections.SparseNoise.bonsai">
<Duration>PT0.2S</Duration>
<ActiveQuads>5</ActiveQuads>
<GridSize>
<Width>10</Width>
<Height>10</Height>
</GridSize>
</Expression>
<Expression xsi:type="Combinator">
<Combinator xsi:type="gl:UpdateTexture">
<gl:TextureName>DynamicVideo</gl:TextureName>
<gl:TextureTarget>Texture2D</gl:TextureTarget>
<gl:InternalFormat>Rgba</gl:InternalFormat>
</Combinator>
</Expression>
</Nodes>
<Edges>
<Edge From="0" To="1" Label="Source1" />
<Edge From="1" To="2" Label="Source1" />
<Edge From="2" To="3" Label="Source1" />
<Edge From="4" To="5" Label="Source1" />
<Edge From="5" To="6" Label="Source1" />
<Edge From="7" To="8" Label="Source1" />
</Edges>
</Workflow>
</WorkflowBuilder>
Loading

0 comments on commit 89241af

Please sign in to comment.