-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathuser_guide.html
356 lines (346 loc) · 26.3 KB
/
user_guide.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
<!DOCTYPE html>
<html lang="en">
<head>
<title>BIAFLOWS NEUBIAS-WG5 : user guide documentation</title>
<!-- Meta -->
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="">
<meta name="author" content="">
<link rel="shortcut icon" href="favicon.ico">
<link href='https://fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800' rel='stylesheet' type='text/css'>
<!-- FontAwesome JS -->
<script defer src="https://use.fontawesome.com/releases/v5.8.2/js/all.js" integrity="sha384-DJ25uNYET2XCl5ZF++U8eNxPWqcKohUUBUpKGlNLMchM7q4Wjg2CUpjHLaL8yYPH" crossorigin="anonymous"></script>
<!-- Global CSS -->
<link rel="stylesheet" href="assets/plugins/bootstrap/css/bootstrap.min.css">
<!-- Plugins CSS -->
<link rel="stylesheet" href="assets/plugins/prism/prism.css">
<link rel="stylesheet" href="assets/plugins/elegant_font/css/style.css">
<!-- Theme CSS -->
<link id="theme-style" rel="stylesheet" href="assets/css/styles.css">
</head>
<body class="body-pink">
<div class="page-wrapper">
<!-- ******Header****** -->
<header id="header" class="header">
<div class="container">
<div class="branding">
<h1 class="logo">
<a href="index.html">
<span aria-hidden="true" class="icon_documents_alt icon"></span>
<span class="text-highlight">BIA</span><span class="text-bold">FLOWS</span>
</a>
</h1>
</div>
</div><!--//container-->
</header><!--//header-->
<div class="doc-wrapper">
<div class="container">
<div id="doc-header" class="doc-header text-center">
<h1 class="doc-title"><i class="icon fa fa-book"></i> <span style="color:#fe7f7f;">BIA</span>FLOWS User Guide</h1>
<div class="meta"><i class="far fa-clock"></i> Last updated: March 31<sup>st</sup>, 2020</div>
</div>
<!--//doc-header-->
<div class="doc-body row">
<div class="doc-content col-md-9 col-12 order-1">
<div class="content-inner">
<section id="introduction-section" class="doc-section">
<h2 class="section-title">Introduction</h2>
<div class="section-block">
<p><span style="color:#fe7f7f;">BIA</span>FLOWS online instance can be accessed from: <a href="https://biaflows.neubias.org" target="_blank">https://biaflows.neubias.org</a>.
<br />It is possible to browse the content in read only mode from the guest account (username: guest; password: guest).<br />
Workflows cannot be launched from the guest account but all results from past workflow runs can be accessed.
<br />The platform has been tested for Chrome, Chromium, Safari and Firefox.</p>
<p>Any feedback is welcome by <a href="mailto:biaflows@neubias.org">e-mail</a> or through our <a href="https://github.com/Neubias-WG5" target="_blank">GitHub repository</a> Issues.</p>
</div>
</section><!--//doc-section-->
<section id="login-section" class="doc-section">
<h2 class="section-title">Login</h2>
<div class="section-block">
<p>Login using guest/guest.
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-login.jpg" alt="screenshot" width="320">
</div>
</p>
</div><!--//section-block-->
</section><!--//doc-section-->
<section id="home-section" class="doc-section">
<h2 class="section-title">Homepage</h2>
<div class="section-block">
<p>The homepage provides an overview of the navigation in <span style="color:#fe7f7f;">BIA</span>FLOWS (also accessible from Help > How to use).</p>
<p><span style="color:#fe7f7f;">BIA</span>FLOWS main sections (Workspace, Problems, Workflows) can be browsed from the top bar menu.</p>
<p>The navigation is summarized in the figure below:</p>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/figure2.png" alt="screenshot"></a>
</div>
</div><!--//section-block-->
</section><!--//doc-section-->
<section id="problems-section" class="doc-section">
<h2 class="section-title">Problems</h2>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-topmenu.jpg" alt="screenshot"></a>
</div>
<div id="problem-list" class="section-block">
<h4>Problem list</h4>
<p>The Problems section brings an overview of all <acronym title="Bio Image Analysis">BIA</acronym> problems currently available in <span style="color:#fe7f7f;">BIA</span>FLOWS online instance. Each Problem recapitulates a common <acronym title="Bio Image Analysis">BIA</acronym> Problem by gathering a set of ground truth annotated images and compatible workflows to process them.</p>
<p>Clicking on a Problem redirects to the list of images available for this Problem.</p>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/problems-list.png" alt="screenshot"></a>
</div>
<p>A Problem class (e.g. object segmentation) is associated to every Problem; <span style="color:#fe7f7f;">BIA</span>FLOWS currently supports 9 Problem classes:</p>
<ul>
<li>Object counting</li>
<li>Object detection</li>
<li>Landmark detection</li>
<li>Object segmentation</li>
<li>Particle tracking</li>
<li>Object tracking</li>
<li>Pixel classification</li>
<li>Filament tree tracing</li>
<li>Filament networks tracing</li>
</ul>
<p>A Problem class specifies the format of ground truth annotations (e.g. skeleton binary mask), as well as the benchmark metrics to be evaluated on all annotated images available for that Problem. The annotation format is identical to the output expected by the workflows of a Problem. See <a href="https://neubias-wg5.github.io/assets/pdfs/Problem%20classes,%20ground%20truth%20annotations%20and%20reported%20metrics.pdf">this document</a> and the contextual help (information icons) from Benchmark results table.</p>
</div><!--//section-block-->
<div id="problem-images" class="section-block">
<h4>Images</h4>
<p>This table lists all the images for the current Problem, where details can be displayed for each image.<br>
<p> Images can be filtered by name by typing a keyword in the search bar and they can be visualized by clicking on their name or thumbnail.</p>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-image-list.png" alt="screenshot" width="480">
</div>
</p>
<p>In case ground truth annotations are stored as image masks (this depends on the problem class), the masks can be displayed by toggling a filter (click on "Show filters" then select Type > Groundtruth). If ground truth annotations are in a different format (e.g. SWC or text file), they can be accessed through the Attached Files available from Image description (blue arrow).</p>
<p>Finally, all images can be downloaded from the <button class="button is-small is-fullwidth layer-field" disabled="disabled">Download</button> button.</p>
</div><!--//section-block-->
<div id="problem-image-viewer" class="section-block">
<h4>Image Viewer</h4>
<p>You can click on an image in the image list to open the Image Viewer.
<h5>Navigation</h5>
Use mouse scroll to zoom in/out, click + drag to move around. <br>
For multidimensional images, C-Z-T slicers automatically appears (when applicable) to navigate through the slices.
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-image-viewer.png" alt="screenshot" width="640">
</div>
</p>
<h5>Layers</h5>
<p>
To overlay an annotation layer, click on <span class="fas fa-copy"></span> in
the right toolbar.
</p>
<p>
Annotation layers are generated from the results of a workflow run.
Several layers can be overlaid on the original image, and they can be hidden by unselecting
the checkbox under the <span class="far fa-eye"></span> symbol, or removed using
<span class="fas fa-times"></span>.</p>
<p>By default, the ground truth layer is selected and visible in <span style="color:#13A1FF">blue</span></p>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/doc_biaflow_user_guide_benchmark_layout.png" alt="screenshot" width="640">
</div>
<p>You can add as many annotation layer as you want. By default, each layer is displayed
with a unique color.</p>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/doc_biaflow_user_guide_benchmark_layout_02.png" alt="screenshot" width="640">
</div>
<p>If you uncheck the check box <em>COLOR ANNOTATION PER LAYER</em>, a unique color is set by object instead of by layer.</p>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/doc_biaflow_user_guide_benchmark_layout_03.png" alt="screenshot" width="640">
</div>
<p>In addition, <em>LAYER OPACITY</em> can be adjusted with the bottom slider.</p>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/doc_biaflow_user_guide_benchmark_layout_06.png" alt="screenshot" width="640">
</div>
<p>
If you do not know which annotation layer to display, click on
<button data-v-0c385051="" class="button is-small is-fullwidth layer-field" disabled="disabled">Add all favorites</button>.
Favorite workflow runs were validated by their maintainer as providing representative results.</p>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/doc_biaflow_user_guide_benchmark_layout_07.png" alt="screenshot" width="640">
</div>
<h5>Multiple viewers</h5>
It is also possible to have multiple viewers opened at once and synchronized (e.g. 4 synchronized views where top left is the original image and others are workflow runs overlays):<br>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-image-multiple-viewer.jpg" alt="screenshot" width="640">
</div>
This feature can be used to open the same image several times and compare different workflow runs through the display of corresponding annotation layers. To do so, select an image from the upper left drop down menu and click on the big <span class="fas fa-plus"></span> blue button (bottom right corner), select a workflow in the annotation layers <span class="fas fa-copy"></span>, then use <span class="fas fa-link"></span> (in the right toolbar) to synchronize the views.</p>
<p>
The current image can be downloaded by clicking on the image information panel <span class="fas fa-info"></span> in the right toolbar, then on <button class="button is-small is-fullwidth layer-field" disabled="disabled">Download</button> (next to More information). For 2D images, it is also possible to switch to previous/next image using corresponding buttons (<button class="button is-small">Previous image</button> / <button class="button is-small">Next image</button>) from this panel.
</p>
<p>Finally, clicking on an annotation in the viewer will display some information (e.g. annotation area and perimeter).</p>
<h5>Image adjustments</h5>
Contrast and gamma correction can be adjusted from the panel <span class="fas fa-adjust"></span> (right toolbar). The image can also be video inverted from this panel.
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/doc_biaflow_user_guide_bright_and_contrast.png" alt="screenshot">
</div>
</div><!--//section-block-->
</section>
<section id="workflows-section" class="doc-section">
<h2 class="section-title">Workflows</h2>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-topmenu-workflow.jpg" alt="screenshot"></a>
</div>
<div id="workflows-list" class="section-block">
<h4>Workflow list</h4>
<p>This page lists all workflow currently available in the platform. For each workflow, the most relevant workflow is the last executable version but all past versions can also be displayed ("Show filters", then Only Last Realeases -> No). Workflows can also be filtered by name or by number of executions from the search bar.
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-workflows-list.png" alt="screenshot">
</div>
Selecting a workflow (click on its name or Open) brings some useful information:
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-workflow-details.jpg" alt="screenshot">
</div>
In particular, it is possible to inspect a workflow input parameters and associated default values (expandable list), the list of Problems this workflow is availble from (with links to problems), GitHub and DockerHub links respectively for the workflow source code and <acronym title="A file encapsulating a workflow and its execution environment to ensure its reproducibility">Workflow Docker image</acronym>, and finally the command line invoked by <span style="color:#fe7f7f;">BIA</span>FLOWS to launch the Workflow Docker image (Execute command).
</p>
</div>
<div id="workflows-workflow-runs" class="section-block">
<h4>Workflow runs</h4>
<p>
In user mode (the user account requires execution rights that are not provided by the guest account), a workflow can be launched by clicking on its name. This opens a workflow parameter dialog box where default parameters are set so as to bring meaningful results for the images of the project. Some parameter description can be obtained while moving the mouse cursor over its name. Click on the blue <button data-v-3442b694="" class="button is-link" disabled="disabled"> Run a workflow </button> button:
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-workflow-run.png" alt="screenshot">
</div>
Upon launch, a job if flagged as "In queue", then "Running" in the job list and a progress bar displays information on the current execution step, as well as the number of annotations created so far. Upon completion, the job status should either turn to success (green) or failed (red). <br>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-workflow-status-inqueue.jpg" alt="screenshot">
</div>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-workflow-status-running.jpg" alt="screenshot">
</div>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-workflow-status-success.jpg" alt="screenshot">
</div>
</p>
<p>If a job is successful, workflow log, results (annotations), and benchmark metrics are available for this run.<br>
The complete report (log file) of the job is can be shown (Execution log <button class="button is-small" disabled="disabled"><span>Show</span></button>):<br>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-workflow-log.jpg" alt="screenshot">
</div>
</p>
<p>
Annotations created by this workflow run can be shown by clicking on Data -> XXX Annotations, opening the Annotations tab where one can filter annotations for all images or individual images: <br>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-workflow-annotations.jpg" alt="screenshot">
</div>
</p>
<p>One can then click on any annotation and further inspect it in the Image viewer where the Workflow layer is overlaid:
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-workflow-annotations-imageviewer.jpg" alt="screenshot">
</div>
</p>
<p>Workflow metrics can be displayed as explained in the next Section.</p>
</div><!--//section-block-->
<div id="workflows-benchmarking" class="section-block">
<h4>Benchmarking</h4>
<p>The Benchmarking Section is located at the bottom of the Workflow runs page and it
is where benchmark results (metrics) from past workflow runs can be inspected.</p>
<p>
A report is generated for all workflow runs selected in the top table
(<i style="color:#61B2E8" class="fas fa-check-square"></i> tick boxes).
</p>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/doc_biaflow_user_guide_benchmark_03.png" alt="screenshot">
</div>
<p>
The metrics are computed for all the images from the Problem for these
workflow runs.
The main metric, displayed in black
<i style="color:black" class="fas fa-info-circle"></i>,
is the most representative to rank the performance of the workflows, the other
metrics are displayed in <span style="color:#7A7A7A">gray</span>
<i style="color:#7A7A7A" class="fas fa-info-circle"></i>.
</p>
<p>
Both selections can be refined using Filters (by Images, by Metrics) and metrics can be explored as
aggregated for all images or selected for individual images:</p>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/doc_biaflow_user_guide_benchmark_01.png" alt="screenshot">
</div>
</p>
<p>The detailed results by image view is useful to, e.g., find jobs with poor metrics and inspect the associated annotation layer to understand where the workflow has failed. To quickly access an image, its thumbnail can be clicked from the list.
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/doc_biaflow_user_guide_benchmark_02.png" alt="screenshot">
</div>
</p>
<p>A textual description of a metric is available by clicking on <span class="fas fa-info-circle"></span> next to the metric name:<br>
<div class="screenshot-holder">
<img class="img-fluid" src="assets/images/biaflows-workflows-metric-descr.jpg" alt="screenshot">
</div>
</p>
</div><!--//section-block-->
</section>
<div id="architecture" name="architecture">
<section id="architecture-section" class="doc-section">
<h2 class="section-title">Architecture</h2>
<p>This section describes BIAFLOWS architecture to deploy reproducible BIA workflows.</p>
<p>BIAFLOWS enables developers to reproducibly package and distribute their workflows by seamlessly configuring their execution environment and specifying standard image and results formats adapted to the BIA problems at hand. Additionally, the functional parameters of the workflows can be documented and default values specified so that users quickly grasp what results should be expected (for the sample images stored in the system) and how they compare to other workflows (both visually and from relevant, documented benchmark metrics).</p>
<img src="assets/images/biaflows-architecture.png" height="560"></a>
<p>Workflows are hosted in a trusted code repository (GitHub) as a set of 4 files:</p>
<ul>
<li>a Dockerfile configuring execution environment (OS, installed software)</li>
<li>a JSON descriptor configuring problem class (e.g. object segmentation) and workflow parameters</li>
<li>the workflow source code</li>
<li>a common wrapper script sequencing all operations</li>
</ul>
<p>Workflow images (Docker images) are automatically built by DockerHub (cloud service) whenever a new workflow version is released from its code repository.</p>
<p>BIAFLOWS instances monitor DockerHub and pull new or updated workflow images to make them available from the web interface.</p>
<p>Workflow images can alternatively be downloaded manually to process local image folders in a (Standalone Local).</p>
<p>See <a class="link" href="https://github.com/Neubias-WG5/W_NucleiSegmentation-ImageJ" target="_blank"><span>this sample workflow repository</span></a>.</p>
</section><!--//doc-section-->
</div>
<section id="add-section" class="doc-section">
<h2 class="section-title">Adding content</h2>
<p>If you want to package your workflows as BIAFLOWS workflows, or populate a local instance of BIAFLOWS, please read <a href="index.html">"Installing and populating BIAFLOWS locally" and "Creating a BIA workflow..."</a>.</p>
<p>If you want to submit new content (image datasets, ground-truth annotations, workflows, ...) to BIAFLOWS online instance, please
write to <a class="link" href="https://forum.image.sc/tags/biaflows" target="_blank"><span>BIAFLOWS forum</span></a> or <a href="mailto:biaflows@neubias.org?subject=New Content Submission BIAFLOWS">contact us</a>. </p>
</section><!--//doc-section-->
</div><!--//content-inner-->
</div><!--//doc-content-->
<div class="doc-sidebar col-md-3 col-12 order-0 d-none d-md-flex">
<div id="doc-nav" class="doc-nav">
<nav id="doc-menu" class="nav doc-menu flex-column sticky">
<a class="nav-link scrollto" href="#introduction-section">Introduction</a>
<a class="nav-link scrollto" href="#login-section">Login</a>
<a class="nav-link scrollto" href="#home-section">Homepage</a>
<a class="nav-link scrollto" href="#problems-section">Problems</a>
<nav class="doc-sub-menu nav flex-column">
<a class="nav-link scrollto" href="#problem-list">Problem list</a>
<a class="nav-link scrollto" href="#problem-images">Images</a>
<a class="nav-link scrollto" href="#problem-image-viewer">Image Viewer</a>
</nav><!--//nav-->
<a class="nav-link scrollto" href="#workflows-section">Workflows</a>
<nav class="doc-sub-menu nav flex-column">
<a class="nav-link scrollto" href="#workflows-list">Workflow list</a>
<a class="nav-link scrollto" href="#workflows-workflow-runs">Workflow runs</a>
<a class="nav-link scrollto" href="#workflows-benchmarking">Benchmarking</a>
</nav>
<a class="nav-link scrollto" href="#architecture-section">Architecture</a>
<a class="nav-link scrollto" href="#add-section">Adding content</a>
</nav><!--//doc-menu-->
</div>
</div><!--//doc-sidebar-->
</div><!--//doc-body-->
</div><!--//container-->
</div><!--//doc-wrapper-->
</div><!--//page-wrapper-->
<footer id="footer" class="footer text-center">
<div class="container">
<!--/* This template is released under the Creative Commons Attribution 3.0 License. Please keep the attribution link below when using for your own project. Thank you for your support. :) If you'd like to use the template without the attribution, you can buy the commercial license via our website: themes.3rdwavemedia.com */-->
<small class="copyright">Designed with <i class="fas fa-heart"></i> by <a href="https://themes.3rdwavemedia.com/" target="_blank">Xiaoying Riley</a> for developers</small>
<p>
<br><br>
<a href="http://neubias.org/" target="_blank"><img src="assets/images/logo-neubias.png" height="50"></a>
<a href="https://cost.eu/COST_Actions/ca/CA15124" target="_blank"><img src="assets/images/logo-cost.png" height="50"></a>
<a href="https://cytomine.org"><img src="assets/images/logo-cytomine-org.png" height="50"></a>
<a href="http://europe.wallonie.be/" target="_blank"><img src="assets/images/logo-feder.jpg" height="50"></a>
</p>
</div><!--//container-->
</footer><!--//footer-->
<!-- Main Javascript -->
<script type="text/javascript" src="assets/plugins/jquery-3.3.1.min.js"></script>
<script type="text/javascript" src="assets/plugins/bootstrap/js/bootstrap.min.js"></script>
<script type="text/javascript" src="assets/plugins/prism/prism.js"></script>
<script type="text/javascript" src="assets/plugins/jquery-scrollTo/jquery.scrollTo.min.js"></script>
<script type="text/javascript" src="assets/plugins/stickyfill/dist/stickyfill.min.js"></script>
<script type="text/javascript" src="assets/js/main.js"></script>
</body>
</html>