-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathapi-docu-seurat-opencpu.Rmd
390 lines (309 loc) · 28.2 KB
/
api-docu-seurat-opencpu.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
---
title: "API Documentation for the Pre-processing of scRNAseq using OpenCPU"
output:
html_document:
df_print: kable
toc: true
toc_float:
collapsed: false
number_sections: true
highlight: null
date: 'Compiled: `r format(Sys.Date(), "%B %d, %Y")`'
---
```{r setup, include=FALSE}
all_times <- list() # store the time for each chunk
knitr::knit_hooks$set(time_it = local({
now <- NULL
function(before, options) {
if (before) {
now <<- Sys.time()
} else {
res <- difftime(Sys.time(), now, units = "secs")
all_times[[options$label]] <<- res
}
}
}))
knitr::opts_chunk$set(
tidy = TRUE,
tidy.opts = list(width.cutoff = 100),
message = FALSE,
warning = FALSE,
time_it = TRUE,
error = TRUE
)
knitr::opts_chunk$set(echo=TRUE)
library(knitr)
library(kableExtra)
```
> This documents the use of OpenCPU for the pre-processing of scRNA sequences using Seurat v5.
# Pre-processing pipeline
The standard steps of the pre-processing of scRNA sequences are as follows:  The steps are performed separately using their own R functions that can be called individually in OpenCPU. But first let's set up our remote access.
# A. Before running the session
To-dos before we process our file:
- Set up the remote connection
- Learn how to use the HTTP methods
- Learn how to read and access the files
### **Setting up on the browser**
To load OpenCPU on the browser, first determine the local server address. If on Windows, type **ipconfig** or if on Linux, **ifconfig**. It is usually the set of numbers located next to "inet". This will be referred to as "local host".
On your browser, load <http://localhost:8080/ocpu>.
### **HTTP Methods**
OpenCPU uses the HTTP methods **GET** (to retrieve a source) and **POST** (for RPC which is valid if targeting a script or function).
```{r echo=FALSE}
Method <- c('GET', 'POST', 'GET')
Target <- c('object', 'object', 'file')
Action <- c('read object', 'call a function', 'read a file')
Parameters <- c('N/A', '"input" or input file', 'N/A')
Example <- c('GET /ocpu/library/writeseuratobject', 'POST /ocpu/library/writeseuratobject/R/write_seurat_object', 'GET /ocpu/library/writeseuratobject/R/write_seurat_object')
http_methods <- data.frame(Method, Target, Action, Parameters, Example)
http_methods %>%
knitr::kable() %>% kableExtra::kable_styling(bootstrap_options = c("striped", "hover", full_width = T))
```
### **API Endpoints**
There pre-processing pipeline for scRNA sequences are available on the server and can be accessed based on the path.
```{r echo=FALSE}
Path <- c('/ocpu/library/{package name}/', '/ocpu/library/{package name}/R/{function}/', '/ocpu/library/{package name}/DESCRIPTION', '/ocpu/tmp/{key}')
What <- c('R packages installed in the global library of the server', 'Function within the package', 'Shows the description/details of the package', ' Temporary session that hold outputs from a function/script RPC')
http_methods <- data.frame(Path, What)
http_methods %>%
knitr::kable() %>% kableExtra::kable_styling(bootstrap_options = c("striped", "hover", full_width = T))
```
### **R Package API**
Any of the /{package}/ libraries support the following endpoints:
```{r echo=FALSE}
Path <- c('../{package name}/info', '../{package name}/R/', '../{package name}/data/', '../{package name}/man/', '../{package name}/html', '../{package name}/*')
What <- c('Show information about this package', 'R objects/scripts exported by this package', 'Data included in this package', 'Manuals or help pages included in the package', 'Simulates the R-base html help pages', 'For all else, interfaces to the files in the package installation directory')
http_methods <- data.frame(Path, What)
http_methods %>%
knitr::kable() %>% kableExtra::kable_styling(bootstrap_options = c("striped", "hover", full_width = T))
```
### **R Object API**
```{r echo=FALSE}
Path <- c("../R/", "../data/", "../R/object")
What <- c('Lists R objects in the package or session', 'List data objects in a package', 'Read object in default format. If object is a function, it can be called using HTTP POST to run the function or HTTP GET to display the function')
http_methods <- data.frame(Path, What)
http_methods %>%
knitr::kable() %>% kableExtra::kable_styling(bootstrap_options = c("striped", "hover", full_width = T))
```
# B. Running a session
Each step of the pipeline have their own functions. These functions can be accessed via their respective packages. A guide on which packages and functions to use are shown below. {width="2000" height="339"}
::: {style="background-color:lightgray;"}
> To perform the quality control step:
>
> > POST ocpu/library/qualitycontrolseurat/R/qc_seurat
>
> where **qualitycontrolseurat** is the package name and **qc_seurat** is the function. Type "input" as the parameter name then upload the respective input file.
:::
### **API Endpoints for the pipeline**
The table lists the packages for the workflow and the function included per package. The <mark>**mandatory parameter name**</mark> for all the function is "**input**" and will only take a <mark>single input file or **mandatory parameter value**</mark>. The functions can be called using **POST** before the API Endpoint.
```{r echo=FALSE}
Package <- c('writeseuratobject', 'qualitycontrolseurat', 'qualitycontrolseurat', 'selectionscalingseurat', 'selectionscalingseurat', 'pcaseurat', 'nonlinearreduction', 'nonlinearreduction', 'nonlinearreduction')
Function <- c('write_seurat_object()', 'qc_seurat()', 'normalization_seurat()', 'variable_features()', 'scaling_seurat()', 'pca_seurat()', 'clusters_seurat()', 'tsne_seurat()', 'umap_seurat()')
input <- c('input', 'input', 'input', 'input', 'input', 'input', 'input', 'input', 'input')
input_value <- c('File with the ff. extensions: “.gz”, “.h5”, “.loom”, “.h5ad”', 'Seurat object with .rds extension/file named “seurat_object.rds”', 'Seurat object after QC/file named “afterQC_seurat.rds”', 'Seurat object after normalization/file named “normalized_seurat.rds”', 'Seurat object after selection of variable features/file named “variable_features.rds”', 'Seurat object after scaling/file named “after_scaling.rds”', 'Seurat object after scaling/file named “after_scaling.rds”', 'Seurat object after clustering/file named “clusters.rds”', 'Seurat object after clustering/file named “clusters.rds”')
endpoint <- c('../library/writeseuratobject/R/write_seurat_object', '../library/qualitycontrolseurat/R/qc_seurat', '../library/qualitycontrolseurat/R/normalization_seurat', '../library/selectionscalingseurat/R/variable_features', '../library/selectionscalingseurat/R/scaling_seurat', '../library/pcaseurat/R/pca_seurat', '../library/nonlinearreduction/R/clusters_seurat', '../library/nonlinearreduction/R/tsne_seurat', '../library/nonlinearreduction/R/umap_seurat')
http_methods <- data.frame(Package, Function, input, input_value, endpoint)
http_methods %>%
knitr::kable(
col.names = c("Package", "Function", "Mandatory parameter name", "Mandatory parameter value", "API Endpoint")
) %>% kableExtra::kable_styling(bootstrap_options = c("striped", "hover", full_width = T))
```
::: {style="background-color:lightgray;"}
> If using the packages from the beginning (writeseuratobject), the mandatory parameter value is the file generated from the step previous. See figure 2 as guide.
:::
### **Input, Output: Data and Formats**
These are the extensions of input and output files.
```{r echo=FALSE}
format <- c("h5", "gz", "loom", "h5ad", "rds", "png", "html")
content <- c('list/matrix', 'zipped matrix', 'matrix', 'annotation data', 'R data', 'image', 'html')
notes <- c("input for the writeseuratobject package", "input for the writeseuratobject package", "input for the writeseuratobject package", "input for the writeseuratobject package", "both input and output of packages", "output only", "output only")
http_methods <- data.frame(format, content, notes)
http_methods %>%
knitr::kable(
col.names = c("Format", "Content-type", "Notes")
) %>% kableExtra::kable_styling(bootstrap_options = c("striped", "hover", full_width = T))
```
# C. After running a session
### **R Sessions API**
```{r echo=FALSE}
format <- c("/ocpu/tmp/{key}/", "/ocpu/tmp/{key}/R", "/ocpu/tmp/{key}/graphics/", "/ocpu/tmp/{key}/graphics/{n}/{format}", "/ocpu/tmp/{key}/source", "/ocpu/tmp/{key}/stdout", "/ocpu/tmp/{key}/console", "/ocpu/tmp/{key}/zip", "/ocpu/tmp/{key}/tar", "/ocpu/tmp/{key}/files/*")
what <- c('List available output for this session', 'R objects stored in this session. Interface using R object API, same as objects in packages', 'Graphics (plots) stored in this session', 'Retrieve plot number {n} in output format {format}. Format is usually one of png, pdf, or svg. The {n} is an integer or “last”', 'Reads the input source code for this session', 'Shows text printed to STDOUT in this session', 'Shows the console input/output for this session (combines source and stdout)', 'Download the entire session as a zip archive', 'Download the entire session as a gzipped tarball', 'Interfaces to the file API in the working dir of the session')
http_methods <- data.frame(format, what)
http_methods %>%
knitr::kable(
col.names = c("Path", "What")
) %>% kableExtra::kable_styling(bootstrap_options = c("striped", "hover", full_width = T))
```
### **HTTP Status Codes**
The codes returned by OpenCPU
```{r echo=FALSE}
format <- c("200 OK", "201 Created", "302 Found", "400 Bad Request", "502 Bad Gateway", "503 Bad Request")
content <- c('On successful GET request', 'On successful POST request', 'Redirect', 'R raised an error', 'Nginx (opencpu-cache) can’t connect to OpenCPU server', 'Serious problem with the server')
notes <- c("Resource content", "Output location", "Redirect Location", " Error message in text/plain", "(Should be resolved by the administrator by looking in error logs)", "(Should be resolved by the administrator by looking in error logs)")
http_methods <- data.frame(format, content, notes)
http_methods %>%
knitr::kable(
col.names = c("HTTP Code", "When", "Returns")
) %>% kableExtra::kable_styling(bootstrap_options = c("striped", "hover", full_width = T))
```
### **API Outputs after running the steps of the pipeline**
+------------------------------------------------------------+--------------------------------------------------+
| API call endpoint | Responses/Output |
+============================================================+==================================================+
| ../library/writeseuratobject/R/write_seurat_object | /ocpu/tmp/{key}/R/write_seurat_object |
| | |
| | /ocpu/tmp/{key}/source |
| | |
| | /ocpu/tmp/{key}/console |
| | |
| | /ocpu/tmp/{key}/info |
| | |
| | /ocpu/tmp/{key}/files/DESCRIPTION |
| | |
| | /ocpu/tmp/{key}/files/seurat_object.rds |
| | |
| | /ocpu/tmp/{key}/files/{input file} |
+------------------------------------------------------------+--------------------------------------------------+
| ../library/qualitycontrolseurat/R/qc_seurat | /ocpu/tmp/{key}/qc_seurat |
| | |
| | /ocpu/tmp/{key}/source |
| | |
| | /ocpu/tmp/{key}/console |
| | |
| | /ocpu/tmp/{key}/info |
| | |
| | /ocpu/tmp/{key}/files/afterQC_seurat.rds |
| | |
| | /ocpu/tmp/{key}/files/DESCRIPTION |
| | |
| | /ocpu/tmp/{key}/files/seurat_object.rds |
| | |
| | /ocpu/tmp/{key}/files/violin_plot.png |
| | |
| | /ocpu/tmp/{key}/files/violin_plots.html |
| | |
| | /ocpu/tmp/{key}/files/scatter_plots.html |
+------------------------------------------------------------+--------------------------------------------------+
| ../library/qualitycontrolseurat/R/normalization_seurat | /ocpu/tmp/{key}/normalization_seurat |
| | |
| | /ocpu/tmp/{key}/source |
| | |
| | /ocpu/tmp/{key}/console |
| | |
| | /ocpu/tmp/{key}/info |
| | |
| | /ocpu/tmp/{key}/files/after_normalization.rds |
| | |
| | /ocpu/tmp/{key}/files/DESCRIPTION |
| | |
| | /ocpu/tmp/{key}/files/afterQC_seurat.rds |
+------------------------------------------------------------+--------------------------------------------------+
| ../library/selectionscalingseurat/R/variable_features | /ocpu/tmp/{key}/variable_features |
| | |
| | /ocpu/tmp/{key}/source |
| | |
| | /ocpu/tmp/{key}/console |
| | |
| | /ocpu/tmp/{key}/info |
| | |
| | /ocpu/tmp/{key}/files/normalized_seurat.rds |
| | |
| | /ocpu/tmp/{key}/files/DESCRIPTION |
| | |
| | /ocpu/tmp/{key}/files/top10_variablefeatures.rds |
| | |
| | /ocpu/tmp/{key}/files/variable_features_plot.png |
| | |
| | /ocpu/tmp/{key}/files/variablefeatures.rds |
+------------------------------------------------------------+--------------------------------------------------+
| ../library/selectionscalingseurat/R/scaling_seurat | /ocpu/tmp/{key}/scaling_seurat |
| | |
| | /ocpu/tmp/{key}/source |
| | |
| | /ocpu/tmp/{key}/console |
| | |
| | /ocpu/tmp/{key}/info |
| | |
| | /ocpu/tmp/{key}/files/after_scaling.rds |
| | |
| | /ocpu/tmp/{key}/files/normalized_seurat.rds |
| | |
| | /ocpu/tmp/{key}/files/DESCRIPTION |
+------------------------------------------------------------+--------------------------------------------------+
| ../library/pcaseurat/R/pca_seurat | /ocpu/tmp/{key}/pca_seurat |
| | |
| | /ocpu/tmp/{key}/source |
| | |
| | /ocpu/tmp/{key}/console |
| | |
| | /ocpu/tmp/{key}/info |
| | |
| | /ocpu/tmp/{key}/graphics/1 |
| | |
| | /ocpu/tmp/{key}/files/clusters.rds |
| | |
| | /ocpu/tmp/{key}/files/DESCRIPTION |
| | |
| | /ocpu/tmp/{key}/files/pca_2d_unlabeled.html |
| | |
| | /ocpu/tmp/{key}/files/pca03_2dplot_labeled.html |
| | |
| | /ocpu/tmp/{key}/files/pca01_2dplot_labeled.html |
| | |
| | /ocpu/tmp/{key}/files/pca_3dplot.html |
| | |
| | /ocpu/tmp/{key}/files/pca_results.png |
| | |
| | /ocpu/tmp/{key}/files/pca_plot_unlabeled.png |
| | |
| | /ocpu/tmp/{key}/files/heatmap.png |
| | |
| | /ocpu/tmp/{key}/files/heatmap_multiple.png |
| | |
| | /ocpu/tmp/{key}/files/elbowplot.png |
+------------------------------------------------------------+--------------------------------------------------+
| ../library/nonlinearreduction/R/clusters_seurat | /ocpu/tmp/{key}/clusters_seurat |
| | |
| | /ocpu/tmp/{key}/source |
| | |
| | /ocpu/tmp/{key}/console |
| | |
| | /ocpu/tmp/{key}/info |
| | |
| | /ocpu/tmp/{key}/files/DESCRIPTION |
| | |
| | /ocpu/tmp/{key}/files/after_scaling.rds |
| | |
| | /ocpu/tmp/{key}/files/clusters.rds |
+------------------------------------------------------------+--------------------------------------------------+
| ../library/nonlinearreduction/R/tsne_seurat | /ocpu/tmp/{key}/tsne_seurat |
| | |
| | /ocpu/tmp/{key}/source |
| | |
| | /ocpu/tmp/{key}/console |
| | |
| | /ocpu/tmp/{key}/info |
| | |
| | /ocpu/tmp/{key}/files/DESCRIPTION |
| | |
| | /ocpu/tmp/{key}/files/clusters.rds |
| | |
| | /ocpu/tmp/{key}/files/tsne_2dplot.html |
| | |
| | /ocpu/tmp/{key}/files/tsne_3dplot.html |
+------------------------------------------------------------+--------------------------------------------------+
| ../library/nonlinearreduction/R/umap_seurat | /ocpu/tmp/{key}/umap_seurat |
| | |
| | /ocpu/tmp/{key}/source |
| | |
| | /ocpu/tmp/{key}/console |
| | |
| | /ocpu/tmp/{key}/info |
| | |
| | /ocpu/tmp/{key}/files/DESCRIPTION |
| | |
| | /ocpu/tmp/{key}/files/clusters.rds |
| | |
| | /ocpu/tmp/{key}/files/umap_2dplot.html |
| | |
| | /ocpu/tmp/{key}/files/umap_3dplot.html |
+------------------------------------------------------------+--------------------------------------------------+