Skip to content

Commit 8f8fdf1

Browse files
committed
Merge branch 'dev-rpackage' into 'master'
Beta release 0.1.0 See merge request eoc_foundation_wip/analysis-pipelines!5
2 parents c8b35bc + ca4cda6 commit 8f8fdf1

File tree

87 files changed

+4203
-2506
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

87 files changed

+4203
-2506
lines changed

DESCRIPTION

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,29 @@
11
Package: analysisPipelines
22
Type: Package
3-
Title: Put analyis reports and pipelines into production
3+
Title: Compose interoperable analysis pipelines, and put them into production
44
Version: 0.1.0
5-
Authors@R:c(
5+
Authors@R: c(
66
person("Naren","Srinivasan", email = "Naren.Srinivasan@mu-sigma.com", role = c("cre","aut")),
7-
person("Neeratyoy","Mallik", email = "Neeratyoy.Mallik@mu-sigma.com", role = c("aut")),
8-
person("Sanjay","", email = "Sanjay@mu-sigma.com", role = c("aut")),
9-
person("Anoop S","", email = "Anoop.S@mu-sigma.com", role = c("ctb")),
10-
person("Vedavyas C","", email = "Vedavyas.C@mu-sigma.com", role = c("ctb"))
7+
person("Sanjay","", email = "Sanjay@mu-sigma.com", role = c("ctb")),
8+
person("Neeratyoy","Mallik", email = "Neeratyoy.Mallik@mu-sigma.com", role = c("ctb")),
9+
person("Anoop S","", email = "Anoop.S@mu-sigma.com", role = c("ctb"))
1110
)
12-
Maintainer: Naren Srinivasan <Naren.Srinivasan@mu-sigma.com>
13-
Description: This package allows data scientists to compose and generate reports as a set of analytical operations. The sequence of generation can be stored as pipelines and reused, specifically for production systems where these tasks are run repetitively. Additionally, the package implements a form of lazy evaluation where the pipelines are run on datasets only when outputs/ reports need to be generated. The package also has functions implemented for working with Spark through SparkR for both traditional Spark jobs on Spark DataFrames, as well as Spark Structured Streaming.
11+
Description: The package aims at enabling data scientists to compose pipelines of analysis which consist of data manipulation, exploratory analysis & reporting, as well as modeling steps. It also aims to enable data scientists to use tools of their choice through an R interface, and compose interoperable pipelines between R, Spark, and Python.
1412
Depends: R (>= 3.4.0), tibble, magrittr, data.table, pipeR, devtools
15-
Imports: ggplot2, dplyr
16-
Suggests: plotly, knitr, rmarkdown, SparkR
13+
Imports: ggplot2, dplyr, futile.logger
14+
Suggests: plotly, knitr, rmarkdown, SparkR, parallel, visNetwork, rjson, DT
1715
Remotes: github::cran/SparkR
1816
Encoding: UTF-8
1917
License: Apache License 2.0
2018
LazyLoad: yes
2119
LazyData: yes
2220
RoxygenNote: 6.0.1
2321
VignetteBuilder: knitr
22+
Collate:
23+
'analysisPipelines_package.R'
24+
'core-functions.R'
25+
'core-functions-batch.R'
26+
'core-streaming-functions.R'
27+
'r-batch-eda-utilities.R'
28+
'spark-structured-streaming-utilities.R'
29+
'zzz.R'

NAMESPACE

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,40 @@
1-
exportPattern("^[[:alpha:]]+")
1+
# Generated by roxygen2: do not edit by hand
2+
3+
export(AnalysisPipeline)
4+
export(BaseAnalysisPipeline)
5+
export(CheckColumnType)
6+
export(StreamingAnalysisPipeline)
7+
export(assessEngineSetUp)
8+
export(bivarPlots)
9+
export(castKafkaStreamAsString)
10+
export(convertKafkaValueFromJson)
11+
export(correlationMatPlot)
12+
export(generateReport)
13+
export(genericPipelineException)
14+
export(getDatatype)
15+
export(getInput)
16+
export(getLoggerDetails)
17+
export(getOutputById)
18+
export(getPipeline)
19+
export(getRegistry)
20+
export(ignoreCols)
21+
export(loadPipeline)
22+
export(loadPredefinedFunctionRegistry)
23+
export(multiVarOutlierPlot)
24+
export(outlierPlot)
25+
export(prepExecution)
26+
export(registerFunction)
27+
export(savePipeline)
28+
export(setInput)
29+
export(setLoggerDetails)
30+
export(sparkRSessionCreateIfNotPresent)
31+
export(univarCatDistPlots)
32+
export(updateObject)
33+
export(visualizePipeline)
34+
exportClasses(AnalysisPipeline)
35+
exportClasses(BaseAnalysisPipeline)
36+
exportClasses(StreamingAnalysisPipeline)
37+
exportMethods(checkSchemaMatch)
38+
exportMethods(generateOutput)
39+
exportMethods(initialize)
40+
import(SparkR)

0 commit comments

Comments
 (0)