Qonnections 2019 Workshop Materials

Content for Qonnections 2019 hands-on workshops. Also, updates to initialization due to recent updates to some packages.

Qlik-Py-Init.bat

@@ -15,9 +15,9 @@ echo Installing required packages... & echo.
 python -m pip install --upgrade setuptools pip
 pip install grpcio grpcio-tools numpy scipy pandas cython joblib
 pip install pystan==2.17
-pip install fbprophet
+pip install fbprophet==0.4.post2
 pip install scikit-learn==0.20.3
-pip install hdbscan
+pip install hdbscan==0.8.22
 pip install skater==1.1.2
 echo.
 echo Creating a new firewall rule for TCP port 50055... & echo.

Qlik-Py-Init.ps1

@@ -9,9 +9,9 @@ Write-Output "Installing required packages..."
 python -m pip install --upgrade setuptools pip
 pip install grpcio grpcio-tools numpy scipy pandas cython joblib
 pip install pystan==2.17
-pip install fbprophet
+pip install fbprophet==0.4.post2
 pip install scikit-learn==0.20.3
-pip install hdbscan
+pip install hdbscan==0.8.22
 pip install skater==1.1.2
 Write-Output "Creating a new firewall rule for TCP port 50055..."
 netsh advfirewall firewall add rule name=Qlik-PyTools dir=in action=allow protocol=TCP localport=50055

README.md

@@ -9,6 +9,7 @@
 - [Pre-requisites](#pre-requisites)
 - [Installation](#installation)
 - [Usage](#usage)
+- [Qonnections 2019 Workshop](#qonnections-2019-workshop)
 
 
 ## Introduction
@@ -136,3 +137,9 @@ Sample Qlik Sense apps are provided and each app includes extensive techniques t
 | [Clustering](docs/Clustering.md) | [Sample App - Clustering with HDBSCAN](docs/Sample_App_Clustering.qvf) | Qlik Sense November 2018 or later with the Dashboard Extension Bundle. |
 | [Forecasting](docs/Prophet.md) | [Sample App - Facebook Prophet (Detailed)](docs/Sample_App_Prophet.qvf)<br><br>[Sample App - Facebook Prophet (Simple)](docs/Sample_App_Forecasting_Simple.qvf) | Qlik Sense November 2018 or later with the Dashboard Extension Bundle. <br/><br/>For the detailed app, use the bookmarks to step through the sheets with relevant selections.<br><br>For calling Prophet through the load script refer to the simple app. If you want to reload the app using Qlik Sense Desktop you will need to download the [data source](docs/VIC-Emergency-Department-Attendances.xlsx), create a data connection named `AttachedFiles` in the app, and point the connection to the folder containing the source file. |
 | [Machine Learning](docs/scikit-learn.md) | [Sample App - Train & Test](docs/Sample-App-scikit-learn-Train-Test.qvf)<br><br>[Sample App - Predict](docs/Sample-App-scikit-learn-Predict.qvf)<br><br>[Sample App - K-fold Cross Validation](docs/Sample-App-scikit-learn-K-fold-Cross-Validation.qvf)<br><br>[Sample App - Parameter Tuning](docs/Sample-App-scikit-learn-Parameter-Tuning.qvf)<br><br>[Sample App - K-fold CV & Parameter Tuning](docs/Sample-App-scikit-learn-K-fold-CV-Grid-Search.qvf) | Make sure you reload the K-fold Cross Validation or Train & Test app before using the Predict app.<br><br>If using Qlik Sense Desktop you will need to download the [data source](docs/HR-Employee-Attrition.xlsx), create a data connection named `AttachedFiles` in the app, and point the connection to the folder containing the source file.<br><br>For the Prediction app you will need Qlik Sense November 2018 or later with the Dashboard Extension Bundle. |
+
+## Qonnections 2019 Workshop
+
+At Qonnections 2019 we ran hands-on workshops with PyTools and Qlik Sense. The content for these workshops, including the sample apps and exercise intructions, is available [here](docs/hands-on/README.md).
+
+The workshop exercises can be used as a tutorial for using this Server Side Extension with Qlik Sense Enterprise or Desktop.

core/__main__.py

@@ -16,7 +16,7 @@
 import ServerSideExtension_pb2 as SSE
 import grpc
 
-# Turn off warnings by default
+# Suppress warnings
 if not sys.warnoptions:
     warnings.simplefilter("ignore")
 

core/_clustering.py

@@ -3,12 +3,18 @@
 import time
 import string
 import locale
+import warnings
 import numpy as np
 import pandas as pd
-import hdbscan
 import _utils as utils
 import ServerSideExtension_pb2 as SSE
 
+# Suppress warnings
+if not sys.warnoptions:
+    warnings.simplefilter("ignore")
+
+import hdbscan
+
 # Add Generated folder to module path
 PARENT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 sys.path.append(os.path.join(PARENT_DIR, 'generated'))

core/_machine_learning.py

@@ -3,6 +3,12 @@
 import joblib
 import numpy as np
 import pandas as pd
+import warnings
+
+# Suppress warnings 
+if not sys.warnoptions:
+    warnings.simplefilter("ignore")
+
 from pathlib import Path
 from sklearn import preprocessing
 from sklearn.base import TransformerMixin

core/_prophet.py

@@ -3,12 +3,18 @@
 import time
 import string
 import locale
+import warnings
 import numpy as np
 import pandas as pd
-from fbprophet import Prophet, plot
 import _utils as utils
 import ServerSideExtension_pb2 as SSE
 
+# Suppress warnings
+if not sys.warnoptions:
+    warnings.simplefilter("ignore")
+
+from fbprophet import Prophet, plot
+
 # Add Generated folder to module path.
 PARENT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 sys.path.append(os.path.join(PARENT_DIR, 'generated'))

core/_sklearn.py

@@ -10,7 +10,7 @@
 import pandas as pd
 from collections import OrderedDict
 
-# Turn off warnings by default
+# Suppress warnings
 if not sys.warnoptions:
     warnings.simplefilter("ignore")
 
@@ -23,27 +23,27 @@
 from sklearn.model_selection import GridSearchCV
 
 from sklearn.decomposition import PCA, KernelPCA, IncrementalPCA, TruncatedSVD, FactorAnalysis, FastICA, NMF, SparsePCA,\
-                                  DictionaryLearning, LatentDirichletAllocation, MiniBatchDictionaryLearning, MiniBatchSparsePCA
+                                DictionaryLearning, LatentDirichletAllocation, MiniBatchDictionaryLearning, MiniBatchSparsePCA
 
 from sklearn.dummy import DummyClassifier, DummyRegressor
 from sklearn.ensemble import AdaBoostClassifier, AdaBoostRegressor, BaggingClassifier,\
-                             BaggingRegressor, ExtraTreesClassifier, ExtraTreesRegressor,\
-                             GradientBoostingClassifier, GradientBoostingRegressor,\
-                             RandomForestClassifier, RandomForestRegressor, VotingClassifier
+                            BaggingRegressor, ExtraTreesClassifier, ExtraTreesRegressor,\
+                            GradientBoostingClassifier, GradientBoostingRegressor,\
+                            RandomForestClassifier, RandomForestRegressor, VotingClassifier
 from sklearn.gaussian_process import GaussianProcessClassifier, GaussianProcessRegressor
 from sklearn.linear_model import ARDRegression, BayesianRidge, ElasticNet, ElasticNetCV, HuberRegressor, Lars, LarsCV,\
-                                 Lasso, LassoCV, LassoLars, LassoLarsCV, LassoLarsIC, LinearRegression, LogisticRegression,\
-                                 LogisticRegressionCV, MultiTaskLasso, MultiTaskElasticNet, MultiTaskLassoCV, MultiTaskElasticNetCV,\
-                                 OrthogonalMatchingPursuit, OrthogonalMatchingPursuitCV, PassiveAggressiveClassifier,\
-                                 PassiveAggressiveRegressor, Perceptron, RANSACRegressor, Ridge, RidgeClassifier, RidgeCV,\
-                                 RidgeClassifierCV, SGDClassifier, SGDRegressor, TheilSenRegressor
+                                Lasso, LassoCV, LassoLars, LassoLarsCV, LassoLarsIC, LinearRegression, LogisticRegression,\
+                                LogisticRegressionCV, MultiTaskLasso, MultiTaskElasticNet, MultiTaskLassoCV, MultiTaskElasticNetCV,\
+                                OrthogonalMatchingPursuit, OrthogonalMatchingPursuitCV, PassiveAggressiveClassifier,\
+                                PassiveAggressiveRegressor, Perceptron, RANSACRegressor, Ridge, RidgeClassifier, RidgeCV,\
+                                RidgeClassifierCV, SGDClassifier, SGDRegressor, TheilSenRegressor
 from sklearn.naive_bayes import BernoulliNB, GaussianNB, MultinomialNB
 from sklearn.neighbors import KNeighborsClassifier, KNeighborsRegressor, RadiusNeighborsClassifier,\
-                              RadiusNeighborsRegressor
+                            RadiusNeighborsRegressor
 from sklearn.neural_network import BernoulliRBM, MLPClassifier, MLPRegressor
 from sklearn.svm import LinearSVC, LinearSVR, NuSVC, NuSVR, SVC, SVR
 from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor, ExtraTreeClassifier,\
-                         ExtraTreeRegressor
+                        ExtraTreeRegressor
 
 from sklearn.cluster import AffinityPropagation, AgglomerativeClustering, Birch, DBSCAN, FeatureAgglomeration, KMeans,\
                             MiniBatchKMeans, MeanShift, SpectralClustering

core/_utils.py

@@ -3,11 +3,17 @@
 import ast
 import string
 import locale
+import warnings
 import numpy as np
 import pandas as pd
-from sklearn import preprocessing
 import ServerSideExtension_pb2 as SSE
 
+# Suppress warnings 
+if not sys.warnoptions:
+    warnings.simplefilter("ignore")
+
+from sklearn import preprocessing
+
 # Add Generated folder to module path.
 PARENT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 sys.path.append(os.path.join(PARENT_DIR, 'generated'))

docs/README.md

@@ -9,6 +9,7 @@
 - [Pre-requisites](#pre-requisites)
 - [Installation](#installation)
 - [Usage](#usage)
+- [Qonnections 2019 Workshop](#qonnections-2019-workshop)
 
 
 ## Introduction
@@ -134,3 +135,9 @@ Sample Qlik Sense apps are provided and each app includes extensive techniques t
 | [Clustering](Clustering.md) | [Sample App - Clustering with HDBSCAN](Sample_App_Clustering.qvf) | Qlik Sense November 2018 or later with the Dashboard Extension Bundle. |
 | [Forecasting](Prophet.md) | [Sample App - Facebook Prophet (Detailed)](Sample_App_Prophet.qvf)<br><br>[Sample App - Facebook Prophet (Simple)](Sample_App_Forecasting_Simple.qvf) | Qlik Sense November 2018 or later with the Dashboard Extension Bundle. <br/><br/>For the detailed app, use the bookmarks to step through the sheets with relevant selections.<br><br>For calling Prophet through the load script refer to the simple app. If you want to reload the app using Qlik Sense Desktop you will need to download the [data source](VIC-Emergency-Department-Attendances.xlsx), create a data connection named `AttachedFiles` in the app, and point the connection to the folder containing the source file. |
 | [Machine Learning](scikit-learn.md) | [Sample App - Train & Test](Sample-App-scikit-learn-Train-Test.qvf)<br><br>[Sample App - Predict](Sample-App-scikit-learn-Predict.qvf)<br><br>[Sample App - K-fold Cross Validation](Sample-App-scikit-learn-K-fold-Cross-Validation.qvf)<br><br>[Sample App - Parameter Tuning](Sample-App-scikit-learn-Parameter-Tuning.qvf)<br><br>[Sample App - K-fold CV & Parameter Tuning](Sample-App-scikit-learn-K-fold-CV-Grid-Search.qvf) | Make sure you reload the K-fold Cross Validation or Train & Test app before using the Predict app.<br><br>If using Qlik Sense Desktop you will need to download the [data source](HR-Employee-Attrition.xlsx), create a data connection named `AttachedFiles` in the app, and point the connection to the folder containing the source file.<br><br>For the Prediction app you will need Qlik Sense November 2018 or later with the Dashboard Extension Bundle. |
+
+## Qonnections 2019 Workshop
+
+At Qonnections 2019 we ran hands-on workshops with PyTools and Qlik Sense. The content for these workshops, including the sample apps and exercise intructions, is available [here](hands-on/README.md).
+
+The workshop exercises can be used as a tutorial for using this Server Side Extension with Qlik Sense Enterprise or Desktop.