Merge pull request #16 from SinclairGurny/0.6_improvements

reorganized files and updated instructions

CHANGELOG.md

@@ -0,0 +1,62 @@
+# Detailed PILOTS README (for PILOTS users)
+
+## Changelog
+
+In version 0.6, we added the following new features to the PILOTS grammar:
+* Support for supervised machine learning with `trainer` grammar.
+* Support for adding new machine learning algorithms.
+* Introduction of a new compiler flag to compile `trainer` (.trn) files.
+* Take-off weight estimation examples.
+
+In version 0.5, we added the following new features to the PILOTS grammar:
+* Support for multiple models of analytical redundancy (for details, see our [DASC2019 paper](http://wcl.cs.rpi.edu/papers/DASC2019_imai.pdf)). Example programs to support this feature are available under [examples/multi_redundancy](./examples/multi_redundancy) directory.
+  - Support for multiple error values under the `errors` section.
+  - Support for multiple `estimate` clauses under the `signatures`and `modes` sections
+  (as a side effect of this new feature, the `when` clause is now deprecated).
+  - Enhancement of the `modes` section to support general boolean expressions.
+  - Scoped naming for output variables using the `namespace` command line option (e.g., output variable `x`   from a child program with `--namespace A` option can be referred to as `A.x` in the parent program).
+    Example is available [here](./examples/multi_redundancy/aoaspeedcheck3/run_speedcheck).
+* Reserved `mode` variable to store the estimated mode.
+* Introduction of `constants` section to declare constant values.
+* Support for `x^n` expression to denote the n-th power of x.
+
+In version 0.5, there are some new features to the PILOTS runtime system and its utilities:
+* Arguments to PILOTS programs are now handled by argparse4j.
+* Log messaging with Java Logging APIs. Logging levels are configurable through
+  [`logging.properties`](logging.properties).
+* [`MultiChartsServer`](./pilots/util/MultiChartsServer.java) to show multiple plots in one window. Settings for multiple plots can be configured through a single yaml file ([example yaml file](./examples/airfrancesim/charts_conf.yaml)). Due to this functionality, JFreeChart and JCommon must be updated to v1.0.19 and v1.0.23 respectively.
+
+
+## Limitations
+
+* Language only supports error signatures that are a constant, or a linear function. Examples are:
+  - S: e = 100;
+  - S(K): e = K, abs(K) > 25;
+  - S(K): e = 2*t + K, K < 0, k > -100;
+
+* The `when` clause introduced in our [Cluster Computing journal article](http://wcl.cs.rpi.edu/papers/pilots-cluster.pdf) is deprecated due to the support for multiple `estimate` clauses under the `signatures` and `modes` sections in v0.5. The previous implementation of the `when` clause assumed that there is only one `estimate` clause per signature, but we decided to give priority to the support for multiple `estimate` clauses in v0.5.
+
+
+## Future Work
+
+* Examples of potential additions for federated learning and methods to create ensemble models can be seen in the [future work](https://github.com/RPI-WCL/pilots/tree/master/examples/future_work) example
+
+* Types of error signatures we would like to support eventually:
+    -   S(K): e = K
+    -   S(K): e = K*t^2
+    -   S:    e = t
+    -   S(K1,K2): e = K1*t + K2, K1 != 0, K2 > threshold
+
+* Better syntax support for intervals (in signature constraints)
+    - i.e., K in (0,10) or K not in (-10,10)
+
+* Simulation mode with support for dynamic simulation inputs
+    - Multiple files, each corresponding to a particular input
+    - Different directories for the different times
+    - Forecast data
+
+* Enhanced error analysis accuracy
+    - Different interpolation methods
+    - More types of error signatures
+
+* Resolve the limitation for the `when` clause described above

README.md

@@ -1,36 +1,16 @@
-# README for PILOTS ver 0.6
+# PILOTS ver 0.6
 
 PILOTS (**P**rogramm**I**ng **L**anguage for spati**O**-**T**emporal data **S**treaming applications) is a highly declarative programming language for spatio-temporal streaming applications.
 It is capable of detecting and correcting data errors through user-defined *error signatures*.
 
 PILOTS has been successfully applied to avionics applications. Most notably, we have shown that PILOTS can fix data errors caused by pitot tube sensor failures which occurred in [Air France Flight 447 accident](http://wcl.cs.rpi.edu/papers/bdse2013.pdf). For more information, visit [the PILOTS web site](http://wcl.cs.rpi.edu/pilots/) and look at [related papers](https://wcl.cs.rpi.edu/bib/Keyword/DATA-STREAMING.html).
 
-In version 0.6, we added the following new features to the PILOTS grammar:
-* Support for supervised machine learning with `trainer` grammar.
-* Support for adding new machine learning algorithms.
-* Introduction of a new compiler flag to compile `trainer` (.trn) files.
-* Take-off weight estimation examples.
-
-In version 0.5, we added the following new features to the PILOTS grammar:
-* Support for multiple models of analytical redundancy (for details, see our [DASC2019 paper](http://wcl.cs.rpi.edu/papers/DASC2019_imai.pdf)). Example programs to support this feature are available under [examples/multi_redundancy](./examples/multi_redundancy) directory.
-  - Support for multiple error values under the `errors` section.
-  - Support for multiple `estimate` clauses under the `signatures`and `modes` sections
-  (as a side effect of this new feature, the `when` clause is now deprecated).
-  - Enhancement of the `modes` section to support general boolean expressions.
-  - Scoped naming for output variables using the `namespace` command line option (e.g., output variable `x`   from a child program with `--namespace A` option can be referred to as `A.x` in the parent program).
-    Example is available [here](./examples/multi_redundancy/aoaspeedcheck3/run_speedcheck).
-* Reserved `mode` variable to store the estimated mode.
-* Introduction of `constants` section to declare constant values.
-* Support for `x^n` expression to denote the n-th power of x.
-
-In version 0.5, there are some new features to the PILOTS runtime system and its utilities:
-* Arguments to PILOTS programs are now handled by argparse4j.
-* Log messaging with Java Logging APIs. Logging levels are configurable through
-  [`logging.properties`](logging.properties).
-* [`MultiChartsServer`](./pilots/util/MultiChartsServer.java) to show multiple plots in one window. Settings for multiple plots can be configured through a single yaml file ([example yaml file](./examples/airfrancesim/charts_conf.yaml)). Due to this functionality, JFreeChart and JCommon must be updated to v1.0.19 and v1.0.23 respectively.
-
 *Note: The following commands shown for the command line are assumed to be implemented in the bash shell* 
 
+## 0. Prerequisite Knowledge
+
+Basics of using a terminal
+Installing programs for use on the terminal
 
 ## 1. Software Requirements
 
@@ -39,7 +19,11 @@ In version 0.5, there are some new features to the PILOTS runtime system and its
 * (Optional) [JavaCC](http://javacc.org/) if you want to modify the PILOTS grammar.
 For machine learning component of PILOTS:
 * Python 3
-* Python libraries: numpy, scipy, scikit-learnnumpy, pickle, and flask
+* Python libraries: numpy, scipy, scikit-learn, pickle, and flask
+Using pip they can be installed using the following line:
+```
+pip3 install numpy scipy scikit-learn pickle flask
+```
 
 ## 2. Downloading PILOTS Library
 
@@ -65,6 +49,7 @@ For machine learning component of PILOTS:
   export CLASSPATH=.:$PILOTS_HOME/lib/*
   alias plc='java pilots.compiler.PilotsCompiler'
   alias plcsim='java pilots.compiler.PilotsCompiler --sim'
+  alias pltserver='$PILOTS_HOME/pilots/util/model/server.sh'
   ```
 
 * **Building a PILOTS jar file**
@@ -75,46 +60,20 @@ For machine learning component of PILOTS:
   ~~~
   pilots.jar will be created under the `$PILOTS_HOME/lib` directory.
 
+## 4. Running a PILOTS program
 
-* **(Optional) Model setup**
-
-	In PILOTS, the model function is a function which uses a machine learning model to generate an output based on given input. If you decide to use the models in PILOTS, please see [pilots/util/model/](https://github.com/RPI-WCL/pilots/tree/master/pilots/util/model) for instruction.
-
-* **Running example PILOTS programs**: See [PILOTS tutorial](https://wcl.cs.rpi.edu/pilots/tutorial/index.html).
-
-* **Running examples with learning model**: See the READMEs in examples such as [Prediction Twice](https://github.com/RPI-WCL/pilots/tree/master/examples/prediction_twice) or [N4207P Sim](https://github.com/RPI-WCL/pilots/tree/master/examples/N4207P_sim).
-
-
-## 4. Limitations
-
-* Language only supports error signatures that are a constant, or a linear function. Examples are:
-  - S: e = 100;
-  - S(K): e = K, abs(K) > 25;
-  - S(K): e = 2*t + K, K < 0, k > -100;
-
-* The `when` clause introduced in our [Cluster Computing journal article](http://wcl.cs.rpi.edu/papers/pilots-cluster.pdf) is deprecated due to the support for multiple `estimate` clauses under the `signatures` and `modes` sections in v0.5. The previous implementation of the `when` clause assumed that there is only one `estimate` clause per signature, but we decided to give priority to the support for multiple `estimate` clauses in v0.5. 
-
-
-## 5. Future Work
+* First read about PILOTS in the [wiki](https://github.com/RPI-WCL/pilots/wiki)
+* Follow along the README in a simple program: [Twice](https://github.com/RPI-WCL/pilots/tree/master/examples/twice), [Twice Sim](https://github.com/RPI-WCL/pilots/tree/master/examples/twicesim)
+* Try some other programs [Aoacheck](https://github.com/RPI-WCL/pilots/tree/master/examples/aoacheck)
 
-* Examples of potential additions for federated learnign and methods to create ensemble models can be seen in the [future work](https://github.com/RPI-WCL/pilots/tree/master/examples/future_work) example
+## 5. Running a PILOTS program with machine learning
 
-* Types of error signatures we would like to support eventually:
-    -   S(K): e = K
-    -   S(K): e = K*t^2
-    -   S:    e = t
-    -   S(K1,K2): e = K1*t + K2, K1 != 0, K2 > threshold
+In PILOTS, the __model__ function is a function which uses a machine learning model to generate an output based on a given input. 
 
-* Better syntax support for intervals (in signature constraints)
-    - i.e., K in (0,10) or K not in (-10,10)
+* First try the [Prediction Twice](https://github.com/RPI-WCL/pilots/tree/master/examples/prediction_twice) program
+* Try the [N4207P Sim](https://github.com/RPI-WCL/pilots/tree/master/examples/N4207P_sim) program
+* Experiment with the [Takeoff Weight Estimation](https://github.com/RPI-WCL/pilots/tree/master/examples/takeoff_weight_est) program
+* Try making your own machine learning algorithm, please see [pilots/util/model/](https://github.com/RPI-WCL/pilots/tree/master/models) for instruction
 
-* Simulation mode with support for dynamic simulation inputs
-    - Multiple files, each corresponding to a particular input
-    - Different directories for the different times
-    - Forecast data
 
-* Enhanced error analysis accuracy
-    - Different interpolation methods
-    - More types of error signatures
 
-* Resolve the limitation for the `when` clause described in [Limitations](#4-limitations)

examples/N4207P_sim/N4207P.plt

@@ -1,4 +1,4 @@
-program weight_tester_err;
+program N4207P;
     inputs
         v_a, prs, tmp, alt, curr_w (t) using closest (t);
 	est_w (t) using model(weight_model_exp, v_a, prs, tmp, alt);

examples/N4207P_sim/README.md

@@ -7,14 +7,13 @@ To train this model, we use two seperate terminals.
 
 In **Terminal 1**, compile the weight estimation model using the commands:
 ```
-$ plc -t weight_exp.trn
+$ plc -t Weight_exp.trn
 $ javac Weight_model_exp.java
 ```
 
 In **Terminal 2**, launch the machine learning server [server.sh](../../pilots/util/model/server.sh):
 ```
-$ cd ../../pilots/util/model
-$ ./server.sh
+$ pltserver
 ```
 
 In **Terminal 2**, run the training programs:
@@ -40,8 +39,7 @@ $ ./outputHandler
 
 In **Terminal 4**, launch the machine learning server [server.sh](../../pilots/util/model/server.sh):
 ```
-$ cd ../../pilots/util/model
-$ ./server.sh
+$ pltserver
 ```
 
 In **Terminal 1**, run the compiled PILOTS program using the command:

examples/N4207P_sim/Weight_exp.trn

@@ -0,0 +1,25 @@
+trainer weight_model_exp;
+  data
+    /* N4207P_less.csv is another file with less training data,
+     * using N4207P_less.csv model must interpolate from data that
+     * less resembles current conditions
+     */
+    v_a, prs, tmp, alt, curr_w using
+      file("TakeoffWeightExperiment/N4207P.csv", 
+            vind_kias,prs,tmp,alt_msl,curr_w );
+  model
+    features: v_a, prs, tmp, alt;
+    labels: curr_w;
+    algorithm:
+      WeightEstimator(verbose: false, cutoff: 0.0, da_close: 0.1, w_close: 0.1);
+        /* verbose - progress messages (default: false),
+         * cutoff - ignore all datapoints with airspeed less than cutoff (default: 0.0)
+         *          [used to ignore slow starting acceleration]
+         * da_close - percent difference in values to estimate delta_acc (default: 0.1%)
+         *           [used to improve ability to account for changes in weight,
+         *             larger values means dataset is less uniform]
+         * w_close - perfect difference in values to estimate delta_da (default: 0.1%)
+         *           [used to improve ability to account for changes in density altitude
+         *             larger values means dataset is less uniform]
+         */
+end;

examples/N4207P_sim/run

@@ -1,6 +1,6 @@
 java -classpath .:$PILOTS_HOME/lib/* \
     -Djava.util.logging.config.file="$PILOTS_HOME/logging.properties" \
-    Weight_tester_err \
+    N4207P \
     --currloctime pilots.runtime.SimTimeService \
     --timerange "2020-02-08 133400-0500~2020-02-08 133427-0500" \
     --inputport 8888 \

examples/airfrancesim/README.md

@@ -8,9 +8,9 @@ $ plcsim AirFrance.plt
 $ javac AirFrance.java
 ```
 
-In **Terminal 2**, launch [chartserver](./chartserver):
+In **Terminal 2**, launch [outputHandler](./outputHandler):
 ```
-$ ./chartserver
+$ ./outputHandler
 ```
 
 In **Terminal 1**, run the compiled PILOTS program using the command:

examples/aoacheck/README.md

@@ -8,9 +8,9 @@ $ plc AoaCheck.plt
 $ javac AoaCheck.java
 ```
 
-In **Terminal 2**, launch [chartserver](./chartserver):
+In **Terminal 2**, launch [outputHandler](./outputHandler):
 ```
-$ ./chartserver
+$ ./outputHandler
 ```
 
 In **Terminal 1**, run the compiled PILOTS program using the command:

examples/deprecated/squared/Squared.java

@@ -0,0 +1,60 @@
+import java.util.*;
+import java.util.logging.*;
+import java.text.*;
+import java.net.Socket;
+import pilots.runtime.*;
+import pilots.runtime.errsig.*;
+
+public class Squared extends PilotsRuntime {
+    private static Logger LOGGER = Logger.getLogger(Squared.class.getName());
+    private int currentMode;
+    private int currentModeCount;
+    private Timer timer;
+    private long[] nextSendTimes;
+
+
+    public Squared(String args[]) {
+        super(args);
+
+        timer = new Timer();
+        nextSendTimes = new long[1];
+        Arrays.fill(nextSendTimes, 0L);
+    }
+
+    public void produceOutputs() {
+        try {
+            openOutput(0, "o");
+        } catch (Exception ex) {
+            ex.printStackTrace();
+        }
+
+        final int interval = 1000;
+        Map<String, Double> data = new HashMap<>();
+        timer.scheduleAtFixedRate(new TimerTask() {
+            public void run() {
+                // Inputs
+                data.put("x", getData("x", new Method(Method.CLOSEST, "t")));
+                LOGGER.fine("Inputs: " + "x=" + data.get("x"));
+
+                // Outputs computation
+                data.put("o", data.get("x")*data.get("x"));
+
+                // Data transfer
+                Date now = getTime();
+                try {
+                    sendData(0, data.get("o"));
+                    LOGGER.info("Outputs: " + now + " " + "o=" + data.get("o") + " ");
+                } catch (Exception ex) {
+                    ex.printStackTrace();
+                }
+
+            }
+        }, 0, interval);
+    }
+
+    public static void main(String[] args) {
+        Squared app = new Squared(args);
+        app.startServer();
+        app.produceOutputs();
+    }
+}

examples/future_work/ensemble_methods.plt → examples/future_work/Ensemble_methods.plt

@@ -1,4 +1,4 @@
-program  ensemble;
+program ensemble_methods;
   constants
     w1 = 0.2;
     w2 = 0.5;

examples/future_work/README.md

@@ -6,12 +6,12 @@ As proposed in **Learning Models from Avionics Data Streams**.
 
 ## Federated Learning
 
-[federated_trainer.trn](./federated_trainer.trn) shows an example of a PILOTS program which trains a neural network
+[Federated_trainer.trn](./Federated_trainer.trn) shows an example of a PILOTS program which trains a neural network
 across 10 different computation nodes over 20 rounds using the Federated Averaging algorithm.
 
 ## Ensemble Models
 
-[ensemble_methods.plt](./ensemble_methods.plt) shows the proposed ensemble methods: __average__, __majority__, __weighted__, and __decision__.
+[Ensemble_methods.plt](./Ensemble_methods.plt) shows the proposed ensemble methods: __average__, __majority__, __weighted__, and __decision__.
 The program combines the results of three different models which calculate density altitude.
 Currently, this functionality can be performed using the EnsembleMethods model.
 However, the **EnsembleMethods** models lacks the clarity and simplicity shown in **Ensemble_methods.plt**.

examples/multi_redundancy/aoaspeedcheck1/README.md

@@ -8,12 +8,12 @@ $ plcsim AoaSpeedCheck1.plt
 $ javac AoaSpeedCheck1.java
 ```
 
-In **Terminal 2**, launch [chartserver](./chartserver):
+In **Terminal 2**, launch [outputHandler](./outputHandler):
 ```
-$ ./chartserver
+$ ./outputHandler
 ```
 
-In **Terminal 3**, in order to show the ground truth data, execute [groundtruth_producer](./groundtruth_producer) (this will input data directly to chartserver):
+In **Terminal 3**, in order to show the ground truth data, execute [groundtruth_producer](./groundtruth_producer) (this will input data directly to outputHandler):
 ```
 $ ./groundtruth_producer
 ```