Tests
Below we detail some preliminary tests conducted on three MyPart prototypes.
- Smoke tests overview
- Smoke test 1
- Smoke test 2
- PSL sanity check
- PSL voltage threshold test
- Real World Test
- Discussion
- Future Tests
One of the particle sources we tested our sensors with was wood smoke. The size distribution for burning wood tends to be in the sub-micron range, which is particularly important to sense accurately with respect to human health. In addition, these particles are easy to generate and decay naturally after being injected into our chamber, which allows us to collect a large amount of data across a broad range of concentrations.
We ran two tests with smoke in our testing chamber to find the correlation between MyPart sensors and commercial sensors. To produce smoke, we set fire to a wooden skewer, blow the flame out, then waft a small quantity of smoke into the chamber before sealing the lid. A set of fans inside the chamber mix the particulates. The sensors run an automated test sequence, all sampling once every minute, until the particles in the chamber decay to ambient concentrations.
| Device 1 | Device 2 | r^2 | slope |
|---|---|---|---|
| MetOne | MyPart 0 | 0.99074 | 0.016 |
| MetOne | MyPart 1 | 0.99657 | 0.0465 |
| MetOne | MyPart 2 | 0.99381 | 0.0401 |
| MyPart 0 | MyPart 1 | 0.99267 | 2.8933 |
| MyPart 0 | MyPart 2 | 0.99038 | 2.4938 |
| MyPart 1 | MyPart 2 | 0.99641 | 0.8616 |
We see very strong correlation between the three MyPart sensors and the MetOne. In addition, we see very strong inter-device correlations between the MyPart sensors.
Sensor 0 is seeing only 30-40% of the particles seen by the sensors 1 and 2. Of the three sensors fabricated, sensor 0 also had the highest noise floor. Though the source of the higher noise is currently unknown, it is likely the cause of the decreased sensitivity.
| Device 1 | Device 2 | r^2 | slope |
|---|---|---|---|
| MetOne | MyPart 0 | 0.98355 | 0.0094 |
| MetOne | MyPart 1 | 0.99355 | .0442 |
| MetOne | MyPart 2 | 0.98992 | .04 |
| MyPart 0 | MyPart 1 | 0.99403 | 4.6847 |
| MyPart 0 | MyPart 2 | 0.99484 | 4.2443 |
| MyPart 1 | MyPart 2 | 0.99684 | 0.9047 |
In this smoke test, we again see very good correlations between the MyPart sensors and the MetOne, as well as between other MyPart sensors.
In addition to being noisier, the noise floor for MyPart 0 also seemed to be inconsistent. The noise floor for MyPart 0 increased before this test, and the voltage cutoff for a sensed particle had to be increased accordingly. As a result, MyPart 0 saw fewer particles compared to MyPart 1 and 2, but still showed good correlation versus the MetOne.
In both of the previous smoke tests, the Dylos DC1100 Pro was tested alongside the MyPart sensors and the MetOne. There is a characteristic bend in the curve of the Dylos data when plotted against the MetOne and the multiple MyPart sensors. Tim Dye of Sonoma Technologies suggested that this was due to the Dylos switching to a different internal calibration curve at higher particle concentrations. Similar behavior was observed in our other smoke tests.
In this test, the raw voltage readings of the three MyPart sensors were recorded when the chamber was filled with 0.5um PSL beads, and also for when the chamber was filled with 3.0um PSL beads. The data was analyzed in Matlab, and the number and amplitude of the peaks were found using the built-in findpeaks function. Below, we plot the voltage distribution for each sensor, with the distributions for both particle sizes on each plot.
Though there is an overlap between the distribution of the two particle sizes (the 50th percentile for the large particles corresponds to roughly the 90th percentile for the large particles for all three sensors), there is a difference in the signal response for the two sizes of particles.
These plots show one source of ambiguity for optical particle counter particle sizing. Though an extremely high voltage peak likely corresponds to a large particle, a lower voltage peak could be a result of a small particle striking the laser at the center or from a larger particle striking the laser slightly off axis. Because we are trying to extract particle size information from intensity of light scattered, other sources of ambiguity include the composition, color, and shape of particle. However, the difference in the distribution suggests that some amount of sizing information can still be recovered.
Using the results here, we set the voltage threshold for a large particle to be at 3 volts to use for our first outdoor test. By setting such a high threshold, the particles sensed as large particles are likely to be actual large particles. However, the amount of large particles detected is sacrificed.
Since factors such as ambient light can affect the readings of optical particle counters, it is important to validate effectiveness in real world environments with varied lighting conditions. Testing in these environments is less controlled, but more representative of real use cases.
We brought the same set of sensors (MyPart 0-2, Dylos, MetOne) on a series of tests outside the lab. We carried them to various locations, some indoors in other buildings, some outdoors. At each location, we gathered two data samples using the same automated testing setup as the previous tests, and recorded the conditions.
Sample locations include:
- Near a group of restaurants.
- A grassy field.
- Inside a library.
- Next to a busy road.
- An old classroom building.
| Device 1 | Device 2 | r^2 | slope |
|---|---|---|---|
| MetOne | MyPart 0 | 0.663 | 0.0134 |
| Dylos | MyPart 0 | 0.651 | 0.0439 |
| MetOne | MyPart 1 | 0.889 | 0.0368 |
| Dylos | MyPart 1 | 0.959 | 0.1227 |
| MetOne | MyPart 2 | 0.905 | 0.0433 |
| Dylos | MyPart 2 | 0.951 | 0.1435 |
| MyPart 0 | MyPart 1 | 0.630 | 2.84 |
| MyPart 0 | MyPart 2 | 0.754 | 3.23 |
| MyPart 1 | MyPart 2 | 0.922 | 1.1545 |
Compared to the test chamber experiments, we see lower, but still good correlation between MyPart 1 and 2 vs the MetOne and the Dylos. At these lower concentrations, MyPart 0 did not perform as well for the reasons discussed previously. The r^2 tended to be higher between the MyPart and the Dylos compared to the MyPart and MetOne. The MyPart has lower resolution than the MetOne, whereas the difference in resolution with the Dylos is smaller. The difference in resolution becomes more noticeable at these lower concentrations.
The normalized data plot shows only MyPart sensors 1 and 2, and is normalized to the Dylos all counts data.
Unfortunately, the extremely high voltage threshold we chose for identifying large particles did not yield a good correlation with the MetOne. However, the Dylos does yield a good correlation versus the MetOne. This suggests that the threshold for what is considered a large particle can be lowered to better balance the amount of particles sensed versus how sure we are that it is a large particle.
MyPart sensors 1 and 2 saw roughly 20-25 times less the amount of particles seen by the MetOne in both indoor and outdoor tests. Though MetOne has higher resolution than the MyPart, it is not necessarily required for measuring particles in ambient environments. (One of the applications of the MetOne HHPC-6 is to monitor the cleanliness of cleanrooms).
The major improvement that should be made to the MyPart sensors is improving the analog front end, which would likely increase sensitivity as well as reduce the inconsistencies between sensors. This could be a good place to start. Another way of increasing the accuracy of the MyPart, especially at lower concentrations, is to explore adaptive sampling. When the particle concentration is high, the MyPart sensor can sense this after a few seconds of sampling, and terminate sampling early. Conversely, if the particle concentration is low, the MyPart can integrate for a longer period of time. The data gathered would be normalized based on the integration time. With the adaptive sampling technique, we can find a more optimal balance between accuracy and battery consumption.
Please see https://github.com/rutian/MyPart/wiki/Potential-Improvements for a list of other potential improvements.
- More outdoor tests, testing with a lower voltage cutoff for large particles.
- Testing in more challenging lighting conditions to test the amount of ambient light rejection.
- Indoor ambient particle test (run test for whole day, sampling every 10 minutes).
- Multiple color shells and measure light level.
- Since the optical particle counter can be affected by ambient light, it’s important to block as much as possible from getting inside the sensor. Black PLA does a good job of this, however it is possible that other colors may not be as effective.