Energy calibration for the electron-detector

[SteveF161]

I'm trying to open one of the reference data sets (the one from the 9.44g of KCl) and I'm getting a pickle error:
` df = pd.read_pickle("./data/KCL_9-44g_1x3x3cm_touchingdiodecase_pulses_2019-07-08_23-07-18___947___11-47.pkl")

File "/home/steve/.local/lib/python3.6/site-packages/pandas/io/pickle.py", line 182, in read_pickle
return pickle.load(f)

UnpicklingError: invalid load key, 'v'.

`
Can you confirm what format the data is in and what versions of pandas/pickle/python were used?

(Background: The data analysis python script isn't finding any pulses in the data files that I record with the JavaScript pulse detector. I've looked at the data in my files and I see a lot of values around -32000, which is the sort of thing I'd expect to see from some kind of 15 bit signed/16 bit unsigned conversion problem. I was trying to compare against a reference data set to test the hypothesis.)

[ozel]

Have you checked out the files using git LFS as explained in this section: https://github.com/ozel/DIY_particle_detector#reference-measurements-and-plots?
Since large binary files should generally rather not be stored in plain git, I put them there.
Alternatively, you can download the files from the Zenodo archive link as a whole or from the Github website one by one: https://github.com/ozel/DIY_particle_detector/tree/master/data_analysis_and_reference_measurements/diode_detector/data

I will come back regarding minimum module versions, but Python 3.6 should be good in principle.

Please note: since you are not using the same USB soundcard like me, one of the 1-2 $ low-cost CM108 USB dongles described in Soundcards, you will certainly have different pulse amplitudes maybe even saturation depending on the gain settings.
In order to adjust your microphone input gain in the system setting for a useful range, I would suggest to either use the WaveWatch Processing sketch or pulse_recorder.py, both described more detailed in the new Wiki page Software.
The latter python script requires the pyqtgraph module and in turn Qt, but those large dependencies can be well handled if you install your whole python environment via anaconda (very very much recommended).
You'll need to adjust the input gain settings such, that -300 is a threshold level where the noise is well below that (like +/-200 on average).

[SteveF161]

D'Oh! If I'd looked, I'd have seen that the one I had downloaded was only 133 bytes! The full 7.6MB file works correctly and I can see the data in it is very different to mine.

My data does look suspect but the fact that the JavaScript detector sees around 500 pulses over 3 hours makes me think that sound card is working OK and the problem might be data conversion.

The laptop that I'm using is running Lubuntu on an early model Intel Atom processor. I've not yet found a Python Qt library that installs on it. I may well spend a few pounds on the cheap soundcard and link it to my Raspberry Pi 4 on the grounds that (a) I'm pretty sure I can get a working Qt for it and (b) it's probably a more capable machine than the antiquated laptop.

[ozel]

Did you try installing pyqtgraph with Anaconda?

As I wrote before, please have a look at the WaveWatch program: https://github.com/ozel/DIY_particle_detector/wiki/Software#processing-based-audio-oscilloscope
Processing is based on Java and should run flawlessly on your old Intel Atom system. It allows you to zoom in on the x-axis such that you can actually check for the correct pulse shape (see the screenshot, that I just uploaded on that wiki page). At the moment, it doesn't show the raw value of the trigger level due to its fancy voltage calibration feature but I can fix that quickly in the next couple of days if it helps you.

[SteveF161]

Update: I bought the cheap sound card on Amazon (£7.99 : Sabrent AU-MMSA). I fitted to the laptop and I get a much cleaner signal. the noise level is far lower and the electron pulses stand out perfectly. However, the data that I save is still unreadable in the analysis script which continues to recognise zero pulses.

Next, I moved the sound card to my raspberry pi4. As expected, the pi4 definitely has more oomph than my antiquated laptop. Again I had a clean signal with very obvious peaks but the saved data files are still unreadable in the analysis script.

I downloaded the python data recording script (...and the interminable dependencies...) to the pi4. It works very well, though the noise level appears much higher as it is scaled differently. Most importantly, the saved data files work in the analysis script. It might be worth taking a look at the JS script to see if there is anything odd or if this is just expected but unreasonable behaviour of different sound cards

I'm in the middle of tinkering with the sensitivity levels to cut out false pulses. I'm currently seeing around 0.9CPS which is about 30-50 times higher than with the other app and the energy spectrum looks nothing like I'd expect for beta. Clearly, tuning is required.

[ozel]

Good to hear that works now for you!
This scale difference in the oscilloscope view is what I meant. The WaveWatch Processing sketch did not work for you on the EeePC?
If I find some time, I will try to give the web browser oscilloscope a little bit more love and functionality, but it is really rather painful to develop. That said, we can consider the situation already quite lucky since it still works after one year which means the javascript WebAudio API seems to have stabilized after all...

In principle, the generated .msgp files should be loadable in the analyser python script. Could you attach one of your files that did not work in the next comment? Just zip it and drop it on the comment field.

Since you've built the electron-detector version so far, looking at the energy spectrum will not provide a lot of interesting information in general. See the Wiki entry on Energy Spectra.

In general, the recording of waveforms and corresponding analyse_and_plot_pulses.py script is really meant for being used in combination with the alpha-spectrometer variant only. Due to a few different component values, much longer pulses are produced in both cases, alphas and electrons! I haven't used the analyse script myself with electron-detector data and it is very likely that it does not yield useful results because important pulse characteristics are different (steepness of pulse slopes and width of pulses).
The other aspect is as discussed the threshold level (THL variable), which is by default at -300 in both scripts, the recorder and analyse python scripts and is only valid for the alpha-spectrometer variant. The electron-detector variant got quite some higher noise value, larger than -1000 is typical if I remember well.

You may have seen the electron energy spectrum in my paper, figure 7.
That however was as well recorded with the alpha-spectrometer variant of the detector, and not with the electron-detector version that you built so far. Conversion from one to the other is doable, by the way. Even unsoldering of the diodes works with tweezers/small pliers and if one pulls while a third-hand navigates the soldering iron.
But if you have spare PCBs, using those would be better of course.

[SteveF161]

There's a non-working file [attached.]
DIY_Particle_Detector_2363-pulses_2020-09-02_14-42.zip

I'm working with THL around -2500 on the electron detector but I'm not yet sure if this is still too small. The average noise level looks to be about -1000 but there are quite a few higher peaks which I think may be EM noise from vehicles on the busy road outside my house.

My sense-check on the data is to look at the energy spectra which I'd (naively) expect to be convex rather than concave near the origin. The fact that I'm seeing concave makes me suspicious of the settings.

Anyway, I have a long collection run ongoing at the moment and I'll see what that contains.

[ozel]

thanks for the file!
You could as well post screenshots from good pulses and suspected noise hits. Otherwise, it is really difficult to help.
EM noise can be an issue (e.g. a running laundry machine or old fridge close by) or just a little bit of light leaking inside the case will as well increase the noise but in a very subtle way (while too much light just saturates everything). You can be suspicious once you see regular 50/60 or 100/120 Hz noise... one segment on the x-axis is 1/48000 seconds wide, so you can roughly estimate the frequencies.

The pulses that trigger really need to have that distinctive, asymmetric and bipolar shape as in the pictures, e.g. https://github.com/ozel/DIY_particle_detector/wiki/Software#processing-based-audio-oscilloscope
If the pulses do not look like that, you can really skip further recording or have to increase/lower the threshold until only good ones trip the trigger. The analyse script does not magically find something within the noise beyond us humans, that is for sure. ;-)

As I wrote, the energy spectra generated by the analyse script may very well be completely off when used with the electron-detector variant since the script is not made for that variant of the detector. Just looking at the pulse shapes is really the way to go here. If you want, you can as well post the spectrum here. If it were recorded with the alpha-spectrometer variant, it should reassemble figure 7 b) in my paper. (link above)

[SteveF161]

I've attached a couple of the "good pulse" screenshots, one from each capture method. I can't post shots of the "bad" pulses as they only seem to be a problem on the python script collector and they don't actually show up on the display. It may be that the display is not refreshing as often as it should. I can send a data file when the collection run is finished.

drive-download-20200911T164921Z-001.zip

[ozel]

those pulses look perfect, note that you can also zoom into them in the pyqtgraph window by changing the window config settings, via right-click or the gear symbol. Or just pull the window very wide.

the only additional info that the analyse script can provide in your case with an electron-detector is finding additional pulses within the recorded waveform snippets (4096 values each). But with a low-intensity source like KCl you should not really see more than one pulse in 4096 value snippet on average.

Have you adapted the threshold value THL in the analyse script? It has to be the same one as the one used in the recorder. Running the analyse script on your data with the original THL of - 300 is expected to do a lot of things wrong as it is intended for the alpha-spectrometer variant only. And as I wrote before, there are further reasons why it will not properly find electron-detector pulses anyway.

[SteveF161]

Thanks for the response. I'm guessing that the calibration of the electron detector is quite different to that of the alpha detector. I'm seeing an energy spectrum of vaguely the right shape but the energy looks completely wrong. I'd expect the maximum around 0.65MeV and nothing above ~1.36MeV except for the single gamma peak at 1.4 MeV. (I'm not seeing the gamma peak but I remember reading that the detector has low efficiency for gamma, so I'm not worrying about this at present.)

I had to tinker with the script to get all the pulses to display but I don't think I've changed anything significant. I've mainly adjusted THL and re-included it in the peak pulse calculation as it is quite significant compared to the sizes of many pulses.

I may try to hack a bit further to get better pulse detection for electrons (...which will require me to do some background study in pulse detection first!...) and to calibrate against known properties of KCl spectrum.

I've attached some of the spectra.
K40 with electron detector - sample graphs.zip

[ozel]

The shape of one of your spectra looks actually quite ok, ignoring the x-axis labels of course.
But it depends on your modifications of the code if it is a real one. The important parameters for tweaking are:

min_g = -20 # steepness of falling edge slope, almost vertical
min_length = 44  # about 0.9 ms at 48 khz sample rate
max_length = 120 # about 2.5 ms, tolerates some alpha-pileup

These are the default settings for the alpha-spectrometer. The electron-detector has much shorter pulses (25 - 100 us max.) as you can see here:

The length values are in numbers of samples, so those depend on the sample rate.
If your spectra were recorded with those same min/max length settings, they could not come from the real pulses. They would need to be set much shorter. Also, min_g may need to be adapted. The extra options like SHOW_DETECTED_PULSES are there to help with debugging trigger parameters. If the pulses are not all well-aligned such that there is no clear "valley", the parameters do not fit to the pulse shape. C.f. the green "overlayed pulses" figure 7a in the paper.

Regarding energy calibration and the possibility to observe characteristic gamma/x-ray peaks with the electron-detector:
The recorded electron "spectrum" is basically a convolution of the diode geometry, amplifier/threshold noise, Compton scattering of gamma particles (see below) and continuous beta decay energy spectra (if emitted from the source nuclide, but mostly both kinds are emitted as with e.g. potassium-40 in KCl).

The main aspects between the detector signals of both different detector variants are those two things: The different pulse geometry described above and a different lower energy threshold. It should be higher than the 33 keV of the alpha-spectrometer due to the 4 diodes (increased capacitance, reducing SNR) and different amplification factors/filtering of the circuit. I would guess something between two and three times as much: 66 keV - 100 keV.
Above 100 keV, Compton scattering of gamma photons (=incoherent scattering) completely dominates in silicon sensors as you can see in this diagram attached: (generated from https://physics.nist.gov/PhysRefData/Xcom/html/xcom1.html).

Since the detector silicon is very thin compared to its area (50 µm vs 7 mm² per diode), it is much much better in collecting electrons from Compton scattering than recording characteristic gamma energies via photo-absorption. As those scattered electrons typically "go for a walk" and loop around rather laterally in this energy range as possed to advancing straight into the silicon - this is what leads to the long thin high energy tail approaching up to about 600 keV.
The only chance to see a characteristic photon peak would be from low-energy sources like the Americium-241 peak at 60 keV. But I doubt that lies within the electron-detector's threshold. It should be well observable with the alpha-spectrometer variant, though. Possibly after blocking the americium alphas with a sheet of paper.
If I find some time and you'd like to continue investigating this as well, I could confirm the lower energy threshold of the electron-detector with an x-ray machine in the next weeks. Together with the known maximum of 500 - 600 keV, which is given by the diode geometry and independent from the rest of the circuit, we could establish a rough energy calibration based on those two corner numbers. One of my longer-term goals is to put the pulse detection code in its own library module and have it accelerated by Numba. That would make it more flexible and speed the processing up a lot.

For a better overall picture, I would really encourage you to have a look at the corresponding paper that I've linked to a couple of times. While its main part is focusing on the alpha-spectrometer and fancy pixel detectors, the first part discusses the detector physics of the silicon diodes - the silicon chips inside those diodes (BPW34 and PBX61) are the same with both DIY particle detector variants (despite the different diode packaging).

[SteveF161]

Hi, and thanks for response. I had read the paper but - on the first reading - the drop-off in sensitivity didn't register. I now understand what I'm seeing and why.

I had already looked into the setting for min_g and max_g and some of the other parameters as the electron pulses where so short. The time from zero level to peak minimum seems to be only 2 or 3 audio samples. I'm trying to balance this with a trigger level closer to the noise level (-2000 vs -1200 average noise) but this seems sometimes to lead to a lot of pulses being rejected

[ozel]

ah yes, the THL threshold level is evaluated at the very beginning - before all other paramters - to cut out the noise. It is most important compared to the other three parameters that I listed. The setting of THL is being evaluated more than once in the decision path at the moment which could be reduced at some point.