Add qc scripts into the pipline

[martinjohndyer]

During a standard observing night we always use some version of a qc.py script to check the RNO levels in the bias frames. But it's a little messy just running a standalone python script that lives in /home/observer/qc, especially when it's in with various older versions which can get confusing. Those scripts are also not version controlled or really maintained anywhere, and there are a few things it might be nice to upgrade e.g. replacing pgplot with matplotlib.

This was also partially inspired by the idea to better manage a load of the scripts from the instrument DRPCs, my thought was that scripts like tweakPointing.py or autofocus.py should live in some sort of separate package ("obs-utils"?) rather than just in /home/observer/bin. But having had a chat with @vikdhillon and Pablo, I think there's no reason we couldn't add the qc scripts directly into the pipeline. Although they're mainly used when observing it's not unreasonable for someone to want to check e.g. noise levels in the biases if they're reducing later down the line. And it would simplify things for the observer on the night by removing this one standalone script.

qc.py (or equivalent - I think it's /home/observer/qc/ultracam/qc.py for ULTRACAM, /home/observer/qc/ultraspec/uspec_qc_hicam.py for ULTRASPEC and /home/observer/reduce/qc/hcam_qc.py for HiPERCAM) is probably the only one that's used regularly, but there are various other analysis scripts in those directories measuring gain or linearity etc. Those sort of image analysis scripts I think should be included too, either as options in a big qc script or as standalones.

I think this would be in scope of adding into the pipeline, a lot of it's probably already there. From a quick look through stats (and mstats) does a similar thing printing out only basic frame statistics, and there's ncal which is also looking at noise levels. I'd something I'd be happy to look into.

Any thoughts @StuartLittlefair?

[StuartLittlefair]

Having thought about this for a day or two I'm totally in support of bringing these user-space scripts under version control.

I'm not sure that the qc scripts as they are currently written should live in the pipeline software. They are too instrument specific and there's a lot of code that links to databases of historical data which is either local or held on my cPanel account.

Putting them in this repository would involve a fair amount of extra work to refactor the code into a single script, rationalise the way we store historical data and transfer the archival data into the new system. This all seems like a lot of work to me, and there are possibly other things that are more a of a priority?

Also - as you say, if a user wants to check noise levels in biasses etc there are already tools for that (e.g stats), so we're also duplicating functionality which makes things confusing.

So my feeling is that these scripts should go in a different repository, along with general user scripts that probably need putting under version control like the auto guiding scripts.