Min model error

[brendan-m-murphy]

Changes:

• added an option for passing arbitrary keyword arguments to run_hbmcmc.py,
• added kwargs to fixedbasisMCMC, so that additional keyword args can be used (currently they are just passed to inferpymc)
• added min_error as an argument to inferpymc; the default value is 0.0, so it has no effect, but it can be specified in the ini file or via the command line.

Syntax for run_hbmcmc.py:

python run_hbmcmc.py -c example.ini --kwargs='{"outputpath": "/user/home/ab12345", "min_error": 20.0}'

In this case, the outputpath argument in example.ini would be replaced with "/user/home/ab12345", and the argument min_error=20.0 would be passed to inferpymc.

Note that the value passed to --kwargs is a "dictionary in string form". Use single quotes for the outside of the string, and use double quotes for the keys inside the curly braces, as well as for any values that are strings. The argument is parsed by json.loads, which will convert a json in string format to a dictionary. (Note: do not put integer or float keyword values in double quotes, unless you want them to be parsed as strings.)

[EricSaboya]

Looks good to me.

[aliceramsden]

Hi @brendan-m-murphy, I'm getting an error in get_data when it tries to read in obs when there isn't any available that month. It gets to line 153 in get_data (unit = float(site_data[site].mf.units) and fails because site_data is None. I think the try/except loop isn't catching this because it's a TypeError instead of a SearchError, maybe? (I'm not very confident about how error messages work in this case!) Is there any way of setting that try/except loop to work with either type of error in get_data?

[brendan-m-murphy]

Hi @aliceramsden , I guess it depends what we want to happen in this case. If we already have a value for units from a previous site, then it makes sense to keep going, but if we don't have a value for units already, the step that adds BC will fail I think.

Another issue is that I thought get_obs_surface never actually returns None unless you're using OpenGHG "on the cloud", which isn't a mode we actually use.

Could you tell me start/end date, species, and sites? If the species is fixed, we only need to get the units once. (I guess it would be best if we could specify it ahead of time so it doesn't need to be inferred.)

[brendan-m-murphy]

Sorry I just realised that skipping would sort this problem in the short term. Although I don't think it's the ideal behaviour, since either we know the units from some site, or none of the sites will be used and we'll have no data for that inversion period.

[brendan-m-murphy]

I think the try/except loop isn't catching this because it's a TypeError instead of a SearchError, maybe? (I'm not very confident about how error messages work in this case!) Is there any way of setting that try/except loop to work with either type of error in get_data?

You're right about this: the except clause is only catching SearchErrors. The reason for narrowing the type of exceptions to catch is that I was only expecting SearchError to happen if data isn't found, but it looks like we might get None in some cases... which should probably be fixed in openghg...

[brendan-m-murphy]

So apparently get_obs_surface will return None if the data you retrieve has a time coordinate, but the time coordinate is empty. So I guess we just want to skip this case.