Make observable.observableTransformation and parameter.prior required in v2?

[matthiaskoenig]

Hi all,

It would be great if we could mark some columns as required to improve clarity. This would allow for stricter validation and reduce the need to constantly refer to the specifications for default values. It would also make the tables more readable, as users wouldn’t need to be aware of implicit defaults.

Specifically, it would be helpful to make the following columns required:

• observable.observableTransformation (currently defaults to lin)
• parameter.prior (currently defaults to uniform)

In a practical context, it is much faster to fill this out then deal with all the implicit consequences.
Let me know what you think.

Best Matthias

[dweindl]

You are proposing to not only make the (potentially empty) columns mandatory, but also to require explicitly specifying all values there, right?

I think having to specify too many things might be scary for newcomers. I think reasonable default values aren't too bad. I think it's nice if one can start with specifying a minimum amount of information and then add things as needed.

For prior, the issue is that we use presence or absence of that to switch between maximum a posteriori and maximum likelihood estimation.

If we'd choose to make more things mandatory, observableTransformation would come only after noiseDistribution for me. Instead of making observableTransformation mandatory, I would rather consider dropping it completely and adding log-distributions to noiseDistribution instead, and making noiseDistribution mandatory.

I am not strongly opposed, but for now, I don't see the advantages outweighing the disadvantages.

[matthiaskoenig]

For prior, the issue is that we use presence or absence of that to switch between maximum a posteriori and maximum likelihood estimation.

So the switch could be done by setting all priors uniform instead.

If we'd choose to make more things mandatory, observableTransformation would come only after noiseDistribution for me. Instead of making observableTransformation mandatory, I would rather consider dropping it completely and adding log-distributions to noiseDistribution instead, and making noiseDistribution mandatory.

I am not an expert here, but for me it sounds reasonable. I have the feeling that a lot of discussions are around what do all the transformations mean for the objective function and the optimization. By getting rid of these transformations (such as parameterScale and perhaps observableTransformation) and making them more explicit (such as putting it in the noiseDistribution) makes things much easier to understand (especially for non-experts such as me).

[JanHasenauer]

A uniform prior and no prior is not the same as a uniform prior requires normalization. Formally, a uniform prior can also not be applied on unbounded domains, meaning that we would have to restrict the problem class.

The observableTransformation we could indeed remove be extending the spectrum of noise distribution. I like this very much and would recommend to go ahead with this.

[matthiaskoenig]

I am also strongly in favor of removing the observableTransformation. This will make implementing PETab support much easier.
The main problem for users and developers is to understand how all the different transformations and priors work together. Getting rid of these transformations and clearly specifying the noise distributions will be much easier to implement and understand.

I feel that tools currently misuse the prior field. To me, no prior' and uniform prior' seem different (and are interpreted differently, e.g. by switching methods as @dweindl mentioned). Perhaps it is better to be more explicit here and allow setting none as a prior to clearly indicate that no prior information is specified, rather than leaving the prior empty (which is officially a uniform prior by default, but is interpreted as none in some contexts).

In summary, it would be great to remove oberservableTransformation and be more explicit by allowing none as a prior to clearly indicate that no prior information is specified. The prior field should then be required.