Support for custom priors via Prior class #488

williambdean · 2025-06-16T17:20:30Z

This using pymc-extras to allow for custom priors. Might need to adjust the remaining a bit to work.

Closes #387

@drbenvincent did some work to:

get remote tests passing
add docstrings
add tests

causalpy/pymc_models.py

drbenvincent

This is very cool!

As far as I can tell, these changes are currently entirely invisible to the end user, right?

Before we do a new release I will follow up with another PR that adds some documentation. This will most likely be in the form of just giving a couple of worked examples

Main request is to add custom-ness to the Dirichlet for the WeightedSumFitter class. Like I say in a comment, I mostly see people wanting to customise the hyperparams, not the distribution itself.

drbenvincent · 2025-06-20T10:21:33Z

causalpy/pymc_models.py

    Inference data...
    """  # noqa: W605

+    default_priors = {


need to add @property decorator here? Or is that remembered from it being done in the PyMCModel base class?

Getting an Pylance warning: Type "dict[str, Prior]" is not assignable to declared type "property"

What line of code bring that on? Maybe having a setter will help?

Agree, if you want a property, maybe we can have a setter method? (not a blocker for now and maybe create an issue?)

causalpy/pymc_models.py

drbenvincent · 2025-06-20T10:40:06Z

causalpy/pymc_models.py

            n_predictors = X.shape[1]
            X = pm.Data("X", X, dims=["obs_ind", "coeffs"])
            y = pm.Data("y", y[:, 0], dims="obs_ind")
            beta = pm.Dirichlet("beta", a=np.ones(n_predictors), dims="coeffs")


We definitely want a custom prior for the Dirichlet. I think the Dirichlet would be used always (or nearly always), but there are plenty of real world use cases where the user might want to change the hyper parameters (currently a=np.ones(n_predictors)).

Alright. since it is function of data, we will have to handle differently

I've address this with dc20e3e. If user wants to change, they can add to the "priors" at initialization.
Take a look and let me know what you think.

codecov · 2025-06-20T11:08:22Z

Codecov Report

❌ Patch coverage is 99.34211% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 95.43%. Comparing base (77f8d59) to head (4be4cdd).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
causalpy/pymc_models.py	96.42%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #488      +/-   ##
==========================================
+ Coverage   95.19%   95.43%   +0.23%     
==========================================
  Files          28       28              
  Lines        2457     2586     +129     
==========================================
+ Hits         2339     2468     +129     
  Misses        118      118

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

drbenvincent · 2025-06-20T11:25:30Z

tests pass :)

juanitorduz · 2025-07-12T08:42:58Z

Excited about this PR!

drbenvincent · 2025-07-16T20:07:41Z

causalpy/pymc_models.py

+        n_predictors = X.shape[1]
+
+        return {
+            "beta": Prior("Dirichlet", a=np.ones(n_predictors), dims="coeffs"),


Just realised that n_predictors will equal length of the "coeffs" dim. Does that have to be the case in fact? If so, do we need priors_from_data?

Not saying I don't want it, it could be really cool. But just wondering more generally if it's needed or not. Will try to think more with a fresh head in the morning, but does this spark off any thoughts?

drbenvincent · 2025-07-17T19:39:10Z

So I spent some time doing a gnarly merge from main into this dev branch. There were some widespread changes in main, so it took me a while to get all the tests passing. But it should make your life easier :)

Remote doctests failing. They pass for me locally. Seems maybe like an import error in pymc? Any ideas @williambdean ?

- Updated LinearRegression default_priors to support multi-unit architecture - Resolved print_coefficients method conflicts - Added patsy import back for dmatrix usage - Regenerated interrogate_badge.svg with updated coverage

- Use Prior class create_likelihood_variable method for model definitions - Updated print_coefficients to handle both sigma and y_hat_sigma variables - Simplified PropensityScore to use Prior class exclusively - Deleted interrogate badge to be regenerated

drbenvincent · 2025-09-29T08:30:17Z

This seems to be the reason why the remote tests are failing: pymc-devs/pymc-extras#541

drbenvincent · 2025-09-29T08:41:28Z

The issue is that pymc-extras 0.3.0+ requires Python >=3.11, but the CI is running on Python 3.10.18. That's why pip can't find a compatible version.

Added '# pragma: no cover' to NotImplementedError and ValueError branches in PyMCModel to exclude them from test coverage reporting.

drbenvincent · 2025-09-29T09:16:39Z

Remote tests back to passing :)

juanitorduz

Great! There is a comment by @williambdean which we could tackle later (please create an issue :) )

juanitorduz · 2025-09-29T10:50:38Z

causalpy/pymc_models.py

    Inference data...
    """  # noqa: W605

+    default_priors = {


Agree, if you want a property, maybe we can have a setter method? (not a blocker for now and maybe create an issue?)

drbenvincent · 2025-10-03T08:47:59Z

Thanks for the review @juanitorduz. My feeling at the moment is...

The current workflow is that computation is kicked off automatically when creating an experiment object instance. As in, you don't first create the experiment then manually call a fit method, for example. Because we have that workflow, then having a setter doesn't makes sense.

Changed PyMCModel.default_priors from a @property method to a class attribute to match how it's implemented in all subclasses (LinearRegression, WeightedSumFitter, PropensityScore). The property decorator was causing Pylance type warnings since subclasses were defining default_priors as dictionaries rather than properties. Since priors are static configuration data that don't depend on instance state, using a class attribute is more appropriate and provides better performance (no method call overhead). This change maintains backward compatibility while making the codebase consistent and eliminating type checker warnings.

Can deal with this (if at all) in another PR

williambdean added 2 commits June 16, 2025 13:18

add pymc-extras to environment

b35001b

add default_priors and support for custom priors

b7300e7

williambdean commented Jun 16, 2025

View reviewed changes

causalpy/pymc_models.py Outdated Show resolved Hide resolved

williambdean commented Jun 16, 2025

View reviewed changes

causalpy/pymc_models.py Outdated Show resolved Hide resolved

williambdean and others added 3 commits June 16, 2025 13:52

get pymc_models tests to pass

a60035e

add dim to y_hat

367c922

fix for sigma -> y_hat_sigma

a9f821c

drbenvincent previously requested changes Jun 20, 2025

View reviewed changes

fix failing doctest

91aee00

drbenvincent mentioned this pull request Jul 5, 2025

Enable multiple treated units in synthetic control quasi experiments #494

Merged

add support for priors from data

dc20e3e

drbenvincent reviewed Jul 16, 2025

View reviewed changes

trying to merge main into this feature branch and resolve conflicts

f51f994

drbenvincent added 3 commits September 27, 2025 10:55

Add regenerated interrogate badge with updated coverage

57ba733

update pymc-extras version pin in attempt to fix failing remote tests

b57810a

drbenvincent mentioned this pull request Sep 29, 2025

Drop support for Python 3.10 #528

Closed

drbenvincent added 2 commits September 29, 2025 10:02

Merge branch 'main' into pr/488

1a0b078

Add pragma no cover to exception branches

787a10e

Added '# pragma: no cover' to NotImplementedError and ValueError branches in PyMCModel to exclude them from test coverage reporting.

drbenvincent added 3 commits September 29, 2025 10:18

update pymc-extras version pin to match that in pyproject.toml

bcba49f

add docstrings to the priors_from_data methods

0650644

add tests

3c659d3

drbenvincent added the major label Sep 29, 2025

juanitorduz approved these changes Sep 29, 2025

View reviewed changes

Convert default_priors property to class attribute

4be4cdd

drbenvincent merged commit 540893b into pymc-labs:main Oct 3, 2025
6 checks passed

drbenvincent mentioned this pull request Oct 3, 2025

Draft (new feature) : Model to estimate when a intervention had effect #480

Open

williambdean deleted the custom-priors branch October 3, 2025 13:53

Support for custom priors via Prior class #488

Support for custom priors via Prior class #488

Conversation

williambdean commented Jun 16, 2025 • edited by drbenvincent Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

drbenvincent left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

drbenvincent commented Jun 20, 2025

Uh oh!

juanitorduz commented Jul 12, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

drbenvincent commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

drbenvincent commented Sep 29, 2025

Uh oh!

drbenvincent commented Sep 29, 2025

Uh oh!

drbenvincent commented Sep 29, 2025

Uh oh!

juanitorduz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

drbenvincent commented Oct 3, 2025

Uh oh!

Uh oh!

Uh oh!

williambdean commented Jun 16, 2025 •

edited by drbenvincent

Loading

codecov bot commented Jun 20, 2025 •

edited

Loading

drbenvincent commented Jul 17, 2025 •

edited

Loading