Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensemble model results need to be attached to model id rather than number #629

Closed
sabinala opened this issue Nov 19, 2024 · 2 comments
Closed
Assignees

Comments

@sabinala
Copy link
Contributor

sabinala commented Nov 19, 2024

Currently, ensemble results are given according to the model number. As in model_0/weight or model_0/persistent_beta_c_param here:

image

TA4 requests that these results include the model id number instead because, per Tom Szendry: The issue with the model order is that we are doing index mapping on indexes that can be updated which is unideal
In other scenarios we may have this result.csv without the given call as well so there may be no way to trace back to model_0, but there will always be a way to trace back to ac3c5752-eb02-4ffa-8325-6b4ae9c7891f.

@sabinala sabinala self-assigned this Nov 19, 2024
@sabinala
Copy link
Contributor Author

This might require a change in MIRA's "load model" functions, because as far as I can tell, we never get the model id in the first place.

Here is where we are loading the models in compiled_dynamics. I added print statements to view what is loaded, as in:

    @load.register(str)
    @classmethod
    def _load_from_url_or_path(cls, path: str):
        if "https://" in path:
            model = mira.sources.amr.model_from_url(path)
            print(f"MODEL from url: {model}")
        else:
            model = mira.sources.amr.model_from_json_file(path)
            print(f"MODEL from json: {model}")
        return cls.load(model)

The output does not include the model id. I get templates, parameters, initials, observables, and annotations only. The model id is the first line of the json file (see here for example).

MODEL from url: templates=[ControlledConversion(rate_law=I*S*kappa*(beta_c + (-beta_c + beta_s)/(1 + exp(-k*(-t + t0))))/total_population, name='t1', display_name=None, type='ControlledConversion', controller=Concept(name='I', display_name='I', description=None, identifiers={'ido': '0000511'}, context={}, units=Unit(expression=person)), subject=Concept(name='S', display_name='S', description=None, identifiers={'ido': '0000514'}, context={}, units=Unit(expression=person)), outcome=Concept(name='E', display_name='E', description=None, identifiers={'apollosv': '0000154'}, context={}, units=Unit(expression=person)), provenance=[]), NaturalConversion(rate_law=E*delta, name='t2', display_name=None, type='NaturalConversion', subject=Concept(name='E', display_name='E', description=None, identifiers={'apollosv': '0000154'}, context={}, units=Unit(expression=person)), outcome=Concept(name='I', display_name='I', description=None, identifiers={'ido': '0000511'}, context={}, units=Unit(expression=person)), provenance=[]), NaturalConversion(rate_law=I*gamma*(1 - hosp), name='t3', display_name=None, type='NaturalConversion', subject=Concept(name='I', display_name='I', description=None, identifiers={'ido': '0000511'}, context={}, units=Unit(expression=person)), outcome=Concept(name='R', display_name='R', description=None, identifiers={'ido': '0000592'}, context={}, units=Unit(expression=person)), provenance=[]), NaturalConversion(rate_law=I*gamma*hosp, name='t4', display_name=None, type='NaturalConversion', subject=Concept(name='I', display_name='I', description=None, identifiers={'ido': '0000511'}, context={}, units=Unit(expression=person)), outcome=Concept(name='H', display_name='H', description=None, identifiers={'ido': '0000511'}, context={'property': 'ncit:C25179'}, units=Unit(expression=person)), provenance=[]), NaturalConversion(rate_law=H*(1 - death_hosp)/los, name='t5', display_name=None, type='NaturalConversion', subject=Concept(name='H', display_name='H', description=None, identifiers={'ido': '0000511'}, context={'property': 'ncit:C25179'}, units=Unit(expression=person)), outcome=Concept(name='R', display_name='R', description=None, identifiers={'ido': '0000592'}, context={}, units=Unit(expression=person)), provenance=[]), NaturalConversion(rate_law=H*death_hosp/los, name='t6', display_name=None, type='NaturalConversion', subject=Concept(name='H', display_name='H', description=None, identifiers={'ido': '0000511'}, context={'property': 'ncit:C25179'}, units=Unit(expression=person)), outcome=Concept(name='D', display_name='D', description=None, identifiers={'ncit': 'C28554'}, context={}, units=Unit(expression=person)), provenance=[])] parameters={'beta_c': Parameter(name='beta_c', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=1/day), value=0.4, distribution=Distribution(type='Uniform1', parameters={'minimum': 0.1, 'maximum': 0.8})), 'beta_s': Parameter(name='beta_s', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=1/day), value=1.0, distribution=None), 'k': Parameter(name='k', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=1), value=5.0, distribution=None), 'kappa': Parameter(name='kappa', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=1/day), value=0.4, distribution=Distribution(type='Uniform1', parameters={'minimum': 0.05, 'maximum': 0.8})), 't0': Parameter(name='t0', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=day), value=89.0, distribution=None), 'total_population': Parameter(name='total_population', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=person), value=19340000.0, distribution=None), 'I0': Parameter(name='I0', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=person), value=10.0, distribution=Distribution(type='Uniform1', parameters={'minimum': 1.0, 'maximum': 15.0})), 'delta': Parameter(name='delta', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=1/day), value=0.25, distribution=None), 'gamma': Parameter(name='gamma', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=1/day), value=0.2, distribution=Distribution(type='Uniform1', parameters={'minimum': 0.1, 'maximum': 0.5})), 'hosp': Parameter(name='hosp', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=1/day), value=0.1, distribution=Distribution(type='Uniform1', parameters={'minimum': 0.005, 'maximum': 0.2})), 'death_hosp': Parameter(name='death_hosp', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=1/day), value=0.07, distribution=Distribution(type='Uniform1', parameters={'minimum': 0.01, 'maximum': 0.1})), 'los': Parameter(name='los', display_name=None, description=None, identifiers={}, context={}, units=Unit(expression=day), value=5.0, distribution=None)} initials={'S': Initial(concept=Concept(name='S', display_name='S', description=None, identifiers={'ido': '0000514'}, context={}, units=Unit(expression=person)), expression=-I0 + total_population), 'I': Initial(concept=Concept(name='I', display_name='I', description=None, identifiers={'ido': '0000511'}, context={}, units=Unit(expression=person)), expression=I0), 'E': Initial(concept=Concept(name='E', display_name='E', description=None, identifiers={'apollosv': '0000154'}, context={}, units=Unit(expression=person)), expression=40.0), 'R': Initial(concept=Concept(name='R', display_name='R', description=None, identifiers={'ido': '0000592'}, context={}, units=Unit(expression=person)), expression=0.0), 'H': Initial(concept=Concept(name='H', display_name='H', description=None, identifiers={'ido': '0000511'}, context={'property': 'ncit:C25179'}, units=Unit(expression=person)), expression=0.0), 'D': Initial(concept=Concept(name='D', display_name='D', description=None, identifiers={'ncit': 'C28554'}, context={}, units=Unit(expression=person)), expression=0.0)} observables={'infected': Observable(name='infected', display_name='infected', description=None, identifiers={}, context={}, units=None, expression=I), 'exposed': Observable(name='exposed', display_name='exposed', description=None, identifiers={}, context={}, units=None, expression=E), 'hospitalized': Observable(name='hospitalized', display_name='hospitalized', description=None, identifiers={}, context={}, units=None, expression=H), 'dead': Observable(name='dead', display_name='dead', description=None, identifiers={}, context={}, units=None, expression=D)} annotations=Annotations(name='SEIRHD model NPI Type 1', description='SEIRHD model NPI Type 1', license=None, authors=[], references=[], time_scale=None, time_start=None, time_end=None, locations=[], pathogens=[], diseases=[], hosts=[], model_types=[]) time=Time(name='t', units=Unit(expression=day))

@SamWitty
Copy link
Contributor

From our conversations with Uncharted yesterday it sounds like they are going to be handling the bookkeeping of model ids. Closing this issue as a result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

2 participants