Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove all uses of Step.__call__ #1499

Merged
merged 4 commits into from
Nov 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changes/1499.stpipe.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Remove Step.__call__ usage (which will be deprecated in stpipe).
4 changes: 2 additions & 2 deletions docs/roman/outlier_detection/outlier_examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ the outlier detection step.
# read the file list into a ModelLibrary object
mc = ModelLibrary(["img_1.asdf", "img_2.asdf"])
step = OutlierDetectionStep()
step.process(mc)
Copy link
Collaborator Author

@braingram braingram Nov 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a use of __call__ but process isn't a recommended way to run a step.

step.run(mc)

2. To run the outlier detection step (with the default parameters) on an ASN file
called "asn_sample.json" with the following content:
Expand Down Expand Up @@ -56,7 +56,7 @@ the outlier detection step.
# read the file list into a ModelLibrary object
mc = ModelLibrary("asn_sample.json")
step = OutlierDetectionStep()
step.process(mc)
step.run(mc)

#. To run the outlier detection step (with the default parameters) on an ASN file
called "asn_sample.json" (the files listed in the association file must have been
Expand Down
8 changes: 1 addition & 7 deletions docs/roman/stpipe/call_via_run.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,6 @@ parameter file will be ignored.
pipe.output_dir = '/my/data/pipeline_outputs'

# Run the pipeline
result = pipe('r0000101001001001001_01101_0001_WFI01_uncal.asdf')

# Or, execute the pipeline using the run method
result = pipe.run('r0000101001001001001_01101_0001_WFI01_uncal.asdf')

To run a single step:
Expand All @@ -41,8 +38,5 @@ To run a single step:
step.save_results = True
step.output_dir = '/my/data/jump_data'

# Execute by calling the instance directly
result = step('r0000101001001001001_01101_0001_WFI01_linearity.asdf')

# Or, execute using the run method
# Execute using the run method
result = step.run('r0000101001001001001_01101_0001_WFI01_linearity.asdf')
6 changes: 3 additions & 3 deletions docs/roman/stpipe/user_pipeline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -183,10 +183,10 @@ step in the example above, one can do::
From Python
-----------

Once the pipeline has been configured (as above), just call the
instance to run it.
Once the pipeline has been configured (as above) it can be executed
using run.

pipe(input_data)
pipe.run(input_data)

Caching details
---------------
Expand Down
5 changes: 0 additions & 5 deletions docs/roman/stpipe/user_step.rst
Original file line number Diff line number Diff line change
Expand Up @@ -243,8 +243,3 @@ the step. The previous example could be re-written as::

mystep = FlatFieldStep(override_sflat='sflat.asdf')
output = mystep.run(input)

Using the ``.run()`` method is the same as calling the instance directly.
They are equivalent::

output = mystep(input)
10 changes: 5 additions & 5 deletions docs/roman/tweakreg/tweakreg_examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ or a Roman datamodel `ImageModel`.

from romancal.tweakreg.tweakreg_step import TweakRegStep
step = TweakRegStep()
step.call([img])
step.run([img])

.. note::
If the input is a single Roman ``DataModel``,
either ``step.call([img])`` or ``step.call(img)`` will work. For multiple elements as input,
either ``step.run([img])`` or ``step.run(img)`` will work. For multiple elements as input,
they must be passed in as either a list or a ModelLibrary.

#. To run TweakReg in a Python session on an association file with the default parameters:
Expand All @@ -22,7 +22,7 @@ or a Roman datamodel `ImageModel`.

from romancal.tweakreg.tweakreg_step import TweakRegStep
step = TweakRegStep()
step.call("asn_file.json")
step.run("asn_file.json")

#. To run TweakReg on a Roman's exposure with default astrometric parameters and save
the absolute catalog data:
Expand All @@ -34,7 +34,7 @@ or a Roman datamodel `ImageModel`.
step.save_abs_catalog = True # save the catalog data used for absolute astrometry
step.abs_refcat = 'GAIADR3' # use Gaia DR3 for absolute astrometry
step.catalog_path = '/path/for/the/abs/catalog' # save the Gaia catalog to this path
step.call([img])
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This use of call didn't use the step instance so the custom parameters were ignored. I changed the other uses in this file to run to match.

step.run([img])

#. To run TweakReg using a custom source catalog with the default parameters:

Expand Down Expand Up @@ -93,4 +93,4 @@ or a Roman datamodel `ImageModel`.
step.use_custom_catalogs = True # use custom catalogs
step.catalog_format = "ascii.ecsv" # custom catalogs format
step.catfile = '/path/to/catfile/catfilename' # path to datamodel:catalog mapping
step.call([img])
step.run([img])
6 changes: 3 additions & 3 deletions romancal/outlier_detection/tests/test_outlier_detection.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ def test_outlier_do_detection_write_files_to_custom_location(tmp_path, base_imag
# set output dir for all files created by the step
outlier_step.output_dir = tmp_path.as_posix()

outlier_step(input_models)
outlier_step.run(input_models)

# meta.filename for the median image created by OutlierDetection.do_detection()
median_path = tmp_path / "drizzled_median.asdf"
Expand Down Expand Up @@ -183,7 +183,7 @@ def test_find_outliers(tmp_path, base_image, on_disk):
# make sure files are written out to disk
outlier_step.in_memory = not on_disk

result = outlier_step(input_models)
result = outlier_step.run(input_models)

expected_crs = [img_0_input_coords, img_1_input_coords, None]
with result:
Expand Down Expand Up @@ -222,7 +222,7 @@ def test_identical_images(tmp_path, base_image, caplog):
# make sure files are written out to disk
outlier_step.in_memory = False

result = outlier_step(input_models)
result = outlier_step.run(input_models)

# assert that log shows no new outliers detected
assert "0 pixels marked as outliers" in {x.message for x in caplog.records}
Expand Down
22 changes: 11 additions & 11 deletions romancal/pipeline/exposure_pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -124,12 +124,12 @@
log.info(f"Processing a WFI exposure {input_filename}")

self.dq_init.suffix = "dq_init"
result = self.dq_init(input)
result = self.dq_init.run(input)

Check warning on line 127 in romancal/pipeline/exposure_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/exposure_pipeline.py#L127

Added line #L127 was not covered by tests

if input_filename:
result.meta.filename = input_filename

result = self.saturation(result)
result = self.saturation.run(result)

Check warning on line 132 in romancal/pipeline/exposure_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/exposure_pipeline.py#L132

Added line #L132 was not covered by tests

# Test for fully saturated data
if is_fully_saturated(result):
Expand All @@ -155,16 +155,16 @@
results.append(result)
return results

result = self.refpix(result)
result = self.linearity(result)
result = self.dark_current(result)
result = self.rampfit(result)
result = self.assign_wcs(result)
result = self.refpix.run(result)
result = self.linearity.run(result)
result = self.dark_current.run(result)
result = self.rampfit.run(result)
result = self.assign_wcs.run(result)

Check warning on line 162 in romancal/pipeline/exposure_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/exposure_pipeline.py#L158-L162

Added lines #L158 - L162 were not covered by tests

if result.meta.exposure.type == "WFI_IMAGE":
result = self.flatfield(result)
result = self.photom(result)
result = self.source_catalog(result)
result = self.flatfield.run(result)
result = self.photom.run(result)
result = self.source_catalog.run(result)

Check warning on line 167 in romancal/pipeline/exposure_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/exposure_pipeline.py#L165-L167

Added lines #L165 - L167 were not covered by tests
tweakreg_input.append(result)
log.info(
f"Number of models to tweakreg: {len(tweakreg_input), n_members}"
Expand All @@ -185,7 +185,7 @@
# Now that all the exposures are collated, run tweakreg
# Note: this does not cover the case where the asn mixes imaging and spectral
# observations. This should not occur on-prem
result = self.tweakreg(results)
result = self.tweakreg.run(results)

Check warning on line 188 in romancal/pipeline/exposure_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/exposure_pipeline.py#L188

Added line #L188 was not covered by tests

log.info("Roman exposure calibration pipeline ending...")

Expand Down
14 changes: 7 additions & 7 deletions romancal/pipeline/mosaic_pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,11 +72,11 @@
if file_type == "asn":
input = ModelLibrary(input, on_disk=self.on_disk)
self.flux.suffix = "flux"
result = self.flux(input)
result = self.flux.run(input)

Check warning on line 75 in romancal/pipeline/mosaic_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/mosaic_pipeline.py#L75

Added line #L75 was not covered by tests
self.skymatch.suffix = "skymatch"
result = self.skymatch(result)
result = self.skymatch.run(result)

Check warning on line 77 in romancal/pipeline/mosaic_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/mosaic_pipeline.py#L77

Added line #L77 was not covered by tests
self.outlier_detection.suffix = "outlier_detection"
result = self.outlier_detection(result)
result = self.outlier_detection.run(result)

Check warning on line 79 in romancal/pipeline/mosaic_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/mosaic_pipeline.py#L79

Added line #L79 was not covered by tests
#
# check to see if the product name contains a skycell name & if true get the skycell record
product_name = input.asn["products"][0]["name"]
Expand Down Expand Up @@ -131,19 +131,19 @@
wcs_file = asdf.open(self.resample.output_wcs)
self.suffix = "i2d"
self.output_file = input.asn["products"][0]["name"]
result = self.resample(result)
result = self.resample.run(result)

Check warning on line 134 in romancal/pipeline/mosaic_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/mosaic_pipeline.py#L134

Added line #L134 was not covered by tests
self.sourcecatalog.output_file = self.output_file
result_catalog = self.sourcecatalog(result)
result_catalog = self.sourcecatalog.run(result)

Check warning on line 136 in romancal/pipeline/mosaic_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/mosaic_pipeline.py#L136

Added line #L136 was not covered by tests
else:
log.info("resampling a mosaic file is not yet supported")
exit(0)

else:
self.resample.suffix = "i2d"
self.output_file = input.asn["products"][0]["name"]
result = self.resample(result)
result = self.resample.run(result)

Check warning on line 144 in romancal/pipeline/mosaic_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/mosaic_pipeline.py#L144

Added line #L144 was not covered by tests
self.sourcecatalog.output_file = self.output_file
result_catalog = self.sourcecatalog(result) # noqa: F841
result_catalog = self.sourcecatalog.run(result) # noqa: F841

Check warning on line 146 in romancal/pipeline/mosaic_pipeline.py

View check run for this annotation

Codecov / codecov/patch

romancal/pipeline/mosaic_pipeline.py#L146

Added line #L146 was not covered by tests
self.suffix = "i2d"
if input_filename:
result.meta.filename = self.output_file
Expand Down