ModelContainer

class jwst.datamodels.container.ModelContainer(init=None, asn_exptypes=None, asn_n_members=None, **kwargs)

Bases: Sequence

A list-like container for holding DataModels.

This functions like a list for holding DataModel objects. It can be iterated through like a list, DataModels within the container can be addressed by index, and the datamodels can be grouped into a list of lists for grouped looping, useful for NIRCam where grouping together all detectors of a given exposure is useful for some pipeline steps.

Notes

When ASN table’s members contain attributes listed in RECOGNIZED_MEMBER_FIELDS, ModelContainer will read those attribute values and update the corresponding attributes in the meta of input models.

Example of ASN table with additional model attributes to supply custom catalogs.

"products": [
    {
        "name": "resampled_image",
        "members": [
            {
                "expname": "input_image1_cal.fits",
                "exptype": "science",
                "tweakreg_catalog": "custom_catalog1.ecsv",
                "group_id": "custom_group_id_number_1",
            },
            {
                "expname": "input_image2_cal.fits",
                "exptype": "science",
                "tweakreg_catalog": "custom_catalog2.ecsv",
                "group_id": 2
            },
            {
                "expname": "input_image3_cal.fits",
                "exptype": "science",
                "tweakreg_catalog": "custom_catalog3.ecsv",
                "group_id": Null
            },
        ]
    }
]

Warning

Input files will be updated in-place with new meta attribute values when ASN table’s members contain additional attributes.

Warning

Custom group_id affects how models are grouped both for tweakreg and skymatch steps. If one wants to group models in one way for the tweakreg step and in a different way for the skymatch step, one will need to run each step separately with their own ASN tables.

Note

group_id can be an integer, a string, or Null. When group_id is Null, it is converted to None in Python and it will be assigned a group ID based on various exposure attributes - see models_grouped property for more details.

Examples

>>> container = ModelContainer('example_asn.json')
>>> for model in container:
...     print(model.meta.filename)

Say the association was a NIRCam dithered dataset. The models_grouped attribute is a list of lists, the first index giving the list of exposure groups, with the second giving the individual datamodels representing each detector in the exposure (2 or 8 in the case of NIRCam).

>>> total_exposure_time = 0.0
>>> for group in container.models_grouped:
...     total_exposure_time += group[0].meta.exposure.exposure_time

>>> c = ModelContainer()
>>> m = datamodels.open('myfile.fits')
>>> c.append(m)

Initialize the container.

Parameters:

• init (file path, list of DataModels, or None) – If a file path, initialize from an association table. If a list, can be a list of DataModels of any type If None, initializes an empty ModelContainer instance, to which DataModels can be added via the append() method.

• asn_exptypes (str) – List of exposure types from the asn file to read into the ModelContainer, if None read all the given files.

• asn_n_members (int) – Open only the first N qualifying members.

• **kwargs (dict) – Additional keyword arguments passed to datamodel_open(), such as memmap, guess, strict_validation, etc. See datamodels.open() for a full list of available keyword arguments.

Attributes Summary

crds_observatory	Return the observatory name for CRDS queries.
group_names	List all the group names in the container.
models_grouped	Assign a grouping ID by exposure, if not already assigned.

Methods Summary

append(model)
close()	Close all datamodels.
copy([memo])	Make a deep copy of the container.
extend(model)
from_asn(asn_data)	Load fits files from a JWST association file.
get_crds_parameters()	Get CRDS parameters for this container.
ind_asn_type(asn_exptype)	Determine the indices of models corresponding to asn_exptype.
insert(index, model)
pop([index])
read_asn(filepath)	Load fits files from a JWST association file.
save([path, save_model_func])	Write out models in container to FITS or ASDF.

Attributes Documentation

crds_observatory

Return the observatory name for CRDS queries.

Returns:

The observatory name for CRDS queries.

Return type:

str

group_names

List all the group names in the container.

Returns:

A list of group names.

Return type:

list

models_grouped

Assign a grouping ID by exposure, if not already assigned.

If model.meta.group_id does not exist or it is None, then data from different detectors of the same exposure will be assigned the same group ID, which allows grouping by exposure in the tweakreg and skymatch steps. The following metadata is used when determining grouping:

meta.observation.program_number meta.observation.observation_number meta.observation.visit_number meta.observation.visit_group meta.observation.sequence_id meta.observation.activity_id meta.observation.exposure_number

If a model already has model.meta.group_id set, that value will be used for grouping.

Returns:

A list of lists of datamodels grouped by exposure.

Return type:

list

Methods Documentation

append(model)

close()

Close all datamodels.

copy(memo=None)

Make a deep copy of the container.

Parameters:

memo (dict) – Keeps track of elements that have already been copied to avoid infinite recursion.

Returns:

A deep copy of the container and all the models in it.

Return type:

ModelContainer

extend(model)

from_asn(asn_data)

Load fits files from a JWST association file.

Parameters:

asn_data (Association) – An association dictionary

get_crds_parameters()

Get CRDS parameters for this container.

Notes

stpipe requires ModelContainer to have a crds_observatory attribute in order to pass through step.run(), but it is never accessed.

ind_asn_type(asn_exptype)

Determine the indices of models corresponding to asn_exptype.

Parameters:

asn_exptype (str) – Exposure type as defined in an association, e.g. “science”.

Returns:

ind – Indices of models in ModelContainer._models matching asn_exptype.

Return type:

list

insert(index, model)

pop(index=-1)

static read_asn(filepath)

Load fits files from a JWST association file.

Parameters:

filepath (str) – The path to an association file.

Returns:

An association dictionary

Return type:

dict

save(path=None, save_model_func=None, **kwargs)

Write out models in container to FITS or ASDF.

Parameters:

• path (str or None) –

  • If None, the meta.filename is used for each model.

  • If a string, the string is used as a root and an index is appended, along with the ‘.fits’ extension.

• save_model_func (func or None) – Alternate function to save each model instead of the models save method. Takes one argument, the model, and keyword argument idx for an index.

• **kwargs (dict) – Additional parameters to be passed to the save method of each model.

Returns:

output_paths – List of output file paths of where the models were saved.

Return type:

[str[, …]]