Skip to content

Specifying models

Jump to bottom
Gary Bernstein edited this page Apr 10, 2017 · 4 revisions

The PhotoFit and WCSFit programs optimize the parameters of mappings from cataloged magnitudes and coordinates, respectively, into calibrated magnitudes or sky positions that are consistent across exposures and across array positions. We need to tell these programs what functions and parameters define the maps the are optimizing.

PixelMaps and PhotoMaps

An astrometric (photometric) transformation is embodied as an instance of the class PixelMap (PhotoMap) defined in the astrometry (photometry) repository. There are LaTeX documents defining these classes in detail in the doc/ directories of each repository. A quick summary is given here.

PixelMap is a base class representing any parameterized transformation from an input 2d position (xpix,ypix) to an output pair (xworld,yworld). Derived from this are the atomic transformations such as the Identity map, a LinearMap, PolyMap, etc. Each has some number of free parameters (0 in the case of Identity).

The ColorTerm is a PixelMap which defines its transformation by being given a reference color and some other PixelMap* pm. Its transformation is defined by scaling pm's transformation linearly with the object's color:

      double c = color-reference; 
      pm->toWorld(xpix,ypix,xworld,yworld);
      xworld = xpix + c*(xworld-xpix);
      yworld = ypix + c*(yworld-ypix);

Its free parameters are those of the PixelMap being scaled.

Most important is the composite map, which represents the serial application of any number of other PixelMaps, known as its elements. Its free parameters are the union of those of its elements. This allows WCSFit to fit models that are assembled from building blocks that may be shared by other catalogs using the same device, or by other devices in the same exposure, etc. The PixelMapCollection class allows us to treat as a unit all of the atomic maps that are involved in the astrometric solution, and the definitions of all the composite combinations that yield the final transformation for each input catalog. Every atomic or composite member of the collection has a unique name.

The PhotoMap system is completely analogous, with the slight difference that a PhotoMap takes as arguments not just the input magnitude, but also a PhotoArguments object holding all the variables upon which the transformation might depend:

  class PhotoArguments {
  public:
    double xDevice;
    double yDevice;
    double xExposure;
    double yExposure;
    double color;
  };

There is one class in the astrometric system, WCS, which has no photometric counterpart. A world coordinate system (WCS) is a PixelMap which includes an association of the output (xworld,yworld) coordinates to a definitive location on the celestial sphere. To do this, each WCS supplements its PixelMap with a Projection object which defines the applicable map between the ICRS celestial sphere and the (xworld,yworld) system. Most commonly we will use the Gnomonic projection about a pole on the telescope axis or at the center of the field. See the documentation in the astrometry repository.

Pixel/PhotoMap serialized formats

Every PhotoMap, PixelMap, and WCS can be (de)serialized to (from) YAML format. The PixelMapCollection and PhotoMapCollection classes know how to read and write themselves to/from these files, which we usually suffix with .astro and .photo, respectively.

The root node of an .astro YAML file is a dictionary (map, in YAML parlance, but I will use "dict" instead to avoid confusion with our pixel maps). Each key in the dict is the name of one atomic or composite PixelMap. The value associated with the key is another dict. The value at the type key says which kind of PixelMap this is, and further key/value pairs give the specifications and parameters of this transformation. Here is an example of part of an astrometric model file:

Y/N10:
  Type: Composite
  Elements:
    - Y/N10/rings
    - Y/N10/lowedge
    - Y/N10/highedge
    - Y/N10/poly
    - Y/N10/color
Y/N10/color:
  Type: Color
  Reference: 0.61
  Function:
    Type: Linear
    Coefficients: [-2.008252062591854e-05, 0.9999740470268519, 1.701381720487465e-06, -2.694100229930706e-06, -3.356349311374476e-06, 0.9999901630314523]
Y/N10/highedge:
  Type: Template
  HasSplit: false
  Filename: astroedges1.yaml
  LowTable: N10/high
  Parameter: 0.5633780401990416
Y/N10/lowedge:
  Type: Template
  HasSplit: false
  Filename: astroedges1.yaml
  LowTable: N10/low
  Parameter: 0.6481938934129615
Y/N10/poly:
  Type: Poly
  XMin: 0
  XMax: 2048
  YMin: 0
  YMax: 4096
  XPoly:
    SumOrder: true
    OrderX: 4
    Coefficients: [-0.1558206509818649, 4.168502331327966e-05, 0.1499827617412307, 3.638945627239186e-06, -2.272610751735557e-05, 4.516582229678883e-05, 3.22288388233415e-07, -3.107921789445971e-06, -2.928767631434694e-07, -1.224585288812819e-05, 5.552970056629137e-08, 3.586154123589477e-07, -1.231460667404401e-06, 3.348128795994819e-07, -2.221665894590155e-06]
  YPoly:
    SumOrder: true
    OrderX: 4
    Coefficients: [-0.2470774629040613, -0.07496679592088366, -1.145523849385519e-05, 1.649850381987998e-05, -1.513763009659413e-05, 2.305987859413278e-05, 8.673768555875327e-07, 6.095134121341583e-07, 5.046683868891156e-06, 5.890847923904075e-08, 3.29613841691984e-07, 1.249286266200388e-06, -7.765236502772289e-07, 7.381100665336743e-07, -8.099278208994903e-08]

We see that this snippet defines a composite map named Y/N10 (for the N10 device in the Y filter) which contains a color term (linear in coordinates) and then templated functional forms for tree rings and edge distortions, and finally a 4th-order polynomial map from the pixel coordinates (which will leave us in an array-wide coordinate system).

The output of a typical WCSFit or PhotoFit run will be a file containing hundreds or thousands of such specifications. But we can also use a much more compact version of the same format to specify, on input, the structure of the model that we want to fit.

Default values

All of the atomic pixel/photo maps have a Parameters or Coefficients key that holds the values of the fitted parameters for the map. But when specifying the model structure we don't know these value and may not even have a good guess. Each of the types of atomic maps have default values for their parameters, which will be adopted if the relevant key/value pair is absent. Furthermore, the WCSFit code knows which parameters have been defaulted. The codes will initialize the defaulted values by setting them to best match the input WCS (i.e. the one from SCAMP).

Variable substitution

A WCSFit or PhotoFit may involve hundreds or thousands of exposures and thousands of extensions. If we had to specify each one's model structure we'd go crazy. Thus we have a templating mechanism for the input model specifications. When it comes time to define the map for a given extension, the YAML serialization will be scanned for the appearance of any of the following strings:

  • EXPOSURE
  • DEVICE
  • INSTRUMENT
  • BAND
  • EPOCH

Any occurrence of these strings will be replaced by the (string-valued) name of the relevant quantity for that extension. Furthermore, for an extension that records detections in exposure D256882 and device N10, the codes will look in your input map specifications for a map that has the name

  • D256882/N10 for PhotoFit
  • D256882/N10/base for WCSFit (for the PixelMap; the WCS gets named D256882/N10).

The codes will look through the list of input maps you provide for any names that match this one. You may have given a map with exactly this name. But you also might have given WCSFit a YAML file including this:

EXPOSURE/DEVICE/base:
  Type: Composite
  Elements: [INSTRUMENT/DEVICE, EXPOSURE/dcr, EXPOSURE]

After variable substitution, this is also a match to the desired map! So this will be used. When there are multiple input maps that match a desired name, the one with the fewest variable substitutions is chosen. If the lowest number of variable substitutions is accomplished with multiple input maps, the results are unpredictable. You can therefore have generic rules, but create specialized exceptions to them.

If we use the above serialization for our extension's map, we can see that it will then need to compose three other maps with names z/N10, D256882/dcr, and D256882 (if this data is from z band). Once again it will search your input map specifications for maps with these names, or names that match after variable substitution. This will occur recursively for each compound map.

Examples

To make this more concrete, here are the input map specifications that fully describe the astrometric and photometric models we apply to single epochs of DECam starflat sequences. Note that in both cases, the model for a given extension is divided into two parts, one that is characteristic of the instrument and device but independent of exposure, followed by another that is specific to the exposure and independent of the device, usually a focal-plane-wide polynomial of some kind.

Standard DECam photometric functions

# YAML input file for photometric solution for
# a single epoch of star flats
EXPOSURE/DEVICE:
  Type: Composite
  Elements: [BAND/DEVICE, EXPOSURE]
# The instrumental solution has several parts
BAND/DEVICE:
  Type: Composite
  Elements:
    - BAND/DEVICE/rings
    - BAND/DEVICE/lowedge
    - BAND/DEVICE/highedge
    - BAND/DEVICE/poly
    - BAND/DEVICE/color
# Fit edges and rings using templates
BAND/DEVICE/lowedge:
  Type: Template
  HasSplit: false
  Filename: photoedges1.yaml
  LowTable: BAND/DEVICE_left
  Parameter: 1.
BAND/DEVICE/highedge:
  Type: Template
  HasSplit: false
  Filename: photoedges1.yaml
  LowTable: BAND/DEVICE_right
  Parameter: 1.
BAND/DEVICE/rings:
  Type: Template
  HasSplit: false
  Filename: photorings4.yaml
  LowTable: DEVICE
  Parameter: 1.
# Cubic polynomial per CCD
BAND/DEVICE/poly:
  Type: Poly
  UsePixelCoords: True
  XMin: 0.
  XMax: 2048.
  YMin: 0.
  YMax: 4096.
  Poly:
    SumOrder: true
    OrderX: 3
# And color term that is linear per CCD
BAND/DEVICE/color:
  Type: Color
  Reference: 0.61
  Function: 
    Type: Poly
    UsePixelCoords: True
    XMin: 0.
    XMax: 2048.
    YMin: 0.
    YMax: 4096.
    Poly:
      SumOrder: true
      OrderX: 1
# The exposure solution is just a constant zeropoint
# shift for each exposure
EXPOSURE:
  Type: Constant

Standard DECam astrometric functions

# Startup YAML file specifying the final chromatic model
# for star flat astrometry
EXPOSURE/DEVICE/base:
  Type: Composite
  Elements: [INSTRUMENT/DEVICE, EXPOSURE/dcr, EXPOSURE]
# Exposure has a free overall linear distortion
EXPOSURE:
  Type: Linear
# Plus a color term for differential
# chromatic refraction, that will look like this with
# except we'd fill in the coefficient in advance from
# knowledge of the conditions of the exposure
EXPOSURE/dcr:
  Type: Color
  Function:
    Type: Constant
  Reference: 0.61
# Now specify the chain of transformations for the instrument
INSTRUMENT/DEVICE:
  Type: Composite
  Elements:
    - BAND/DEVICE/rings
    - BAND/DEVICE/lowedge
    - BAND/DEVICE/highedge
    - BAND/DEVICE/poly
    - BAND/color
# Fit edges and rings using templates;
BAND/DEVICE/lowedge:
  Type: Template
  HasSplit: false
  Filename: astroedges1.yaml
  LowTable: DEVICE/low
  Parameter: 1.
BAND/DEVICE/highedge:
  Type: Template
  HasSplit: false
  Filename: astroedges1.yaml
  LowTable: DEVICE/high
  Parameter: 1.
BAND/DEVICE/rings:
  Type: Template
  HasSplit: false
  Filename: astrorings4.yaml
  LowTable: DEVICE
  Parameter: 1.
# Quartic polynomial distortion per CCD
BAND/DEVICE/poly:
  Type: Poly
  XMin: 0
  XMax: 2048
  YMin: 0
  YMax: 4096
  XPoly:
    SumOrder: true
    OrderX: 4
  YPoly:
    SumOrder: true
    OrderX: 4
# And a predetermined lateral color function
# that we might generically ignore
BAND/color:
  Type: Identity
# But specify in g band (we do for r band too)
g/color:
  Type: Color
  Reference: 0.61
  Function:
    Type: Piecewise
    Axis: R
    XCenter: 0
    YCenter: 0
    ArgStart: 0.2
    ArgStep: 0.02
    Parameters: [0,...]  # Skipping the coefficient list here for brevity

Reuse and recycle

Even the above examples are more than you will usually need to specify for your model-fitting. Both WCSFit and PhotoFit have parameters inputMaps which allow you to specify multiple input YAML files specifying your models. This means that you can include in your input specifications some old solution to your model (or parts of it) and these will be read and used if they have names that match those your model needs. So for example if I wanted the starting CCD polynomial solutions to my PhotoFit to be those derived in a previous season, I would just append my previous season's .astro file to the inputMaps parameter after the file given above. I could omit the BAND/DEVICE/poly entry above, but I don't have to because the old solutions will be more specific (no variables in the names) and will take precedence.

In fact you need to delete from the old solution's file any maps specifications that you do not want to take precedence over the templated forms in the input file above.

In the case of PhotoFit, for example, if I am producing updated solutions of the same form as an old one, my input .astro file can be as brief as

EXPOSURE/DEVICE:
  Type: Composite
  Elements: [BAND/DEVICE, EXPOSURE]
EXPOSURE:
  Type: Constant

since the entire format of the instrument solutions (and a good starting value) are already in the old .astro file that we can add to the inputMaps list.

As the WCSFit and PhotoFit pages document, there is also a fixMaps parameter, which allows you to freeze the parameter values for any set of named input maps. We might for instance want to keep the color terms from a previous season, or hold the parameters of the dcr models fixed because they are fully determined by geometry.

Clone this wiki locally