201.1. Object table

201.1. Object table¶

For the Rubin Science Platform at data.lsst.cloud.
Data Release: Data Preview 1
Container Size: large
LSST Science Pipelines version: r29.1.1
Last verified to run: 2025-06-21
Repository: github.com/lsst/tutorial-notebooks

Learning objective: To understand the contents of the Object table and how to access it.

LSST data products: Object

Packages: lsst.rsp, lsst.daf.butler

Credit: Originally developed by the Rubin Community Science team. Please consider acknowledging them if this notebook is used for the preparation of journal articles, software releases, or other notebooks.

Get Support: Everyone is encouraged to ask questions or raise issues in the Support Category of the Rubin Community Forum. Rubin staff will respond to all questions posted there.

1. Introduction¶

The Object table contains forced measurements in the $ugrizy$ deep_coadd images at the sky coordinates of every source detected in any individual Visit image, or in a deep_coadd image in any filter, with signal-to-noise ratio $\geq5$.

The Object table contains only deblended objects (the "children" of deblending only, no "parent" objects), and duplicates have been rejected (e.g., objects in the overlapping edges of deep_coadd images).

• TAP table name: dp1.Object
• butler table name: object
• columns: 1296
• rows: 2,299,757

Related tutorials: The TAP and Butler data access services are demonstrated in the 100-level "How to" tutorials. There is a 200-level tutorial on deep_coadd images.

1.1. Import packages¶

Import standard python packages re, numpy, matplotlib, and astropy.

From the lsst package, import modules for the TAP service and the butler.

In [1]:

import re
import numpy as np
import matplotlib.pyplot as plt
from lsst.rsp import get_tap_service
from lsst.daf.butler import Butler

1.2. Define parameters and functions¶

Create an instance of the TAP service, and assert that it exists.

In [2]:

service = get_tap_service("tap")
assert service is not None

Create an instance of the Rubin data Butler, and assert that it exists.

In [3]:

butler = Butler('dp1', collections="LSSTComCam/DP1")
assert butler is not None

2. Schema (columns)¶

To browse the table schema visit the Rubin schema browser, or use the TAP service via the Portal Aspect or as demonstrated in Section 2.1.

2.1. Retrieve table schema¶

To retrieve the table schema, define a query for the schema columns of the Object table and run the query job.

In [4]:

query = "SELECT column_name, datatype, description, unit " \
        "FROM tap_schema.columns " \
        "WHERE table_name = 'dp1.Object'"
job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

Retrieve the query results and display them as an astropy table with the to_table() attribute.

In [5]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
results

Out[5]:

Table length=1296

column_name	datatype	description	unit
str64	str64	str512	str64
coord_dec	double	Fiducial ICRS Declination of centroid used for database indexing	deg
coord_decErr	float	Error in fiducial ICRS Declination of centroid	deg
coord_ra	double	Fiducial ICRS Right Ascension of centroid used for database indexing	deg
coord_ra_dec_Cov	float	Covariance between fiducial ICRS Right Ascension and Declination of centroid	deg**2
coord_raErr	float	Error in fiducial ICRS Right Ascension of centroid	deg
deblend_failed	boolean	Deblender failed to deblend this source
deblend_incompleteData	boolean	One or more bands were not deblended due to an inability to model the PSF.
deblend_isolatedParent	boolean	Deblender skipped this footprint because there was only a single peak
deblend_iterations	int	Number of iterations during deblending
deblend_logL	float	Log likelihood of the entire blend in scarlet_lite.
deblend_masked	boolean	Deblender skipped this source because there were too many masked pixels.
deblend_nChild	int	Number of children this object has (defaults to 0)
deblend_nPeaks	int	Number of peaks this parent footprint has (even if the deblender failed or skipped this blend)
deblend_parentTooBig	boolean	Deblender skipped this source because the parent footprint was too large.
deblend_peak_center_x	int	x-coordinate of the peak after source detection	pixel
deblend_peak_center_y	int	y-coordinate of the peak after source detection	pixel
deblend_skipped	boolean	Deblender skipped this source
deblend_tooManyPeaks	boolean	Deblender skipped this source because there were too many peaks in the Footprint.
detect_fromBlend	boolean	This source is deblended from a parent with more than one child.
detect_isDeblendedModelSource	boolean	True if source has no children and is in the inner region of a coadd patch and is in the inner region of a coadd tract and is not detected in a pseudo-filter (see config.pseudoFilterList) and is a deblended child
detect_isIsolated	boolean	This source is not a part of a blend.
ebv	float	E(B-V) at coord_ra/coord_dec per Schlegel, Finkbeiner & Davis (1998)	mag
...	...	...	...
z_psfFlux_flag_noGoodPixels	boolean	Not enough non-rejected pixels in data to attempt the fit. Forced on z-band.
z_psfFluxErr	float	Flux uncertainty derived from linear least-squares fit of PSF model. Forced on z-band.	nJy
z_psfMag	float	AB magnitude of the z-band psfFlux.	mag
z_psfMagErr	float	Uncertainty in magnitudes of the z-band psfFlux.	mag
z_psfModel_TwoGaussian_chisq_reduced	float	Reduced chi-squared of the best-fit model parameters (chi divided the number of data points) for the two-Gaussian PSF model (z-band).
z_psfModel_TwoGaussian_gauss1_fluxfrac	float	Fraction of the total flux (normalized to unity) of the first Gaussian component in the two-Gaussian PSF model (z-band).
z_psfModel_TwoGaussian_gauss1_rho	float	Ellipse rho (correlation coefficient) of the first Gaussian component in the two-Gaussian PSF model (z-band).
z_psfModel_TwoGaussian_gauss1_sigma_x	float	Standard deviation of the first Gaussian component (x-axis) in the two-Gaussian PSF model (z-band).	pixel
z_psfModel_TwoGaussian_gauss1_sigma_y	float	Standard deviation of the first Gaussian component (y-axis) in the two-Gaussian PSF model (z-band).	pixel
z_psfModel_TwoGaussian_gauss2_rho	float	Ellipse rho (correlation coefficient) of the second Gaussian component in the two-Gaussian PSF model (z-band).
z_psfModel_TwoGaussian_gauss2_sigma_x	float	Standard deviation of the second Gaussian component (x-axis) in the two-Gaussian PSF model (z-band).	pixel
z_psfModel_TwoGaussian_gauss2_sigma_y	float	Standard deviation of the second Gaussian component (y-axis) in the two-Gaussian PSF model (z-band).	pixel
z_psfModel_TwoGaussian_n_iter	int	Number of iterations in the non-linear fit for the two-Gaussian PSF model (z-band).
z_psfModel_TwoGaussian_no_inputs_flag	boolean	Flag set for objects not fit because there were no coadd PSF inputs for the two-Gaussian PSF model (z-band).
z_psfModel_TwoGaussian_unknown_flag	boolean	Flag set for failures with an unexpected or unknown cause for the two-Gaussian PSF model (z-band).
z_ra	double	Position in right ascension, measured on z-band.	deg
z_ra_dec_Cov	float	Covariance between right ascension and declination, measured on z-band.	deg**2
z_raErr	float	Error in right ascension, measured on z-band.	deg
z_sersicFlux	float	z-band flux from the multiband Sersic model fit.	nJy
z_sersicFluxErr	float	Error on the z-band flux from the multiband Sersic model fit.	nJy
z_sizeExtendedness	float	Moments-based measure of an object to be a galaxy. Measured on z-band.
z_sizeExtendedness_flag	boolean	Set to 1 for any fatal failure. Measured on z-band.

The table displayed above has been truncated.

Option to print every column name as a list.

In [6]:

# for col in results['column_name']:
#     print(col)

Option to use the regular expressions package re to find all column names that hold the psfFlux for the six filters.

In [7]:

# for col in results['column_name']:
#     if re.fullmatch('[ugrizy]_psfFlux', col):
#         print(col)

Option to search column names that contain the string temp.

In [8]:

# temp = 'Err'
# temp = 'pixelFlag'
# temp = 'extend'
# temp = 'cModel'
# for col in results['column_name']:
#     if re.search(temp, col):
#         print(col)

Delete the job, but not the results.

In [9]:

del query
job.delete()

2.2. Key columns¶

Of the $>$1000 columns of the Object table, a few tens will be the most commonly used.

2.2.1. Object Id¶

The long integer that uniquely identifies each row of the Object table.

• objectId

2.2.2. Coordinates¶

The sky coordinates in decimal degrees for each object:

• coord_ra, coord_raErr
• coord_dec, coord_decErr

Per-filter coordinates in degrees are in columns [f]_ra, [f]_dec.

Per-filter centroids in pixels are in columns [f]_centroid_x, [f]_centroid_y.

The all-sky tesselation regions are stored in columns tract and patch.

2.2.3. Photometry¶

Fluxes ($f$) are in nanoJanskys and can be converted to AB magnitudes ($m$) with:

$m = -2.5\log(f) + 31.4$

Magnitudes

Magnitude conversions have been done for the TAP tables, for the PSF and cModel fluxes. These columns are not available via the Butler.

• [f]_psfMag, [f]_psfMagErr
• [f]_cModelMag, [f]_cModelMagErr

PSF fluxes

A forced fit of the Point Spread Function (PSF) at the object's coordinates in each image.

PSF fluxes are best to use for point-like sources (e.g., stars).

• [f]_psfFlux, [f]_psfFluxErr

cModel fluxes

The SDSS Composite Model, the linear combination of the best fit exponential and de Vaucouleurs profile fits in each band.

The cModel fluxes are best used for extended objects (e.g., galaxies).

• [f]_cModelFlux, [f]_cModelFluxErr

Other extended-object photometry measurements

• [f]_sersicFlux: Sersic flux (NED documentation)
• [f]_gaap*: GaaP fluxes (Kuijken et al. 2008)
• [f]_kronFlux: Kron flux (NED documentation)
• [f]_ap*: aperture fluxes (radii; pixels)

Milky Way dust extinction

The value of E(B-V) at the coordinates coord_ra, coord_dec (Schlegel, Finkbeiner & Davis (1998)).

• ebv

2.2.4. Shapes¶

HSM moments

Hirata-Seljak-Mandelbaum (HSM) moments (Hirata & Seljak 2003, Mandelbaum et al. 2005):

• shape_xx, _yy, and _xy
• [f]_ixx, _iyy, and _ixy

Extendedness (star/galaxy separation)

If the product of the cModel flux and a configurable flux ratio (0.985) is less than the PSF flux, the object is "not extended" (extendedness = 0, or False). If it is greater than the PSF flux, the object is considered "extended" (extendedness = 1, or True)

• refExtendedness (extendedness in band refBand)
• [f]_extendedness (extendedness in filter [f])

2.2.5. Flags¶

Pixel flags

A variety of flags indicating whether pixels that are saturated, or affected by cosmic rays, contributed to the object's measurements.

• [f]_pixelFlags_*

Measurement flags

The flux and shape measurements mentioned above have associated flag columns suffixed with _flag.

Deblending flags

If the object was deblended from a parent with more than one child, this flag is True:

• detect_fromBlend

The objectId of the parent.

• parentObjectId

Blendedness flag

A measure of how much the flux is affected by neighbors $1 - \frac{f_{\rm child}}{f_{\rm parent}}$. This uses the absolute value of the instrumental flux to try to obtain a de-noised value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70.

• [f]_blendedness

2.3. Descriptions and units¶

For a subset of the key columns show the table of their descriptions and units.

In [10]:

col_list = set(['objectId', 'coord_ra', 'coord_dec',
                'r_psfFlux', 'r_psfFluxErr',
                'r_cModelFlux', 'r_cModelFluxErr',
                'r_extendedness', 'r_blendedness'])
tx = [i for i, item in enumerate(results['column_name']) if item in col_list]
results[tx]

Out[10]:

Table length=9

column_name	datatype	description	unit
str64	str64	str512	str64
coord_dec	double	Fiducial ICRS Declination of centroid used for database indexing	deg
coord_ra	double	Fiducial ICRS Right Ascension of centroid used for database indexing	deg
objectId	long	Unique id. Unique ObjectID
r_blendedness	float	Measure of how much the flux is affected by neighbors, (1 - child_flux/parent_flux). Operates on the absolute value of the pixels to try to obtain a de-noised value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details. Measured on r-band.
r_cModelFlux	float	Flux from the final cmodel fit. Forced on r-band.	nJy
r_cModelFluxErr	float	Flux uncertainty from the final cmodel fit. Forced on r-band.	nJy
r_extendedness	float	Set to 1 for extended sources, 0 for point sources. Measured on r-band.
r_psfFlux	float	Flux derived from linear least-squares fit of PSF model. Forced on r-band.	nJy
r_psfFluxErr	float	Flux uncertainty derived from linear least-squares fit of PSF model. Forced on r-band.	nJy

Clean up.

In [11]:

del col_list, tx, results

3. Data access¶

The Object table is available via the TAP service and the butler.

Recommended access method: TAP.

3.1. Advisory: avoid full-table queries¶

Avoid full-table queries. Always include spatial constraints.

The Object table is a large, inclusive, union set of measurements made in the deeply coadded images at the locations of all objects detected in any image.

The DP1 data release Object table is relatively small and full-table TAP queries can run in minutes. However, skipping spatial constraints is not a good habit to form, because future data release Object tables will contain billions of rows.

3.2. TAP (Table Access Protocol)¶

The Object table is stored in Qserv and accessible via the TAP services using ADQL queries.

Include spatial constraints: Qserv stores catalog data sharded by coordinate (RA, Dec), so ADQL query statements that include constraints by coordinate do not requre a whole-catalog search and are typically faster (and can be much faster) than ADQL query statements which only include constraints for other columns. Use either an ADQL cone or polygon search for faster queries (do not use WHERE ... BETWEEN statements to set boundaries on RA and Dec).

3.2.1. Demo query¶

Define a query to return the nine "key columns" from Section 2.3.

Impose spatial constraints: search within 0.2 degrees of the center of the Extended Chandra Deep Field South (ECDFS) field, RA, Dec = $53.13 -28.10$.

In [12]:

query = "SELECT objectId, coord_ra, coord_dec, r_extendedness, r_blendedness, " \
        "r_psfMag, r_psfMagErr, r_cModelMag, r_cModelMagErr " \
        "FROM dp1.Object " \
        "WHERE CONTAINS(POINT('ICRS', coord_ra, coord_dec), " \
        "CIRCLE('ICRS', 53.13, -28.10, 0.2)) = 1 " \
        "ORDER BY coord_ra ASC "
job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

Fetch the results as an astropy table.

In [13]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
print(len(results))

53880

Option to display the table.

In [14]:

# results

As an example, plot the coordinates and the magnitude vs. the error for objects with an r-band cModel magnitude brighter than 25.

In [15]:

tx = np.where(results['r_cModelMag'] <= 25.0)[0]
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(7, 3))
ax1.plot(results['coord_ra'][tx], results['coord_dec'][tx],
         'o', ms=2, mew=0, alpha=0.4, color='grey')
ax1.set_xlabel('Right Ascension')
ax1.set_ylabel('Declination')
ax2.plot(results['r_cModelMag'][tx], results['r_cModelMagErr'][tx],
         'o', ms=2, mew=0, alpha=0.4, color='grey')
ax2.set_xlabel('Apparent Magnitude')
ax2.set_ylabel('Error')
plt.tight_layout()
plt.show()

Figure 1: At left, the RA vs. Dec of retrieved objects is shown as a circle of grey points. At right, the r-band magnitude vs. its uncertainty shows how magnitude error increases for fainter objects.

Clean up.

In [16]:

job.delete()
del results

3.2.2. Joinable tables¶

The Object table can be joined to the ForcedSource table on the column objectId.

The ForcedSource table contains forced PSF photometry in the processed visit images and difference images at the sky coordinates of every object.

The following query joins the Object and ForcedSource tables. Columns returned include the object and forcedSource unique identifiers, and the r-band PSF flux from the Object table. The query is for objects that are within 0.1 degrees of the ECDFS field center, are not extended (are point-like), and have an apparent magnitude $20 < r < 25$ mag.

In [17]:

query = "SELECT o.objectId, o.r_psfFlux " \
        "FROM dp1.Object AS o " \
        "JOIN dp1.ForcedSource AS fs ON o.objectId = fs.objectId " \
        "WHERE CONTAINS(POINT('ICRS', o.coord_ra, o.coord_dec), " \
        "CIRCLE('ICRS', 53.13, -28.10, 0.1)) = 1 " \
        "AND o.refExtendedness = 0 " \
        "AND o.r_psfMag > 20 AND o.r_psfMag < 25 " \
        "ORDER BY o.objectId ASC"
job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

In [18]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
print(len(results))

132368

Show that this query returned data for 177 unique objects (as denoted by their objectId), with an average of $\sim752$ forced source measurements each.

In [19]:

values, counts = np.unique(results['objectId'], return_counts=True)
print(len(values))
print(np.mean(counts))

176
752.0909090909091

Option to display the table.

In [20]:

# results

Clean up.

In [21]:

job.delete()
del results, values, counts

3.3. Butler¶

TAP is the recommended way to access the object table, but the Butler is a convenient way to retrieve all the objects in a given tract.

Show that the only dimension for the object table is the skymap's tract, and that it is required.

In [22]:

butler.get_dataset_type('object')

Out[22]:

DatasetType('object', {skymap, tract}, ArrowAstropy)

In [23]:

butler.get_dataset_type('object').dimensions.required

Out[23]:

{skymap, tract}

3.3.1. Demo query¶

Include spatial constraints: The Butler object table contents are stored and retrieved by individual tract.

Retrieve all dataset_refs for object tables for tracts that overlap the coordinates near the center of the ECDFS field.

In [24]:

query = "tract.region OVERLAPS POINT(53.13, -28.10)"
refs = butler.query_datasets("object", where=query)

Show that one tract overlaps the coordinates.

In [25]:

for ref in refs:
    print(ref.dataId)

{skymap: 'lsst_cells_v1', tract: 5063}

Define the columns to retrieve.

In [26]:

col_list = ['objectId', 'coord_ra', 'coord_dec',
            'r_psfFlux', 'r_psfFluxErr',
            'r_cModelFlux', 'r_cModelFluxErr',
            'r_extendedness', 'r_blendedness']

Get the data from the butler.

In [27]:

results = butler.get(refs[0],
                     parameters={'columns': col_list})

Option to display the results.

In [28]:

# results

Constrain the results to non-extended objects with r-band magnitudes between 20 and 25 mag.

In [29]:

tx = np.where((results['r_extendedness'] == 0)
              & (results['r_cModelFlux'] > 360.0)
              & (results['r_cModelFlux'] <= 36000.0))[0]
print(len(tx))

8585

Plot the sky coordinates and cModel fluxes and flux errors for the subset of objects.

In [30]:

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(7, 3))
ax1.plot(results['coord_ra'][tx], results['coord_dec'][tx],
         'o', ms=2, mew=0, alpha=0.1, color='grey')
ax1.set_xlabel('Right Ascension')
ax1.set_ylabel('Declination')
ax2.plot(-2.5 * np.log10(results['r_cModelFlux'][tx]) + 31.4,
         results['r_cModelFluxErr'][tx]/results['r_cModelFlux'][tx],
         'o', ms=2, mew=0, alpha=0.1, color='grey')
ax2.set_xlabel('cModel Mag')
ax2.set_ylabel('Error/Flux')
plt.tight_layout()
plt.show()

Figure 2: At left, the RA vs. Dec of retrieved objects as grey points. At right, the r-band cModel Flux vs. its uncertainty shows how flux error increases for fainter objects.

In [31]:

del query, refs, col_list, results, tx

In [ ]: