206.1. Deblender outputs

206.1. Deblender outputs¶

For the Rubin Science Platform at data.lsst.cloud.
Data Release: DP1
Container Size: Large
LSST Science Pipelines version: Weekly v29.2.0
Last verified to run: 2025-10-22
Repository: github.com/lsst/tutorial-notebooks

Learning objective: This notebook demonstrates the meaning and basic use of deblender data products cataloged in the Object table.

LSST data products: Object table, deep_coadd

Packages: lsst.rsp, lsst.daf.butler, lsst.geom, lsst.afw.display

Credit: Originally developed by Christina Williams and the Rubin Community Science team. This notebook is partly based on DP0.2 notebook 10 on deblender data products, and benefited from helpful discussions with Fred Moolekamp. The deblender packages are based on Melchior et al. 2018, and the Scarlet software written by Peter Melchior and Fred Moolekamp. Further documentation on Scarlet is available at that site.

Get Support: Everyone is encouraged to ask questions or raise issues in the Support Category of the Rubin Community Forum. Rubin staff will respond to all questions posted there.

1. Introduction¶

This notebook is an introduction to the data products in the Object table that are associated with the deblending process.

In order to build its source catalog and measure photometry of objects in images, the LSST Science Pipeline uses the multi-wavelength deblending algorithm Scarlet (Melchior et al. 2018) to identify independent detected components within source blends and separate the flux from independent objects.

A "parent" source is identified as a region of connected pixels above a given signal-to-noise ratio (S/N) in the deepCoadd image. These regions are called footprints. Each footprint may have one or more peaks, which the deblender uses to infer the number and positions of objects blended in each footprint. The deblended sub-peaks are referred to as "children" of the parent in the properties saved by the deblender in the Object table.

The Scarlet deblender separates the flux among components by modeling the flux and shape of each source making up the blend.

This tutorial will identify a heavily blended parent object with many children and explore how to interpret the deblender-related measurements in the Object table. It will also demonstrate how to use deblender-related columns in the Object table to identify unique deblended child sources, and provide an example of how to access metadata that is available for each deblended source (e.g. the deblended footprints and pixel-weight maps).

Note: There have been updates to the deblending implementation since DP1 so expect the API to be slightly different (improved) in future data releases, including having more information available to understand how blended an object is with its neighbors and useful cuts to remove potentially more difficult blends from analysis.

References:

• An introduction to all of the available deblending flags, and the processes that produce the deblender data products discussed in this tutorial can be found in this overview of the deblending flags.
• A general discussion of blending impacts to LSST can be found in this article titled "The challenge of blending in large sky surveys" by Melchior et al. 2021.
• Some more in-depth presentations on the deblending implementation in the LSST Science Pipelines are available from recorded talks during the Rubin Project and Community Workshop 2022's session on "Deblending Plans and Challenges".

Related tutorials: A notebook demonstrating how to work with blend footprints is available as 206.2 Deblender footprints.

1.1. Import packages¶

The matplotlib (and especially sublibrary matplotlib.pyplot), numpy, and astropy libraries are widely used Python libraries for plotting, scientific computing, and astronomical data analysis.

The lsst.rsp package provides access to the Table Access Protocol (TAP) service for queries to the DP0 catalogs.

The lsst.afw.display library provides access to image visualization routines and the lsst.daf.butler library is used to access data products via the butler.

In [1]:

import numpy as np
import matplotlib.pyplot as plt

from lsst.daf.butler import Butler

from lsst.rsp import get_tap_service
import lsst.geom as geom
import lsst.afw.display as afwDisplay

1.2. Define functions and parameters¶

The following cutout_coadd function enables the visualization of deblended products by making a cutout.

In [2]:

def cutout_coadd(butler, ra, dec, band='r', datasetType='deepCoadd',
                 skymap=None, cutoutSideLength=51, **kwargs):
    """
    Produce a cutout from a coadd at the given ra, dec position.

    Adapted from DC2 tutorial notebook by Michael Wood-Vasey.

    Parameters
    ----------
    butler: lsst.daf.persistence.Butler
        Helper object providing access to a data repository
    ra: float
        Right ascension of the center of the cutout, in degrees
    dec: float
        Declination of the center of the cutout, in degrees
    band: string
        Filter of the image to load
    datasetType: string ['deepCoadd']
        Which type of coadd to load.  Doesn't support 'calexp'
    skymap: lsst.afw.skyMap.SkyMap [optional]
        Pass in to avoid the Butler read.  Useful if you have lots of them.
    cutoutSideLength: float [optional]
        Size of the cutout region in pixels.

    Returns
    -------
    MaskedImage
    """
    radec = geom.SpherePoint(ra, dec, geom.degrees)
    cutoutSize = geom.ExtentI(cutoutSideLength, cutoutSideLength)

    if skymap is None:
        skymap = butler.get("skyMap")

    tractInfo = skymap.findTract(radec)
    patchInfo = tractInfo.findPatch(radec)
    xy = geom.PointI(tractInfo.getWcs().skyToPixel(radec))
    bbox = geom.BoxI(xy - cutoutSize // 2, cutoutSize)
    print("bbox = ", bbox, "xy = ", xy, "cutoutSize = ", cutoutSize)
    patch = tractInfo.getSequentialPatchIndex(patchInfo)
    tract = tractInfo.getId()

    parameters = {'bbox': bbox}

    query = """band.name = '{}' AND patch = {} AND tract = {}
        """.format('i', patch, tract)
    print(query)

    dataset_refs = butler.query_datasets("deep_coadd", where=query)

    cutout_image = butler.get(dataset_refs[0], parameters=parameters)

    return cutout_image

Set the backend for afwDisplay to matplotlib.

In [3]:

afwDisplay.setDefaultBackend('matplotlib')

Get an instance of the TAP service, and assert that it exists.

In [4]:

service = get_tap_service("tap")
assert service is not None

Instantiate the butler.

In [5]:

butler = Butler('dp1', collections="LSSTComCam/DP1")
assert butler is not None

2. The deblender data products¶

The object catalog contains a number of columns that can be used to identify and characterize blended objects in the deepCoadd images. Prior to deblending, blends are identified (referred to as parents), and are then deblended into child objects by the LSST deblender package, Scarlet Lite. Only the children are ultimately stored in the Object table; the parent objects are excluded. Of course, isolated single sources that do not require deblending are also stored in the Object table and are not considered parent sources.

In this tutorial we will focus on the deblending data products for objects stored in the Object table.

The following describes key parameters in the Object table: 1) boolean flags that are set by the deblender, Scarlet Lite; 2) measurements or properties of deblended sources that can be used to characterize (de)blended sources; 3) blendedness metrics performed after the deblender completes; 4) obsolete keywords still present in the Object table.

Scarlet blend flags:

detect_fromBlend (boolean): If True, this source is deblended from a parent with more than one child.

detect_isIsolated (boolean): If True, this source was not part of a blend.

Scarlet Lite blend measurements:

parentObjectId (long): Unique ID of parent source.

footprintArea (int): Number of pixels in the deblended object's detection "footprint", in the "reference band".

<f>_deblend_fluxOverlap (dbl): The total flux from neighboring objects that overlaps with this sources footprint in the deconvolved space. See also <f>_deblend_fluxOverlapFraction Fraction of flux from neighbors / source flux in the deconvolved footprint.

Blendedness parameters:

<f>_blendedness (dbl): For each filter (f = u, g, r, i, z, or y), a measure of how much the flux is affected by neighbors, (1 - child_flux/parent_flux). The measurement is in the observed, not point spread function (PSF)-deconvolved space. Operates on the absolute value of the pixels to try to obtain a de-noised value. These are not performed by the deblender (Scarlet Lite) but rather from a blendedness algorithm (see section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details).

<f>_deblend_blendedness (dbl): Blendedness in filter in the PSF-deconvolved space.

<f>_blendedness_flag (boolean): Flag set for any failure in the blendedness algorithm.

Obsolete or non-informative parameters:

deblend_failed (boolean): Deblender failed to deblend this source. Note that in DP1, only child objects that were successfully deblended will appear in the Object table. Thus, parent objects which failed to deblend are not included, and this flag is always 0. In future data releases, a parent catalog may exist with informative keywords for the deblender's performance.

In addition to deblend_failed, the following columns from DP0.2 have now been deprecated, although still appear in the Object table (artifacts from when the parents were included in the same catalog). This includes deblend_nChild, which was a property of parents but does not contain useful information in DP1 because the parent blends themselves are now excluded from the Object table. Similarly, deblend_nPeaks, which used to contain the number of peaks a parent object has in DP0.2, still appears in DP1, but always contains the value 1 because only child objects are in the Object table.

2.1. Identify blended objects¶

As with all explorative TAP queries searching for examples, it is essential to first start with a small area while developing queries (to confirm the query is good), and then iteratively increase the area as needed (because the full query could take hours without any spatial constraint on the search). This is because the tables are indexed by Right Ascension and declination coordinates (R.A., Dec) and queries with constraints only on measurement columns (such as deblending-related parameters) are inefficient. In other words, Qserv (the backend hosting the TAP-accessible tables) is "spatially sharded".

Set the location for the query to be in the center of the Extended Chandra Deep Field South (ECDFS) field.

In [6]:

ra = 53.2
dec = -28.1

Query for objects in a small region of ECDFS for exploration of the deblender data products stored in the Object table.

In [7]:

query = "SELECT objectId, coord_ra, coord_dec, x, y, tract, patch, " + \
        "refExtendedness, detect_fromBlend, detect_isIsolated, " + \
        "parentObjectId, deblend_failed, i_deblend_fluxOverlapFraction, " + \
        "footprintArea, i_blendedness, i_deblend_blendedness, i_blendedness_flag " + \
        "FROM dp1.Object " + \
        "WHERE CONTAINS(POINT('ICRS', coord_ra, coord_dec ), " + \
        "CIRCLE('ICRS', " + str(ra) + ", " + str(dec) + ", .1)) = 1 "

Run the TAP query.

In [8]:

job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)

Job phase is COMPLETED

Store the resulting objects and columns from the Object table to a table called results.

In [9]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()

2.2. Identify isolated objects¶

A sample of objects that are not near any neighbors, and therefore were not deblended, can be identified using the detect_isIsolated keyword. Similarly, a set of objects that were identified as part of a blend can be identified using the detect_fromBlend keyword.

In [10]:

whiso = np.where(results['detect_isIsolated'] == 1)[0]
whblend = np.where(results['detect_fromBlend'] == 1)[0]

Generate a cutout of the i-band deep_coadd imaging that overlaps the queried objects, centered at (RA, Dec) = 53.11, -28.07 degrees, with a cutout edge size of 200 pixels.

In [11]:

cutout = cutout_coadd(butler, 53.11, -28.07,
                      band='i', datasetType='deepCoadd',
                      cutoutSideLength=200)

bbox =  (minimum=(14543, 5011), maximum=(14742, 5210)) xy =  (14643, 5111) cutoutSize =  (200, 200)
band.name = 'i' AND patch = 14 AND tract = 5063

Display the cutout with isolated objects identified by red circles and objects that were identified from blends identified as cyan plus signs. To ensure that afwDisplay will make the low surface brightness connecting the blends together visible, set the scale to linear, set the min and max pixel values to be relatively low and also indicate with units=Absolute that these should be in actual count units.

In [12]:

fig = plt.figure(figsize=(6, 6))
display = afwDisplay.Display(frame=fig)

display.scale('linear', min=0.1, max=10, unit='Absolute')

display.mtv(cutout.image)
with display.Buffering():
    for ci in range(len(whiso)):
        display.dot('o', results['x'][whiso][ci], results['y'][whiso][ci],
                    size=10, ctype=afwDisplay.RED)
    for ci in range(len(whblend)):
        display.dot('+', results['x'][whblend][ci], results['y'][whblend][ci],
                    size=8, ctype=afwDisplay.CYAN)
plt.gca().axis('off')
display.show_colorbar(False)
plt.show()

Figure 1: The cutout image displayed in greyscale, with cyan + marking objects that are blended in the deep_coadd, red circles marking objects that are isolated (and is not part of a blend).

3. Explore blended objects¶

Use the TAP service to find a group of objects that were heavily blended and part of the same parent object.

3.1. Blend with several children¶

First, identify a set of unique parentObjectIds, and the number of objects that are deblended from those parentObjectIds, (stored as counts). Then, select as an example a blend that was made up of 15 children.

To identify the most blended object in the query, replace the second code line with: most_common_index_in_unique = np.argmax(counts)

In [13]:

unique_IDs, counts = np.unique(results['parentObjectId'], return_counts=True)

most_common_index_in_unique = np.where(counts == 15)[0]

most_common_value = unique_IDs[most_common_index_in_unique[0]]

whparent = np.where(results['parentObjectId'] == most_common_value)[0]
max_parentId = results['parentObjectId'][whparent][0]

Use TAP to retrieve all children for the selected parent.

In [14]:

query = "SELECT objectId, coord_ra, coord_dec, x, y, parentObjectId, i_blendedness_flag, " + \
        "i_cModelFlux/i_cModelFluxErr AS i_S2N, footprintArea, i_deblend_blendedness, " + \
        "i_blendedness, deblend_failed, i_deblend_fluxOverlap, i_deblend_fluxOverlapFraction " + \
        "FROM dp1.Object " + \
        "WHERE CONTAINS(POINT('ICRS', coord_ra, coord_dec ), " + \
        "CIRCLE('ICRS', " + str(ra) + ", " + str(dec) + ", .1)) = 1 " + \
        "AND parentObjectId = " + str(max_parentId)

Run the TAP query.

In [15]:

job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)

Job phase is COMPLETED

Store the query results in a table called children.

In [16]:

assert job.phase == 'COMPLETED'
children = job.fetch_result().to_table()
children

Out[16]:

Table length=15

objectId	coord_ra	coord_dec	x	y	parentObjectId	i_blendedness_flag	i_S2N	footprintArea	i_deblend_blendedness	i_blendedness	deblend_failed	i_deblend_fluxOverlap	i_deblend_fluxOverlapFraction
	deg	deg	pix	pix				pix
int64	float64	float64	float64	float64	int64	bool	float64	int32	float32	float32	bool	float32	float32
611254316728073284	53.294864915499765	-28.126076964734423	11707.922594133186	4098.302973641447	611254316728059699	False	7.096778393068185	1127	0.0	0.0466138	False	0.0	0.0
611254316728073280	53.30000004299527	-28.127485719212654	11626.443568882292	4072.8016766671863	611254316728059699	False	7.809782722792145	597	0.0	0.0271333	False	0.0	0.0
611254316728073281	53.29739851633408	-28.127993986892044	11667.758939611198	4063.7235506255665	611254316728059699	True	-0.019130034522410754	454	0.0	--	False	0.0	0.0
611254316728073282	53.296165745419366	-28.128734914905575	11687.351831657825	4050.4189893817747	611254316728059699	False	7.423539277962396	1162	0.0246679	0.266039	False	3.46231	0.0168952
611254316728073283	53.29660129461913	-28.125536808717257	11680.340399977047	4107.979685606347	611254316728059699	False	7.419048613431707	881	0.0	0.0227583	False	0.0	0.0
611254316728073279	53.295658871229705	-28.125294285218384	11695.29454317496	4112.371287173421	611254316728059699	True	-0.06455831873487836	422	0.0	--	False	0.0	0.0
611254316728073285	53.2967491477205	-28.129172760001698	11678.103579404262	4042.5209388886897	611254316728059699	True	6.25844073372527	993	0.0102184	0.103867	False	38.0434	0.196383
611254316728073286	53.29294969335254	-28.12786049935853	11738.380787078966	4066.2472628100695	611254316728059699	False	5.4996531838805645	388	0.0	0.057179	False	0.0	0.0
611254316728073287	53.29414955777493	-28.126644370416713	11719.296237432669	4088.1078306569243	611254316728059699	False	3.561928011397123	608	0.0	0.618377	False	0.0	0.0
611254316728073273	53.29566789404083	-28.127286563178075	11695.211494752568	4076.505898078906	611254316728059699	False	983.0246787788672	3793	0.00166738	0.00233513	False	2336.91	0.059042
611254316728073274	53.29411430010412	-28.127641516595084	11719.885868289113	4070.158073828561	611254316728059699	False	180.3119748899455	2401	0.048463	0.0133469	False	5735.77	0.996521
611254316728073275	53.299318362969274	-28.127070560818673	11637.252647255584	4080.2942815238393	611254316728059699	False	29.476632628687174	898	0.0	0.00107619	False	0.0	0.0
611254316728073276	53.29821288343833	-28.12659024085694	11654.787788395472	4088.971541652779	611254316728059699	False	13.644255885327834	848	0.0	0.000571488	False	0.0	0.0
611254316728073277	53.29420641670363	-28.12891471823645	11718.46169470667	4047.235296448674	611254316728059699	False	11.529717469086624	904	0.0	0.00755404	False	0.0	0.0
611254316728073278	53.29670099675805	-28.128213784960245	11678.83885814897	4059.7858143524772	611254316728059699	False	9.164728767215426	1381	0.0115476	0.0656418	False	33.8051	0.16126

The table above confirms that the children returned by the query were deblended from the same parent source, since they all show the same parentObjectId.

A single source is identified with high i_deblend_fluxOverlapFraction (i.e. flux contribution from neighbors) of almost 62%. Note however that in this case, the deblender indicates that the deblended flux is uncontaminated in the PSF-decovolved space, i_deblend_blendedness = 0, and the blendedness flag returns False, indicating this object was successfully deblended. However, these blendedness metrics before and after deblending can assist in quality assessment.

Below, identify this child with heavy blendedness of 50%.

In [17]:

whheavy = np.where(children['i_blendedness'] > 0.5)[0]

children[whheavy]

Out[17]:

Table length=1

objectId	coord_ra	coord_dec	x	y	parentObjectId	i_blendedness_flag	i_S2N	footprintArea	i_deblend_blendedness	i_blendedness	deblend_failed	i_deblend_fluxOverlap	i_deblend_fluxOverlapFraction
	deg	deg	pix	pix				pix
int64	float64	float64	float64	float64	int64	bool	float64	int32	float32	float32	bool	float32	float32
611254316728073287	53.29414955777493	-28.126644370416713	11719.296237432669	4088.1078306569243	611254316728059699	False	3.561928011397123	608	0.0	0.618377	False	0.0	0.0

Next, visualize these test cases along with the other child sources of this parentObjectId. Use the cutout_coadd function to obtain a cutout image to visualize the blend. Set the size of the cutout to be 2x the square root of the footprintArea.

In [18]:

cutout = cutout_coadd(butler, children['coord_ra'][0],
                      children['coord_dec'][0],
                      band='i', datasetType='deepCoadd',
                      cutoutSideLength=2.0
                      * np.sqrt(np.sum(children['footprintArea'])))

bbox =  (minimum=(11579, 3969), maximum=(11837, 4227)) xy =  (11708, 4098) cutoutSize =  (259, 259)
band.name = 'i' AND patch = 13 AND tract = 5063

Display the deepCoadd cutout made for this parent, with all deblended children identified with red circles. Identify the children for which the blendedness metric (before deblending) was 50% with a cyan circle.

In [19]:

fig = plt.figure(figsize=(6, 6))
display = afwDisplay.Display(frame=fig)
display.scale('linear', min=0.1, max=10, unit='Absolute')
display.mtv(cutout.image)
with display.Buffering():
    for ci in range(len(children)):
        display.dot('o', children['x'][ci], children['y'][ci],
                    size=10, ctype=afwDisplay.RED)

    for ci in range(len(whheavy)):
        display.dot('o', children['x'][whheavy][ci], children['y'][whheavy][ci],
                    size=8, ctype=afwDisplay.CYAN)

plt.gca().axis('off')
display.show_colorbar(False)
plt.show()

Figure 2: The cutout image displayed in greyscale, with red circles marking the deblended children, cyan circles indicating any children whose contamination from neighbors prior to deblending is high (>50% of flux from a neighbor).

4. Heavy blends¶

It is useful to be able to quantify how much flux in the blended source is attributed to a neighbor. Mandelbaum et al. 2018 quantitatively explored the criteria for acceptable blendedness as a metric of deblender performance. In this section, explore examples where sources are heavily contaminated (<f>_blendedness >50%) by the flux of neighbors, and then compare to the deblended contamination fraction <f>_deblend_blendedness.

Below, identify which child objects are above this high threshold of PSF-convolved blending, and see what fraction of objects in the catalog meet this criteria.

In [20]:

whheavy = np.where(results['i_blendedness'] > 0.5)[0]

print('Fraction of objects that are heavily blended is', len(whheavy)/len(results))

results[whheavy]

Fraction of objects that are heavily blended is 0.05161951296398455

Out[20]:

Table length=655

objectId	coord_ra	coord_dec	x	y	tract	patch	refExtendedness	detect_fromBlend	detect_isIsolated	parentObjectId	deblend_failed	i_deblend_fluxOverlapFraction	footprintArea	i_blendedness	i_deblend_blendedness	i_blendedness_flag
	deg	deg	pix	pix									pix
int64	float64	float64	float64	float64	int64	int64	float32	bool	bool	int64	bool	float32	int32	float32	float32	bool
611254385447554247	53.119850049117744	-28.051975483125325	14486.0	5435.0	5063	14	--	True	False	611254385447538062	False	0.66276	524	0.516076	0.033251	True
611254385447554149	53.26707422689095	-28.052525983120294	12147.19488817705	5423.057028110923	5063	14	1.0	True	False	611254385447538011	False	0.0	646	0.659665	0.0	True
611254385447554100	53.11494062961147	-28.05414302365997	14564.0	5396.0	5063	14	--	True	False	611254385447537986	False	0.0	547	0.677394	0.0	True
611254385447554110	53.12217959030314	-28.05341923831319	14449.0	5409.0	5063	14	--	True	False	611254385447537990	False	0.00730981	746	0.839532	0.000447392	True
611254385447554213	53.18551883210311	-28.05276228387117	13442.791494017694	5420.278680042868	5063	14	1.0	True	False	611254385447538035	False	0.0	897	0.682092	0.0	True
611254385447554107	53.12181583464017	-28.05216568916782	14454.772337652805	5431.567466914587	5063	14	1.0	True	False	611254385447537990	False	0.0	627	0.544059	0.0	True
611254385447554211	53.18449852208436	-28.051902637100135	13458.9880245452	5435.766632426703	5063	14	1.0	True	False	611254385447538035	False	0.0	528	1.0	0.0	True
611254385447554805	53.2622516340637	-28.020704670509932	12223.0	5996.0	5063	14	--	True	False	611254385447538197	False	0.103061	1853	0.64032	0.00168151	True
611254385447554837	53.259709153133514	-28.021629104567182	12263.424918675286	5979.416326628661	5063	14	0.0	True	False	611254385447538197	False	0.483975	1321	0.851791	0.0196931	True
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
611254385447552406	53.13923919667152	-28.073251863620435	14178.14221769388	5051.8846379722145	5063	14	1.0	True	False	611254385447537542	False	0.0	730	0.642831	0.0	False
611254385447552283	53.166290265122754	-28.063606369646777	13748.380048357492	5225.289368678812	5063	14	1.0	True	False	611254385447537530	False	0.0	762	0.763668	0.0	True
611254385447552322	53.165407612669085	-28.06412493425716	13762.406312760688	5215.963375123078	5063	14	--	True	False	611254385447537530	False	0.503917	3075	0.602695	0.0487601	True
611254385447552533	53.10150455225182	-28.070266762970615	14777.474868149957	5105.782963569278	5063	14	0.0	True	False	611254385447537604	False	0.0	543	1.0	0.0	True
611254385447552582	53.14907820440253	-28.069837771559264	14021.838873568724	5113.271342707839	5063	14	1.0	True	False	611254385447537632	False	0.39208	626	0.615249	0.00921541	True
611254385447552473	53.11102953587668	-28.072315260226656	14626.192599131047	5068.8833580167775	5063	14	1.0	True	False	611254385447537585	False	0.0	376	0.607354	0.0	True
611254385447552449	53.09375744811367	-28.0725858012339	14900.52740424963	5064.046597712321	5063	14	0.0	True	False	611254385447537565	False	0.0	311	1.0	0.0	True
611254385447552488	53.247432176772186	-28.072149835301676	12459.685567715162	5070.229972516796	5063	14	0.0	True	False	611254385447537597	False	1.71397	607	0.699938	0.111303	True
611254454167027841	53.087105917175	-28.099354253839042	15006.172415517243	4582.170027669604	5063	15	--	True	False	611254454167013588	False	0.0	1012	0.648493	0.0	True

In [21]:

fig, (ax, ax2) = plt.subplots(ncols=2, nrows=1,
                              width_ratios=[0.8, 0.2], figsize=(10, 6))

ax.plot(results['i_blendedness'][whheavy], results['i_deblend_blendedness'][whheavy],
        '+', alpha=0.5, label='i_blendedness_flag = False')

ax2.hist((results['i_deblend_blendedness'][whheavy]), orientation="horizontal",
         bins=np.linspace(0, 1.1, 11), align='left',
         histtype="step", color='blue', stacked=True, fill=False, label=None)

ax.legend()
ax.set_ylim([-0.1, 1.1])
ax2.set_ylim([-0.1, 1.1])
ax.set_xlabel('i_blendedness')
ax.set_ylabel('i_deblend_blendedness')
ax2.set_xlabel('Number of objects')
plt.show()

Figure 3: This figure shows the blendedness metric (fraction of flux contributed by neighbors) before deblending vs the deblend_blendedness metric that identifies the fraction of flux contamination after deblending, for heavily blended objects (>50% flux from one or more neighbors). The right panel shows a histogram of the deblend_blendedness, indicating that the deblender is successful at separating flux, such that deblend_blendedness = 0, in the overwhelming majority of objects.

Below, take a closer look at the objects for which the deblender did not effectively separate the flux from neighbors.

In [22]:

wh = np.where((results['i_blendedness'][whheavy] == 1)
              & (results['i_deblend_blendedness'][whheavy] == 1))[0]
results[whheavy][wh]

Out[22]:

Table length=7

objectId	coord_ra	coord_dec	x	y	tract	patch	refExtendedness	detect_fromBlend	detect_isIsolated	parentObjectId	deblend_failed	i_deblend_fluxOverlapFraction	footprintArea	i_blendedness	i_deblend_blendedness	i_blendedness_flag
	deg	deg	pix	pix									pix
int64	float64	float64	float64	float64	int64	int64	float32	bool	bool	int64	bool	float32	int32	float32	float32	bool
611254385447544541	53.20949616587542	-28.172079076941582	13064.0	3272.0	5063	14	--	True	False	611254385447534971	False	3.03895e+19	800	1.0	1.0	True
611254385447544333	53.15827111350926	-28.184446622615013	13877.0	3050.0	5063	14	--	True	False	611254385447534885	False	4.39223e+21	1078	1.0	1.0	True
611254385447549206	53.18980321551127	-28.119724512611672	13375.723811530852	4214.783959504233	5063	14	0.0	True	False	611254385447536499	False	1.13236e+20	381	1.0	1.0	True
611254385447549812	53.12049510141591	-28.11173172929737	14476.038394057774	4359.281431858267	5063	14	0.0	True	False	611254385447536655	False	2.25389e+20	825	1.0	1.0	True
611254385447551337	53.08850219033574	-28.09403090209418	14984.0	4678.0	5063	14	--	True	False	611254385447537140	False	1.46538e+21	715	1.0	1.0	True
611254385447553259	53.14641831409112	-28.059632707558045	14064.0	5297.0	5063	14	--	True	False	611254385447537876	False	1.92496e+20	590	1.0	1.0	True
611254385447552110	53.17513260519447	-28.079017674217436	13608.120706299507	4947.76417469945	5063	14	0.0	True	False	611254385447537432	False	2.26052e+20	509	1.0	1.0	True

Note that by selecting objects where the i_blendedness and i_deblend_blendedness metrics are high, the i_deblend_fluxOverlapFraction is also high. Below, pick one of the above objects which has a high value of i_deblend_fluxOverlapFraction as an example.

In [23]:

max_parentId = results['parentObjectId'][whheavy][wh][3]
print(max_parentId)

611254385447536655

Define a TAP query that will return all children with the same parentObjectId as the above source.

In [24]:

query = "SELECT objectId, coord_ra, coord_dec, x, y, parentObjectId, i_blendedness_flag, " + \
        "i_cModelFlux/i_cModelFluxErr AS i_S2N, footprintArea, i_deblend_blendedness, " + \
        "i_blendedness, deblend_failed, i_deblend_fluxOverlapFraction " + \
        "FROM dp1.Object " + \
        "WHERE CONTAINS(POINT('ICRS', coord_ra, coord_dec ), " + \
        "CIRCLE('ICRS', " + str(ra) + ", " + str(dec) + ", .1)) = 1 " + \
        "AND parentObjectId = " + str(max_parentId)

Run the TAP query.

In [25]:

job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)

Job phase is COMPLETED

Save the results to a table named bigblend.

In [26]:

assert job.phase == 'COMPLETED'
bigblend = job.fetch_result().to_table()
bigblend

Out[26]:

Table length=96

objectId	coord_ra	coord_dec	x	y	parentObjectId	i_blendedness_flag	i_S2N	footprintArea	i_deblend_blendedness	i_blendedness	deblend_failed	i_deblend_fluxOverlapFraction
	deg	deg	pix	pix				pix
int64	float64	float64	float64	float64	int64	bool	float64	int32	float32	float32	bool	float32
611254385447549797	53.1288427976875	-28.10480712598396	14343.457205267834	4483.896967629499	611254385447536655	False	215.44854614877488	1163	0.0	0.00350331	False	0.0
611254385447549796	53.11888362555566	-28.107791439571333	14501.6065137152	4430.220912669169	611254385447536655	False	250.63591035174784	2025	3.21865e-06	0.000266227	False	0.000313662
611254385447549795	53.11611224989274	-28.113955866474047	14545.635880058906	4319.260279373383	611254385447536655	False	322.89613376597396	3663	0.0793223	0.0341204	False	0.506552
611254385447549794	53.11700075484309	-28.1079685721846	14531.503430249773	4427.03966030183	611254385447536655	False	291.69423067404375	1563	6.4373e-06	0.0004873	False	0.00192911
611254385447549793	53.1180600864281	-28.109246128997558	14514.68908284418	4404.037068458728	611254385447536655	False	348.8334477051298	3984	3.23057e-05	9.18724e-06	False	0.00858761
611254385447549792	53.11158995129432	-28.097739682971145	14617.380160351358	4611.197577154995	611254385447536655	False	506.0188839436898	3151	0.000519693	0.000509103	False	0.0593205
611254385447549791	53.12623991171105	-28.103492711561	14384.77949530281	4507.572471862002	611254385447536655	False	757.3897367079634	3508	0.000226974	0.000384537	False	0.00732551
611254385447549790	53.12814080136739	-28.103523183723958	14354.596220940137	4507.014080406694	611254385447536655	False	1352.8618903658546	4672	6.71744e-05	6.82606e-05	False	0.0349882
611254385447549789	53.11828000026783	-28.112432694459184	14511.211557368695	4346.671920236903	611254385447536655	False	2091.330592818051	8486	0.00108224	5.54428e-06	False	0.0687492
...	...	...	...	...	...	...	...	...	...	...	...	...
611254385447549832	53.125084219842584	-28.101914774087426	14403.12165942666	4535.984030352416	611254385447536655	False	20.910674548838095	1365	0.0	0.0259139	False	0.0
611254385447549831	53.113438156916516	-28.095569206991563	14588.02356178145	4650.264074694331	611254385447536655	False	22.907193688433754	1065	0.0	0.0286494	False	0.0
611254385447549830	53.12218227063278	-28.111791938258616	14449.250843199985	4358.190112733937	611254385447536655	False	34.48630226669729	3145	0.0340317	0.179712	False	0.385945
611254385447549829	53.1318061296595	-28.106943745200972	14296.41795031805	4445.417256691017	611254385447536655	False	22.45457143896183	1127	0.0	0.0013955	False	0.0
611254385447549828	53.11686873975791	-28.100558805740587	14533.568026965002	4560.429811437709	611254385447536655	False	23.334000662629794	851	0.0	0.00102294	False	0.0
611254385447549827	53.11699399268731	-28.10543098758147	14531.599954931451	4472.72099149102	611254385447536655	False	25.66002105212157	916	0.0	0.00145569	False	0.0
611254385447549826	53.12571971486296	-28.11439955931827	14393.099916741849	4311.231098268632	611254385447536655	False	27.86640540363285	1213	0.00416863	0.0188595	False	0.0386981
611254385447549825	53.11654664325399	-28.110218231774027	14538.723255255323	4386.5432456915005	611254385447536655	False	36.848724531259194	1432	0.00419945	0.0242835	False	0.0636848
611254385447549824	53.12443454460638	-28.101495927169392	14413.435494369105	4543.527206441226	611254385447536655	False	28.603272655627084	941	0.0	0.00712143	False	0.0

Use the cutout_coadd function to request an image cutout of this blend, and identify any objects which still have 1% of flux contributed by a neighbor after the deblending process (i_deblend_blendedness > 0.01).

In [27]:

cutout = cutout_coadd(butler, bigblend['coord_ra'][0],
                      bigblend['coord_dec'][0],
                      band='i', datasetType='deepCoadd',
                      cutoutSideLength=2.0 * np.sqrt(np.sum(bigblend['footprintArea'])))

whheavy = np.where(bigblend['i_deblend_blendedness'] > 0.01)[0]

bbox =  (minimum=(13941, 4082), maximum=(14744, 4885)) xy =  (14343, 4484) cutoutSize =  (804, 804)
band.name = 'i' AND patch = 14 AND tract = 5063

In [28]:

fig = plt.figure(figsize=(6, 6))
display = afwDisplay.Display(frame=fig)
display.scale('linear', min=0.01, max=5, unit='Absolute')
display.mtv(cutout.image)
with display.Buffering():
    for ci in range(len(bigblend)):
        display.dot('o', bigblend['x'][ci], bigblend['y'][ci],
                    size=10, ctype=afwDisplay.RED)

    for ci in range(len(whheavy)):
        display.dot('o', bigblend['x'][whheavy][ci], bigblend['y'][whheavy][ci],
                    size=9, ctype=afwDisplay.CYAN)

plt.gca().axis('off')
display.show_colorbar(False)
plt.show()

Figure 4: This figure shows a heavily blended parent for which deblended objects have a fraction of flux contributed by neighbors (red circles). Cyan circles indicate objects whose deblend_blendedness metric is not 0, and remains > 1% contamination even after deblending. They are typically faint objects near the outskirts of the biggest and brightest objects where contamination might be high.

5. Exercises for the learner¶

Use the deblender data products and blend metrics in the Object table to identify what fraction of sources in ECDFS are contaminated by neighbor flux at the 5% level in the observed frame? What fraction have 5% flux contamination in the PSF-deconvolved (i.e. deblended) frame?