201.10. Visit table

201.10. Visit table¶

For the Rubin Science Platform at data.lsst.cloud.
Data Release: Data Preview 1
Container Size: large
LSST Science Pipelines version: r29.1.1
Last verified to run: 2025-06-21
Repository: github.com/lsst/tutorial-notebooks

Learning objective: To understand the contents of the Visit table and how to access it.

LSST data products: Visit

Packages: lsst.rsp, lsst.daf.butler

Credit: Originally developed by the Rubin Community Science team. Please consider acknowledging them if this notebook is used for the preparation of journal articles, software releases, or other notebooks.

Get Support: Everyone is encouraged to ask questions or raise issues in the Support Category of the Rubin Community Forum. Rubin staff will respond to all questions posted there.

1. Introduction¶

A Visit refers to an observation of the sky with the camera.

The Visit table contains data about the individual observations, such as the coordinates, date, time, filter, camera rotation, exposure time, and airmass. This table does not contain the image data or data for astrophysical sources.

The contents of the Visit table are associated with the boresight of the entire focal plane. By contrast, the CcdVisit table contains data by individual detectors of the camera, instead of the entire focal plane.

• TAP table name: dp1.Visit
• Butler table name: visit_table
• columns: 16
• rows: 1786

Related tutorials: The tutorial for the CcdVisit table is also in this series. The TAP and Butler data access services are demonstrated in the 100-level "How to" tutorials.

1.1. Import packages¶

Import standard python packages re, numpy, matplotlib, and astropy.

From the lsst package, import modules for the TAP service and the Butler.

In [1]:

import re
import numpy as np
import matplotlib.pyplot as plt
from lsst.rsp import get_tap_service
from lsst.daf.butler import Butler
from lsst.utils.plotting import (get_multiband_plot_colors,
                                 get_multiband_plot_symbols)

1.2. Define parameters and functions¶

Create an instance of the TAP service, and assert that it exists.

In [2]:

service = get_tap_service("tap")
assert service is not None

Create an instance of the Rubin data Butler, and assert that it exists.

In [3]:

butler = Butler('dp1', collections="LSSTComCam/DP1")
assert butler is not None

Define the colors and symbols to represent the LSST filters in plots.

In [4]:

filter_names = ['u', 'g', 'r', 'i', 'z', 'y']
filter_colors = get_multiband_plot_colors()
filter_symbols = get_multiband_plot_symbols()

2. Schema (columns)¶

To browse the table schema visit the Rubin schema browser, or use the TAP service via the Portal Aspect or as demonstrated in Section 2.1.

2.1. Retrieve table schema¶

To retrieve the table schema, define a query for the schema columns of the Visit table and run the query job.

In [5]:

query = "SELECT column_name, datatype, description, unit " \
        "FROM tap_schema.columns " \
        "WHERE table_name = 'dp1.Visit'"
job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

Retrieve the query results and display them as an astropy table with the to_table() attribute.

In [6]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
assert len(results) == 16
results

Out[6]:

Table length=16

column_name	datatype	description	unit
str64	str64	str512	str64
airmass	double	Airmass of the observed line of sight.
altitude	double	Altitude of focal plane center at the middle of the visit.	deg
azimuth	double	Azimuth of focal plane center at the middle of the visit.	deg
band	char	Name of the band used to take the visit where this source was measured. Abstract filter that is not associated with a particular instrument.
dec	double	Declination of focal plane center	deg
decl	double	Deprecated duplicate of dec.	deg
expMidpt	char	Midpoint time for exposure at the fiducial center of the focal plane array. TAI, accurate to 10ms.
expMidptMJD	double	Midpoint time for exposure at the fiducial center of the focal plane array in MJD. TAI, accurate to 10ms.	d
expTime	double	Spatially-averaged duration of visit, accurate to 10ms.	s
obsStart	char	Start time of the visit at the fiducial center of the focal plane array, TAI, accurate to 10ms.
obsStartMJD	double	Start of the exposure in MJD, TAI, accurate to 10ms.	d
physical_filter	char	ID of physical filter, the filter associated with a particular instrument.
ra	double	Right Ascension of focal plane center.	deg
skyRotation	double	Sky rotation angle.	deg
visit	long	Unique identifier.
zenithDistance	double	Zenith distance at the middle of the visit.	deg

The table displayed above has not been trucated, as it is only 16 rows long.

Option to use the regular expressions package re to search for columns for which the description contains the string temp.

In [7]:

# temp = 'time'
# temp = 'filt'
# temp = 'exp'
# for c, desc in enumerate(results['description']):
#     if re.search(temp, desc):
#         print(results['column_name'][c])

Delete the job and the results.

In [8]:

del query, results
job.delete()

3. Data access¶

The Visit table is available via the TAP service and the Butler.

Recommended access method: TAP.

3.1. TAP (Table Access Protocol)¶

The Visit table is stored in Qserv and accessible via the TAP services using ADQL queries.

3.1.2. Demo query¶

Avoid full-table queries. Although the DP1 Visit table is relatively small, it is good practice to always include spatial constraints and only retrieve necessary columns.

Search for g- and r-band visits within 3 degrees of the center of the Extended Chandra Deep Field South (ECDFS) field, RA, Dec = $53.13 -28.10$. Return the visit identifier, RA, Dec, and the midpoint time of the exposure in MJD.

In [9]:

query = "SELECT visit, ra, dec, band, expMidptMJD " \
        "FROM dp1.Visit " \
        "WHERE CONTAINS(POINT('ICRS', ra, dec), " \
        "CIRCLE('ICRS', 53.13, -28.10, 3)) = 1 " \
        "AND (band = 'r' OR band = 'g') " \
        "ORDER BY expMidptMJD ASC "
job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

Fetch the results as an astropy table.

In [10]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
print(len(results))

467

Option to display the table.

In [11]:

# results

As an example, plot the RA and Dec for the g- and r-band visits, and the cumulative distribution of observation MJDs.

In [12]:

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(7, 3))
for filt in filter_names:
    fx = np.where(results['band'] == filt)[0]
    if len(fx) > 0:
        ax1.plot(results['ra'][fx], results['dec'][fx],
                 filter_symbols[filt], ms=5, mew=0, alpha=0.4,
                 color=filter_colors[filt], label=filt)
ax1.set_xlabel('Right Ascension')
ax1.set_ylabel('Declination')
ax1.legend(loc='best', handletextpad=0)
ax2.hist(results['expMidptMJD'], len(results),
         histtype='step', cumulative=True)
ax2.set_xlabel('MJD')
ax2.set_ylabel('Number of Visits')
plt.tight_layout()
plt.show()

Figure 1: At left, the RA vs. Dec of the g- and r-band visits (central boresight of the focal plane). At right, the cumulative number of visits as a function of time in MJD.

Clean up.

In [13]:

job.delete()
del query, results

3.2.2. Joinable tables¶

The Visit table can be joined with CcdVisit on column visitId.

The Visit table can also be joined with the Source, ForcedSource, DiaSource, and ForcedSourceOnDiaObject tables on column visit.

CcdVisit¶

For example, start with the same query as in Section 3.1.2. Add the constraint on the data of observation to return only visits in one night.

Join the CcdVisit table and return the ccdVisitId, detector, seeing and zeropoint for all of the processed, calibrated visit_images associated with the visits that match the query constraints. Recall that with LSSTComCam there will be 9 detectors (visit_images) for each visit.

In [14]:

query = "SELECT v.visit, v.expMidptMJD, v.band, " \
        "c.ccdVisitId, c.detector, c.seeing, c.zeropoint " \
        "FROM dp1.Visit AS v " \
        "JOIN dp1.CcdVisit AS c ON c.visitId = v.visit " \
        "WHERE CONTAINS(POINT('ICRS', v.ra, v.dec), " \
        "CIRCLE('ICRS', 53.13, -28.10, 3)) = 1 " \
        "AND (v.band = 'r' OR v.band='g') " \
        "AND v.expMidptMJD > 60623.0 " \
        "AND v.expMidptMJD < 60624.0 " \
        "ORDER BY v.expMidptMJD ASC "
job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

In [15]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
print(len(results))

360

Option to display the table.

In [16]:

# results

Assert that 40 unique visits were identified by the query, and that the length of the results table is the number of unique visits times the number of detectors per visit, which is 9 for LSSTComCam.

In [17]:

values, counts = np.unique(results['visit'], return_counts=True)
assert len(values) == 40
assert len(values)*9 == len(results)

Print the visitId for the first visit, and the number of detectors (rows of the results table) associated with it.

In [18]:

print('Visit, number of detectors: ', values[0], counts[0])

Visit, number of detectors:  2024110800246 9

In [19]:

del query, results, values, counts
job.delete()

Source¶

Joins to the ForcedSource, DiaSource, and ForcedSourceOnDiaObject tables would similarly use the column visit and examples for each are not shown.

As an example, start with a similar spatial query from Section 3.1.2, but constrain resulting visits to r-band and a shorter time window. Return the counts of the number of detected sources in each visit.

In [20]:

query = "SELECT v.visit, v.expMidptMJD, v.band, " \
        "COUNT(s.sourceId) " \
        "FROM dp1.Visit AS v " \
        "JOIN dp1.Source AS s ON s.visit = v.visit " \
        "WHERE CONTAINS(POINT('ICRS', v.ra, v.dec), " \
        "CIRCLE('ICRS', 53.13, -28.10, 3)) = 1 " \
        "AND v.band = 'r' " \
        "AND v.expMidptMJD > 60623.250 " \
        "AND v.expMidptMJD < 60623.270 " \
        "GROUP BY v.visit " \
        "ORDER BY v.expMidptMJD ASC "
job = service.submit_job(query)
job.run()
job.wait(phases=['COMPLETED', 'ERROR'])
print('Job phase is', job.phase)
if job.phase == 'ERROR':
    job.raise_if_error()

Job phase is COMPLETED

In [21]:

assert job.phase == 'COMPLETED'
results = job.fetch_result().to_table()
results

Out[21]:

Table length=8

visit	expMidptMJD	band	COUNT4
	d
int64	float64	object	int64
2024110800246	60623.25932903928	r	27708
2024110800247	60623.25989537033	r	27446
2024110800250	60623.262127748836	r	27834
2024110800251	60623.2626942534	r	27474
2024110800254	60623.2648977835	r	28241
2024110800255	60623.265465752265	r	27057
2024110800258	60623.2676935763	r	26917
2024110800259	60623.268262389945	r	27709

In [22]:

del query, results
job.delete()

3.2. Butler¶

Show that the only dimension for the visit_table is instrument, and that it is required.

In [23]:

butler.get_dataset_type('visit_table')

Out[23]:

DatasetType('visit_table', {instrument}, ArrowAstropy)

In [24]:

butler.get_dataset_type('visit_table').dimensions.required

Out[24]:

{instrument}

Show that there is only one dataset reference for the "LSSTComCam" visit_table.

In [25]:

dataset_refs = butler.query_datasets("visit_table", instrument="LSSTComCam")
print(len(dataset_refs))
print(dataset_refs[0])

1
visit_table@{instrument: 'LSSTComCam'} [sc=ArrowAstropy] (run=LSSTComCam/runs/DRP/DP1/v29_0_0/DM-50260/20250416T185152Z id=78262edb-b510-47d1-9162-ce6169c9b5c8)

Retrieve the entire visit_table.

In [26]:

visit_table = butler.get(dataset_refs[0])

Confirm that the visit_table has 1786 rows.

In [27]:

assert len(visit_table) == 1786
print(len(visit_table))

1786

Option to display the table (automatically truncated).

In [28]:

# visit_table

Display one row of the table.

In [29]:

tx = np.where(visit_table['visit'] == 2024110800250)[0]
visit_table[tx[0]]

Out[29]:

Row index=5

visitId	visit	physical_filter	band	ra	dec	decl	skyRotation	azimuth	altitude	zenithDistance	airmass	expTime	expMidpt	expMidptMJD	obsStart	obsStartMJD
int64	int64	str4	str1	float64	float64	float64	float64	float64	float64	float64	float64	float64	datetime64[ns]	float64	datetime64[ns]	float64
2024110800250	2024110800250	r_03	r	53.1891579665548	-28.2085120405563	-28.2085120405563	102.799582415553	272.690602893552	73.4458487097539	16.554151290246097	1.0431643110249	30.0	2024-11-09T06:17:27.837500000	60623.262127748836	2024-11-09T06:17:12.837500000	60623.26195413773

Clean up.

In [30]:

del dataset_refs, visit_table, tx

In [ ]: