Introduction

Each dataset has a catalog consisting of one or more tables that store metadata associated with objects in the dataset. The catalog data resides in the following category of tables.

• Internal tables - Tables that are auto-created and populated as part of data ingestion through the execution of pipelines.

• Imported tables - Tables created and populated when users import external catalog information by uploading files in a supported format(like CSV or COCO).

  • SaaS mode: The catalog file must be imported on the web portal UI. The details on the structure of the CSV file are described here.

  • Local mode(deprecated): CSV file must be imported using the 'adectl import' command. Catalog import on the web portal UI import is not supported. The details on the structure of the CSV file are described here.

• External tables - Tables in an external catalog database are registered on the External catalog page. This is available only in SaaS mode. The details on the fields that must be present in external tables are described here.

To open a catalog and view the catalog-related functionalities:

1. Navigate to Data >  Repo > Datasets.

2. Locate the required dataset, and click the Catalog button.

The video below provides an introduction to navigating the catalog page.

Catalog actions

The 3-dots menu shown below provides the following actions.

• Create Table: Create a new table to import the catalog. The Quick import catalog creates the table with auto-inferred types and is the recommended flow for most use cases. Use this option only for special situations where the auto-inferred type of the column needs to be overridden, or explicit default value or non-null constraints are required.

• Create View: A view joins multiple tables, flexibly combining catalog information in various tables. Refer to Catalog views for more details.

• Create/Update Default View: Every dataset has a default view associated with it, which populates the default results shown on the catalog page. For datasets created before the 1.18 release, this default view may not exist. Use this action to create a default view or rerun the query against the default view to fetch the latest query results.

• Queries: Lists query execution history and allows direct fetch of the results without rerunning the query. Refer to Queries for more details.

• Import Catalog: Initiate a Quick Import Catalog.

• Import Jobs: Track the status of import catalog jobs.

  

• Table/View Actions: Lists all tables and views with actions against each table/view.

  

  • Dataset Tables

    • View Import Jobs: Track the status of import catalog jobs on a specific table.

    • Import Catalog: Use this option to append new entries to the existing catalog table incrementally.

    • Delete table: Delete table with all its entries.

    • Show Details: Show a detailed view with description and column details.
      

  • Views

    • Delete: Delete a view. This does not impact the entries in the underlying tables used to define the view.

    • Show Details: The read-only screen has details about the view.
      

    • Edit View: Edit properties of the view.

• Download: Download the query results as a CSV file.