Pipeline-Dataset attach/detach operations
  • 21 Aug 2023
  • 1 Minute to read
  • Dark
    Light
  • PDF

Pipeline-Dataset attach/detach operations

  • Dark
    Light
  • PDF

Article Summary

A pipeline must be attached to a dataset for it to be executed on the data present in the dataset. The decoupling between pipeline and dataset allows each pipeline to be maintained as a reusable entity that can be independently attached to any dataset. A pipeline can be attached to multiple datasets, and a dataset can have multiple pipelines attached. The attached pipelines are available for execution using the 'adectl run' command.
To run a specific pipeline, please use 'adectl run -m <pipeline_name>' or 'adectl run -p <pipeline_id>'. 

Attach pipeline to dataset

  1. Navigate to the 3-dot icon on the card corresponding to the pipeline to which the dataset(s) needs to be attached and click on the 'Attach datasets' option.
  2. Select the list of datasets to be attached from the drop-down provided.
  3. Select the policy for this attachment that determines the mode(scheduled Vs triggered) of ingestion and the compute resources where ingestion will be done.
    1. Schedule policy(BETA): The ingestion will be triggered as per the provided schedule on the selected cluster. Currently, only one pre-provisioned cluster, 'AkridataEdgeCluster', is available for selection, and this list will be extended with user-registered clusters in the future. The schedule is specified using a cron string. 
    2. On-demand policy(BETA): In this mode, the ingestion is triggered by the user as needed on the selected cluster.
    3. Manual adectl run: In this mode, the compute resource for ingestion is to be provisioned by the user and ingestion must be triggered using the adectl command line utility.
  4. Click on the 'Attach' button.

Note:

Once a pipeline is attached to a video dataset, ingestion frame rate is set. This frame rate will be used for all pipelines attached to this dataset, and it will be displayed on the dataset info section.

Detach pipeline from dataset

  1. Navigate to the 3-dot icon on the card corresponding to the pipeline to which the dataset(s) needs to be attached and click on the 'Detach datasets' option.
  2. Select a list of datasets to detach from the drop-down provided.
  3. Click on 'Detach' button.
Detach pipeline from dataset
A detach operation does not delete the already registered data in the dataset but disallows any future data ingestion using the detached pipeline.

Was this article helpful?