Text search

Data Explorer offers various visual input-based search techniques, including full image search, patch search, and full object search. Text search augments these capabilities with support for free-form text inputs. To create a job with text search capabilities, follow the steps below.

1. A dataset will get text search capability through a pre-registered pipeline named 'AkridataImageClipPipeline'(for the Image data type) and 'AkridataVideoClipPipeline'(for the Video data type). Read the article to learn more about pipelines.

2. The corresponding pipeline must be attached to the dataset. Read the article to learn more about pipeline attachments to datasets.

3. The pipeline must be run to produce features that allow text search. Read the article to learn more about executing a pipeline.

4. The text feature pipeline will populate a catalog on the dataset catalog page. Read the article to learn more about catalog operations. 

5. Create a view that joins the catalog table from a pipeline that ingested image features and the text-search capable pipeline. The file_id and frame_idx_in_file fields must be used to join the catalog tables. Read the articleto learn more about catalog views.

   1. The screenshot below shows two pipeline tables that can be used to create a view that supports text search jobs.
      

      

   2. Create a view with file_id and frame_idx_in_file as the join columns. The 'frame_idx_in_file' column is not needed for the image type of the dataset.
      

6. Once the view is created, run the catalog query and create a visualize job.

   1. The icon highlighted below can identify a job with text search support.
      

      

   2. The job visualization shows a 'Text Search' tab shown below.

      

7. Run text prompt search as per the following steps:

   1. Enter text prompt.

   2. Enter the required number of results.

   3. Optionally select the filters and tunables for further reduction in search space. Refer to the article for details about tunables and filters.  

   4. Click the 'Search' button.
       

8. Use the following controls on the text search results:

   1. Group Images: For Video datasets, successive images that are expected to be similar can be grouped together using this control to reduce redundancy in the results.
      

   2. Sim score settings: You can view the sim search match score and filter results above a threshold score.
       

   3. Highlight points: View the results on a 2D plot that shows the relative position of the results in the content feature space.