Text search
  • 03 Sep 2023
  • 2 Minutes to read
  • Dark
    Light
  • PDF

Text search

  • Dark
    Light
  • PDF

Article Summary

This feature is currently in the BETA stage.
Data Explorer provides different visual input-based search techniques like full image search, patch search and full object search. Text search augments these capabilities with support for free-form text inputs. To create a job with text search capabilities, follow the below steps.

  1. A dataset will get text search capability through a pre-registered pipeline named 'AkridataImageClipPipeline'(for the Image data type) and 'AkridataVideoClipPipeline'(for the Video data type). Read the article to learn more about pipelines.
  2. The corresponding pipeline must be attached to the dataset. Read the article to learn more about pipeline attachments to datasets.
  3. The pipeline must be run to produce features that allow text search. Read the article to learn more about executing a pipeline.
  4. The text feature pipeline will populate a catalog on the dataset catalog page. Read the article to learn more about catalog operations. 
  5. Create a view that joins the catalog table from a pipeline that ingested image features and the text-search capable pipeline. The file_id and frame_idx_in_file fields must be used to join the catalog tables. Read the article to learn more about catalog views.
    1. The screenshot below shows two pipeline tables that can be used to create a view that supports text search jobs.
    2. Create a view with file_id and frame_idx_in_file as the join columns. The 'frame_idx_in_file' column is not needed for the image type of the dataset.
  6. Once the view is created, run the catalog query and create a visualize job.
    1. The icon highlighted below can identify a job with text search support.
    2. The job visualization shows a 'Text Search' tab shown below.
  7. Run text prompt search as per the below steps(Refer controls 1 to 4 on the image)
    1. Enter text prompt
    2. Optionally select search space reduction options to narrow the search space. In the screenshot, the catalog filters applied in the 'Selection' panel can be applied to narrow the target search space.
    3. Enter the required number of results.
    4. Optionally select the filters and tunables for further reduction in search space. Please refer to the article for details about tunables and filters.
    5. Click the 'Search' button.
  8. When the search is complete, the following controls are available (Refer to steps 5 to 7 on the above image)
    1. Group Images: For Video datasets, successive images that are expected to be similar can be grouped together using this control to reduce redundancy in the results.
    2. Sim score settings: You can view the sim search match score and filter results above a threshold score.
    3. Highlight points: View the results on a 2D plot showing the relative position of the result in the content feature space.

Was this article helpful?