Catalog structure for analyze jobs
  • 14 Dec 2022
  • 2 Minutes to read
  • PDF

Catalog structure for analyze jobs

  • PDF

Data Explorer supports the Analyze job used for model performance analysis of an image classification or object detection model. This analysis requires ground truth and prediction information to be available in the catalog.

Analyze job for classification models

Create a comma-separated file similar to the following sample.

file_path,frame_idx_in_file,score,prediction_class,ground_truth_class
sdnet_analyze/Walls/Cracked/7069-154.jpg,0,0.9209516048431396,non-cracked,
sdnet_analyze/Walls/Cracked/7069-154.jpg,0,0.07904839515686035,cracked,
sdnet_analyze/Walls/Cracked/7069-154.jpg,0,,,cracked
sdnet_analyze/Walls/Cracked/7069-183.jpg,0,0.9039033353328705,non-cracked,
sdnet_analyze/Walls/Cracked/7069-183.jpg,0,0.09609666466712952,cracked,
sdnet_analyze/Walls/Cracked/7069-183.jpg,0,,,cracked

The first line should be the header line with the names of the fields. As described in general structure, fields must be present that represent the absolute file path or file path relative to the container URI. Additionally, for Video datasets, a field that identifies frames within the video file must also be present. 

  1. score(float): Probability of the image being detected correctly. The number must be between 0 and 1.
  2. prediction_class(string): A predicted class label for input data.
  3. ground_truth_class(string): Ground truth class label.

Note 

  1. Ground truth and prediction must be imported into the catalog as two separate rows, with each prediction in its row.
  2. Only one ground truth per image/video frame is supported.
  3. The prediction and corresponding confidence score must be present in the same row.

Analyze Job for Object Detection Models

Create a comma-separated file that must be in the following format.

file_path,frame_idx_in_file,gt_class,gt_box,pd_class,pd_box,pd_score
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2011_000753.jpg,0,car,"{'x_left': 0.812666902937676, 'x_right': 0.9925274478643895, 'y_top': 0.5254668930645845, 'y_bottom': 0.9725518810661693}","","",""
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2011_000753.jpg,0,traffic-light,"{'x_left': 0.25302248112220327, 'x_right': 0.9489973045211927, 'y_top': 0.3737651964479025, 'y_bottom': 0.4424076757749313}","","",""
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2011_000753.jpg,0,truck,"{'x_left': 0.821515851648852, 'x_right': 0.9616102232755864, 'y_top': 0.04334663063624866, 'y_bottom': 0.413286574667091}","","",""
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2011_000753.jpg,0,"","",traffic-light,"{'x_left': 0.08149561800260563, 'x_right': 0.9668002399072667, 'y_top': 0.25440137329378776, 'y_bottom': 0.8837804257735178}",0.5483175214891738
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2011_000753.jpg,0,"","",truck,"{'x_left': 0.06777141251658725, 'x_right': 0.9858757232625857, 'y_top': 0.0036388237825385195, 'y_bottom': 0.8911654794471764}",0.18220726194488202
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2011_000753.jpg,0,"","",car,"{'x_left': 0.4113943543536298, 'x_right': 0.9943189514949846, 'y_top': 0.15488761456109107, 'y_bottom': 0.9785810352571493}",0.17037061975029022
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2010_005364.jpg,0,truck,"{'x_left': 0.7211499479874045, 'x_right': 0.9808525551062757, 'y_top': 0.9075398530491968, 'y_bottom': 0.965455589700479}","","",""
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2010_005364.jpg,0,car,"{'x_left': 0.9463358841265729, 'x_right': 0.9740749436395301, 'y_top': 0.7288394870413163, 'y_bottom': 0.7581365702756696}","","",""
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2010_005364.jpg,0,pedestrian,"{'x_left': 0.14286272565145897, 'x_right': 0.9203391402990826, 'y_top': 0.643751508026241, 'y_bottom': 0.8944058141852372}","","",""
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2010_005364.jpg,0,traffic-light,"{'x_left': 0.35961338630076967, 'x_right': 0.40658112854256917, 'y_top': 0.8308644102469491, 'y_bottom': 0.8732949131563695}","","",""
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2010_005364.jpg,0,bus,"{'x_left': 0.5833657273948736, 'x_right': 0.8374848805983268, 'y_top': 0.30027764239612054, 'y_bottom': 0.3814474567028219}","","",""
VOCtrainval_11-May-2012/VOCdevkit/VOC2012/JPEGImages/2010_005364.jpg,0,"","",truck,"{'x_left': 0.1696867615155303, 'x_right': 0.9984250232135757, 'y_top': 0.23597941793683774, 'y_bottom': 0.9982859463247853}",0.8464981925824753

The first line should be the header line with the names of the fields. As described in general structure, fields must be present that represent the absolute file path or file path relative to the container URI. Additionally, for Video datasets, a field that identifies frames within the video file must also be present. 

  1. gt_class(string): Ground truth class label
  2. gt_box(string):Ground truth bounding box co-ordinates as a string.
    1. The string should be formatted as a JSON with x_left, x_right, y_top, and y_bottom.  
    2. All key values must be normalized between 0 and 1 using the width and height of the image.
    3. For example {'x_left': 0.5, 'y_top': 0.75, 'x_right': 0.7, 'y_bottom': 0.95} represents a valid bounding box.
  3. pd_class(string): a predicted class label for input data.
  4. pd_box(string): prediction bounding box coordinates in the format same as ground truth bounding box coordinates.
  5. pd_score(float): prediction confidence score.

Note:

  1. Each row in the catalog should specify one ground truth or one prediction label if an object has multiple ground truth and prediction labels.
  2. The ground truth label and ground truth bounding box should be in the same row.
  3. The prediction confidence and bounding box should be in the same row as the prediction label.

Was this article helpful?