Skip to content

Comments

Fixes IoU calculation - off by 1 bug#343

Closed
dsuthar-nvidia wants to merge 2 commits intoNVIDIA:caffe-0.16from
dsuthar-nvidia:caffe-0.16
Closed

Fixes IoU calculation - off by 1 bug#343
dsuthar-nvidia wants to merge 2 commits intoNVIDIA:caffe-0.16from
dsuthar-nvidia:caffe-0.16

Conversation

@dsuthar-nvidia
Copy link

Fixes IoU calculation - off by 1 bug. There should be +1 when calculating the area of a rectangle from left,top,right,bottom pixel values. Otherwise this seemingly benign lines can result in wrong FP/TP calculation for some edge cases.

Fixes IoU calculation - off by 1 bug. There should be +1 when calculating the area of a rectangle from left,top,right,bottom pixel values.
@drnikolaev drnikolaev requested a review from lukeyeager May 24, 2017 06:02
@lukeyeager
Copy link
Member

@ssarathy @drendleman - can either of you review this?

@drendleman
Copy link

@dansuthar Can you describe the edge case in a little more detail? Just looking over this makes it seem that you're removing the possibility that there can be zero intersection between the two bounding boxes

@lukeyeager lukeyeager removed their request for review May 24, 2017 16:43
@dsuthar-nvidia
Copy link
Author

@drendleman For example, if my IoU threshold is 0.5, following will get classified as FP because as per current method it gets IoU of 0.4942

iou([ 278, 299, 302, 326], [ 286.5757, 301.9515, 300.103, 326.7515])

The correct IoU should be 0.51185.

Both methods still correctly calculate non overlapping rectangles like ([ 0, 0, 5, 5], [ 10, 10, 15, 15]).

@drendleman
Copy link

drendleman commented May 24, 2017

I don't think this is the right solution. This code fails for the case of [0, 0, 10, 10] and [0, 5, 10, 10] which should have an IOU of exactly 0.5 - instead I get 0.54~. In the case of [0, 0, 10, 10] and [0, 10, 20, 20] I get 0.03~ instead of 0.

We follow OpenCV's Rect convention in that the TL point is inclusive and the BR point is not

@dsuthar-nvidia
Copy link
Author

dsuthar-nvidia commented May 25, 2017

I don't think challenges like PASCAL VOC and KITTI, when specifies bboxes in [left, top, right, bottom] format, excludes the TL point from the area calculation. Please have a look at their VOCevaldet.m file line 79 in their devkit.
Challenges like WIDER face detection also have bboxes as small as 2x2.

True negatives are never used in precision and recall calculations.
@drendleman
Copy link

drendleman commented May 25, 2017

Lines 187-227 from the Kitti devkit's IOU calculation assumes BR is exclusive and TL is inclusive. Pascal VOC I am not intimately familiar with, perhaps the two challenges differ?

/*=======================================================================
EVALUATION HELPER FUNCTIONS
=======================================================================*/

// criterion defines whether the overlap is computed with respect to both areas (ground truth and detection)
// or with respect to box a or b (detection and "dontcare" areas)
inline double boxoverlap(tBox a, tBox b, int32_t criterion=-1){

  // overlap is invalid in the beginning
  double o = -1;

  // get overlapping area
  double x1 = max(a.x1, b.x1);
  double y1 = max(a.y1, b.y1);
  double x2 = min(a.x2, b.x2);
  double y2 = min(a.y2, b.y2);

  // compute width and height of overlapping area
  double w = x2-x1;
  double h = y2-y1;

  // set invalid entries to 0 overlap
  if(w<=0 || h<=0)
    return 0;

  // get overlapping areas
  double inter = w*h;
  double a_area = (a.x2-a.x1) * (a.y2-a.y1);
  double b_area = (b.x2-b.x1) * (b.y2-b.y1);

  // intersection over union overlap depending on users choice
  if(criterion==-1)     // union
    o = inter / (a_area+b_area-inter);
  else if(criterion==0) // bbox_a
    o = inter / a_area;
  else if(criterion==1) // bbox_b
    o = inter / b_area;

  // overlap
  return o;
}

@dsuthar-nvidia
Copy link
Author

Ok, looks like those two challenges have different methods of computing IoU despite having same label format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants