Opencv Speed Traffic Sign Detection

I have a problem detecting speed traffic signs with opencv 2.4 for Android. I do the following: 'capture frame -> convert it to HSV -> extract red areas -> detect signs

Solution 1:

You can find a review of traffic signs detection methods here and here.

You'll see that there are 2 ways you can achieve this:

1. Color-based (like what you're doing now)
2. Shape-based

In my experience, I found that shape-based methods works pretty good, because the color may change a lot under different lighting conditions, camera quality, etc.

Since you need to detect speed traffic signs, which I assume are always circular, you can use an ellipse detector to find all circular objects in your image, and then apply some validation to determine if it's a traffic sign or not.

Why ellipse detection?

Well, since you're looking for perspective distorted circles, you are in fact looking for ellipses. Real-time ellipse detection is an interesting (although limited) research topic. I'll point you out to 2 papers with C++ source code available (which you can use in you app through native JNI calls):

1. L. Libuda, I. Grothues, K.-F. Kraiss, Ellipse detection in digital image data using geometric features, in: J. Braz, A. Ranchordas, H. Arajo, J. Jorge (Eds.), Advances in Computer Graphics and Computer Vision, volume 4 of Communications in Computer and Information Science, Springer Berlin Heidelberg, 2007, pp. 229-239. link, code

2. M. Fornaciari, A. Prati, R. Cucchiara, "A fast and effective ellipse detector for embedded vision applications", Pattern Recognition, 2014 link, code

---

UPDATE

I tried the method 2) without any preprocessing. You can see that at least the sign with the red border is detected very good:

Solution 2:

Referencing to your text:

This is converted to HSV, as you can see red areas look the same color as nearby trees. Thats how I'm suppose to know it's red but I can't.

I want to show you my result of basically what you did (simple operations should be easily transferable to android openCV):

// convert to HSV
    cv::Mat hsv;
    cv::cvtColor(input,hsv,CV_BGR2HSV);

    std::vector<cv::Mat> channels;
    cv::split(hsv,channels);

    // opencv = hue values are divided by 2 to fit 8 bit rangefloat red1 = 25/2.0f;
    // red has one part at the beginning and one part at the end of the range (I assume 0° to 25° and 335° to 360°)float red2 = (360-25)/2.0f;

    // compute both thresholds
    cv::Mat thres1 = channels[0] < red1;
    cv::Mat thres2 = channels[0] > red2;

    // choose some minimum saturation
    cv::Mat saturationThres = channels[1] > 50;

    // combine the results
    cv::Mat redMask = (thres1 | thres2) & saturationThres;

    // display result
    cv::imshow("red", redMask);

These are my results:

From your result, please mind that findContours alters the input image, so maybe you extracted the ellipse but just don't see it in the image anymore, if you saved the image AFTER findContours.

Solution 3:

privatevoidfindEllipses(Mat input){
    MatthresholdOutput=newMat();
    intthresh=150;

    List<MatOfPoint> contours = newArrayList<MatOfPoint>();
    MatOfInt4hierarchy=newMatOfInt4();

    Imgproc.threshold(source, thresholdOutput, thresh, 255, Imgproc.THRESH_BINARY);
    //Imgproc.Canny(source, thresholdOutput, 50, 180);
    Imgproc.findContours(source, contours, hierarchy, Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
//      source = thresholdOutput;
    RotatedRect minEllipse[] = newRotatedRect[contours.size()];

    for(int i=0; i<contours.size();i++){
        MatOfPoint2f temp=newMatOfPoint2f(contours.get(i).toArray());

        if(temp.size().height > minEllipseSize && temp.size().height < maxEllipseSize){
            doublea= Imgproc.fitEllipse(temp).size.height;
            doubleb= Imgproc.fitEllipse(temp).size.width;
            if(Math.abs(a - b) < 10)
                minEllipse[i] = Imgproc.fitEllipse(temp);
        }
    }

    detectedObjects.clear();
    for( inti=0; i< contours.size(); i++ ){
        Scalarcolor=newScalar(180, 255, 180);
        if(minEllipse[i] != null){
            detectedObjects.add(newDetectedObject(minEllipse[i].center));
            DetectedObjectdetectedObj=newDetectedObject(minEllipse[i].center);
            Core.ellipse(source, minEllipse[i], color, 2, 8);
        }
    }
}

Solution 4:

have you tried using opencv ORB? it works really well. I created a haar cascade for a traffic sign (roundabout in my case) and used opencv ORB to match features and remove any false positives. For image recognition used Google's tensorflow and results were spectacular.