c++ - OpenCV Template Matching for Playing Cards - Stack Overflow

admin管理员组

文章数量:1122846

For my bridge training lessons I 'm trying to analyse a deal played in Bridge Base Online, by capturing the image and then try to find where are the 52 cards.

The reference image is this one (capture.jpg)

And I have also 52 images (41x68) of the cards, like the five of spades:

Now when doing pattern matching in OpenCV:

Mat1f result;
matchTemplate(org_gray, template_gray, result, TM_CCOEFF_NORMED);
double thresh = 0.8;
threshold(result, result, thresh, 1., THRESH_BINARY);

Mat1b resb;
result.convertTo(resb, CV_8U, 255);

std::vector<std::vector<Point>> contours;
findContours(resb, contours, RETR_LIST, CHAIN_APPROX_SIMPLE);

for (int i = 0; i < contours.size(); ++i)
{
    Mat1b mask(result.rows, result.cols, uchar(0));
    drawContours(mask, contours, i, Scalar(255), cv::FILLED);

    Point max_point,min_point;
    double max_val;
    minMaxLoc(result, NULL, &max_val, &min_point, &max_point, mask);

    rectangle(img, Rect(max_point.x, max_point.y, ptpw->m.cols, ptpw->m.rows), Scalar(0, 255, 0), 2);
}
imshow("b", ptpw->m);
imshow("a",img);

The result is this one. It did detect the location of S5 but it also detected 7 more cards...

How can I enhance my algorithm?

If I raise the threshold to 0.87 it finds only one card, but the 5 of clubs instead. I can adjust the threshold to find only one card, but why the 5C instead of 5S?

Thanks.

For my bridge training lessons I 'm trying to analyse a deal played in Bridge Base Online, by capturing the image and then try to find where are the 52 cards.

The reference image is this one (capture.jpg)

And I have also 52 images (41x68) of the cards, like the five of spades:

Now when doing pattern matching in OpenCV:

Mat1f result;
matchTemplate(org_gray, template_gray, result, TM_CCOEFF_NORMED);
double thresh = 0.8;
threshold(result, result, thresh, 1., THRESH_BINARY);

Mat1b resb;
result.convertTo(resb, CV_8U, 255);

std::vector<std::vector<Point>> contours;
findContours(resb, contours, RETR_LIST, CHAIN_APPROX_SIMPLE);

for (int i = 0; i < contours.size(); ++i)
{
    Mat1b mask(result.rows, result.cols, uchar(0));
    drawContours(mask, contours, i, Scalar(255), cv::FILLED);

    Point max_point,min_point;
    double max_val;
    minMaxLoc(result, NULL, &max_val, &min_point, &max_point, mask);

    rectangle(img, Rect(max_point.x, max_point.y, ptpw->m.cols, ptpw->m.rows), Scalar(0, 255, 0), 2);
}
imshow("b", ptpw->m);
imshow("a",img);

The result is this one. It did detect the location of S5 but it also detected 7 more cards...

How can I enhance my algorithm?

If I raise the threshold to 0.87 it finds only one card, but the 5 of clubs instead. I can adjust the threshold to find only one card, but why the 5C instead of 5S?

Thanks.

• c++
• opencv
• computer-vision
• template-matching
• game-automation

Share Improve this question Follow edited Nov 21, 2024 at 11:41 Christoph Rackwitz 15.1k55 gold badges3838 silver badges4949 bronze badges asked Nov 21, 2024 at 8:46 Michael ChourdakisMichael Chourdakis 11k33 gold badges4646 silver badges8787 bronze badges 9

• This question is similar to: matchTemplate() missing detections and giving false positives, what can I do?. If you believe it’s different, please edit the question, make it clear how it’s different and/or how the answers on that question are not helpful for your problem. – Christoph Rackwitz Commented Nov 21, 2024 at 9:05
• matchTemplate doesn't test multiple scales. – Christoph Rackwitz Commented Nov 21, 2024 at 9:07
• @ChristophRackwitz If I use TM_SQDIFF_NORMED I get 50 completely out of place contours. The question is actually why, with the best threshold of 0.87 , it finds first the 5 club instead of the 5 spade. – Michael Chourdakis Commented Nov 21, 2024 at 9:10
• 1 I've been hasty. I'll look into this more thoroughly. – Christoph Rackwitz Commented Nov 21, 2024 at 9:14
• the one "exact" instance on the left isn't exact because the template has a wide white border while the instance is narrower, having the border of an adjacent card within the area of the template's best fit. that makes the match worse. – Christoph Rackwitz Commented Nov 21, 2024 at 9:20

 |  Show 4 more comments

1 Answer 1

Sorted by: Reset to default 1

As is the case with everyone who asks, the matching mode is bad. For perfect data without brightness variations, the TM_SQDIFF* modes are the best choice.

matchTemplate(org_gray, template_gray, result, TM_SQDIFF_NORMED);
double thresh = 0.1;
threshold(result, result, thresh, 1., THRESH_BINARY_INV);

You will get a difference if the instance is red/gray, so you should consider converting to grayscale by simply taking the green or blue channel of the image (red text on white background is just all bright in the red channel). You could do that by splitting the source channels or with mixChannels, or even with transform and a custom mixing matrix.

Further, your template has a wide white border. It's too wide. On the expected perfect match, it overlaps onto the image of a neighboring card (black border, black number), reducing the quality of the match.

Trim it down. That'll do better.

SQDIFF_NORMED on the original template:

With template cropped more closely:

Even here it's not perfect because of JPEG compression artefacts. I also think the card instance's rounded edge intrudes on the square template.

---

And then you'll need Non-Maximum Suppression (NMS). If you just threshold the scores array, you'll get adjacent "on" pixels for the same detection, or even extrema that are almost adjacent for some reason. So don't just threshold, but do a little more. This is a general recipe with steps that were needed in various situations.

Sketch in Python:

nms_threshold = 0.10 # looking for minima
nms_radius = 5
localmin = cv.erode(scores, None, iterations=nms_radius)
extrema = (scores == localmin) & (scores <= nms_threshold)
extrema = cv.morphologyEx(extrema.astype(np.uint8), cv.MORPH_CLOSE, None, iterations=nms_radius)
# and then connected components, with stats for centroid
# will also handle minima larger than 1 pixel
(nlabels, labels, stats, centroids) = cv.connectedComponentsWithStats(extrema.astype(np.uint8))

本文标签： cOpenCV Template Matching for Playing CardsStack Overflow