opencv - How to find orienation of an object in a 2d Image - Stack Overflow

admin管理员组

文章数量:1122832

I have to detect some objects (Sofa and table in my case) and replace that objects from a 3d model. I am using YOLO to detect and LaMa to remove object and then trying to place the 3d model on the same place with same orientation from where the object is removed.

So, I tried PCA approach in OpenCV to find the angle of orientation but in some image it is working fine but not for all. Below is my code:

import cv2
import numpy as np
from ultralytics import YOLO
from math import atan2, cos, sin, sqrt, pi
import matplotlib.pyplot as plt
from PIL import Image
from matplotlib.patches import Polygon

img = cv2.imread("/home/spxlpt087/Downloads/image_with_info.jpg")
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

model = YOLO("/home/spxlpt087/Downloads/yoloV11_1_orig_2.pt")
result = model.predict(img, conf = 0.45)
cordinates = result[0].boxes.xywh.cpu().numpy()

roi_new = img[int(y1):int(y1)+int(h), int(x1):int(x1)+int(w)]
mask_cordinates = result[0].masks.xy[1]
height, width = img.shape[:2]
binary_mask = np.zeros((height, width), dtype=np.uint8)

polygon_points = np.int32(mask_cordinates)
cv2.fillPoly(binary_mask, [polygon_points], color=255)
contours, _ = cv2.findContours(binary_mask, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
contour_image = img.copy()  # Replace `img` with your original image
cv2.drawContours(contour_image, contours, -1, (0, 255, 0), 2)

for c in contours:
    sz = len(c)
    data_pts = np.empty((sz, 2), dtype = np.float64)
    for i in range(data_pts.shape[0]):
        data_pts[i,0] = c[i,0,0]
        data_pts[i,1] = c[i,0,1]

    mean = np.empty((0))
    mean, eigenvectors, eigenvalues = cv2.PCACompute2(data_pts, mean)

    angle_g = np.arctan2(eigenvectors[0, 1], eigenvectors[0, 0]) #* 180 / np.pi
    angle_doc = atan2(eigenvectors[0,1], eigenvectors[0,0])
     
    # Store the center of the object
    cntr = (int(mean[0,0]), int(mean[0,1]))

    cv2.circle(contour_image, cntr, 5, (0, 0, 255), 20)  # Red circle for center

    # Plot the data points
    for pt in data_pts:
        pt_int = tuple(map(int, pt))
        cv2.circle(contour_image, pt_int, 2, (255, 0, 0), 20)

plt.figure(figsize=(10, 8))
plt.imshow(cv2.cvtColor(contour_image, cv2.COLOR_BGR2RGB))
plt.title("Contour with PCA Center and Data Points")
plt.axis('off')
plt.show()

I am getting same value for angle_g and angle_doc. I am attaching image which graphically represents this. When I am passing this values to the front end guy who is working on three.js to place the 3d model is not abe to fir the 3d properly. I am attaching an image which shows that the sofa is oncorrectly placed.

I did some digging and I found this article :-

1. /
2. .x/d1/dee/tutorial_introduction_to_pca.html

But results are not correct. Also, the sofa placed is not very wrong but a slight off orientation bt when I am removing the table in from of the sofa and calculating the angle using above approach it is place just towards similar way the sofa placed. In origional image the objects are placed facing each other but when 3D objects are placed they are placed facing opposite that in same direction.

本文标签： opencvHow to find orienation of an object in a 2d ImageStack Overflow