javascript - Is there any way to auto crop the face after doing face detection with face-api.js? - Stack Overflow

admin管理员组

文章数量:1332382

I've implemented face-API in my react project which is detecting a single face with detectSingleFace from the picture.

Now I want to move one step further. I want face-api to auto-crop the face after detection. So, I can store it in some server, state or local storage. Is there any way to do so?

Here you can see a screenshot example I want to achieve One side is a picture another side is the auto cropped face(which I want to implement).

Here is my live code link in codesandbox

Below is my code module for face-api

PhotoFaceDetection.js

import React, { useState, useEffect, useRef } from "react";
import * as faceapi from "face-api.js";
import Img from "./assets/mFace.jpg";
import "./styles.css";

const PhotoFaceDetection = () => {
  const [initializing, setInitializing] = useState(false);
  const [image, setImage] = useState(Img);
  const canvasRef = useRef();
  const imageRef = useRef();

  // I want to store cropped image in this state
  const [pic, setPic] = useState();

  useEffect(() => {
    const loadModels = async () => {
      setInitializing(true);
      Promise.all([
        // models getting from public/model directory
        faceapis.tinyFaceDetector.load("/models"),
        faceapis.faceLandmark68Net.load("/models"),
        faceapis.faceRecognitionNet.load("/models"),
        faceapis.faceExpressionNet.load("/models")
      ])
        .then(console.log("success", "/models"))
        .then(handleImageClick)
        .catch((e) => console.error(e));
    };
    loadModels();
  }, []);

  const handleImageClick = async () => {
    if (initializing) {
      setInitializing(false);
    }
    canvasRef.current.innerHTML = faceapi.createCanvasFromMedia(
      imageRef.current
    );
    const displaySize = {
      width: 500,
      height: 350
    };
    faceapi.matchDimensions(canvasRef.current, displaySize);
    const detections = await faceapi.detectSingleFace(
      imageRef.current,
      new faceapi.TinyFaceDetectorOptions()
    );
    const resizeDetections = faceapi.resizeResults(detections, displaySize);
    canvasRef.current
      .getContext("2d")
      .clearRect(0, 0, displaySize.width, displaySize.height);
    faceapi.draw.drawDetections(canvasRef.current, resizeDetections);
    console.log(
      `Width ${detections.box._width} and Height ${detections.box._height}`
    );
    setPic(detections);
    console.log(detections);
  };

  return (
    <div className="App">
      <span>{initializing ? "Initializing" : "Ready"}</span>
      <div className="display-flex justify-content-center">
        <img ref={imageRef} src={image} alt="face" crossorigin="anonymous" />
        <canvas ref={canvasRef} className="position-absolute" />
      </div>
    </div>
  );
};

export default PhotoFaceDetection;

I've implemented face-API in my react project which is detecting a single face with detectSingleFace from the picture.

Now I want to move one step further. I want face-api to auto-crop the face after detection. So, I can store it in some server, state or local storage. Is there any way to do so?

Here you can see a screenshot example I want to achieve One side is a picture another side is the auto cropped face(which I want to implement).

Here is my live code link in codesandbox

Below is my code module for face-api

PhotoFaceDetection.js

import React, { useState, useEffect, useRef } from "react";
import * as faceapi from "face-api.js";
import Img from "./assets/mFace.jpg";
import "./styles.css";

const PhotoFaceDetection = () => {
  const [initializing, setInitializing] = useState(false);
  const [image, setImage] = useState(Img);
  const canvasRef = useRef();
  const imageRef = useRef();

  // I want to store cropped image in this state
  const [pic, setPic] = useState();

  useEffect(() => {
    const loadModels = async () => {
      setInitializing(true);
      Promise.all([
        // models getting from public/model directory
        faceapis.tinyFaceDetector.load("/models"),
        faceapis.faceLandmark68Net.load("/models"),
        faceapis.faceRecognitionNet.load("/models"),
        faceapis.faceExpressionNet.load("/models")
      ])
        .then(console.log("success", "/models"))
        .then(handleImageClick)
        .catch((e) => console.error(e));
    };
    loadModels();
  }, []);

  const handleImageClick = async () => {
    if (initializing) {
      setInitializing(false);
    }
    canvasRef.current.innerHTML = faceapi.createCanvasFromMedia(
      imageRef.current
    );
    const displaySize = {
      width: 500,
      height: 350
    };
    faceapi.matchDimensions(canvasRef.current, displaySize);
    const detections = await faceapi.detectSingleFace(
      imageRef.current,
      new faceapi.TinyFaceDetectorOptions()
    );
    const resizeDetections = faceapi.resizeResults(detections, displaySize);
    canvasRef.current
      .getContext("2d")
      .clearRect(0, 0, displaySize.width, displaySize.height);
    faceapi.draw.drawDetections(canvasRef.current, resizeDetections);
    console.log(
      `Width ${detections.box._width} and Height ${detections.box._height}`
    );
    setPic(detections);
    console.log(detections);
  };

  return (
    <div className="App">
      <span>{initializing ? "Initializing" : "Ready"}</span>
      <div className="display-flex justify-content-center">
        <img ref={imageRef} src={image} alt="face" crossorigin="anonymous" />
        <canvas ref={canvasRef} className="position-absolute" />
      </div>
    </div>
  );
};

export default PhotoFaceDetection;

• javascript
• reactjs
• react-hooks
• face-recognition
• face-api

Share Follow asked May 15, 2021 at 6:47 aishcriptaishcript 56311 gold badge1515 silver badges3131 bronze badges 2

• 1 Just looked at your sandbox and it seems like you've already got it working, is that correct? You could post that as an answer. – bas Commented May 15, 2021 at 16:29
• 1 @BasvanderLinden sure – aishcript Commented May 16, 2021 at 12:35

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 7

After doing a lot of R&D I figured it out. For future readers who may face an issue here is the guide. I've created another function that will get the original image reference and the bounded box dimension i.e. width and height. After that, I've used faceapi method to extract faces and then with the help of the toDataURL method I actually converted it to base64 file which can be rendered to any image src or can be stored anywhere. This is the function I was explaining above

async function extractFaceFromBox(imageRef, box) {
    const regionsToExtract = [
      new faceapi.Rect(box.x, box.y, box.width, box.height)
    ];
    let faceImages = await faceapi.extractFaces(imageRef, regionsToExtract);

    if (faceImages.length === 0) {
      console.log("No face found");
    } else {
      const outputImage = "";
      faceImages.forEach((cnv) => {
        outputImage.src = cnv.toDataURL();
        setPic(cnv.toDataURL());
      });
      // setPic(faceImages.toDataUrl);
      console.log("face found ");
      console.log(pic);
    }
  }

Then I call the above function inside my main function where I used faceapi face detection tiny model. extractFaceFromBox(imageRef.current, detections.box);

You can also visit live code here to check plete implementation

本文标签： javascriptIs there any way to auto crop the face after doing face detection with faceapijsStack Overflow