react native - RN MLKit does not recognize custom TFLite model - Stack Overflow

admin管理员组

文章数量:1332873

I am unable to get results of my TFLite model, which was previously used successfully with the react-native-fast-tflite library, so I am confident the model works. Any ideas, perhaps my setup is incorrect and the custom model docs are incorrect?

const MODELS: AssetRecord = {
  // the name you'll use to refer to the model
  mfi: {
    // the relative path to the model file
    model: require('../src/assets/tflite/model.tflite'),
    options: {
      // the options you want to use for this model
      shouldEnableMultipleObjects: false,
      shouldEnableClassification: false,
      detectorMode: 'singleImage',
      // maxPerObjectLabelCount: 1,
    },
  },
};

 const { ObjectDetectionModelContextProvider } = useObjectDetectionModels({
    assets: MODELS,
    loadDefaultModel: false,
  });

<ObjectDetectionModelContextProvider>
                                <Stack
                                  initialRouteName="(app)/(auth)/home"
                                  screenOptions={{
                                    headerShown: false,
                                    presentation: 'fullScreenModal',
                                  }}
                                />

Then, I consume the model:

const model = useObjectDetector('mfi');
  const [modelLoaded, setModelLoaded] = useState(model?.isLoaded() ?? false);
  useEffect(() => {
    async function loadModel() {
      if (!model || modelLoaded) return;
      await model.load();
      setModelLoaded(true);
    }

    loadModel();
  }, [model, modelLoaded]);

and invoke the detection from camera photo:

const capturePhoto = async () => {
    if (!cameraRef?.current) {
      setMessage([
        'error',
        'Something went wrong. Unable to capture photo. Please reload & try again.',
      ]);
      return;
    }
    const photo = await cameraRef.current?.takePhoto({
      flash: flash,
    });

    const detectionResult = await model?.detectObjects(photo.path);
    console.log(JSON.stringify(detectionResult));

    setResult(detectionResult);
    setPhoto(photo.path);

  };

my output of the detectionResult is: [{"frame":{"origin":{"x":1064,"y":301},"size":{"y":2050,"x":2075}},"labels":[],"trackingID":0}]

So there are no labels in the result...which means it didn't recognize content of my photo.

本文标签： react nativeRN MLKit does not recognize custom TFLite modelStack Overflow