在 Android 上使用 ML Kit 偵測臉孔

你可以使用 ML Kit 偵測圖片和影片中的臉孔。

事前準備

  1. 如果還沒試過 將 Firebase 新增至您的 Android 專案
  2. 將 ML Kit Android 程式庫的依附元件新增至模組 (應用程式層級) Gradle 檔案 (通常是 app/build.gradle):
    apply plugin: 'com.android.application'
    apply plugin: 'com.google.gms.google-services'
    
    dependencies {
      // ...
    
      implementation 'com.google.firebase:firebase-ml-vision:24.0.3'
      // If you want to detect face contours (landmark detection and classification
      // don't require this additional model):
      implementation 'com.google.firebase:firebase-ml-vision-face-model:20.0.1'
    }
  3. 選用 (建議選用):將應用程式設為自動下載 將應用程式從 Play 商店安裝到裝置上

    方法是在應用程式的 AndroidManifest.xml 檔案:

    <application ...>
      ...
      <meta-data
          android:name="com.google.firebase.ml.vision.DEPENDENCIES"
          android:value="face" />
      <!-- To use multiple models: android:value="face,model2,model3" -->
    </application>
    敬上 如果您未啟用安裝期間模型下載功能,模型就會 。您在 下載完成不會產生任何結果。

輸入圖片規範

為了讓 ML Kit 準確偵測臉孔,輸入圖片必須包含臉孔 以充足的像素資料表示基本上, 至少需要 100x100 像素如要偵測 臉部輪廓線,則 ML Kit 需要較高的解析度輸入: 至少應為 200 x 200 像素。

如果您在即時應用程式中偵測臉孔,您可能還需要 將輸入圖片的整體尺寸納入考量較小的圖片 加快處理速度,因此為了縮短延遲時間,擷取解析度較低的圖片 (請謹記上述準確率規定),並確保 拍攝主體的臉孔會盡量佔滿圖片。另請參閱 即時效能改善秘訣

圖片焦點不佳可能會降低準確性。如果沒有可接受的結果 請試著要求使用者重新擷取圖片

臉部與相機相對的方向也會影響臉部表情 ML Kit 偵測到的特徵詳情請見 臉部偵測 概念

1. 設定臉部偵測工具

為圖像套用臉部偵測功能之前,如果想變更 臉部偵測器的預設設定,請用 FirebaseVisionFaceDetectorOptions 物件。 您可以變更下列設定:

設定
效能模式 FAST (預設) |ACCURATE

改善偵測臉孔的速度或精確度。

偵測地標 NO_LANDMARKS (預設) |ALL_LANDMARKS

是否嘗試辨識臉部「地標」:眼睛、耳朵、鼻子、 臉頰、嘴巴等

偵測輪廓線 NO_CONTOURS (預設) |ALL_CONTOURS

是否偵測臉部特徵的輪廓。輪廓線是 只會偵測到圖片中最醒目的臉孔。

將臉孔分類 NO_CLASSIFICATIONS (預設) |ALL_CLASSIFICATIONS

是否將臉孔分類 (例如「微笑」)、 和「睜開雙眼」

臉孔最小尺寸 float (預設:0.1f)

待偵測臉孔的最小尺寸 (相對於圖片)。

啟用臉部追蹤功能 false (預設) |true

是否要指派臉孔 ID,以用於追蹤 圖像中的人物臉孔。

請注意,啟用輪廓偵測功能後,只有一張臉孔 因此臉部追蹤功能無法產生實用的結果。為此 原因及加快偵測速度,請勿同時啟用兩個輪廓線 偵測及臉部追蹤

例如:

Java

// High-accuracy landmark detection and face classification
FirebaseVisionFaceDetectorOptions highAccuracyOpts =
        new FirebaseVisionFaceDetectorOptions.Builder()
                .setPerformanceMode(FirebaseVisionFaceDetectorOptions.ACCURATE)
                .setLandmarkMode(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS)
                .setClassificationMode(FirebaseVisionFaceDetectorOptions.ALL_CLASSIFICATIONS)
                .build();

// Real-time contour detection of multiple faces
FirebaseVisionFaceDetectorOptions realTimeOpts =
        new FirebaseVisionFaceDetectorOptions.Builder()
                .setContourMode(FirebaseVisionFaceDetectorOptions.ALL_CONTOURS)
                .build();

Kotlin+KTX

// High-accuracy landmark detection and face classification
val highAccuracyOpts = FirebaseVisionFaceDetectorOptions.Builder()
        .setPerformanceMode(FirebaseVisionFaceDetectorOptions.ACCURATE)
        .setLandmarkMode(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS)
        .setClassificationMode(FirebaseVisionFaceDetectorOptions.ALL_CLASSIFICATIONS)
        .build()

// Real-time contour detection of multiple faces
val realTimeOpts = FirebaseVisionFaceDetectorOptions.Builder()
        .setContourMode(FirebaseVisionFaceDetectorOptions.ALL_CONTOURS)
        .build()

2. 執行臉部偵測工具

如要偵測圖片中的臉孔,請建立 FirebaseVisionImage 物件 從 Bitmapmedia.ImageByteBuffer、位元組陣列或 裝置。然後,將 FirebaseVisionImage 物件傳遞至 FirebaseVisionFaceDetectordetectInImage 方法。

如要使用臉部辨識功能,圖片尺寸應至少為 480x360 像素。如果您可以即時辨識臉孔,就必須擷取影格 達到這個最低解析度將有助於縮短延遲時間

  1. 透過所需位置建立 FirebaseVisionImage 物件。 圖片。

    • 要使用 FirebaseVisionImage 物件 media.Image 物件,例如從 裝置的相機,請傳遞 media.Image 物件和圖片的 旋轉至 FirebaseVisionImage.fromMediaImage()

      如果您使用 CameraX 程式庫、OnImageCapturedListenerImageAnalysis.Analyzer 類別會計算旋轉值 因此只需將旋轉模型 轉換為 ML Kit 的 呼叫前 ROTATION_ 常數 FirebaseVisionImage.fromMediaImage()

      Java

      private class YourAnalyzer implements ImageAnalysis.Analyzer {
      
          private int degreesToFirebaseRotation(int degrees) {
              switch (degrees) {
                  case 0:
                      return FirebaseVisionImageMetadata.ROTATION_0;
                  case 90:
                      return FirebaseVisionImageMetadata.ROTATION_90;
                  case 180:
                      return FirebaseVisionImageMetadata.ROTATION_180;
                  case 270:
                      return FirebaseVisionImageMetadata.ROTATION_270;
                  default:
                      throw new IllegalArgumentException(
                              "Rotation must be 0, 90, 180, or 270.");
              }
          }
      
          @Override
          public void analyze(ImageProxy imageProxy, int degrees) {
              if (imageProxy == null || imageProxy.getImage() == null) {
                  return;
              }
              Image mediaImage = imageProxy.getImage();
              int rotation = degreesToFirebaseRotation(degrees);
              FirebaseVisionImage image =
                      FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
              // Pass image to an ML Kit Vision API
              // ...
          }
      }

      Kotlin+KTX

      private class YourImageAnalyzer : ImageAnalysis.Analyzer {
          private fun degreesToFirebaseRotation(degrees: Int): Int = when(degrees) {
              0 -> FirebaseVisionImageMetadata.ROTATION_0
              90 -> FirebaseVisionImageMetadata.ROTATION_90
              180 -> FirebaseVisionImageMetadata.ROTATION_180
              270 -> FirebaseVisionImageMetadata.ROTATION_270
              else -> throw Exception("Rotation must be 0, 90, 180, or 270.")
          }
      
          override fun analyze(imageProxy: ImageProxy?, degrees: Int) {
              val mediaImage = imageProxy?.image
              val imageRotation = degreesToFirebaseRotation(degrees)
              if (mediaImage != null) {
                  val image = FirebaseVisionImage.fromMediaImage(mediaImage, imageRotation)
                  // Pass image to an ML Kit Vision API
                  // ...
              }
          }
      }

      如果您沒有使用相機程式庫來提供圖像旋轉角度, 可根據裝置旋轉角度和相機方向計算 感應器:

      Java

      private static final SparseIntArray ORIENTATIONS = new SparseIntArray();
      static {
          ORIENTATIONS.append(Surface.ROTATION_0, 90);
          ORIENTATIONS.append(Surface.ROTATION_90, 0);
          ORIENTATIONS.append(Surface.ROTATION_180, 270);
          ORIENTATIONS.append(Surface.ROTATION_270, 180);
      }
      
      /**
       * Get the angle by which an image must be rotated given the device's current
       * orientation.
       */
      @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
      private int getRotationCompensation(String cameraId, Activity activity, Context context)
              throws CameraAccessException {
          // Get the device's current rotation relative to its "native" orientation.
          // Then, from the ORIENTATIONS table, look up the angle the image must be
          // rotated to compensate for the device's rotation.
          int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation();
          int rotationCompensation = ORIENTATIONS.get(deviceRotation);
      
          // On most devices, the sensor orientation is 90 degrees, but for some
          // devices it is 270 degrees. For devices with a sensor orientation of
          // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
          CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
          int sensorOrientation = cameraManager
                  .getCameraCharacteristics(cameraId)
                  .get(CameraCharacteristics.SENSOR_ORIENTATION);
          rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;
      
          // Return the corresponding FirebaseVisionImageMetadata rotation value.
          int result;
          switch (rotationCompensation) {
              case 0:
                  result = FirebaseVisionImageMetadata.ROTATION_0;
                  break;
              case 90:
                  result = FirebaseVisionImageMetadata.ROTATION_90;
                  break;
              case 180:
                  result = FirebaseVisionImageMetadata.ROTATION_180;
                  break;
              case 270:
                  result = FirebaseVisionImageMetadata.ROTATION_270;
                  break;
              default:
                  result = FirebaseVisionImageMetadata.ROTATION_0;
                  Log.e(TAG, "Bad rotation value: " + rotationCompensation);
          }
          return result;
      }

      Kotlin+KTX

      private val ORIENTATIONS = SparseIntArray()
      
      init {
          ORIENTATIONS.append(Surface.ROTATION_0, 90)
          ORIENTATIONS.append(Surface.ROTATION_90, 0)
          ORIENTATIONS.append(Surface.ROTATION_180, 270)
          ORIENTATIONS.append(Surface.ROTATION_270, 180)
      }
      /**
       * Get the angle by which an image must be rotated given the device's current
       * orientation.
       */
      @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
      @Throws(CameraAccessException::class)
      private fun getRotationCompensation(cameraId: String, activity: Activity, context: Context): Int {
          // Get the device's current rotation relative to its "native" orientation.
          // Then, from the ORIENTATIONS table, look up the angle the image must be
          // rotated to compensate for the device's rotation.
          val deviceRotation = activity.windowManager.defaultDisplay.rotation
          var rotationCompensation = ORIENTATIONS.get(deviceRotation)
      
          // On most devices, the sensor orientation is 90 degrees, but for some
          // devices it is 270 degrees. For devices with a sensor orientation of
          // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
          val cameraManager = context.getSystemService(CAMERA_SERVICE) as CameraManager
          val sensorOrientation = cameraManager
                  .getCameraCharacteristics(cameraId)
                  .get(CameraCharacteristics.SENSOR_ORIENTATION)!!
          rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360
      
          // Return the corresponding FirebaseVisionImageMetadata rotation value.
          val result: Int
          when (rotationCompensation) {
              0 -> result = FirebaseVisionImageMetadata.ROTATION_0
              90 -> result = FirebaseVisionImageMetadata.ROTATION_90
              180 -> result = FirebaseVisionImageMetadata.ROTATION_180
              270 -> result = FirebaseVisionImageMetadata.ROTATION_270
              else -> {
                  result = FirebaseVisionImageMetadata.ROTATION_0
                  Log.e(TAG, "Bad rotation value: $rotationCompensation")
              }
          }
          return result
      }

      然後,請傳遞 media.Image 物件和 將旋轉值轉換為 FirebaseVisionImage.fromMediaImage()

      Java

      FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation);

      Kotlin+KTX

      val image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation)
    • 如要從檔案 URI 建立 FirebaseVisionImage 物件,請傳遞 應用程式環境和檔案 URI FirebaseVisionImage.fromFilePath()。如果您要 使用 ACTION_GET_CONTENT 意圖提示使用者選取 取自圖片庫應用程式中的圖片。

      Java

      FirebaseVisionImage image;
      try {
          image = FirebaseVisionImage.fromFilePath(context, uri);
      } catch (IOException e) {
          e.printStackTrace();
      }

      Kotlin+KTX

      val image: FirebaseVisionImage
      try {
          image = FirebaseVisionImage.fromFilePath(context, uri)
      } catch (e: IOException) {
          e.printStackTrace()
      }
    • 要使用 FirebaseVisionImage 物件 ByteBuffer 或位元組陣列,請先計算圖片 旋轉 (方法如上所述) media.Image 輸入欄位。

      接著建立 FirebaseVisionImageMetadata 物件 包含圖片的高度、寬度、色彩編碼格式 和輪替金鑰

      Java

      FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
              .setWidth(480)   // 480x360 is typically sufficient for
              .setHeight(360)  // image recognition
              .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
              .setRotation(rotation)
              .build();

      Kotlin+KTX

      val metadata = FirebaseVisionImageMetadata.Builder()
              .setWidth(480) // 480x360 is typically sufficient for
              .setHeight(360) // image recognition
              .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
              .setRotation(rotation)
              .build()

      使用緩衝區或陣列和中繼資料物件 FirebaseVisionImage 物件:

      Java

      FirebaseVisionImage image = FirebaseVisionImage.fromByteBuffer(buffer, metadata);
      // Or: FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata);

      Kotlin+KTX

      val image = FirebaseVisionImage.fromByteBuffer(buffer, metadata)
      // Or: val image = FirebaseVisionImage.fromByteArray(byteArray, metadata)
    • 要使用 FirebaseVisionImage 物件 Bitmap 物件:

      Java

      FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);

      Kotlin+KTX

      val image = FirebaseVisionImage.fromBitmap(bitmap)
      Bitmap 物件代表的圖片必須 保持直立,不用另外旋轉。
  2. 取得 FirebaseVisionFaceDetector 的執行個體:

    Java

    FirebaseVisionFaceDetector detector = FirebaseVision.getInstance()
            .getVisionFaceDetector(options);

    Kotlin+KTX

    val detector = FirebaseVision.getInstance()
            .getVisionFaceDetector(options)
  3. 最後,將圖片傳遞至 detectInImage 方法:

    Java

    Task<List<FirebaseVisionFace>> result =
            detector.detectInImage(image)
                    .addOnSuccessListener(
                            new OnSuccessListener<List<FirebaseVisionFace>>() {
                                @Override
                                public void onSuccess(List<FirebaseVisionFace> faces) {
                                    // Task completed successfully
                                    // ...
                                }
                            })
                    .addOnFailureListener(
                            new OnFailureListener() {
                                @Override
                                public void onFailure(@NonNull Exception e) {
                                    // Task failed with an exception
                                    // ...
                                }
                            });

    Kotlin+KTX

    val result = detector.detectInImage(image)
            .addOnSuccessListener { faces ->
                // Task completed successfully
                // ...
            }
            .addOnFailureListener { e ->
                // Task failed with an exception
                // ...
            }

3. 取得系統偵測到的臉孔資訊

如果臉部辨識作業成功,系統會顯示 FirebaseVisionFace 物件將傳遞至成功 接聽程式。每個 FirebaseVisionFace 物件都代表偵測到的臉孔 在圖片中定義文字您可以在輸入中取得每個臉孔的定界座標 以及你設定臉部偵測器 發現。例如:

Java

for (FirebaseVisionFace face : faces) {
    Rect bounds = face.getBoundingBox();
    float rotY = face.getHeadEulerAngleY();  // Head is rotated to the right rotY degrees
    float rotZ = face.getHeadEulerAngleZ();  // Head is tilted sideways rotZ degrees

    // If landmark detection was enabled (mouth, ears, eyes, cheeks, and
    // nose available):
    FirebaseVisionFaceLandmark leftEar = face.getLandmark(FirebaseVisionFaceLandmark.LEFT_EAR);
    if (leftEar != null) {
        FirebaseVisionPoint leftEarPos = leftEar.getPosition();
    }

    // If contour detection was enabled:
    List<FirebaseVisionPoint> leftEyeContour =
            face.getContour(FirebaseVisionFaceContour.LEFT_EYE).getPoints();
    List<FirebaseVisionPoint> upperLipBottomContour =
            face.getContour(FirebaseVisionFaceContour.UPPER_LIP_BOTTOM).getPoints();

    // If classification was enabled:
    if (face.getSmilingProbability() != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) {
        float smileProb = face.getSmilingProbability();
    }
    if (face.getRightEyeOpenProbability() != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) {
        float rightEyeOpenProb = face.getRightEyeOpenProbability();
    }

    // If face tracking was enabled:
    if (face.getTrackingId() != FirebaseVisionFace.INVALID_ID) {
        int id = face.getTrackingId();
    }
}

Kotlin+KTX

for (face in faces) {
    val bounds = face.boundingBox
    val rotY = face.headEulerAngleY // Head is rotated to the right rotY degrees
    val rotZ = face.headEulerAngleZ // Head is tilted sideways rotZ degrees

    // If landmark detection was enabled (mouth, ears, eyes, cheeks, and
    // nose available):
    val leftEar = face.getLandmark(FirebaseVisionFaceLandmark.LEFT_EAR)
    leftEar?.let {
        val leftEarPos = leftEar.position
    }

    // If contour detection was enabled:
    val leftEyeContour = face.getContour(FirebaseVisionFaceContour.LEFT_EYE).points
    val upperLipBottomContour = face.getContour(FirebaseVisionFaceContour.UPPER_LIP_BOTTOM).points

    // If classification was enabled:
    if (face.smilingProbability != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) {
        val smileProb = face.smilingProbability
    }
    if (face.rightEyeOpenProbability != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) {
        val rightEyeOpenProb = face.rightEyeOpenProbability
    }

    // If face tracking was enabled:
    if (face.trackingId != FirebaseVisionFace.INVALID_ID) {
        val id = face.trackingId
    }
}

臉部輪廓範例

啟用臉部輪廓偵測功能後,畫面上會列出 偵測到的臉部特徵這些點代表 而不是每個特徵的分數查看臉孔 偵測概念總覽,進一步瞭解輪廓如何 。

下圖說明這些點如何對應到表面 (按一下 可放大的圖片):

即時臉部偵測

如要在即時應用程式中使用臉部偵測功能,請按照下列步驟操作: 實現最佳影格速率:

  • 設定臉部偵測工具, 臉部輪廓偵測或分類及地標偵測,但兩者只能擇一:

    輪廓偵測
    地標偵測
    分類
    地標偵測與分類
    模型偵測和地標偵測
    模型偵測與分類
    輪廓偵測、地標偵測與分類

  • 啟用 FAST 模式 (預設為啟用)。

  • 建議以較低的解析度拍攝圖片。請特別注意 這個 API 的圖片尺寸規定

  • 限制對偵測工具的呼叫。如果新的影片影格 因此請在偵測器執行時捨棄影格。
  • 使用偵測工具的輸出內容將圖像重疊 先從 ML Kit 取得結果,然後算繪圖片 並疊加單一步驟這麼一來,您的應用程式就會算繪到顯示途徑 每個輸入影格只能建立一次
  • 如果你使用 Camera2 API, ImageFormat.YUV_420_888 格式。

    如果使用舊版 Camera API,請以 ImageFormat.NV21 格式。