您可以使用ML Kit來偵測影像和影片中的人臉。
在你開始之前
- 如果您尚未將 Firebase 新增至您的 Android 專案中,請將其新增至您的 Android 專案中。
- 將 ML Kit Android 函式庫的依賴項新增至模組(應用程式層級)Gradle 檔案(通常
app/build.gradle
):apply plugin: 'com.android.application' apply plugin: 'com.google.gms.google-services' dependencies { // ... implementation 'com.google.firebase:firebase-ml-vision:24.0.3' // If you want to detect face contours (landmark detection and classification // don't require this additional model): implementation 'com.google.firebase:firebase-ml-vision-face-model:20.0.1' }
- 可選但建議:將您的應用程式配置為在從 Play 商店安裝應用程式後自動將 ML 模型下載到裝置。
為此,請將以下聲明新增至應用程式的
AndroidManifest.xml
檔案中:<application ...> ... <meta-data android:name="com.google.firebase.ml.vision.DEPENDENCIES" android:value="face" /> <!-- To use multiple models: android:value="face,model2,model3" --> </application>
如果您不啟用安裝時模型下載,系統將在您首次執行偵測器時下載模型。您在下載完成之前發出的請求不會產生任何結果。
輸入影像指南
為了讓 ML Kit 準確地偵測人臉,輸入影像必須包含由足夠的像素資料表示的人臉。一般來說,您想要在影像中偵測的每張臉應至少為 100x100 像素。如果要偵測人臉輪廓,ML Kit 需要更高解析度的輸入:每張人臉至少應為 200x200 像素。
如果您在即時應用程式中偵測人臉,您可能還需要考慮輸入影像的整體尺寸。較小的影像可以更快地處理,因此為了減少延遲,請以較低的解析度擷取影像(記住上述精度要求)並確保主體的臉部盡可能佔據影像。另請參閱提高即時效能的提示。
影像聚焦不良會影響準確性。如果您沒有獲得可接受的結果,請嘗試要求使用者重新捕捉影像。
臉部相對於相機的方向也會影響 ML Kit 偵測到的臉部特徵。請參閱人臉偵測概念。
1.配置人臉偵測器
在將臉部偵測套用至影像之前,如果您想要變更任何臉部偵測器的預設設置,請使用FirebaseVisionFaceDetectorOptions
物件指定這些設定。您可以更改以下設定:設定 | |
---|---|
性能模式 | FAST (預設)| ACCURATE 偵測人臉時優先考慮速度或準確性。 |
檢測地標 | NO_LANDMARKS (預設)| ALL_LANDMARKS 是否嘗試辨識臉部「地標」:眼睛、耳朵、鼻子、臉頰、嘴巴等。 |
檢測輪廓 | NO_CONTOURS (預設)| ALL_CONTOURS 是否偵測臉部特徵輪廓。僅檢測影像中最突出的臉部的輪廓。 |
將臉孔進行分類 | NO_CLASSIFICATIONS (預設)| ALL_CLASSIFICATIONS 是否將臉孔分類為「微笑」和「睜開眼睛」等類別。 |
最小臉部尺寸 | float (預設值: 0.1f )要偵測的臉部相對於影像的最小尺寸。 |
啟用人臉追蹤 | false (預設)| true 是否為人臉分配 ID,可用於跨影像追蹤人臉。 請注意,啟用輪廓偵測後,僅偵測到一張臉部,因此臉部追蹤不會產生有用的結果。因此,為了提高偵測速度,請勿同時啟用輪廓偵測和臉部追蹤。 |
例如:
Java
// High-accuracy landmark detection and face classification FirebaseVisionFaceDetectorOptions highAccuracyOpts = new FirebaseVisionFaceDetectorOptions.Builder() .setPerformanceMode(FirebaseVisionFaceDetectorOptions.ACCURATE) .setLandmarkMode(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS) .setClassificationMode(FirebaseVisionFaceDetectorOptions.ALL_CLASSIFICATIONS) .build(); // Real-time contour detection of multiple faces FirebaseVisionFaceDetectorOptions realTimeOpts = new FirebaseVisionFaceDetectorOptions.Builder() .setContourMode(FirebaseVisionFaceDetectorOptions.ALL_CONTOURS) .build();
Kotlin+KTX
// High-accuracy landmark detection and face classification val highAccuracyOpts = FirebaseVisionFaceDetectorOptions.Builder() .setPerformanceMode(FirebaseVisionFaceDetectorOptions.ACCURATE) .setLandmarkMode(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS) .setClassificationMode(FirebaseVisionFaceDetectorOptions.ALL_CLASSIFICATIONS) .build() // Real-time contour detection of multiple faces val realTimeOpts = FirebaseVisionFaceDetectorOptions.Builder() .setContourMode(FirebaseVisionFaceDetectorOptions.ALL_CONTOURS) .build()
2. 運行人臉偵測器
若要偵測影像中的人臉,請從Bitmap
、 media.Image
、 ByteBuffer
、位元組陣列或裝置上的檔案建立FirebaseVisionImage
物件。然後,將FirebaseVisionImage
物件傳遞給FirebaseVisionFaceDetector
的detectInImage
方法。對於人臉識別,您應該使用尺寸至少為480x360像素的影像。如果您要即時辨識人臉,以此最低解析度擷取幀可以幫助減少延遲。
從您的映像建立
FirebaseVisionImage
物件。若要從
media.Image
物件建立FirebaseVisionImage
物件(例如從裝置的相機擷取影像時),請將media.Image
物件和影像的旋轉傳遞給FirebaseVisionImage.fromMediaImage()
。如果您使用CameraX函式庫,則
OnImageCapturedListener
和ImageAnalysis.Analyzer
類別會為您計算旋轉值,因此您只需在呼叫FirebaseVisionImage.fromMediaImage()
之前將旋轉轉換為 ML Kit 的ROTATION_
常數之一:Java
private class YourAnalyzer implements ImageAnalysis.Analyzer { private int degreesToFirebaseRotation(int degrees) { switch (degrees) { case 0: return FirebaseVisionImageMetadata.ROTATION_0; case 90: return FirebaseVisionImageMetadata.ROTATION_90; case 180: return FirebaseVisionImageMetadata.ROTATION_180; case 270: return FirebaseVisionImageMetadata.ROTATION_270; default: throw new IllegalArgumentException( "Rotation must be 0, 90, 180, or 270."); } } @Override public void analyze(ImageProxy imageProxy, int degrees) { if (imageProxy == null || imageProxy.getImage() == null) { return; } Image mediaImage = imageProxy.getImage(); int rotation = degreesToFirebaseRotation(degrees); FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation); // Pass image to an ML Kit Vision API // ... } }
Kotlin+KTX
private class YourImageAnalyzer : ImageAnalysis.Analyzer { private fun degreesToFirebaseRotation(degrees: Int): Int = when(degrees) { 0 -> FirebaseVisionImageMetadata.ROTATION_0 90 -> FirebaseVisionImageMetadata.ROTATION_90 180 -> FirebaseVisionImageMetadata.ROTATION_180 270 -> FirebaseVisionImageMetadata.ROTATION_270 else -> throw Exception("Rotation must be 0, 90, 180, or 270.") } override fun analyze(imageProxy: ImageProxy?, degrees: Int) { val mediaImage = imageProxy?.image val imageRotation = degreesToFirebaseRotation(degrees) if (mediaImage != null) { val image = FirebaseVisionImage.fromMediaImage(mediaImage, imageRotation) // Pass image to an ML Kit Vision API // ... } } }
如果您不使用為您提供影像旋轉的相機庫,您可以根據裝置的旋轉和裝置中相機感測器的方向來計算它:
Java
private static final SparseIntArray ORIENTATIONS = new SparseIntArray(); static { ORIENTATIONS.append(Surface.ROTATION_0, 90); ORIENTATIONS.append(Surface.ROTATION_90, 0); ORIENTATIONS.append(Surface.ROTATION_180, 270); ORIENTATIONS.append(Surface.ROTATION_270, 180); } /** * Get the angle by which an image must be rotated given the device's current * orientation. */ @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) private int getRotationCompensation(String cameraId, Activity activity, Context context) throws CameraAccessException { // Get the device's current rotation relative to its "native" orientation. // Then, from the ORIENTATIONS table, look up the angle the image must be // rotated to compensate for the device's rotation. int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation(); int rotationCompensation = ORIENTATIONS.get(deviceRotation); // On most devices, the sensor orientation is 90 degrees, but for some // devices it is 270 degrees. For devices with a sensor orientation of // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees. CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE); int sensorOrientation = cameraManager .getCameraCharacteristics(cameraId) .get(CameraCharacteristics.SENSOR_ORIENTATION); rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360; // Return the corresponding FirebaseVisionImageMetadata rotation value. int result; switch (rotationCompensation) { case 0: result = FirebaseVisionImageMetadata.ROTATION_0; break; case 90: result = FirebaseVisionImageMetadata.ROTATION_90; break; case 180: result = FirebaseVisionImageMetadata.ROTATION_180; break; case 270: result = FirebaseVisionImageMetadata.ROTATION_270; break; default: result = FirebaseVisionImageMetadata.ROTATION_0; Log.e(TAG, "Bad rotation value: " + rotationCompensation); } return result; }
Kotlin+KTX
private val ORIENTATIONS = SparseIntArray() init { ORIENTATIONS.append(Surface.ROTATION_0, 90) ORIENTATIONS.append(Surface.ROTATION_90, 0) ORIENTATIONS.append(Surface.ROTATION_180, 270) ORIENTATIONS.append(Surface.ROTATION_270, 180) } /** * Get the angle by which an image must be rotated given the device's current * orientation. */ @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) @Throws(CameraAccessException::class) private fun getRotationCompensation(cameraId: String, activity: Activity, context: Context): Int { // Get the device's current rotation relative to its "native" orientation. // Then, from the ORIENTATIONS table, look up the angle the image must be // rotated to compensate for the device's rotation. val deviceRotation = activity.windowManager.defaultDisplay.rotation var rotationCompensation = ORIENTATIONS.get(deviceRotation) // On most devices, the sensor orientation is 90 degrees, but for some // devices it is 270 degrees. For devices with a sensor orientation of // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees. val cameraManager = context.getSystemService(CAMERA_SERVICE) as CameraManager val sensorOrientation = cameraManager .getCameraCharacteristics(cameraId) .get(CameraCharacteristics.SENSOR_ORIENTATION)!! rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360 // Return the corresponding FirebaseVisionImageMetadata rotation value. val result: Int when (rotationCompensation) { 0 -> result = FirebaseVisionImageMetadata.ROTATION_0 90 -> result = FirebaseVisionImageMetadata.ROTATION_90 180 -> result = FirebaseVisionImageMetadata.ROTATION_180 270 -> result = FirebaseVisionImageMetadata.ROTATION_270 else -> { result = FirebaseVisionImageMetadata.ROTATION_0 Log.e(TAG, "Bad rotation value: $rotationCompensation") } } return result }
然後,將
media.Image
物件和旋轉值傳遞給FirebaseVisionImage.fromMediaImage()
:Java
FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
Kotlin+KTX
val image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation)
- 若要從檔案 URI 建立
FirebaseVisionImage
對象,請將套用上下文和檔案 URI 傳遞給FirebaseVisionImage.fromFilePath()
。當您使用ACTION_GET_CONTENT
意圖提示使用者從其圖庫應用程式中選擇影像時,這非常有用。Java
FirebaseVisionImage image; try { image = FirebaseVisionImage.fromFilePath(context, uri); } catch (IOException e) { e.printStackTrace(); }
Kotlin+KTX
val image: FirebaseVisionImage try { image = FirebaseVisionImage.fromFilePath(context, uri) } catch (e: IOException) { e.printStackTrace() }
- 若要從
ByteBuffer
或位元組數組建立FirebaseVisionImage
對象,請先按照上面針對media.Image
輸入所述計算圖像旋轉。然後,建立一個
FirebaseVisionImageMetadata
對象,其中包含圖像的高度、寬度、顏色編碼格式和旋轉:Java
FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder() .setWidth(480) // 480x360 is typically sufficient for .setHeight(360) // image recognition .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21) .setRotation(rotation) .build();
Kotlin+KTX
val metadata = FirebaseVisionImageMetadata.Builder() .setWidth(480) // 480x360 is typically sufficient for .setHeight(360) // image recognition .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21) .setRotation(rotation) .build()
使用緩衝區或陣列以及元資料物件來建立
FirebaseVisionImage
物件:Java
FirebaseVisionImage image = FirebaseVisionImage.fromByteBuffer(buffer, metadata); // Or: FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata);
Kotlin+KTX
val image = FirebaseVisionImage.fromByteBuffer(buffer, metadata) // Or: val image = FirebaseVisionImage.fromByteArray(byteArray, metadata)
- 要從
Bitmap
物件建立FirebaseVisionImage
物件:Java
FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);
Kotlin+KTX
val image = FirebaseVisionImage.fromBitmap(bitmap)
Bitmap
物件表示的影像必須是直立的,不需要額外旋轉。
取得
FirebaseVisionFaceDetector
的實例:Java
FirebaseVisionFaceDetector detector = FirebaseVision.getInstance() .getVisionFaceDetector(options);
Kotlin+KTX
val detector = FirebaseVision.getInstance() .getVisionFaceDetector(options)
最後,將圖像傳遞給
detectInImage
方法:Java
Task<List<FirebaseVisionFace>> result = detector.detectInImage(image) .addOnSuccessListener( new OnSuccessListener<List<FirebaseVisionFace>>() { @Override public void onSuccess(List<FirebaseVisionFace> faces) { // Task completed successfully // ... } }) .addOnFailureListener( new OnFailureListener() { @Override public void onFailure(@NonNull Exception e) { // Task failed with an exception // ... } });
Kotlin+KTX
val result = detector.detectInImage(image) .addOnSuccessListener { faces -> // Task completed successfully // ... } .addOnFailureListener { e -> // Task failed with an exception // ... }
3. 取得偵測到的人臉資訊
如果人臉辨識操作成功,FirebaseVisionFace
物件的清單將傳遞給成功偵聽器。每個FirebaseVisionFace
物件代表在影像中偵測到的一張臉。對於每張臉,您都可以獲得其在輸入影像中的邊界座標,以及配置臉部偵測器要尋找的任何其他資訊。例如: Java
for (FirebaseVisionFace face : faces) { Rect bounds = face.getBoundingBox(); float rotY = face.getHeadEulerAngleY(); // Head is rotated to the right rotY degrees float rotZ = face.getHeadEulerAngleZ(); // Head is tilted sideways rotZ degrees // If landmark detection was enabled (mouth, ears, eyes, cheeks, and // nose available): FirebaseVisionFaceLandmark leftEar = face.getLandmark(FirebaseVisionFaceLandmark.LEFT_EAR); if (leftEar != null) { FirebaseVisionPoint leftEarPos = leftEar.getPosition(); } // If contour detection was enabled: List<FirebaseVisionPoint> leftEyeContour = face.getContour(FirebaseVisionFaceContour.LEFT_EYE).getPoints(); List<FirebaseVisionPoint> upperLipBottomContour = face.getContour(FirebaseVisionFaceContour.UPPER_LIP_BOTTOM).getPoints(); // If classification was enabled: if (face.getSmilingProbability() != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) { float smileProb = face.getSmilingProbability(); } if (face.getRightEyeOpenProbability() != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) { float rightEyeOpenProb = face.getRightEyeOpenProbability(); } // If face tracking was enabled: if (face.getTrackingId() != FirebaseVisionFace.INVALID_ID) { int id = face.getTrackingId(); } }
Kotlin+KTX
for (face in faces) { val bounds = face.boundingBox val rotY = face.headEulerAngleY // Head is rotated to the right rotY degrees val rotZ = face.headEulerAngleZ // Head is tilted sideways rotZ degrees // If landmark detection was enabled (mouth, ears, eyes, cheeks, and // nose available): val leftEar = face.getLandmark(FirebaseVisionFaceLandmark.LEFT_EAR) leftEar?.let { val leftEarPos = leftEar.position } // If contour detection was enabled: val leftEyeContour = face.getContour(FirebaseVisionFaceContour.LEFT_EYE).points val upperLipBottomContour = face.getContour(FirebaseVisionFaceContour.UPPER_LIP_BOTTOM).points // If classification was enabled: if (face.smilingProbability != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) { val smileProb = face.smilingProbability } if (face.rightEyeOpenProbability != FirebaseVisionFace.UNCOMPUTED_PROBABILITY) { val rightEyeOpenProb = face.rightEyeOpenProbability } // If face tracking was enabled: if (face.trackingId != FirebaseVisionFace.INVALID_ID) { val id = face.trackingId } }
臉部輪廓範例
啟用臉部輪廓偵測後,您將獲得偵測到的每個臉部特徵的點列表。這些點代表特徵的形狀。有關如何表示輪廓的詳細信息,請參閱人臉檢測概念概述。
下圖說明了這些點如何映射到臉部(點擊圖像放大):
即時人臉偵測
如果您想在即時應用程式中使用人臉偵測,請遵循以下指南以獲得最佳幀速率:
將人臉偵測器配置為使用人臉輪廓偵測或分類和地標偵測,但不能同時使用兩者:
輪廓檢測
地標檢測
分類
地標檢測與分類
輪廓檢測和地標檢測
輪廓檢測和分類
輪廓檢測、地標檢測與分類啟用
FAST
模式(預設為啟用)。考慮以較低解析度擷取影像。但是,也要記住此 API 的圖像尺寸要求。
- 對檢測器的節流呼叫。如果偵測器運作時有新的視訊幀可用,則丟棄該幀。
- 如果您使用偵測器的輸出將圖形疊加在輸入影像上,請先從 ML Kit 取得結果,然後一步渲染影像並疊加。透過這樣做,每個輸入幀只需渲染到顯示表面一次。
如果您使用 Camera2 API,請以
ImageFormat.YUV_420_888
格式擷取影像。如果您使用較舊的相機 API,請以
ImageFormat.NV21
格式擷取影像。