在 Android 上使用 Firebase ML 辨識地標

您可以使用 Firebase ML 來辨識圖片中的知名地標。

事前準備

  1. 如果您尚未將 Firebase 新增至 Android 專案，請先完成這項作業。
  2. 在模組 (應用程式層級) Gradle 檔案 (通常是 <project>/<app-module>/build.gradle.kts 或 <project>/<app-module>/build.gradle) 中，新增 Android 版 Firebase ML Vision 程式庫的依附元件。建議您使用 Firebase Android BoM 來控管程式庫版本管理。 
    dependencies {
    // Import the BoM for the Firebase platform
    implementation(platform("com.google.firebase:firebase-bom:33.1.1"))

    // Add the dependency for the Firebase ML Vision library
    // When using the BoM, you don't specify versions in Firebase library dependencies
    implementation 'com.google.firebase:firebase-ml-vision'
}

    如果使用 Firebase Android BoM，應用程式就會一律使用相容的 Firebase Android 程式庫版本。

    (替代方法) 新增 Firebase 程式庫依附元件，「不必」使用 BoM

    如果選擇不使用 Firebase BoM，請務必在依附元件行中指定各個 Firebase 程式庫版本。

    請注意，如果在應用程式中使用多個 Firebase 程式庫，強烈建議您使用 BoM 管理程式庫版本，確保所有版本都相容。

    dependencies {
    // Add the dependency for the Firebase ML Vision library
    // When NOT using the BoM, you must specify versions in Firebase library dependencies
    implementation 'com.google.firebase:firebase-ml-vision:24.1.0'
}
    在尋找 Kotlin 專用的程式庫模組嗎？自 2023 年 10 月 (Firebase BoM 32.5.0) 起，Kotlin 和 Java 開發人員都可以依賴主程式庫模組 (詳情請參閱這項計畫的常見問題)。

  3. 如果您尚未為專案啟用以 Cloud 為基礎的 API，請立即啟用：

    1. 開啟 Firebase 控制台的「Firebase ML API」頁面。

    2. 如果您尚未將專案升級至 Blaze 定價方案，請按一下「升級」來執行這項操作。只有在專案並非採用 Blaze 方案時，系統才會提示您升級。)

      只有 Blaze 等級的專案可以使用以 Cloud 為基礎的 API。

    3. 如果尚未啟用雲端式 API，請按一下「啟用雲端式 API」。

設定地標偵測工具

根據預設，Cloud 偵測工具會使用 STABLE 版本的模型並傳回最多 10 筆結果。如要變更這兩項設定，請使用 FirebaseVisionCloudDetectorOptions 物件指定。

舉例來說，如要變更這兩項預設設定，請建構 FirebaseVisionCloudDetectorOptions 物件，如以下範例所示：

Kotlin+KTX

val options = FirebaseVisionCloudDetectorOptions.Builder()
    .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL)
    .setMaxResults(15)
    .build()
MainActivity.kt

Java

FirebaseVisionCloudDetectorOptions options =
        new FirebaseVisionCloudDetectorOptions.Builder()
                .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL)
                .setMaxResults(15)
                .build();
MainActivity.java

如要使用預設設定，您可以在下一個步驟中使用 FirebaseVisionCloudDetectorOptions.DEFAULT。

執行地標偵測工具

如要辨識圖片中的地標，請從 Bitmap、media.Image、ByteBuffer、位元組陣列或裝置上的檔案建立 FirebaseVisionImage 物件。接著，將 FirebaseVisionImage 物件傳遞至 FirebaseVisionCloudLandmarkDetector 的 detectInImage 方法。

  1. 從圖片建立 FirebaseVisionImage 物件。

    • 如要從 media.Image 物件建立 FirebaseVisionImage 物件 (例如從裝置的相機擷取圖片)，請將 media.Image 物件和圖片的旋轉角度傳遞至 FirebaseVisionImage.fromMediaImage()。

      如果您使用 CameraX 程式庫，OnImageCapturedListener 和 ImageAnalysis.Analyzer 類別會為您計算旋轉值，因此您只需在呼叫 FirebaseVisionImage.fromMediaImage() 之前，將旋轉角度轉換為 Firebase ML 的其中一個 ROTATION_ 常數：

      Kotlin+KTX

      private class YourImageAnalyzer : ImageAnalysis.Analyzer {
    private fun degreesToFirebaseRotation(degrees: Int): Int = when(degrees) {
        0 -> FirebaseVisionImageMetadata.ROTATION_0
        90 -> FirebaseVisionImageMetadata.ROTATION_90
        180 -> FirebaseVisionImageMetadata.ROTATION_180
        270 -> FirebaseVisionImageMetadata.ROTATION_270
        else -> throw Exception("Rotation must be 0, 90, 180, or 270.")
    }

    override fun analyze(imageProxy: ImageProxy?, degrees: Int) {
        val mediaImage = imageProxy?.image
        val imageRotation = degreesToFirebaseRotation(degrees)
        if (mediaImage != null) {
            val image = FirebaseVisionImage.fromMediaImage(mediaImage, imageRotation)
            // Pass image to an ML Vision API
            // ...
        }
    }
}

      Java

      private class YourAnalyzer implements ImageAnalysis.Analyzer {

    private int degreesToFirebaseRotation(int degrees) {
        switch (degrees) {
            case 0:
                return FirebaseVisionImageMetadata.ROTATION_0;
            case 90:
                return FirebaseVisionImageMetadata.ROTATION_90;
            case 180:
                return FirebaseVisionImageMetadata.ROTATION_180;
            case 270:
                return FirebaseVisionImageMetadata.ROTATION_270;
            default:
                throw new IllegalArgumentException(
                        "Rotation must be 0, 90, 180, or 270.");
        }
    }

    @Override
    public void analyze(ImageProxy imageProxy, int degrees) {
        if (imageProxy == null || imageProxy.getImage() == null) {
            return;
        }
        Image mediaImage = imageProxy.getImage();
        int rotation = degreesToFirebaseRotation(degrees);
        FirebaseVisionImage image =
                FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
        // Pass image to an ML Vision API
        // ...
    }
}

      如果您不使用相機程式庫提供圖片旋轉角度，可以透過裝置旋轉角度和裝置的相機感應器方向計算圖片：

      Kotlin+KTX

      private val ORIENTATIONS = SparseIntArray()

init {
    ORIENTATIONS.append(Surface.ROTATION_0, 90)
    ORIENTATIONS.append(Surface.ROTATION_90, 0)
    ORIENTATIONS.append(Surface.ROTATION_180, 270)
    ORIENTATIONS.append(Surface.ROTATION_270, 180)
}
/**
 * Get the angle by which an image must be rotated given the device's current
 * orientation.
 */
@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
@Throws(CameraAccessException::class)
private fun getRotationCompensation(cameraId: String, activity: Activity, context: Context): Int {
    // Get the device's current rotation relative to its "native" orientation.
    // Then, from the ORIENTATIONS table, look up the angle the image must be
    // rotated to compensate for the device's rotation.
    val deviceRotation = activity.windowManager.defaultDisplay.rotation
    var rotationCompensation = ORIENTATIONS.get(deviceRotation)

    // On most devices, the sensor orientation is 90 degrees, but for some
    // devices it is 270 degrees. For devices with a sensor orientation of
    // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
    val cameraManager = context.getSystemService(CAMERA_SERVICE) as CameraManager
    val sensorOrientation = cameraManager
        .getCameraCharacteristics(cameraId)
        .get(CameraCharacteristics.SENSOR_ORIENTATION)!!
    rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360

    // Return the corresponding FirebaseVisionImageMetadata rotation value.
    val result: Int
    when (rotationCompensation) {
        0 -> result = FirebaseVisionImageMetadata.ROTATION_0
        90 -> result = FirebaseVisionImageMetadata.ROTATION_90
        180 -> result = FirebaseVisionImageMetadata.ROTATION_180
        270 -> result = FirebaseVisionImageMetadata.ROTATION_270
        else -> {
            result = FirebaseVisionImageMetadata.ROTATION_0
            Log.e(TAG, "Bad rotation value: $rotationCompensation")
        }
    }
    return result
}
      VisionImage.kt

      Java

      private static final SparseIntArray ORIENTATIONS = new SparseIntArray();
static {
    ORIENTATIONS.append(Surface.ROTATION_0, 90);
    ORIENTATIONS.append(Surface.ROTATION_90, 0);
    ORIENTATIONS.append(Surface.ROTATION_180, 270);
    ORIENTATIONS.append(Surface.ROTATION_270, 180);
}

/**
 * Get the angle by which an image must be rotated given the device's current
 * orientation.
 */
@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
private int getRotationCompensation(String cameraId, Activity activity, Context context)
        throws CameraAccessException {
    // Get the device's current rotation relative to its "native" orientation.
    // Then, from the ORIENTATIONS table, look up the angle the image must be
    // rotated to compensate for the device's rotation.
    int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation();
    int rotationCompensation = ORIENTATIONS.get(deviceRotation);

    // On most devices, the sensor orientation is 90 degrees, but for some
    // devices it is 270 degrees. For devices with a sensor orientation of
    // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
    CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
    int sensorOrientation = cameraManager
            .getCameraCharacteristics(cameraId)
            .get(CameraCharacteristics.SENSOR_ORIENTATION);
    rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;

    // Return the corresponding FirebaseVisionImageMetadata rotation value.
    int result;
    switch (rotationCompensation) {
        case 0:
            result = FirebaseVisionImageMetadata.ROTATION_0;
            break;
        case 90:
            result = FirebaseVisionImageMetadata.ROTATION_90;
            break;
        case 180:
            result = FirebaseVisionImageMetadata.ROTATION_180;
            break;
        case 270:
            result = FirebaseVisionImageMetadata.ROTATION_270;
            break;
        default:
            result = FirebaseVisionImageMetadata.ROTATION_0;
            Log.e(TAG, "Bad rotation value: " + rotationCompensation);
    }
    return result;
}
      VisionImage.java

      接著，將 media.Image 物件和旋轉值傳遞至 FirebaseVisionImage.fromMediaImage()：

      Kotlin+KTX

      val image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation)
      VisionImage.kt

      Java

      FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
      VisionImage.java
    • 如要從檔案 URI 建立 FirebaseVisionImage 物件，請將應用程式結構定義和檔案 URI 傳遞至 FirebaseVisionImage.fromFilePath()。如要使用 ACTION_GET_CONTENT 意圖提示使用者從圖片庫應用程式選取圖片，這項功能就非常實用。

      Kotlin+KTX

      val image: FirebaseVisionImage
try {
    image = FirebaseVisionImage.fromFilePath(context, uri)
} catch (e: IOException) {
    e.printStackTrace()
}
      VisionImage.kt

      Java

      FirebaseVisionImage image;
try {
    image = FirebaseVisionImage.fromFilePath(context, uri);
} catch (IOException e) {
    e.printStackTrace();
}
      VisionImage.java
    • 如要使用 ByteBuffer 或位元組陣列建立 FirebaseVisionImage 物件，請先按照上文說明計算 media.Image 輸入內容的圖片旋轉角度。

      接著，建立包含圖片高度、寬度、色彩編碼格式和旋轉圖片的 FirebaseVisionImageMetadata 物件：

      Kotlin+KTX

      val metadata = FirebaseVisionImageMetadata.Builder()
    .setWidth(480) // 480x360 is typically sufficient for
    .setHeight(360) // image recognition
    .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
    .setRotation(rotation)
    .build()
      VisionImage.kt

      Java

      FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
        .setWidth(480)   // 480x360 is typically sufficient for
        .setHeight(360)  // image recognition
        .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
        .setRotation(rotation)
        .build();
      VisionImage.java

      使用緩衝區或陣列以及中繼資料物件來建立 FirebaseVisionImage 物件：

      Kotlin+KTX

      val image = FirebaseVisionImage.fromByteBuffer(buffer, metadata)
// Or: val image = FirebaseVisionImage.fromByteArray(byteArray, metadata)
      VisionImage.kt

      Java

      FirebaseVisionImage image = FirebaseVisionImage.fromByteBuffer(buffer, metadata);
// Or: FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata);
      VisionImage.java
    • 如要透過 Bitmap 物件建立 FirebaseVisionImage 物件：

      Kotlin+KTX

      val image = FirebaseVisionImage.fromBitmap(bitmap)
      VisionImage.kt

      Java

      FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);
      VisionImage.java
      Bitmap 物件表示的圖片必須直立，無須額外旋轉。

  2. 取得 FirebaseVisionCloudLandmarkDetector 的執行個體：

    Kotlin+KTX 

    val detector = FirebaseVision.getInstance()
    .visionCloudLandmarkDetector
// Or, to change the default settings:
// val detector = FirebaseVision.getInstance()
//         .getVisionCloudLandmarkDetector(options)
    LandmarkRecognitionActivity.kt

    Java 

    FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()
        .getVisionCloudLandmarkDetector();
// Or, to change the default settings:
// FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()
//         .getVisionCloudLandmarkDetector(options);
    LandmarkRecognitionActivity.java

  3. 最後，將圖片傳遞至 detectInImage 方法：

    Kotlin+KTX 

    val result = detector.detectInImage(image)
    .addOnSuccessListener { firebaseVisionCloudLandmarks ->
        // Task completed successfully
        // ...
    }
    .addOnFailureListener { e ->
        // Task failed with an exception
        // ...
    }
    LandmarkRecognitionActivity.kt

    Java 

    Task<List<FirebaseVisionCloudLandmark>> result = detector.detectInImage(image)
        .addOnSuccessListener(new OnSuccessListener<List<FirebaseVisionCloudLandmark>>() {
            @Override
            public void onSuccess(List<FirebaseVisionCloudLandmark> firebaseVisionCloudLandmarks) {
                // Task completed successfully
                // ...
            }
        })
        .addOnFailureListener(new OnFailureListener() {
            @Override
            public void onFailure(@NonNull Exception e) {
                // Task failed with an exception
                // ...
            }
        });
    LandmarkRecognitionActivity.java

取得已辨識地標的相關資訊

如果地標辨識作業成功，系統會將 FirebaseVisionCloudLandmark 物件清單傳遞至成功事件監聽器。每個 FirebaseVisionCloudLandmark 物件都代表圖片中可識別的一個地標。您可以取得每個地標的邊界座標、輸入圖片中的地標名稱、經緯度、其知識圖譜實體 ID (如有)，以及相符項目的可信度分數。例如：

Kotlin+KTX 

for (landmark in firebaseVisionCloudLandmarks) {
    val bounds = landmark.boundingBox
    val landmarkName = landmark.landmark
    val entityId = landmark.entityId
    val confidence = landmark.confidence

    // Multiple locations are possible, e.g., the location of the depicted
    // landmark and the location the picture was taken.
    for (loc in landmark.locations) {
        val latitude = loc.latitude
        val longitude = loc.longitude
    }
}
LandmarkRecognitionActivity.kt

Java 

for (FirebaseVisionCloudLandmark landmark: firebaseVisionCloudLandmarks) {

    Rect bounds = landmark.getBoundingBox();
    String landmarkName = landmark.getLandmark();
    String entityId = landmark.getEntityId();
    float confidence = landmark.getConfidence();

    // Multiple locations are possible, e.g., the location of the depicted
    // landmark and the location the picture was taken.
    for (FirebaseVisionLatLng loc: landmark.getLocations()) {
        double latitude = loc.getLatitude();
        double longitude = loc.getLongitude();
    }
}
LandmarkRecognitionActivity.java

後續步驟