在 Android 上使用 ML Kit 識別地標

This page is about an old version of the Landmark Recognition API, which was part of ML Kit for Firebase. For the latest docs, see the latest version in the Firebase ML section.

您可以使用 ML Kit 識別圖像中的知名地標。

使用 ML Kit 訪問 Cloud ML 功能需遵守Google Cloud Platform 許可協議和服務特定條款，並相應收費。有關結算信息，請參閱 Firebase定價頁面。

在你開始之前

1. 如果您還沒有，請將 Firebase 添加到您的 Android 項目中。
2. 將 ML Kit Android 庫的依賴項添加到您的模塊（應用級）Gradle 文件（通常是app/build.gradle ）：

   apply plugin: 'com.android.application'
   apply plugin: 'com.google.gms.google-services'
   
   dependencies {
     // ...
   
     implementation 'com.google.firebase:firebase-ml-vision:24.0.3'
   }
   

3. 如果您尚未為您的項目啟用基於雲的 API，請立即執行此操作：

   1. 打開 Firebase 控制台的ML Kit API 頁面。

   2. 如果您尚未將項目升級到 Blaze 定價計劃，請單擊升級以執行此操作。 （僅當您的項目不在 Blaze 計劃中時，系統才會提示您升級。）

      只有 Blaze 級項目可以使用基於雲的 API。

   3. 如果尚未啟用基於雲的 API，請單擊啟用基於雲的 API 。
   在將使用 Cloud API 的應用程序部署到生產環境之前，您應該採取一些額外的步驟來防止和減輕未經授權的 API 訪問的影響。

配置地標檢測器

默認情況下，雲檢測器使用模型的STABLE版本並返回最多 10 個結果。如果您想更改這些設置中的任何一個，請使用FirebaseVisionCloudDetectorOptions對象指定它們。

例如，要更改這兩個默認設置，請構建一個FirebaseVisionCloudDetectorOptions對象，如下例所示：

Java

FirebaseVisionCloudDetectorOptions options =
        new FirebaseVisionCloudDetectorOptions.Builder()
                .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL)
                .setMaxResults(15)
                .build();
MainActivity.java

Kotlin+KTX

val options = FirebaseVisionCloudDetectorOptions.Builder()
        .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL)
        .setMaxResults(15)
        .build()
MainActivity.kt

要使用默認設置，您可以在下一步中使用FirebaseVisionCloudDetectorOptions.DEFAULT 。

運行地標檢測器

要識別圖像中的地標，請從Bitmap 、 media.Image 、 ByteBuffer 、字節數組或設備上的文件創建一個FirebaseVisionImage對象。然後，將FirebaseVisionImage對像傳遞給FirebaseVisionCloudLandmarkDetector的detectInImage方法。

1. 從您的圖像創建一個FirebaseVisionImage對象。

   • 要從media.Image對象創建FirebaseVisionImage對象，例如從設備的攝像頭捕獲圖像時，請將media.Image對象和圖像的旋轉傳遞給FirebaseVisionImage.fromMediaImage() 。

     如果您使用CameraX庫， OnImageCapturedListener和ImageAnalysis.Analyzer類會為您計算旋轉值，因此您只需在調用FirebaseVisionImage.fromMediaImage()之前將旋轉轉換為 ML Kit 的ROTATION_常量之一：

     Java

     private class YourAnalyzer implements ImageAnalysis.Analyzer {
     
         private int degreesToFirebaseRotation(int degrees) {
             switch (degrees) {
                 case 0:
                     return FirebaseVisionImageMetadata.ROTATION_0;
                 case 90:
                     return FirebaseVisionImageMetadata.ROTATION_90;
                 case 180:
                     return FirebaseVisionImageMetadata.ROTATION_180;
                 case 270:
                     return FirebaseVisionImageMetadata.ROTATION_270;
                 default:
                     throw new IllegalArgumentException(
                             "Rotation must be 0, 90, 180, or 270.");
             }
         }
     
         @Override
         public void analyze(ImageProxy imageProxy, int degrees) {
             if (imageProxy == null || imageProxy.getImage() == null) {
                 return;
             }
             Image mediaImage = imageProxy.getImage();
             int rotation = degreesToFirebaseRotation(degrees);
             FirebaseVisionImage image =
                     FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
             // Pass image to an ML Kit Vision API
             // ...
         }
     }
     

     Kotlin+KTX

     private class YourImageAnalyzer : ImageAnalysis.Analyzer {
         private fun degreesToFirebaseRotation(degrees: Int): Int = when(degrees) {
             0 -> FirebaseVisionImageMetadata.ROTATION_0
             90 -> FirebaseVisionImageMetadata.ROTATION_90
             180 -> FirebaseVisionImageMetadata.ROTATION_180
             270 -> FirebaseVisionImageMetadata.ROTATION_270
             else -> throw Exception("Rotation must be 0, 90, 180, or 270.")
         }
     
         override fun analyze(imageProxy: ImageProxy?, degrees: Int) {
             val mediaImage = imageProxy?.image
             val imageRotation = degreesToFirebaseRotation(degrees)
             if (mediaImage != null) {
                 val image = FirebaseVisionImage.fromMediaImage(mediaImage, imageRotation)
                 // Pass image to an ML Kit Vision API
                 // ...
             }
         }
     }
     

     如果您不使用為您提供圖像旋轉的相機庫，您可以根據設備的旋轉和設備中相機傳感器的方向來計算它：

     Java

     private static final SparseIntArray ORIENTATIONS = new SparseIntArray();
     static {
         ORIENTATIONS.append(Surface.ROTATION_0, 90);
         ORIENTATIONS.append(Surface.ROTATION_90, 0);
         ORIENTATIONS.append(Surface.ROTATION_180, 270);
         ORIENTATIONS.append(Surface.ROTATION_270, 180);
     }
     
     /**
      * Get the angle by which an image must be rotated given the device's current
      * orientation.
      */
     @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
     private int getRotationCompensation(String cameraId, Activity activity, Context context)
             throws CameraAccessException {
         // Get the device's current rotation relative to its "native" orientation.
         // Then, from the ORIENTATIONS table, look up the angle the image must be
         // rotated to compensate for the device's rotation.
         int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation();
         int rotationCompensation = ORIENTATIONS.get(deviceRotation);
     
         // On most devices, the sensor orientation is 90 degrees, but for some
         // devices it is 270 degrees. For devices with a sensor orientation of
         // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
         CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
         int sensorOrientation = cameraManager
                 .getCameraCharacteristics(cameraId)
                 .get(CameraCharacteristics.SENSOR_ORIENTATION);
         rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;
     
         // Return the corresponding FirebaseVisionImageMetadata rotation value.
         int result;
         switch (rotationCompensation) {
             case 0:
                 result = FirebaseVisionImageMetadata.ROTATION_0;
                 break;
             case 90:
                 result = FirebaseVisionImageMetadata.ROTATION_90;
                 break;
             case 180:
                 result = FirebaseVisionImageMetadata.ROTATION_180;
                 break;
             case 270:
                 result = FirebaseVisionImageMetadata.ROTATION_270;
                 break;
             default:
                 result = FirebaseVisionImageMetadata.ROTATION_0;
                 Log.e(TAG, "Bad rotation value: " + rotationCompensation);
         }
         return result;
     }
     VisionImage.java

     Kotlin+KTX

     private val ORIENTATIONS = SparseIntArray()
     
     init {
         ORIENTATIONS.append(Surface.ROTATION_0, 90)
         ORIENTATIONS.append(Surface.ROTATION_90, 0)
         ORIENTATIONS.append(Surface.ROTATION_180, 270)
         ORIENTATIONS.append(Surface.ROTATION_270, 180)
     }
     /**
      * Get the angle by which an image must be rotated given the device's current
      * orientation.
      */
     @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
     @Throws(CameraAccessException::class)
     private fun getRotationCompensation(cameraId: String, activity: Activity, context: Context): Int {
         // Get the device's current rotation relative to its "native" orientation.
         // Then, from the ORIENTATIONS table, look up the angle the image must be
         // rotated to compensate for the device's rotation.
         val deviceRotation = activity.windowManager.defaultDisplay.rotation
         var rotationCompensation = ORIENTATIONS.get(deviceRotation)
     
         // On most devices, the sensor orientation is 90 degrees, but for some
         // devices it is 270 degrees. For devices with a sensor orientation of
         // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
         val cameraManager = context.getSystemService(CAMERA_SERVICE) as CameraManager
         val sensorOrientation = cameraManager
                 .getCameraCharacteristics(cameraId)
                 .get(CameraCharacteristics.SENSOR_ORIENTATION)!!
         rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360
     
         // Return the corresponding FirebaseVisionImageMetadata rotation value.
         val result: Int
         when (rotationCompensation) {
             0 -> result = FirebaseVisionImageMetadata.ROTATION_0
             90 -> result = FirebaseVisionImageMetadata.ROTATION_90
             180 -> result = FirebaseVisionImageMetadata.ROTATION_180
             270 -> result = FirebaseVisionImageMetadata.ROTATION_270
             else -> {
                 result = FirebaseVisionImageMetadata.ROTATION_0
                 Log.e(TAG, "Bad rotation value: $rotationCompensation")
             }
         }
         return result
     }
     VisionImage.kt

     然後，將media.Image對象和旋轉值傳遞給FirebaseVisionImage.fromMediaImage() ：

     Java

     FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
     VisionImage.java

     Kotlin+KTX

     val image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation)
     VisionImage.kt

   • 要從文件 URI 創建FirebaseVisionImage對象，請將應用上下文和文件 URI 傳遞給FirebaseVisionImage.fromFilePath() 。當您使用ACTION_GET_CONTENT意圖提示用戶從他們的圖庫應用中選擇圖像時，這很有用。

     Java

     FirebaseVisionImage image;
     try {
         image = FirebaseVisionImage.fromFilePath(context, uri);
     } catch (IOException e) {
         e.printStackTrace();
     }
     VisionImage.java

     Kotlin+KTX

     val image: FirebaseVisionImage
     try {
         image = FirebaseVisionImage.fromFilePath(context, uri)
     } catch (e: IOException) {
         e.printStackTrace()
     }
     VisionImage.kt

   • 要從ByteBuffer或字節數組創建FirebaseVisionImage對象，請首先按照上面針對media.Image輸入的描述計算圖像旋轉。

     然後，創建一個FirebaseVisionImageMetadata對象，其中包含圖像的高度、寬度、顏色編碼格式和旋轉：

     Java

     FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder()
             .setWidth(480)   // 480x360 is typically sufficient for
             .setHeight(360)  // image recognition
             .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
             .setRotation(rotation)
             .build();
     VisionImage.java

     Kotlin+KTX

     val metadata = FirebaseVisionImageMetadata.Builder()
             .setWidth(480) // 480x360 is typically sufficient for
             .setHeight(360) // image recognition
             .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21)
             .setRotation(rotation)
             .build()
     VisionImage.kt

     使用緩衝區或數組以及元數據對象來創建FirebaseVisionImage對象：

     Java

     FirebaseVisionImage image = FirebaseVisionImage.fromByteBuffer(buffer, metadata);
     // Or: FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata);
     VisionImage.java

     Kotlin+KTX

     val image = FirebaseVisionImage.fromByteBuffer(buffer, metadata)
     // Or: val image = FirebaseVisionImage.fromByteArray(byteArray, metadata)
     VisionImage.kt

   • 要從Bitmap對象創建FirebaseVisionImage對象：

     Java

     FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);
     VisionImage.java

     Kotlin+KTX

     val image = FirebaseVisionImage.fromBitmap(bitmap)
     VisionImage.kt

     Bitmap對象表示的圖像必須是直立的，不需要額外的旋轉。

2. 獲取FirebaseVisionCloudLandmarkDetector的實例：

   Java

   FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()
           .getVisionCloudLandmarkDetector();
   // Or, to change the default settings:
   // FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()
   //         .getVisionCloudLandmarkDetector(options);

   Kotlin+KTX

   val detector = FirebaseVision.getInstance()
           .visionCloudLandmarkDetector
   // Or, to change the default settings:
   // val detector = FirebaseVision.getInstance()
   //         .getVisionCloudLandmarkDetector(options)

3. 最後，將圖像傳遞給detectInImage方法：

   Java

   Task<List<FirebaseVisionCloudLandmark>> result = detector.detectInImage(image)
           .addOnSuccessListener(new OnSuccessListener<List<FirebaseVisionCloudLandmark>>() {
               @Override
               public void onSuccess(List<FirebaseVisionCloudLandmark> firebaseVisionCloudLandmarks) {
                   // Task completed successfully
                   // ...
               }
           })
           .addOnFailureListener(new OnFailureListener() {
               @Override
               public void onFailure(@NonNull Exception e) {
                   // Task failed with an exception
                   // ...
               }
           });

   Kotlin+KTX

   val result = detector.detectInImage(image)
           .addOnSuccessListener { firebaseVisionCloudLandmarks ->
               // Task completed successfully
               // ...
           }
           .addOnFailureListener { e ->
               // Task failed with an exception
               // ...
           }

獲取有關已識別地標的信息

如果地標識別操作成功， FirebaseVisionCloudLandmark對象列表將傳遞給成功偵聽器。每個FirebaseVisionCloudLandmark對像都代表圖像中識別的地標。對於每個地標，您可以獲得其在輸入圖像中的邊界坐標、地標的名稱、其緯度和經度、其知識圖實體 ID （如果可用）以及匹配的置信度分數。例如：

Java

for (FirebaseVisionCloudLandmark landmark: firebaseVisionCloudLandmarks) {

    Rect bounds = landmark.getBoundingBox();
    String landmarkName = landmark.getLandmark();
    String entityId = landmark.getEntityId();
    float confidence = landmark.getConfidence();

    // Multiple locations are possible, e.g., the location of the depicted
    // landmark and the location the picture was taken.
    for (FirebaseVisionLatLng loc: landmark.getLocations()) {
        double latitude = loc.getLatitude();
        double longitude = loc.getLongitude();
    }
}

Kotlin+KTX

for (landmark in firebaseVisionCloudLandmarks) {

    val bounds = landmark.boundingBox
    val landmarkName = landmark.landmark
    val entityId = landmark.entityId
    val confidence = landmark.confidence

    // Multiple locations are possible, e.g., the location of the depicted
    // landmark and the location the picture was taken.
    for (loc in landmark.locations) {
        val latitude = loc.latitude
        val longitude = loc.longitude
    }
}

下一步

• 在將使用 Cloud API 的應用程序部署到生產環境之前，您應該採取一些額外的步驟來防止和減輕未經授權的 API 訪問的影響。