您可以使用 Firebase ML 識別圖片中的知名地標。
事前準備
- 如果您尚未將 Firebase 新增至 Android 專案,請新增 Firebase。
-
在模組 (應用程式層級) Gradle 檔案 (通常是
<project>/<app-module>/build.gradle.kts
或<project>/<app-module>/build.gradle
) 中,加入 Android 版 Firebase ML Vision 程式庫的依附元件。建議您使用 Firebase Android BoM 來控管程式庫版本。dependencies { // Import the BoM for the Firebase platform implementation(platform("com.google.firebase:firebase-bom:33.7.0")) // Add the dependency for the Firebase ML Vision library // When using the BoM, you don't specify versions in Firebase library dependencies implementation 'com.google.firebase:firebase-ml-vision' }
只要使用 Firebase Android BoM,應用程式就會一律使用相容的 Firebase Android 程式庫版本。
(替代做法) 不使用 BoM 新增 Firebase 程式庫依附元件
如果您選擇不使用 Firebase BoM,則必須在依附元件行中指定每個 Firebase 程式庫版本。
請注意,如果您在應用程式中使用多個 Firebase 程式庫,強烈建議您使用 BoM 來管理程式庫版本,確保所有版本皆相容。
dependencies { // Add the dependency for the Firebase ML Vision library // When NOT using the BoM, you must specify versions in Firebase library dependencies implementation 'com.google.firebase:firebase-ml-vision:24.1.0' }
-
如果您尚未為專案啟用雲端 API,請立即啟用:
- 開啟 Firebase 控制台的 Firebase ML API 頁面。
-
如果您尚未將專案升級至 Blaze 定價方案,請按一下「Upgrade」進行升級 (只有在專案未採用 Blaze 方案時,系統才會提示您升級)。
只有 Blaze 級別專案可以使用雲端 API。
- 如果您尚未啟用雲端 API,請按一下「啟用雲端 API」。
設定地標偵測器
根據預設,Cloud 偵測器會使用 STABLE
版本的模型,並傳回最多 10 個結果。如果您要變更這兩項設定,請使用 FirebaseVisionCloudDetectorOptions
物件指定設定。
舉例來說,如要同時變更兩個預設設定,請建構 FirebaseVisionCloudDetectorOptions
物件,如下例所示:
Kotlin
val options = FirebaseVisionCloudDetectorOptions.Builder() .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL) .setMaxResults(15) .build()
Java
FirebaseVisionCloudDetectorOptions options = new FirebaseVisionCloudDetectorOptions.Builder() .setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL) .setMaxResults(15) .build();
如要使用預設設定,您可以在下一個步驟中使用 FirebaseVisionCloudDetectorOptions.DEFAULT
。
執行地標偵測器
如要辨識圖片中的地標,請從Bitmap
、media.Image
、ByteBuffer
、位元組陣列或裝置上的檔案建立 FirebaseVisionImage
物件。接著,將 FirebaseVisionImage
物件傳遞至 FirebaseVisionCloudLandmarkDetector
的 detectInImage
方法。
從圖片建立
FirebaseVisionImage
物件。-
如要從
media.Image
物件建立FirebaseVisionImage
物件 (例如從裝置相機擷取圖片時),請將media.Image
物件和圖片的旋轉角度傳遞至FirebaseVisionImage.fromMediaImage()
。如果您使用 CameraX 程式庫,
OnImageCapturedListener
和ImageAnalysis.Analyzer
類別會為您計算旋轉值,因此您只需在呼叫FirebaseVisionImage.fromMediaImage()
之前,將旋轉值轉換為 Firebase ML 的ROTATION_
常數:Kotlin
private class YourImageAnalyzer : ImageAnalysis.Analyzer { private fun degreesToFirebaseRotation(degrees: Int): Int = when(degrees) { 0 -> FirebaseVisionImageMetadata.ROTATION_0 90 -> FirebaseVisionImageMetadata.ROTATION_90 180 -> FirebaseVisionImageMetadata.ROTATION_180 270 -> FirebaseVisionImageMetadata.ROTATION_270 else -> throw Exception("Rotation must be 0, 90, 180, or 270.") } override fun analyze(imageProxy: ImageProxy?, degrees: Int) { val mediaImage = imageProxy?.image val imageRotation = degreesToFirebaseRotation(degrees) if (mediaImage != null) { val image = FirebaseVisionImage.fromMediaImage(mediaImage, imageRotation) // Pass image to an ML Vision API // ... } } }
Java
private class YourAnalyzer implements ImageAnalysis.Analyzer { private int degreesToFirebaseRotation(int degrees) { switch (degrees) { case 0: return FirebaseVisionImageMetadata.ROTATION_0; case 90: return FirebaseVisionImageMetadata.ROTATION_90; case 180: return FirebaseVisionImageMetadata.ROTATION_180; case 270: return FirebaseVisionImageMetadata.ROTATION_270; default: throw new IllegalArgumentException( "Rotation must be 0, 90, 180, or 270."); } } @Override public void analyze(ImageProxy imageProxy, int degrees) { if (imageProxy == null || imageProxy.getImage() == null) { return; } Image mediaImage = imageProxy.getImage(); int rotation = degreesToFirebaseRotation(degrees); FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation); // Pass image to an ML Vision API // ... } }
如果您未使用可提供圖片旋轉角度的相機程式庫,可以根據裝置旋轉角度和裝置中相機感應器的方向計算:
Kotlin
private val ORIENTATIONS = SparseIntArray() init { ORIENTATIONS.append(Surface.ROTATION_0, 90) ORIENTATIONS.append(Surface.ROTATION_90, 0) ORIENTATIONS.append(Surface.ROTATION_180, 270) ORIENTATIONS.append(Surface.ROTATION_270, 180) } /** * Get the angle by which an image must be rotated given the device's current * orientation. */ @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) @Throws(CameraAccessException::class) private fun getRotationCompensation(cameraId: String, activity: Activity, context: Context): Int { // Get the device's current rotation relative to its "native" orientation. // Then, from the ORIENTATIONS table, look up the angle the image must be // rotated to compensate for the device's rotation. val deviceRotation = activity.windowManager.defaultDisplay.rotation var rotationCompensation = ORIENTATIONS.get(deviceRotation) // On most devices, the sensor orientation is 90 degrees, but for some // devices it is 270 degrees. For devices with a sensor orientation of // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees. val cameraManager = context.getSystemService(CAMERA_SERVICE) as CameraManager val sensorOrientation = cameraManager .getCameraCharacteristics(cameraId) .get(CameraCharacteristics.SENSOR_ORIENTATION)!! rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360 // Return the corresponding FirebaseVisionImageMetadata rotation value. val result: Int when (rotationCompensation) { 0 -> result = FirebaseVisionImageMetadata.ROTATION_0 90 -> result = FirebaseVisionImageMetadata.ROTATION_90 180 -> result = FirebaseVisionImageMetadata.ROTATION_180 270 -> result = FirebaseVisionImageMetadata.ROTATION_270 else -> { result = FirebaseVisionImageMetadata.ROTATION_0 Log.e(TAG, "Bad rotation value: $rotationCompensation") } } return result }
Java
private static final SparseIntArray ORIENTATIONS = new SparseIntArray(); static { ORIENTATIONS.append(Surface.ROTATION_0, 90); ORIENTATIONS.append(Surface.ROTATION_90, 0); ORIENTATIONS.append(Surface.ROTATION_180, 270); ORIENTATIONS.append(Surface.ROTATION_270, 180); } /** * Get the angle by which an image must be rotated given the device's current * orientation. */ @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) private int getRotationCompensation(String cameraId, Activity activity, Context context) throws CameraAccessException { // Get the device's current rotation relative to its "native" orientation. // Then, from the ORIENTATIONS table, look up the angle the image must be // rotated to compensate for the device's rotation. int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation(); int rotationCompensation = ORIENTATIONS.get(deviceRotation); // On most devices, the sensor orientation is 90 degrees, but for some // devices it is 270 degrees. For devices with a sensor orientation of // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees. CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE); int sensorOrientation = cameraManager .getCameraCharacteristics(cameraId) .get(CameraCharacteristics.SENSOR_ORIENTATION); rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360; // Return the corresponding FirebaseVisionImageMetadata rotation value. int result; switch (rotationCompensation) { case 0: result = FirebaseVisionImageMetadata.ROTATION_0; break; case 90: result = FirebaseVisionImageMetadata.ROTATION_90; break; case 180: result = FirebaseVisionImageMetadata.ROTATION_180; break; case 270: result = FirebaseVisionImageMetadata.ROTATION_270; break; default: result = FirebaseVisionImageMetadata.ROTATION_0; Log.e(TAG, "Bad rotation value: " + rotationCompensation); } return result; }
接著,將
media.Image
物件和旋轉值傳遞至FirebaseVisionImage.fromMediaImage()
:Kotlin
val image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation)
Java
FirebaseVisionImage image = FirebaseVisionImage.fromMediaImage(mediaImage, rotation);
- 如要從檔案 URI 建立
FirebaseVisionImage
物件,請將應用程式背景資訊和檔案 URI 傳遞至FirebaseVisionImage.fromFilePath()
。這在您使用ACTION_GET_CONTENT
意圖,提示使用者從相片庫應用程式中選取圖片時,非常實用。Kotlin
val image: FirebaseVisionImage try { image = FirebaseVisionImage.fromFilePath(context, uri) } catch (e: IOException) { e.printStackTrace() }
Java
FirebaseVisionImage image; try { image = FirebaseVisionImage.fromFilePath(context, uri); } catch (IOException e) { e.printStackTrace(); }
- 如要從
ByteBuffer
或位元組陣列建立FirebaseVisionImage
物件,請先計算圖片旋轉角度,如上文所述的media.Image
輸入資料。接著,請建立
FirebaseVisionImageMetadata
物件,其中包含圖片的高度、寬度、顏色編碼格式和旋轉角度:Kotlin
val metadata = FirebaseVisionImageMetadata.Builder() .setWidth(480) // 480x360 is typically sufficient for .setHeight(360) // image recognition .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21) .setRotation(rotation) .build()
Java
FirebaseVisionImageMetadata metadata = new FirebaseVisionImageMetadata.Builder() .setWidth(480) // 480x360 is typically sufficient for .setHeight(360) // image recognition .setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21) .setRotation(rotation) .build();
使用緩衝區或陣列和中繼資料物件,建立
FirebaseVisionImage
物件:Kotlin
val image = FirebaseVisionImage.fromByteBuffer(buffer, metadata) // Or: val image = FirebaseVisionImage.fromByteArray(byteArray, metadata)
Java
FirebaseVisionImage image = FirebaseVisionImage.fromByteBuffer(buffer, metadata); // Or: FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata);
- 如要從
Bitmap
物件建立FirebaseVisionImage
物件,請按照下列步驟操作:Kotlin
val image = FirebaseVisionImage.fromBitmap(bitmap)
Java
FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(bitmap);
Bitmap
物件所代表的圖片必須是直立的,不需要額外旋轉。
-
取得
FirebaseVisionCloudLandmarkDetector
的例項:Kotlin
val detector = FirebaseVision.getInstance() .visionCloudLandmarkDetector // Or, to change the default settings: // val detector = FirebaseVision.getInstance() // .getVisionCloudLandmarkDetector(options)
Java
FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance() .getVisionCloudLandmarkDetector(); // Or, to change the default settings: // FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance() // .getVisionCloudLandmarkDetector(options);
最後,將圖片傳遞至
detectInImage
方法:Kotlin
val result = detector.detectInImage(image) .addOnSuccessListener { firebaseVisionCloudLandmarks -> // Task completed successfully // ... } .addOnFailureListener { e -> // Task failed with an exception // ... }
Java
Task<List<FirebaseVisionCloudLandmark>> result = detector.detectInImage(image) .addOnSuccessListener(new OnSuccessListener<List<FirebaseVisionCloudLandmark>>() { @Override public void onSuccess(List<FirebaseVisionCloudLandmark> firebaseVisionCloudLandmarks) { // Task completed successfully // ... } }) .addOnFailureListener(new OnFailureListener() { @Override public void onFailure(@NonNull Exception e) { // Task failed with an exception // ... } });
取得已辨識地標的相關資訊
如果地標辨識作業成功,系統會將FirebaseVisionCloudLandmark
物件清單傳遞至成功事件監聽器。每個 FirebaseVisionCloudLandmark
物件都代表圖片中辨識到的地標。您可以為每個地標取得輸入圖片中的邊界座標、地標名稱、經緯度、知識圖形實體 ID (如有),以及比對的信心分數。例如:
Kotlin
for (landmark in firebaseVisionCloudLandmarks) { val bounds = landmark.boundingBox val landmarkName = landmark.landmark val entityId = landmark.entityId val confidence = landmark.confidence // Multiple locations are possible, e.g., the location of the depicted // landmark and the location the picture was taken. for (loc in landmark.locations) { val latitude = loc.latitude val longitude = loc.longitude } }
Java
for (FirebaseVisionCloudLandmark landmark: firebaseVisionCloudLandmarks) { Rect bounds = landmark.getBoundingBox(); String landmarkName = landmark.getLandmark(); String entityId = landmark.getEntityId(); float confidence = landmark.getConfidence(); // Multiple locations are possible, e.g., the location of the depicted // landmark and the location the picture was taken. for (FirebaseVisionLatLng loc: landmark.getLocations()) { double latitude = loc.getLatitude(); double longitude = loc.getLongitude(); } }
後續步驟
- 在將使用 Cloud API 的應用程式部署至正式環境之前,您應採取一些額外步驟,防止及減輕未經授權的 API 存取行為。