您可以使用ML Kit來識別圖像中的文字。 ML Kit 既有適合識別影像中文字(例如路牌文字)的通用 API,也有針對識別文件文字而最佳化的 API。通用 API 具有設備端模型和基於雲端的模型。文件文字辨識僅作為基於雲端的模型可用。請參閱概述以了解雲端模型和設備端模型的比較。
在你開始之前
- 如果您尚未將 Firebase 新增至您的應用程式中,請按照入門指南中的步驟進行操作。
- 在 Podfile 中包含 ML Kit 函式庫:
pod 'Firebase/MLVision', '6.25.0' # If using an on-device API: pod 'Firebase/MLVisionTextModel', '6.25.0'
安裝或更新專案的 Pod 後,請務必使用其.xcworkspace
開啟 Xcode 專案。 - 在您的應用程式中,導入 Firebase:
迅速
import Firebase
Objective-C
@import Firebase;
如果您想使用基於雲端的模型,並且尚未為您的專案啟用基於雲端的 API,請立即執行以下操作:
- 開啟 Firebase 控制台的ML Kit API 頁面。
如果您尚未將項目升級到 Blaze 定價計劃,請按一下升級來執行此操作。 (只有當您的專案不在 Blaze 計劃中時,系統才會提示您升級。)
只有 Blaze 等級的項目才能使用基於雲端的 API。
- 如果尚未啟用基於雲端的 API,請按一下啟用基於雲端的 API 。
如果您只想使用裝置上的模型,可以跳過此步驟。
現在您已準備好開始識別圖像中的文字。
輸入影像指南
為了讓 ML Kit 準確地識別文本,輸入圖像必須包含由足夠的像素資料表示的文本。理想情況下,對於拉丁文本,每個字元應至少為 16x16 像素。對於中文、日文和韓文文字(僅基於雲端的 API 支援),每個字元應為 24x24 像素。對於所有語言,大於 24x24 像素的字元通常不會帶來準確性優勢。
例如,640x480 的影像可能適合掃描佔據影像整個寬度的名片。要掃描列印在 letter 尺寸紙張上的文檔,可能需要 720x1280 像素的圖像。
影像焦點不佳會損害文字辨識的準確性。如果您沒有獲得可接受的結果,請嘗試要求使用者重新捕捉影像。
如果您在即時應用程式中識別文本,您可能還需要考慮輸入圖像的整體尺寸。較小的圖像可以更快地處理,因此為了減少延遲,請以較低的分辨率捕獲圖像(記住上述精度要求)並確保文字盡可能多地佔據圖像。另請參閱提高即時效能的提示。
辨識圖像中的文字
若要使用裝置上或基於雲端的模型識別圖像中的文本,請按如下所述運行文本識別器。
1. 運行文字辨識器
將圖像作為“UIImage”或“CMSampleBufferRef”傳遞給“VisionTextRecognizer”的“process(_:completion:)”方法:- 透過呼叫
onDeviceTextRecognizer
或cloudTextRecognizer
來取得VisionTextRecognizer
的實例:迅速
要使用設備上的模型:
let vision = Vision.vision() let textRecognizer = vision.onDeviceTextRecognizer()
使用雲端模型:
let vision = Vision.vision() let textRecognizer = vision.cloudTextRecognizer() // Or, to provide language hints to assist with language detection: // See https://cloud.google.com/vision/docs/languages for supported languages let options = VisionCloudTextRecognizerOptions() options.languageHints = ["en", "hi"] let textRecognizer = vision.cloudTextRecognizer(options: options)
Objective-C
要使用設備上的模型:
FIRVision *vision = [FIRVision vision]; FIRVisionTextRecognizer *textRecognizer = [vision onDeviceTextRecognizer];
使用雲端模型:
FIRVision *vision = [FIRVision vision]; FIRVisionTextRecognizer *textRecognizer = [vision cloudTextRecognizer]; // Or, to provide language hints to assist with language detection: // See https://cloud.google.com/vision/docs/languages for supported languages FIRVisionCloudTextRecognizerOptions *options = [[FIRVisionCloudTextRecognizerOptions alloc] init]; options.languageHints = @[@"en", @"hi"]; FIRVisionTextRecognizer *textRecognizer = [vision cloudTextRecognizerWithOptions:options];
使用
UIImage
或CMSampleBufferRef
建立VisionImage
物件。使用
UIImage
:- 如有必要,旋轉影像,使其
imageOrientation
屬性為.up
。 - 使用正確旋轉的
UIImage
建立VisionImage
物件。不要指定任何旋轉元資料 - 必須使用預設值.topLeft
。迅速
let image = VisionImage(image: uiImage)
Objective-C
FIRVisionImage *image = [[FIRVisionImage alloc] initWithImage:uiImage];
使用
CMSampleBufferRef
:建立一個
VisionImageMetadata
對象,該對象指定CMSampleBufferRef
緩衝區中包含的圖像資料的方向。若要取得影像方向:
迅速
func imageOrientation( deviceOrientation: UIDeviceOrientation, cameraPosition: AVCaptureDevice.Position ) -> VisionDetectorImageOrientation { switch deviceOrientation { case .portrait: return cameraPosition == .front ? .leftTop : .rightTop case .landscapeLeft: return cameraPosition == .front ? .bottomLeft : .topLeft case .portraitUpsideDown: return cameraPosition == .front ? .rightBottom : .leftBottom case .landscapeRight: return cameraPosition == .front ? .topRight : .bottomRight case .faceDown, .faceUp, .unknown: return .leftTop } }
Objective-C
- (FIRVisionDetectorImageOrientation) imageOrientationFromDeviceOrientation:(UIDeviceOrientation)deviceOrientation cameraPosition:(AVCaptureDevicePosition)cameraPosition { switch (deviceOrientation) { case UIDeviceOrientationPortrait: if (cameraPosition == AVCaptureDevicePositionFront) { return FIRVisionDetectorImageOrientationLeftTop; } else { return FIRVisionDetectorImageOrientationRightTop; } case UIDeviceOrientationLandscapeLeft: if (cameraPosition == AVCaptureDevicePositionFront) { return FIRVisionDetectorImageOrientationBottomLeft; } else { return FIRVisionDetectorImageOrientationTopLeft; } case UIDeviceOrientationPortraitUpsideDown: if (cameraPosition == AVCaptureDevicePositionFront) { return FIRVisionDetectorImageOrientationRightBottom; } else { return FIRVisionDetectorImageOrientationLeftBottom; } case UIDeviceOrientationLandscapeRight: if (cameraPosition == AVCaptureDevicePositionFront) { return FIRVisionDetectorImageOrientationTopRight; } else { return FIRVisionDetectorImageOrientationBottomRight; } default: return FIRVisionDetectorImageOrientationTopLeft; } }
然後,建立元資料物件:
迅速
let cameraPosition = AVCaptureDevice.Position.back // Set to the capture device you used. let metadata = VisionImageMetadata() metadata.orientation = imageOrientation( deviceOrientation: UIDevice.current.orientation, cameraPosition: cameraPosition )
Objective-C
FIRVisionImageMetadata *metadata = [[FIRVisionImageMetadata alloc] init]; AVCaptureDevicePosition cameraPosition = AVCaptureDevicePositionBack; // Set to the capture device you used. metadata.orientation = [self imageOrientationFromDeviceOrientation:UIDevice.currentDevice.orientation cameraPosition:cameraPosition];
- 使用
CMSampleBufferRef
物件和旋轉元資料建立VisionImage
物件:迅速
let image = VisionImage(buffer: sampleBuffer) image.metadata = metadata
Objective-C
FIRVisionImage *image = [[FIRVisionImage alloc] initWithBuffer:sampleBuffer]; image.metadata = metadata;
- 如有必要,旋轉影像,使其
- 然後,將影像傳遞給
process(_:completion:)
方法:迅速
textRecognizer.process(visionImage) { result, error in guard error == nil, let result = result else { // ... return } // Recognized text }
Objective-C
[textRecognizer processImage:image completion:^(FIRVisionText *_Nullable result, NSError *_Nullable error) { if (error != nil || result == nil) { // ... return; } // Recognized text }];
2. 從識別的文本區塊中提取文本
如果文字辨識操作成功,它將傳回一個 [`VisionText`][VisionText] 物件。 「VisionText」物件包含影像中辨識的全文以及零個或多個 [`VisionTextBlock`][VisionTextBlock] 物件。每個「VisionTextBlock」代表一個矩形文字區塊,其中包含零個或多個[“VisionTextLine”][VisionTextLine]物件。每個 `VisionTextLine` 對象包含零個或多個 [`VisionTextElement`][VisionTextElement] 對象,這些對象表示單字和類似單字的實體(日期、數字等)。對於每個「VisionTextBlock」、「VisionTextLine」和「VisionTextElement」對象,您可以獲得該區域中識別的文字以及該區域的邊界座標。例如:迅速
let resultText = result.text for block in result.blocks { let blockText = block.text let blockConfidence = block.confidence let blockLanguages = block.recognizedLanguages let blockCornerPoints = block.cornerPoints let blockFrame = block.frame for line in block.lines { let lineText = line.text let lineConfidence = line.confidence let lineLanguages = line.recognizedLanguages let lineCornerPoints = line.cornerPoints let lineFrame = line.frame for element in line.elements { let elementText = element.text let elementConfidence = element.confidence let elementLanguages = element.recognizedLanguages let elementCornerPoints = element.cornerPoints let elementFrame = element.frame } } }
Objective-C
NSString *resultText = result.text; for (FIRVisionTextBlock *block in result.blocks) { NSString *blockText = block.text; NSNumber *blockConfidence = block.confidence; NSArray<FIRVisionTextRecognizedLanguage *> *blockLanguages = block.recognizedLanguages; NSArray<NSValue *> *blockCornerPoints = block.cornerPoints; CGRect blockFrame = block.frame; for (FIRVisionTextLine *line in block.lines) { NSString *lineText = line.text; NSNumber *lineConfidence = line.confidence; NSArray<FIRVisionTextRecognizedLanguage *> *lineLanguages = line.recognizedLanguages; NSArray<NSValue *> *lineCornerPoints = line.cornerPoints; CGRect lineFrame = line.frame; for (FIRVisionTextElement *element in line.elements) { NSString *elementText = element.text; NSNumber *elementConfidence = element.confidence; NSArray<FIRVisionTextRecognizedLanguage *> *elementLanguages = element.recognizedLanguages; NSArray<NSValue *> *elementCornerPoints = element.cornerPoints; CGRect elementFrame = element.frame; } } }
提升即時效能的技巧
如果您想使用裝置上的模型來識別即時應用程式中的文本,請遵循以下指南以獲得最佳幀速率:
- 限制對文字辨識器的呼叫。如果文字辨識器在運行時有新的視訊幀可用,則丟棄該幀。
- 如果您使用文字辨識器的輸出在輸入影像上疊加圖形,請先從 ML Kit 取得結果,然後一步渲染影像並疊加。透過這樣做,每個輸入幀只需渲染到顯示表面一次。有關範例,請參閱展示範例應用程式中的PreviewOverlayView和FIRDetectionOverlayView類別。
- 考慮以較低解析度捕捉影像。但是,也要記住此 API 的圖像尺寸要求。
下一步
- 在將使用雲端 API 的應用程式部署到生產環境之前,您應該採取一些額外的步驟來防止和減輕未經授權的 API 存取的影響。
識別文件圖像中的文字
若要識別文檔的文本,請配置並執行基於雲端的文檔文本識別器,如下所述。
下面描述的文件文字辨識 API 提供了一個旨在更方便處理文件影像的介面。但是,如果您喜歡稀疏文字 API 提供的接口,則可以透過將雲端文字辨識器配置為使用密集文字模型來使用稀疏文字 API 來掃描文件。
使用文件文字識別API:
1. 運行文字辨識器
將影像作為UIImage
或CMSampleBufferRef
傳遞給VisionDocumentTextRecognizer
的process(_:completion:)
方法:- 透過呼叫
cloudDocumentTextRecognizer
來取得VisionDocumentTextRecognizer
的實例:迅速
let vision = Vision.vision() let textRecognizer = vision.cloudDocumentTextRecognizer() // Or, to provide language hints to assist with language detection: // See https://cloud.google.com/vision/docs/languages for supported languages let options = VisionCloudDocumentTextRecognizerOptions() options.languageHints = ["en", "hi"] let textRecognizer = vision.cloudDocumentTextRecognizer(options: options)
Objective-C
FIRVision *vision = [FIRVision vision]; FIRVisionDocumentTextRecognizer *textRecognizer = [vision cloudDocumentTextRecognizer]; // Or, to provide language hints to assist with language detection: // See https://cloud.google.com/vision/docs/languages for supported languages FIRVisionCloudDocumentTextRecognizerOptions *options = [[FIRVisionCloudDocumentTextRecognizerOptions alloc] init]; options.languageHints = @[@"en", @"hi"]; FIRVisionDocumentTextRecognizer *textRecognizer = [vision cloudDocumentTextRecognizerWithOptions:options];
使用
UIImage
或CMSampleBufferRef
建立VisionImage
物件。使用
UIImage
:- 如有必要,旋轉影像,使其
imageOrientation
屬性為.up
。 - 使用正確旋轉的
UIImage
建立VisionImage
物件。不要指定任何旋轉元資料 - 必須使用預設值.topLeft
。迅速
let image = VisionImage(image: uiImage)
Objective-C
FIRVisionImage *image = [[FIRVisionImage alloc] initWithImage:uiImage];
使用
CMSampleBufferRef
:建立一個
VisionImageMetadata
對象,該對象指定CMSampleBufferRef
緩衝區中包含的圖像資料的方向。若要取得影像方向:
迅速
func imageOrientation( deviceOrientation: UIDeviceOrientation, cameraPosition: AVCaptureDevice.Position ) -> VisionDetectorImageOrientation { switch deviceOrientation { case .portrait: return cameraPosition == .front ? .leftTop : .rightTop case .landscapeLeft: return cameraPosition == .front ? .bottomLeft : .topLeft case .portraitUpsideDown: return cameraPosition == .front ? .rightBottom : .leftBottom case .landscapeRight: return cameraPosition == .front ? .topRight : .bottomRight case .faceDown, .faceUp, .unknown: return .leftTop } }
Objective-C
- (FIRVisionDetectorImageOrientation) imageOrientationFromDeviceOrientation:(UIDeviceOrientation)deviceOrientation cameraPosition:(AVCaptureDevicePosition)cameraPosition { switch (deviceOrientation) { case UIDeviceOrientationPortrait: if (cameraPosition == AVCaptureDevicePositionFront) { return FIRVisionDetectorImageOrientationLeftTop; } else { return FIRVisionDetectorImageOrientationRightTop; } case UIDeviceOrientationLandscapeLeft: if (cameraPosition == AVCaptureDevicePositionFront) { return FIRVisionDetectorImageOrientationBottomLeft; } else { return FIRVisionDetectorImageOrientationTopLeft; } case UIDeviceOrientationPortraitUpsideDown: if (cameraPosition == AVCaptureDevicePositionFront) { return FIRVisionDetectorImageOrientationRightBottom; } else { return FIRVisionDetectorImageOrientationLeftBottom; } case UIDeviceOrientationLandscapeRight: if (cameraPosition == AVCaptureDevicePositionFront) { return FIRVisionDetectorImageOrientationTopRight; } else { return FIRVisionDetectorImageOrientationBottomRight; } default: return FIRVisionDetectorImageOrientationTopLeft; } }
然後,建立元資料物件:
迅速
let cameraPosition = AVCaptureDevice.Position.back // Set to the capture device you used. let metadata = VisionImageMetadata() metadata.orientation = imageOrientation( deviceOrientation: UIDevice.current.orientation, cameraPosition: cameraPosition )
Objective-C
FIRVisionImageMetadata *metadata = [[FIRVisionImageMetadata alloc] init]; AVCaptureDevicePosition cameraPosition = AVCaptureDevicePositionBack; // Set to the capture device you used. metadata.orientation = [self imageOrientationFromDeviceOrientation:UIDevice.currentDevice.orientation cameraPosition:cameraPosition];
- 使用
CMSampleBufferRef
物件和旋轉元資料建立VisionImage
物件:迅速
let image = VisionImage(buffer: sampleBuffer) image.metadata = metadata
Objective-C
FIRVisionImage *image = [[FIRVisionImage alloc] initWithBuffer:sampleBuffer]; image.metadata = metadata;
- 如有必要,旋轉影像,使其
- 然後,將影像傳遞給
process(_:completion:)
方法:迅速
textRecognizer.process(visionImage) { result, error in guard error == nil, let result = result else { // ... return } // Recognized text }
Objective-C
[textRecognizer processImage:image completion:^(FIRVisionDocumentText *_Nullable result, NSError *_Nullable error) { if (error != nil || result == nil) { // ... return; } // Recognized text }];
2. 從識別的文本區塊中提取文本
如果文字辨識操作成功,將會傳回一個VisionDocumentText
物件。 VisionDocumentText
物件包含影像中識別的全文以及反映已識別文件結構的物件層次結構:對於每個VisionDocumentTextBlock
、 VisionDocumentTextParagraph
、 VisionDocumentTextWord
和VisionDocumentTextSymbol
對象,您可以獲得該區域中識別的文本以及該區域的邊界座標。
例如:
迅速
let resultText = result.text for block in result.blocks { let blockText = block.text let blockConfidence = block.confidence let blockRecognizedLanguages = block.recognizedLanguages let blockBreak = block.recognizedBreak let blockCornerPoints = block.cornerPoints let blockFrame = block.frame for paragraph in block.paragraphs { let paragraphText = paragraph.text let paragraphConfidence = paragraph.confidence let paragraphRecognizedLanguages = paragraph.recognizedLanguages let paragraphBreak = paragraph.recognizedBreak let paragraphCornerPoints = paragraph.cornerPoints let paragraphFrame = paragraph.frame for word in paragraph.words { let wordText = word.text let wordConfidence = word.confidence let wordRecognizedLanguages = word.recognizedLanguages let wordBreak = word.recognizedBreak let wordCornerPoints = word.cornerPoints let wordFrame = word.frame for symbol in word.symbols { let symbolText = symbol.text let symbolConfidence = symbol.confidence let symbolRecognizedLanguages = symbol.recognizedLanguages let symbolBreak = symbol.recognizedBreak let symbolCornerPoints = symbol.cornerPoints let symbolFrame = symbol.frame } } } }
Objective-C
NSString *resultText = result.text; for (FIRVisionDocumentTextBlock *block in result.blocks) { NSString *blockText = block.text; NSNumber *blockConfidence = block.confidence; NSArray<FIRVisionTextRecognizedLanguage *> *blockRecognizedLanguages = block.recognizedLanguages; FIRVisionTextRecognizedBreak *blockBreak = block.recognizedBreak; CGRect blockFrame = block.frame; for (FIRVisionDocumentTextParagraph *paragraph in block.paragraphs) { NSString *paragraphText = paragraph.text; NSNumber *paragraphConfidence = paragraph.confidence; NSArray<FIRVisionTextRecognizedLanguage *> *paragraphRecognizedLanguages = paragraph.recognizedLanguages; FIRVisionTextRecognizedBreak *paragraphBreak = paragraph.recognizedBreak; CGRect paragraphFrame = paragraph.frame; for (FIRVisionDocumentTextWord *word in paragraph.words) { NSString *wordText = word.text; NSNumber *wordConfidence = word.confidence; NSArray<FIRVisionTextRecognizedLanguage *> *wordRecognizedLanguages = word.recognizedLanguages; FIRVisionTextRecognizedBreak *wordBreak = word.recognizedBreak; CGRect wordFrame = word.frame; for (FIRVisionDocumentTextSymbol *symbol in word.symbols) { NSString *symbolText = symbol.text; NSNumber *symbolConfidence = symbol.confidence; NSArray<FIRVisionTextRecognizedLanguage *> *symbolRecognizedLanguages = symbol.recognizedLanguages; FIRVisionTextRecognizedBreak *symbolBreak = symbol.recognizedBreak; CGRect symbolFrame = symbol.frame; } } } }
下一步
- 在將使用雲端 API 的應用程式部署到生產環境之前,您應該採取一些額外的步驟來防止和減輕未經授權的 API 存取的影響。