您现在的位置: IOS教程網 >> IOS系統 >  >> iOS 程式開發 >> iOS 開發綜合

Swift 人臉識別實作:用 AVFoundation 逐幀讀取相機並以 CIDetector 偵測

編輯:iOS 開發綜合

整體架構

要從相機畫面中即時抓出人脸,需要三個零件合作:AVCaptureSession 負責管理擷取流程,AVCaptureVideoDataOutput 透過 delegate 把每一幀資料交給你,CIDetector 則是真的做辨識的那個。三者串起來就是一個每秒數十次的「取幀 → 分析」迴圈。

找到相機並啟動 session

 captureSession.sessionPreset = AVCaptureSessionPresetLow

let devices = AVCaptureDevice.devices()
for device in devices {
    if device.hasMediaType(AVMediaTypeVideo) {
        if device.position == AVCaptureDevicePosition.Front {
            captureDevice = device as? AVCaptureDevice
            if captureDevice != nil {
                println("Capture Device found")
                beginSession()
            }
        }
    }
}

這裡刻意選的是前置鏡頭(Front),因為人脸自拍場景用前置才合理。sessionPreset 設為 Low 是效能考量:偵測人臉不需要高畫質,降低解析度能讓 CIDetector 的運算負擔大幅下降。

beginSession:把裝置接進串流

 func beginSession() {
    var err: NSError? = nil
    captureSession.addInput(AVCaptureDeviceInput(device: captureDevice, error: &err))

    let output = AVCaptureVideoDataOutput()
    let cameraQueue = dispatch_queue_create("cameraQueue", DISPATCH_QUEUE_SERIAL)
    output.setSampleBufferDelegate(self, queue: cameraQueue)
    output.videoSettings = [kCVPixelBufferPixelFormatTypeKey: kCVPixelFormatType_32BGRA]
    captureSession.addOutput(output)

    if err != nil { println("error: \(err?.localizedDescription)") }

    previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
    previewLayer?.videoGravity = "AVLayerVideoGravityResizeAspect"
    previewLayer?.frame = self.view.bounds
    self.view.layer.addSublayer(previewLayer)

    captureSession.startRunning()
}

這裡有個關鍵設計:delegate 回呼的 queue 指定為自己建立的 serial queue,而不是主執行緒。影像處理很耗 CPU,若放在主執行緒會讓界面完全卡住。代價是——任何 UI 操作都必須 dispatch 回主執行緒。

每幀的處理:captureOutput

 func captureOutput(captureOutput: AVCaptureOutput!,
                   didOutputSampleBuffer sampleBuffer: CMSampleBuffer!,
                   fromConnection connection: AVCaptureConnection!) {
    if self.isStart {
        let resultImage = sampleBufferToImage(sampleBuffer)
        let context = CIContext(options: [kCIContextUseSoftwareRenderer: true])
        let detector = CIDetector(ofType: CIDetectorTypeFace,
                                  context: context,
                                  options: [CIDetectorAccuracy: CIDetectorAccuracyHigh])
        let ciImage = CIImage(image: resultImage)
        let results: NSArray = detector.featuresInImage(ciImage,
            options: ["CIDetectorImageOrientation": 6])

        for r in results {
            let face = r as! CIFaceFeature
            let faceImage = UIImage(CGImage: context.createCGImage(ciImage, fromRect: face.bounds),
                                    scale: 1.0, orientation: .Right)
            dispatch_async(dispatch_get_main_queue()) {
                if self.isStart {
                    self.dismissViewControllerAnimated(true, completion: nil)
                    self.callBack!(face: faceImage!)
                }
                self.isStart = false
            }
        }
    }
}

CIDetector 除了 CIDetectorTypeFace,也支援偵測四角目標(QRCode)與矩形,人臉偵測另外還能搭配 CIDetectorEyeBlink、CIDetectorSmile 等屬性辨識眨眼與微笑。CIDetectorImageOrientation 那個 6 對應「右側朝上」,是因為前置鏡頭的原始畫面方向需要校正。

CMSampleBuffer 轉 UIImage

這是整段程式裡最瑣碎但也最必要的部分——把系統給的 CMSampleBuffer 剝成 CGImage 再包成 UIImage。

 private func sampleBufferToImage(sampleBuffer: CMSampleBuffer!) -> UIImage {
    let imageBuffer: CVImageBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)
    CVPixelBufferLockBaseAddress(imageBuffer, 0)

    let baseAddress = CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 0)
    let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer)
    let width  = CVPixelBufferGetWidth(imageBuffer)
    let height = CVPixelBufferGetHeight(imageBuffer)
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let bitsPerComponent = 8
    let bitmapInfo = CGBitmapInfo(
        (CGBitmapInfo.ByteOrder32Little.rawValue | CGImageAlphaInfo.PremultipliedFirst.rawValue) as UInt32)

    let newContext = CGBitmapContextCreate(baseAddress, width, height,
        bitsPerComponent, bytesPerRow, colorSpace, bitmapInfo) as CGContextRef
    let imageRef: CGImageRef = CGBitmapContextCreateImage(newContext)
    return UIImage(CGImage: imageRef, scale: 1.0, orientation: .Right)!
}

取景框動畫

畫面上那個上下掃描的線,用一個 0.01 秒的 timer 加一個邊界判斷就能做出來:

 func animationSate() {
    if upOrdown {
        if line.frame.origin.y >= pickUIImager.frame.origin.y + 200 {
            upOrdown = false
        } else { line.frame.origin.y += 2 }
    } else {
        if line.frame.origin.y <= pickUIImager.frame.origin.y {
            upOrdown = true
        } else { line.frame.origin.y -= 2 }
    }
}

另外用一個只觸發一次的 timer 做 2 秒延遲,讓使用者先對準再開始偵測,避免剛開啟就抓到半張臉。

三個必須修正的問題

一、每幀都建立 CIDetector 與 CIContext。這兩個物件配置成本很高,放在每秒數十次的回呼裡會造成明顯卡頓與記憶體壓力。正確做法是把它們拉出來當成成員屬性,只建立一次。

二、缺 Necessary 的隱私權限。iOS 10 之後存取相機必須在 Info.plist 中設定 NSCameraUsageDescription,缺少它 App 會直接閃退而非跳提示框。

三、Swift 版本過舊。範例中的 AVCaptureDevice.devices()、println、CGContextRef 強轉等寫法在 Swift 2 之後已全部變更。現代寫法應改用 AVCaptureDevice.DiscoverySession、print,並搭配 AVAuthorizationStatus 先請求權限。若要在新專案做人臉相關功能,更建議直接採用 Vision 框架的 VNDetectFaceRectanglesRequest,程式碼短得多、效能也好得多。

 let request = VNDetectFaceRectanglesRequest()
let handler = VNImageRequestHandler(ciImage: ciImage)
try? handler.perform([request])
  1. 上一篇文章:
  2. 下一篇文章:
蘋果刷機越獄教程| IOS教程問題解答| IOS技巧綜合| IOS7技巧| IOS8教程
Copyright © IOS教程網 All Rights Reserved