要從相機畫面中即時抓出人脸,需要三個零件合作:AVCaptureSession 負責管理擷取流程,AVCaptureVideoDataOutput 透過 delegate 把每一幀資料交給你,CIDetector 則是真的做辨識的那個。三者串起來就是一個每秒數十次的「取幀 → 分析」迴圈。
captureSession.sessionPreset = AVCaptureSessionPresetLow
let devices = AVCaptureDevice.devices()
for device in devices {
if device.hasMediaType(AVMediaTypeVideo) {
if device.position == AVCaptureDevicePosition.Front {
captureDevice = device as? AVCaptureDevice
if captureDevice != nil {
println("Capture Device found")
beginSession()
}
}
}
}
這裡刻意選的是前置鏡頭(Front),因為人脸自拍場景用前置才合理。sessionPreset 設為 Low 是效能考量:偵測人臉不需要高畫質,降低解析度能讓 CIDetector 的運算負擔大幅下降。
func beginSession() {
var err: NSError? = nil
captureSession.addInput(AVCaptureDeviceInput(device: captureDevice, error: &err))
let output = AVCaptureVideoDataOutput()
let cameraQueue = dispatch_queue_create("cameraQueue", DISPATCH_QUEUE_SERIAL)
output.setSampleBufferDelegate(self, queue: cameraQueue)
output.videoSettings = [kCVPixelBufferPixelFormatTypeKey: kCVPixelFormatType_32BGRA]
captureSession.addOutput(output)
if err != nil { println("error: \(err?.localizedDescription)") }
previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
previewLayer?.videoGravity = "AVLayerVideoGravityResizeAspect"
previewLayer?.frame = self.view.bounds
self.view.layer.addSublayer(previewLayer)
captureSession.startRunning()
}
這裡有個關鍵設計:delegate 回呼的 queue 指定為自己建立的 serial queue,而不是主執行緒。影像處理很耗 CPU,若放在主執行緒會讓界面完全卡住。代價是——任何 UI 操作都必須 dispatch 回主執行緒。
func captureOutput(captureOutput: AVCaptureOutput!,
didOutputSampleBuffer sampleBuffer: CMSampleBuffer!,
fromConnection connection: AVCaptureConnection!) {
if self.isStart {
let resultImage = sampleBufferToImage(sampleBuffer)
let context = CIContext(options: [kCIContextUseSoftwareRenderer: true])
let detector = CIDetector(ofType: CIDetectorTypeFace,
context: context,
options: [CIDetectorAccuracy: CIDetectorAccuracyHigh])
let ciImage = CIImage(image: resultImage)
let results: NSArray = detector.featuresInImage(ciImage,
options: ["CIDetectorImageOrientation": 6])
for r in results {
let face = r as! CIFaceFeature
let faceImage = UIImage(CGImage: context.createCGImage(ciImage, fromRect: face.bounds),
scale: 1.0, orientation: .Right)
dispatch_async(dispatch_get_main_queue()) {
if self.isStart {
self.dismissViewControllerAnimated(true, completion: nil)
self.callBack!(face: faceImage!)
}
self.isStart = false
}
}
}
}
CIDetector 除了 CIDetectorTypeFace,也支援偵測四角目標(QRCode)與矩形,人臉偵測另外還能搭配 CIDetectorEyeBlink、CIDetectorSmile 等屬性辨識眨眼與微笑。CIDetectorImageOrientation 那個 6 對應「右側朝上」,是因為前置鏡頭的原始畫面方向需要校正。
這是整段程式裡最瑣碎但也最必要的部分——把系統給的 CMSampleBuffer 剝成 CGImage 再包成 UIImage。
private func sampleBufferToImage(sampleBuffer: CMSampleBuffer!) -> UIImage {
let imageBuffer: CVImageBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)
CVPixelBufferLockBaseAddress(imageBuffer, 0)
let baseAddress = CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 0)
let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer)
let width = CVPixelBufferGetWidth(imageBuffer)
let height = CVPixelBufferGetHeight(imageBuffer)
let colorSpace = CGColorSpaceCreateDeviceRGB()
let bitsPerComponent = 8
let bitmapInfo = CGBitmapInfo(
(CGBitmapInfo.ByteOrder32Little.rawValue | CGImageAlphaInfo.PremultipliedFirst.rawValue) as UInt32)
let newContext = CGBitmapContextCreate(baseAddress, width, height,
bitsPerComponent, bytesPerRow, colorSpace, bitmapInfo) as CGContextRef
let imageRef: CGImageRef = CGBitmapContextCreateImage(newContext)
return UIImage(CGImage: imageRef, scale: 1.0, orientation: .Right)!
}
畫面上那個上下掃描的線,用一個 0.01 秒的 timer 加一個邊界判斷就能做出來:
func animationSate() {
if upOrdown {
if line.frame.origin.y >= pickUIImager.frame.origin.y + 200 {
upOrdown = false
} else { line.frame.origin.y += 2 }
} else {
if line.frame.origin.y <= pickUIImager.frame.origin.y {
upOrdown = true
} else { line.frame.origin.y -= 2 }
}
}
另外用一個只觸發一次的 timer 做 2 秒延遲,讓使用者先對準再開始偵測,避免剛開啟就抓到半張臉。
一、每幀都建立 CIDetector 與 CIContext。這兩個物件配置成本很高,放在每秒數十次的回呼裡會造成明顯卡頓與記憶體壓力。正確做法是把它們拉出來當成成員屬性,只建立一次。
二、缺 Necessary 的隱私權限。iOS 10 之後存取相機必須在 Info.plist 中設定 NSCameraUsageDescription,缺少它 App 會直接閃退而非跳提示框。
三、Swift 版本過舊。範例中的 AVCaptureDevice.devices()、println、CGContextRef 強轉等寫法在 Swift 2 之後已全部變更。現代寫法應改用 AVCaptureDevice.DiscoverySession、print,並搭配 AVAuthorizationStatus 先請求權限。若要在新專案做人臉相關功能,更建議直接採用 Vision 框架的 VNDetectFaceRectanglesRequest,程式碼短得多、效能也好得多。
let request = VNDetectFaceRectanglesRequest() let handler = VNImageRequestHandler(ciImage: ciImage) try? handler.perform([request])