在 Basic 與 Playback 教學中,GStreamer 能與 GLib 的主迴圈完美整合,同時監控 pipeline 狀態與處理 UI 是一件很自然的事。但在 iOS、Android 這類不以 GLib 為基礎的平台,就必須自己安排執行緒——絕對不能让 pipeline 的操作阻塞住 UI 執行緒。
本篇涵蓋三件事:
範例的 pipeline 只用了 audiotestsrc 與 autoaudiosink 兩個元件;畫面上有 Play、Pause 兩個按鈕與一個顯示訊息的 Label。
@interface ViewController () {
GStreamerBackend *gst_backend;
}
@end
- (void)viewDidLoad {
[super viewDidLoad];
play_button.enabled = FALSE;
pause_button.enabled = FALSE;
gst_backend = [[GStreamerBackend alloc] init:self];
}
-(IBAction)play:(id)sender { [gst_backend play]; }
-(IBAction)pause:(id)sender { [gst_backend pause]; }
幾個設計意圖值得注意:
-(void) gstreamerInitialized {
dispatch_async(dispatch_get_main_queue(), ^{
play_button.enabled = TRUE;
pause_button.enabled = TRUE;
message_label.text = @"Ready";
});
}
-(void) gstreamerSetUIMessage:(NSString *)message {
dispatch_async(dispatch_get_main_queue(), ^{
message_label.text = message;
});
}
這兩個方法都定義在 GStreamerBackendDelegate 協定中,分別代表「後端就緒」與「後端要說話」。關鍵在於:它們都不是在 UI 執行緒上被呼叫的,所以任何對控制項的操作都必須用 dispatch_async 包回主佇列,否則就是未定義行為。
@implementation GStreamerBackend {
id ui_delegate;
GstElement *pipeline;
GMainContext *context;
GMainLoop *main_loop;
gboolean initialized;
}
-(id) init:(id) uiDelegate {
if (self = [super init]) {
self->ui_delegate = uiDelegate;
GST_DEBUG_CATEGORY_INIT(debug_category, "tutorial-2", 0, "iOS tutorial 2");
gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);
dispatch_async(dispatch_get_global_queue(
DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
[self app_function];
});
}
return self;
}
-(void) dealloc {
if (pipeline) {
gst_element_set_state(pipeline, GST_STATE_NULL);
gst_object_unref(pipeline);
pipeline = NULL;
}
}
init 中除了保存 delegate,還註冊了 GStreamer 的除錯類別並設定等級,訊息就會出現在 Xcode 主控台。而監聽 bus 的工作被丟到一條併發佇列上執行,這是整篇架構的核心。
-(void) play {
if (gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {
[self setUIMessage:"Failed to set pipeline to playing"];
}
}
-(void) pause {
if (gst_element_set_state(pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE) {
[self setUIMessage:"Failed to set pipeline to paused"];
}
}
-(void)setUIMessage:(gchar*) message {
NSString *string = [NSString stringWithUTF8String:message];
if (ui_delegate &&
[ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)]) {
[ui_delegate gstreamerSetUIMessage:string];
}
}
因為 delegate 方法是 optional,呼叫前必須用 respondsToSelector: 確認對方有實作,這是 Cocoa 的標準防禦寫法。
static void error_cb(GstBus *bus, GstMessage *msg, GStreamerBackend *self) {
GError *err;
gchar *debug_info;
gst_message_parse_error(msg, &err, &debug_info);
gchar *message_string = g_strdup_printf(
"Error received from element %s: %s", GST_OBJECT_NAME(msg->src), err->message);
g_clear_error(&err);
g_free(debug_info);
[self setUIMessage:message_string];
g_free(message_string);
gst_element_set_state(self->pipeline, GST_STATE_NULL);
}
static void state_changed_cb(GstBus *bus, GstMessage *msg, GStreamerBackend *self) {
GstState old_state, new_state, pending_state;
gst_message_parse_state_changed(msg, &old_state, &new_state, &pending_state);
// 只關心 pipeline 本身的狀態變化,忽略子元件
if (GST_MESSAGE_SRC(msg) == GST_OBJECT(self->pipeline)) {
gchar *message = g_strdup_printf("State changed to %s",
gst_element_state_get_name(new_state));
[self setUIMessage:message];
g_free(message);
}
}
與 Basic 教學相比,這裡只多了兩點差異:訊息一律走 setUIMessage: 傳給 UI;以及回呼需要 GStreamerBackend 的實例,因此透過 userdata 傳入——這也是下面 app_function 註冊時要特別處理的地方。
-(void) check_initialization_complete {
if (!initialized && main_loop) {
if (ui_delegate &&
[ui_delegate respondsToSelector:@selector(gstreamerInitialized)]) {
[ui_delegate gstreamerInitialized];
}
initialized = TRUE;
}
}
本篇的條件很寬鬆:只要主迴圈已建立、且還沒通知過 UI 就算完成。後續教學加入更多元件後,條件會變得更嚴謹。
context = g_main_context_new();
g_main_context_push_thread_default(context);
pipeline = gst_parse_launch(
"audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
bus = gst_element_get_bus(pipeline);
bus_source = gst_bus_create_watch(bus);
g_source_set_callback(bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
g_source_attach(bus_source, context);
g_source_unref(bus_source);
g_signal_connect(G_OBJECT(bus), "message::error",
(GCallback)error_cb, (__bridge void *)self);
g_signal_connect(G_OBJECT(bus), "message::state-changed",
(GCallback)state_changed_cb, (__bridge void *)self);
gst_object_unref(bus);
main_loop = g_main_loop_new(context, FALSE);
[self check_initialization_complete];
g_main_loop_run(main_loop);
這裡有兩個重點:
本篇主要在建立兩個觀念:用 Dispatch Queue 把 GStreamer 的處理隔離到子執行緒;以及在 ObjC 與 C 之間正確傳遞物件與字串。check_initialization_complete()、app_function()、init:、play、pause、gstreamerInitialized、setUIMessage: 這幾個介面後續會被沿用並小幅修改,建議先熟悉起來。