SDL音視頻渲染

---

目錄

SDL簡(jiǎn)介

SDL窗口顯示

SDL Event事件

SDL Thread

SDL PCM播放

SDL YUV播放

---

1. SDL基本介紹

1. SDL簡(jiǎn)介

SDL（Simple DirectMedia Layer）是一套開(kāi)放源代碼的跨平臺(tái)多媒體開(kāi)發(fā)庫(kù)，使用C語(yǔ)言寫(xiě)成。

SDL提供了數(shù)種控制圖像、聲音、輸出入的函數(shù)，讓開(kāi)發(fā)者只要用相同或是相似的代碼就可以開(kāi)發(fā)出跨多個(gè)平臺(tái)（Linux、 Windows、 Mac OS X等）的應(yīng)用軟件。

目前SDL多用于開(kāi)發(fā)游戲、模擬器、 媒體播放器等多媒體應(yīng)用領(lǐng)域。

學(xué)習(xí)SDL主要用來(lái)輔助學(xué)習(xí)Ffmpeg。

官網(wǎng)：https://www.libsdl.org/

文檔：http://wiki.libsdl.org/Introduction

2. SDL子系統(tǒng)

SDL將功能分成下列數(shù)個(gè)子系統(tǒng)（subsystem）：

SDL_INIT_TIMER：定時(shí)器

SDL_INIT_AUDIO：音頻

SDL_INIT_VIDEO：視頻

SDL_INIT_JOYSTICK：搖桿

SDL_INIT_HAPTIC：觸摸屏

SDL_INIT_GAMECONTROLLER：游戲控制器

SDL_INIT_EVENTS：事件

SDL_INIT_EVERYTHING：包含上述所有選項(xiàng)

---

2. SDL窗口顯示

1. SDL視頻顯示函數(shù)簡(jiǎn)介

SDL_Init()：初始化SDL系統(tǒng)

SDL_CreateWindow()：創(chuàng)建窗口SDL_Window

SDL_CreateRenderer()：創(chuàng)建渲染器SDL_Renderer

SDL_CreateTexture()：創(chuàng)建紋理SDL_Texture

SDL_UpdateTexture()：設(shè)置紋理的數(shù)據(jù)

SDL_RenderCopy()：將紋理的數(shù)據(jù)拷貝給渲染器

SDL_RenderPresent()：顯示

SDL_Delay()：工具函數(shù)，用于延時(shí)

SDL_Quit()：退出SDL系統(tǒng)

2. SDL數(shù)據(jù)結(jié)構(gòu)簡(jiǎn)介

SDL_Window 代表了一個(gè)“窗口”

SDL_Renderer 代表了一個(gè)“渲染器”

SDL_Texture 代表了一個(gè)“紋理”

SDL_Rect 一個(gè)簡(jiǎn)單的矩形結(jié)構(gòu)

一個(gè)窗口可以有多個(gè)render，render可用來(lái)渲染紋理，也可以用來(lái)顯示紋理。

存儲(chǔ)RGB和存儲(chǔ)紋理的區(qū)別：比如一個(gè)從左到右由紅色漸變到藍(lán)色的矩形，用存儲(chǔ)RGB的話(huà)就需要把矩形中每個(gè)點(diǎn)的具體顏色值存儲(chǔ)下來(lái)；而紋理只是一些描述信息，比如記錄了矩形的大小、起始顏色、終止顏色等信息，顯卡可以通過(guò)這些信息推算出矩形塊的詳細(xì)信息。所以相對(duì)于存儲(chǔ)RGB而已，存儲(chǔ)紋理占用的內(nèi)存要少的多。

3. 代碼示例

pro文件設(shè)置動(dòng)態(tài)庫(kù)

TEMPLATE = app CONFIG += console CONFIG -= app_bundle CONFIG -= qtSOURCES += \main.cINCLUDEPATH += "/usr/local/Cellar/sdl2/2.0.12_1/include"# 默認(rèn)是動(dòng)態(tài)庫(kù)的鏈接 LIBS += -L/usr/local/Cellar/sdl2/2.0.12_1/lib -lSDL2

代碼

#include <stdio.h> #include <SDL2/SDL.h>int main() {printf("Hello World!\n");int run = 1;SDL_Window *window = NULL;SDL_Renderer *renderer = NULL;SDL_Texture *texture = NULL;SDL_Rect rect; //長(zhǎng)方形，原點(diǎn)在左上角rect.w = 50;rect.h = 50;SDL_Init(SDL_INIT_VIDEO);window = SDL_CreateWindow("2 WINDOW", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 640, 480,SDL_WINDOW_OPENGL | SDL_WINDOW_RESIZABLE); //創(chuàng)建窗口if (!window) {return -1;}renderer = SDL_CreateRenderer(window, -1, 0); //基于窗口創(chuàng)建渲染器if (!renderer) {return -1;}texture = SDL_CreateTexture(renderer, SDL_PIXELFORMAT_RGB888, SDL_TEXTUREACCESS_TARGET, 640, 480); // 創(chuàng)建紋理if (!texture) {return -1;}int show_count = 0;while (run) {rect.x = rand() % 600;rect.y = rand() % 400;SDL_SetRenderTarget(renderer, texture); //設(shè)置渲染目標(biāo)為紋理SDL_SetRenderDrawColor(renderer, 255, 0, 0, 255); //紋理背景為黑色SDL_RenderClear(renderer); //清屏SDL_RenderDrawRect(renderer, &rect); //繪制一個(gè)長(zhǎng)方形SDL_SetRenderDrawColor(renderer, 0, 255, 255, 255); //長(zhǎng)方形為白色SDL_RenderFillRect(renderer, &rect);SDL_SetRenderTarget(renderer, NULL); //恢復(fù)默認(rèn)，渲染目標(biāo)為窗口SDL_RenderCopy(renderer, texture, NULL, NULL); //拷貝紋理到CPUSDL_RenderPresent(renderer); //輸出到目標(biāo)窗口上SDL_Delay(300);if (show_count++ > 30) {run = 0; //不跑了}}SDL_DestroyTexture(texture);SDL_DestroyRenderer(renderer);SDL_DestroyWindow(window);SDL_Quit();return 0; }

---

3. SDL Event事件

1. 函數(shù)

SDL_WaitEvent()：等待一個(gè)事件

SDL_PushEvent()：發(fā)送一個(gè)事件

SDL_PumpEvents()：將硬件設(shè)備產(chǎn)生的事件放入事件隊(duì)列，用于讀取事件，在調(diào)用該函數(shù)之前，必須調(diào)用SDL_PumpEvents搜集鍵盤(pán)等事件

SDL_PeepEvents()：從事件隊(duì)列提取一個(gè)事件

2. 數(shù)據(jù)結(jié)構(gòu)

SDL_Event：代表一個(gè)事件

3. 代碼示例

代碼

#include <stdio.h> #include <SDL2/SDL.h>#define FF_QUIT_EVENT (SDL_USEREVENT + 2) // 用戶(hù)自定義事件int main() {SDL_Window *window = NULL; // Declare a pointerSDL_Renderer *renderer = NULL;SDL_Init(SDL_INIT_VIDEO); // Initialize SDL2// Create an application window with the following settings:window = SDL_CreateWindow("An SDL2 window", // window titleSDL_WINDOWPOS_UNDEFINED, // initial x positionSDL_WINDOWPOS_UNDEFINED, // initial y position640, // width, in pixels480, // height, in pixelsSDL_WINDOW_SHOWN | SDL_WINDOW_BORDERLESS// flags - see below);// Check that the window was successfully createdif (window == NULL) {// In the case that the window could not be made...printf("Could not create window: %s\n", SDL_GetError());return 1;}/* We must call SDL_CreateRenderer in order for draw calls to affect this window. */renderer = SDL_CreateRenderer(window, -1, 0);/* Select the color for drawing. It is set to red here. */SDL_SetRenderDrawColor(renderer, 255, 0, 0, 255);/* Clear the entire screen to our selected color. */SDL_RenderClear(renderer);/* Up until now everything was drawn behind the scenes.This will show the new, red contents of the window. */SDL_RenderPresent(renderer);SDL_Event event;int b_exit = 0;for (;;) {SDL_WaitEvent(&event);switch (event.type) {case SDL_KEYDOWN: /* 鍵盤(pán)事件 */switch (event.key.keysym.sym) {case SDLK_a:printf("key down a\n");break;case SDLK_s:printf("key down s\n");break;case SDLK_d:printf("key down d\n");break;case SDLK_q:printf("key down q and push quit event\n");SDL_Event event_q;event_q.type = FF_QUIT_EVENT;SDL_PushEvent(&event_q);break;default:printf("key down 0x%x\n", event.key.keysym.sym);break;}break;case SDL_MOUSEBUTTONDOWN: /* 鼠標(biāo)按下事件 */if (event.button.button == SDL_BUTTON_LEFT) {printf("mouse down left\n");} else if (event.button.button == SDL_BUTTON_RIGHT) {printf("mouse down right\n");} else {printf("mouse down %d\n", event.button.button);}break;case SDL_MOUSEMOTION: /* 鼠標(biāo)移動(dòng)事件 */printf("mouse movie (%d,%d)\n", event.button.x, event.button.y);break;case FF_QUIT_EVENT:printf("receive quit event\n");b_exit = 1;break;}if (b_exit)break;}//destory rendererif (renderer)SDL_DestroyRenderer(renderer);// Close and destroy the windowif (window)SDL_DestroyWindow(window);// Clean upSDL_Quit();return 0; }

效果

---

4. SDL Thread

1. 函數(shù)

SDL線(xiàn)程創(chuàng)建： SDL_CreateThread

SDL線(xiàn)程等待： SDL_WaitThead

SDL互斥鎖： SDL_CreateMutex/SDL_DestroyMutex

SDL鎖定互斥： SDL_LockMutex/SDL_UnlockMutex

SDL條件變量(信號(hào)量)： SDL_CreateCond/SDL_DestoryCond

SDL條件變量(信號(hào)量)等待/通知： SDL_CondWait/SDL_CondSingal

2. 代碼示例

代碼

#include <stdio.h> #include <SDL2/SDL.h> #include <unistd.h>SDL_mutex *s_lock = NULL; SDL_cond *s_cond = NULL;int thread_work(void *arg) {SDL_LockMutex(s_lock);printf(" <============thread_work sleep\n"); // sleep(10); // 用來(lái)測(cè)試獲取鎖sleep(10);printf(" <============thread_work wait\n");// 釋放s_lock資源，并等待signal。之所以釋放s_lock是讓別的線(xiàn)程能夠獲取到s_lockSDL_CondWait(s_cond, s_lock); //另一個(gè)線(xiàn)程(1)發(fā)送signal和(2)釋放lock后，這個(gè)函數(shù)退出printf(" <===========thread_work receive signal, continue to do ~_~!!!\n");printf(" <===========thread_work end\n");SDL_UnlockMutex(s_lock);return 0; }#undef main int main() {s_lock = SDL_CreateMutex();s_cond = SDL_CreateCond();SDL_Thread * t = SDL_CreateThread(thread_work,"thread_work",NULL);if(!t){printf(" %s",SDL_GetError);return -1;}for(int i = 0;i< 2;i++){sleep(2);printf("main execute =====>\n");}printf("main SDL_LockMutex(s_lock) before ====================>\n");SDL_LockMutex(s_lock); // 獲取鎖，但是子線(xiàn)程還拿著鎖printf("main ready send signal====================>\n");printf("main SDL_CondSignal(s_cond) before ====================>\n");SDL_CondSignal(s_cond); // 發(fā)送信號(hào)，喚醒等待的線(xiàn)程printf("main SDL_CondSignal(s_cond) after ====================>\n"); // sleep(10);SDL_UnlockMutex(s_lock);// 釋放鎖，讓其他線(xiàn)程可以拿到鎖printf("main SDL_UnlockMutex(s_lock) after ====================>\n");SDL_WaitThread(t, NULL);SDL_DestroyMutex(s_lock);SDL_DestroyCond(s_cond);return 0; }

注：mac系統(tǒng)下sleep需要引用頭文件：#include <unistd.h>，Windows下不需要。

效果

---

5. SDL PCM播放

1. 函數(shù)

打開(kāi)音頻設(shè)備

int SDLCALL SDL_OpenAudio(SDL_AudioSpec * desired,SDL_AudioSpec * obtained); // desired：期望的參數(shù)。 // obtained：實(shí)際音頻設(shè)備的參數(shù)，一般情況下設(shè)置為NULL即可。

SDL_AudioSpec

typedef struct SDL_AudioSpec {int freq; // 音頻采樣率SDL_AudioFormat format; // 音頻數(shù)據(jù)格式Uint8 channels; // 聲道數(shù): 1 單聲道, 2 立體聲Uint8 silence; // 設(shè)置靜音的值， 因?yàn)槁曇舨蓸邮怯蟹?hào)的， 所以0當(dāng)然就是這個(gè)值Uint16 samples; // 音頻緩沖區(qū)中的采樣個(gè)數(shù)，要求必須是2的n次Uint16 padding; // 考慮到兼容性的一個(gè)參數(shù)Uint32 size; // 音頻緩沖區(qū)的大小，以字節(jié)為單位SDL_AudioCallback callback; // 填充音頻緩沖區(qū)的回調(diào)函數(shù)void *userdata; // 用戶(hù)自定義的數(shù)據(jù) } SDL_AudioSpec;

SDL_AudioCallback

// userdata： SDL_AudioSpec結(jié)構(gòu)中的用戶(hù)自定義數(shù)據(jù)，一般情況下可以不用。 // stream：該指針指向需要填充的音頻緩沖區(qū)。 // len：音頻緩沖區(qū)的大小（以字節(jié)為單位） 1024*2*2。 void (SDLCALL * SDL_AudioCallback) (void *userdata, Uint8 *stream, int len);

2. 代碼示例

代碼

/*** SDL2播放PCM*** 本程序使用SDL2播放PCM音頻采樣數(shù)據(jù)。SDL實(shí)際上是對(duì)底層繪圖* API（Direct3D，OpenGL）的封裝，使用起來(lái)明顯簡(jiǎn)單于直接調(diào)用底層* API。* 測(cè)試的PCM數(shù)據(jù)采用采樣率44.1k, 采用精度S16SYS, 通道數(shù)2** 函數(shù)調(diào)用步驟如下:** [初始化]* SDL_Init(): 初始化SDL。* SDL_OpenAudio(): 根據(jù)參數(shù)（存儲(chǔ)于SDL_AudioSpec）打開(kāi)音頻設(shè)備。* SDL_PauseAudio(): 播放音頻數(shù)據(jù)。** [循環(huán)播放數(shù)據(jù)]* SDL_Delay(): 延時(shí)等待播放完成。**/#include <stdio.h> #include <SDL2/SDL.h>// 每次讀取2幀數(shù)據(jù), 以1024個(gè)采樣點(diǎn)一幀 2通道 16bit采樣點(diǎn)為例 #define PCM_BUFFER_SIZE (1024*2*2*2)// 音頻PCM數(shù)據(jù)緩存 static Uint8 *s_audio_buf = NULL; // 目前讀取的位置 static Uint8 *s_audio_pos = NULL; // 緩存結(jié)束位置 static Uint8 *s_audio_end = NULL;//音頻設(shè)備回調(diào)函數(shù) void fill_audio_pcm(void *udata, Uint8 *stream, int len) {SDL_memset(stream, 0, len);if(s_audio_pos >= s_audio_end) // 數(shù)據(jù)讀取完畢{return;}// 數(shù)據(jù)夠了就讀預(yù)設(shè)長(zhǎng)度，數(shù)據(jù)不夠就只讀部分（不夠的時(shí)候剩多少就讀取多少）int remain_buffer_len = s_audio_end - s_audio_pos;len = (len < remain_buffer_len) ? len : remain_buffer_len;// 拷貝數(shù)據(jù)到stream并調(diào)整音量SDL_MixAudio(stream, s_audio_pos, len, SDL_MIX_MAXVOLUME/8);printf("len = %d\n", len);s_audio_pos += len; // 移動(dòng)緩存指針 }// 提取PCM文件 // ffmpeg -i input.mp4 -t 20 -codec:a pcm_s16le -ar 44100 -ac 2 -f s16le 44100_16bit_2ch.pcm // 測(cè)試PCM文件 // ffplay -ar 44100 -ac 2 -f s16le 44100_16bit_2ch.pcm #undef main int main(int argc, char *argv[]) {int ret = -1;FILE *audio_fd = NULL;SDL_AudioSpec spec;const char *path = "44100_16bit_2ch.pcm";// 每次緩存的長(zhǎng)度size_t read_buffer_len = 0;//SDL initializeif(SDL_Init(SDL_INIT_AUDIO)) // 支持AUDIO{fprintf(stderr, "Could not initialize SDL - %s\n", SDL_GetError());return ret;}//打開(kāi)PCM文件audio_fd = fopen(path, "rb");if(!audio_fd){fprintf(stderr, "Failed to open pcm file!\n");goto _FAIL;}s_audio_buf = (uint8_t *)malloc(PCM_BUFFER_SIZE);// 音頻參數(shù)設(shè)置SDL_AudioSpecspec.freq = 44100; // 采樣頻率spec.format = AUDIO_S16SYS; // 采樣點(diǎn)格式spec.channels = 2; // 2通道spec.silence = 0;spec.samples = 1024; // 23.2ms -> 46.4ms 每次讀取的采樣數(shù)量，多久產(chǎn)生一次回調(diào)和 samplesspec.callback = fill_audio_pcm; // 回調(diào)函數(shù)spec.userdata = NULL;//打開(kāi)音頻設(shè)備if(SDL_OpenAudio(&spec, NULL)){fprintf(stderr, "Failed to open audio device, %s\n", SDL_GetError());goto _FAIL;}//play audioSDL_PauseAudio(0);int data_count = 0;while(1){// 從文件讀取PCM數(shù)據(jù)read_buffer_len = fread(s_audio_buf, 1, PCM_BUFFER_SIZE, audio_fd);if(read_buffer_len == 0){break;}data_count += read_buffer_len; // 統(tǒng)計(jì)讀取的數(shù)據(jù)總字節(jié)數(shù)printf("now playing %10d bytes data.\n",data_count);s_audio_end = s_audio_buf + read_buffer_len; // 更新buffer的結(jié)束位置s_audio_pos = s_audio_buf; // 更新buffer的起始位置//the main thread wait for a momentwhile(s_audio_pos < s_audio_end){SDL_Delay(10); // 等待PCM數(shù)據(jù)消耗}}printf("play PCM finish\n");// 關(guān)閉音頻設(shè)備SDL_CloseAudio();_FAIL://release some resourcesif(s_audio_buf)free(s_audio_buf);if(audio_fd)fclose(audio_fd);//quit SDLSDL_Quit();return 0; }

效果，聽(tīng)到音頻聲音

---

6. SDL YUV播放

1. YUV顯示：SDL視頻顯示流程

2. 代碼

代碼

#include <stdio.h> #include <SDL2/SDL.h> #include <string.h>//自定義消息類(lèi)型 #define REFRESH_EVENT (SDL_USEREVENT + 1) // 請(qǐng)求畫(huà)面刷新事件 #define QUIT_EVENT (SDL_USEREVENT + 2) // 退出事件//定義分辨率 // YUV像素分辨率 #define YUV_WIDTH 320 #define YUV_HEIGHT 240 //定義YUV格式 #define YUV_FORMAT SDL_PIXELFORMAT_IYUVint s_thread_exit = 0; // 退出標(biāo)志 = 1則退出int refresh_video_timer(void *data) {while (!s_thread_exit) {SDL_Event event;event.type = REFRESH_EVENT;SDL_PushEvent(&event);SDL_Delay(40);}s_thread_exit = 0;//push quit eventSDL_Event event;event.type = QUIT_EVENT;SDL_PushEvent(&event); }int main() {if (SDL_Init(SDL_INIT_VIDEO)) {fprintf(stderr, "Could not initialize SDL - %s\n", SDL_GetError());return -1;}//SDLSDL_Event event; // 事件SDL_Rect rect; // 矩形SDL_Window *window = NULL; // 窗口SDL_Renderer *renderer = NULL; // 渲染SDL_Texture *texture = NULL; // 紋理SDL_Thread *timer_thread = NULL; // 請(qǐng)求刷新線(xiàn)程u_int32_t pixformat = YUV_FORMAT; // YUV420P，即是SDL_PIXELFORMAT_IYUV// 分辨率// 1. YUV的分辨率int video_width = YUV_WIDTH;int video_hight = YUV_HEIGHT;// 2.顯示窗口的分辨率int win_width = YUV_WIDTH;int win_height = YUV_WIDTH;// YUV文件句柄FILE *video_fd = NULL;const char *yuv_path = "yuv420p_320x240.yuv";size_t video_buff_len = 0;uint8_t *video_buf = NULL; //讀取數(shù)據(jù)后放buff里面// 我們測(cè)試的文件是YUV420P格式uint32_t y_frame_len = video_width * video_hight;uint32_t u_frame_len = video_width * video_hight / 4;uint32_t v_frame_len = video_width * video_hight / 4;uint32_t yuv_frame_len = y_frame_len + u_frame_len + v_frame_len;window = SDL_CreateWindow("simplest YUV Player", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, video_width,video_hight, SDL_WINDOW_OPENGL | SDL_WINDOW_RESIZABLE);if (!window) {fprintf(stderr, "SDL: could not create window, err:%s\n", SDL_GetError());goto _FAIL;}// 基于窗口創(chuàng)建渲染器renderer = SDL_CreateRenderer(window, -1, 0);// 基于渲染器創(chuàng)建紋理texture = SDL_CreateTexture(renderer, pixformat, SDL_TEXTUREACCESS_STREAMING, video_width, video_hight);//分配空間video_buf = (uint8_t *) malloc(yuv_frame_len);if (!video_buf) {fprintf(stderr, "Failed to alloce yuv frame space!\n");goto _FAIL;}video_fd = fopen(yuv_path, "rb");if (!video_fd) {fprintf(stderr, "Failed to open yuv file\n");goto _FAIL;}//創(chuàng)建請(qǐng)求刷新線(xiàn)程timer_thread = SDL_CreateThread(refresh_video_timer, NULL, NULL);while (1) {// 收取SDL系統(tǒng)里面的事件SDL_WaitEvent(&event);if (event.type == REFRESH_EVENT) {video_buff_len = fread(video_buf, 1, yuv_frame_len, video_fd);if (video_buff_len <= 0) {fprintf(stderr, "Failed to read data from yuv file!\n");goto _FAIL;}// 設(shè)置紋理的數(shù)據(jù) video_width = 320， planeSDL_UpdateTexture(texture, NULL, video_buf, video_width);//顯示區(qū)域，可以用個(gè)修改w和h進(jìn)行縮放rect.x = 0;rect.y = 0;float w_ratio = win_width * 1.0 / video_width;float h_ratio = win_height * 1.0 / video_hight;// 320x240 怎么保持原視頻的寬高比例rect.w = video_width * w_ratio;rect.h = video_hight * h_ratio;SDL_RenderClear(renderer);// 將紋理的數(shù)據(jù)拷貝給渲染器SDL_RenderCopy(renderer, texture, NULL, &rect);// 顯示SDL_RenderPresent(renderer);} else if (event.type == SDL_WINDOWEVENT) {//If ResizeSDL_GetWindowSize(window, &win_width, &win_height);printf("SDL_WINDOWEVENT win_width:%d, win_height:%d\n", win_width,win_height);} else if (event.type == SDL_QUIT) {s_thread_exit = 1;} else if (event.type == QUIT_EVENT) {break;}}_FAIL:s_thread_exit = 1; // 保證線(xiàn)程能夠退出// 釋放資源if (timer_thread)SDL_WaitThread(timer_thread, NULL); // 等待線(xiàn)程退出if (video_buf)free(video_buf);if (video_fd)fclose(video_fd);if (texture)SDL_DestroyTexture(texture);if (renderer)SDL_DestroyRenderer(renderer);if (window)SDL_DestroyWindow(window);SDL_Quit();return 0; }

效果

總結(jié)

以上是生活随笔為你收集整理的SDL音视频渲染的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。