在常規的視頻圖像編輯處理場景中，畫中畫是必不可少的一個功能，而 FFmpeg 的 overlay 濾鏡通過進行參數設置就可以輕鬆地達到畫中畫效果，下面就詳細講解一下如何使用 overlay 濾鏡實現畫中畫效果。
在做畫中畫效果實現之前，首先需要了解基本的需求，那麼問題來了，畫中畫就涉及到主畫面和子畫面，會遇到如下幾個問題：
子畫面顯示在主畫面的什麼位置？
子畫面從主畫面的哪個時間點開始顯示？
子畫面從子畫面的哪個時間點開始顯示？
子畫面是按照時間段顯示還是一直顯示？
如果子畫面或者主畫面不等長怎麼辦？
解決以上問題之前，首先了解一下overlay的一些基本參數：
overlay AVOptions: x <string> ..FV set the x expression (default "0") y <string> ..FV set the y expression (default "0") eof_action <int> ..FV Action to take when encountering EOF from secondary input (from 0 to 2) (default repeat) repeat ..FV Repeat the previous frame. endall ..FV End both streams. pass ..FV Pass through the main input. eval <int> ..FV specify when to evaluate expressions (from 0 to 1) (default frame) init ..FV eval expressions once during initialization frame ..FV eval expressions per-frame shortest <boolean> ..FV force termination when the shortest input terminates (default false) format <int> ..FV set output format (from 0 to 5) (default yuv420) yuv420 ..FV yuv422 ..FV yuv444 ..FV rgb ..FV gbrp ..FV auto ..FV repeatlast <boolean> ..FV repeat overlay of the last overlay frame (default true) alpha <int> ..FV alpha format (from 0 to 1) (default straight) straight ..FV premultiplied ..FV
framesync AVOptions: eof_action <int> ..FV Action to take when encountering EOF from secondary input (from 0 to 2) (default repeat) repeat ..FV Repeat the previous frame. endall ..FV End both streams. pass ..FV Pass through the main input. shortest <boolean> ..FV force termination when the shortest input terminates (default false) repeatlast <boolean> ..FV extend last frame of secondary streams beyond EOF (default true)
This filter has support for timeline through the 'enable' option.解決第 1 個問題，主要設置 x, y 這兩個參數就可以達到效果；
解決第 2 個問題，看最後一行，這個濾鏡是支持 timeline 的，通過 enable 配合可以設置可見時間區間；
解決第 4 個問題，與第 2 個問題解決方法相同；
第 5 個問題，如果兩個視頻流時長不等長，可以通過 framesync 參數來處理，具體的說明均在上面，如果英語不好，可以長按英文，然後複製說明內容，貼到翻譯軟體中翻譯一下。
那麼，第 3 個問題怎麼辦呢？第 3 個問題單單用 overlay 濾鏡解決不了，想一下場景：
主畫面播放 3 秒後，子畫面從子畫面的第 0 秒開始顯示出來，顯示 3 秒鐘後不繼續顯示
這種場景，子畫面的 start time 與主畫面的 start time 之間存在了偏移，那麼視頻偏移可以通過 setpts 濾鏡設置
Filter setpts Set PTS for the output video frame. Inputs: Outputs: setpts AVOptions: expr <string> ..FVA.... Expression determining the frame timestamp (default "PTS")pts 設置比較簡單，通過 expr 也就是咱們現在這個公眾號以往的分享內容中可以找到。
overlay 配合 pts 即可解決第三個問題。
下面舉個例子，爭取把三個問題都串起來。
命令行部分
先創建兩個素材,為了查看方便，都創建的是帶時間顯示的視頻。
1. 創建主畫面視頻素材
StevenLiu:ffmpeg StevenLiu$ ffmpeg -f lavfi -i testsrc2=s=800x480 -vcodec libx264 -t 10 input1.mp4ffmpeg version N-94165-g6e988b75df Copyright (c) 2000-2019 the FFmpeg developers built with Apple LLVM version 10.0.0 (clang-1000.11.45.5) configuration: libavutil 56. 30.100 / 56. 30.100 libavcodec 58. 53.101 / 58. 53.101 libavformat 58. 28.101 / 58. 28.101 libavdevice 58. 7.100 / 58. 7.100 libavfilter 7. 56.100 / 7. 56.100 libswscale 5. 4.101 / 5. 4.101 libswresample 3. 4.100 / 3. 4.100 libpostproc 55. 4.100 / 55. 4.100Input Duration: N/A, start: 0.000000, bitrate: N/A Stream Stream mapping: Stream Press [q] to stop, [?] for help[libx264 @ 0x7ff3f2801200] using SAR=1/1[libx264 @ 0x7ff3f2801200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2[libx264 @ 0x7ff3f2801200] profile High, level 3.0[libx264 @ 0x7ff3f2801200] 264 - core 133 r2334M a3ac64b - H.264/MPEG-4 AVC codec - Copyleft 2003-2013 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00Output Metadata: encoder : Lavf58.28.101 Stream Metadata: encoder : Lavc58.53.101 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1frame= 250 fps= 92 q=-1.0 Lsize= 1297kB time=00:00:09.88 bitrate=1075.4kbits/s speed=3.63xvideo:1294kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.224270%[libx264 @ 0x7ff3f2801200] frame I:1 Avg QP:24.72 size: 11441[libx264 @ 0x7ff3f2801200] frame P:180 Avg QP:27.72 size: 5503[libx264 @ 0x7ff3f2801200] frame B:69 Avg QP:30.78 size: 4673[libx264 @ 0x7ff3f2801200] consecutive B-frames: 50.0% 36.8% 8.4% 4.8%[libx264 @ 0x7ff3f2801200] mb I I16..4: 72.5% 13.7% 13.9%[libx264 @ 0x7ff3f2801200] mb P I16..4: 3.9% 0.4% 0.3% P16..4: 8.0% 4.9% 3.6% 0.0% 0.0% skip:78.9%[libx264 @ 0x7ff3f2801200] mb B I16..4: 0.6% 0.1% 0.2% B16..8: 13.2% 3.7% 0.9% direct: 2.2% skip:79.0% L0:47.6% L1:44.4% BI: 8.0%[libx264 @ 0x7ff3f2801200] 8x8 transform intra:8.5% inter:18.7%[libx264 @ 0x7ff3f2801200] coded y,uvDC,uvAC intra: 6.6% 12.4% 11.2% inter: 5.3% 9.2% 7.9%[libx264 @ 0x7ff3f2801200] i16 v,h,dc,p: 82% 17% 2% 0%[libx264 @ 0x7ff3f2801200] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 4% 81% 0% 0% 0% 0% 0% 1%[libx264 @ 0x7ff3f2801200] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 26% 19% 37% 5% 2% 2% 3% 3% 4%[libx264 @ 0x7ff3f2801200] i8c dc,h,v,p: 63% 17% 20% 0%[libx264 @ 0x7ff3f2801200] Weighted P-Frames: Y:0.0% UV:0.0%[libx264 @ 0x7ff3f2801200] ref P L0: 50.7% 7.4% 24.6% 17.2%[libx264 @ 0x7ff3f2801200] ref B L0: 69.9% 27.6% 2.5%[libx264 @ 0x7ff3f2801200] ref B L1: 95.4% 4.6%[libx264 @ 0x7ff3f2801200] kb/s:1059.602. 創建子畫面素材
StevenLiu:ffmpeg StevenLiu$ ffmpeg -f lavfi -i testsrc2=s=320x240 -vcodec libx264 -t 5 input2.mp4ffmpeg version N-94165-g6e988b75df Copyright (c) 2000-2019 the FFmpeg developers built with Apple LLVM version 10.0.0 (clang-1000.11.45.5) configuration: libavutil 56. 30.100 / 56. 30.100 libavcodec 58. 53.101 / 58. 53.101 libavformat 58. 28.101 / 58. 28.101 libavdevice 58. 7.100 / 58. 7.100 libavfilter 7. 56.100 / 7. 56.100 libswscale 5. 4.101 / 5. 4.101 libswresample 3. 4.100 / 3. 4.100 libpostproc 55. 4.100 / 55. 4.100Input Duration: N/A, start: 0.000000, bitrate: N/A Stream Stream mapping: Stream Press [q] to stop, [?] for help[libx264 @ 0x7f8faa805200] using SAR=1/1[libx264 @ 0x7f8faa805200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2[libx264 @ 0x7f8faa805200] profile High, level 1.3[libx264 @ 0x7f8faa805200] 264 - core 133 r2334M a3ac64b - H.264/MPEG-4 AVC codec - Copyleft 2003-2013 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00Output Metadata: encoder : Lavf58.28.101 Stream Metadata: encoder : Lavc58.53.101 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1frame= 125 fps=0.0 q=-1.0 Lsize= 173kB time=00:00:04.88 bitrate= 289.9kbits/s speed=12.4xvideo:171kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.129911%[libx264 @ 0x7f8faa805200] frame I:1 Avg QP:25.53 size: 4375[libx264 @ 0x7f8faa805200] frame P:84 Avg QP:27.86 size: 1519[libx264 @ 0x7f8faa805200] frame B:40 Avg QP:32.16 size: 1056[libx264 @ 0x7f8faa805200] consecutive B-frames: 47.2% 24.0% 19.2% 9.6%[libx264 @ 0x7f8faa805200] mb I I16..4: 66.7% 7.3% 26.0%[libx264 @ 0x7f8faa805200] mb P I16..4: 4.6% 0.4% 0.5% P16..4: 13.5% 8.8% 6.7% 0.0% 0.0% skip:65.5%[libx264 @ 0x7f8faa805200] mb B I16..4: 0.8% 0.1% 0.1% B16..8: 19.7% 6.5% 1.8% direct: 2.6% skip:68.6% L0:45.5% L1:49.1% BI: 5.4%[libx264 @ 0x7f8faa805200] 8x8 transform intra:6.7% inter:34.2%[libx264 @ 0x7f8faa805200] coded y,uvDC,uvAC intra: 7.5% 23.3% 18.9% inter: 5.6% 15.1% 12.2%[libx264 @ 0x7f8faa805200] i16 v,h,dc,p: 75% 21% 4% 0%[libx264 @ 0x7f8faa805200] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 8% 68% 6% 0% 0% 2% 1% 1%[libx264 @ 0x7f8faa805200] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 42% 20% 30% 1% 1% 1% 2% 1% 1%[libx264 @ 0x7f8faa805200] i8c dc,h,v,p: 45% 21% 34% 1%[libx264 @ 0x7f8faa805200] Weighted P-Frames: Y:0.0% UV:0.0%[libx264 @ 0x7f8faa805200] ref P L0: 61.3% 7.5% 18.9% 12.2%[libx264 @ 0x7f8faa805200] ref B L0: 77.5% 18.8% 3.7%[libx264 @ 0x7f8faa805200] ref B L1: 94.8% 5.2%[libx264 @ 0x7f8faa805200] kb/s:278.713. 開始畫中畫操作
StevenLiu:ffmpeg StevenLiu$ ffmpeg -i input1.mp4 -filter_complex "movie=input2.mp4[in2];[in2]setpts=PTS+2/TB[out2];[0:v][out2]overlay=x=20:y=120:enable='between(t,2,4)':shortest=1" output.mp4ffmpeg version N-94165-g6e988b75df Copyright (c) 2000-2019 the FFmpeg developers built with Apple LLVM version 10.0.0 (clang-1000.11.45.5) configuration: libavutil 56. 30.100 / 56. 30.100 libavcodec 58. 53.101 / 58. 53.101 libavformat 58. 28.101 / 58. 28.101 libavdevice 58. 7.100 / 58. 7.100 libavfilter 7. 56.100 / 7. 56.100 libswscale 5. 4.101 / 5. 4.101 libswresample 3. 4.100 / 3. 4.100 libpostproc 55. 4.100 / 55. 4.100Input Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf58.28.101 Duration: 00:00:10.00, start: 0.000000, bitrate: 1062 kb/s Stream Metadata: handler_name : VideoHandlerStream mapping: Stream overlay -> Stream Press [q] to stop, [?] for help[libx264 @ 0x7fab3a827000] using SAR=1/1[libx264 @ 0x7fab3a827000] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2[libx264 @ 0x7fab3a827000] profile High, level 3.0[libx264 @ 0x7fab3a827000] 264 - core 133 r2334M a3ac64b - H.264/MPEG-4 AVC codec - Copyleft 2003-2013 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00Output Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf58.28.101 Stream Metadata: encoder : Lavc58.53.101 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1[Parsed_movie_0 @ 0x7fab3a4072c0] EOF timestamp not reliablebitrate= 873.9kbits/s speed=1.87xframe= 250 fps= 52 q=-1.0 Lsize= 1311kB time=00:00:09.88 bitrate=1087.2kbits/s speed=2.05xvideo:1308kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.226632%[libx264 @ 0x7fab3a827000] frame I:1 Avg QP:22.64 size: 11415[libx264 @ 0x7fab3a827000] frame P:173 Avg QP:26.25 size: 5606[libx264 @ 0x7fab3a827000] frame B:76 Avg QP:30.55 size: 4705[libx264 @ 0x7fab3a827000] consecutive B-frames: 44.4% 43.2% 6.0% 6.4%[libx264 @ 0x7fab3a827000] mb I I16..4: 64.3% 22.3% 13.4%[libx264 @ 0x7fab3a827000] mb P I16..4: 3.2% 0.6% 0.4% P16..4: 9.4% 5.0% 3.4% 0.0% 0.0% skip:77.9%[libx264 @ 0x7fab3a827000] mb B I16..4: 0.4% 0.1% 0.2% B16..8: 14.2% 3.8% 1.0% direct: 2.3% skip:78.1% L0:49.1% L1:44.1% BI: 6.8%[libx264 @ 0x7fab3a827000] 8x8 transform intra:16.0% inter:20.7%[libx264 @ 0x7fab3a827000] coded y,uvDC,uvAC intra: 8.9% 17.4% 15.5% inter: 5.4% 10.0% 8.5%[libx264 @ 0x7fab3a827000] i16 v,h,dc,p: 84% 14% 2% 0%[libx264 @ 0x7fab3a827000] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 13% 68% 0% 1% 1% 1% 0% 1%[libx264 @ 0x7fab3a827000] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 19% 37% 5% 2% 2% 3% 3% 4%[libx264 @ 0x7fab3a827000] i8c dc,h,v,p: 65% 15% 20% 0%[libx264 @ 0x7fab3a827000] Weighted P-Frames: Y:0.0% UV:0.0%[libx264 @ 0x7fab3a827000] ref P L0: 52.4% 7.5% 23.5% 16.6%[libx264 @ 0x7fab3a827000] ref B L0: 73.4% 25.2% 1.5%[libx264 @ 0x7fab3a827000] ref B L1: 97.1% 2.9%[libx264 @ 0x7fab3a827000] kb/s:1071.14StevenLiu:ffmpeg StevenLiu$1我們解析一下命令行參數：
輸入的主文件是 input1.mp4
輸入的子文件是 input2.mp4
將子畫面的所有的時間戳都加上 2 秒鐘的偏移
將子畫面顯示在主畫面的 x 坐標為 20，y 坐標為 120 的位置
子畫面在主畫面顯示時間段是第 2 秒到第 4 秒之間
以主畫面和子畫面中時長最短的那個視頻流長度為結束點
API 調用實現
為了讓覺得用命令行 Low B 確喜歡調用 API 的大佬們開開心心抄代碼，我把代碼部分貼上來：
StevenLiu:ffmpeg StevenLiu$ git diffdiff --git a/doc/examples/transcoding.c b/doc/examples/transcoding.cindex e48837cbd2..cf46a61492 100644--- a/doc/examples/transcoding.c+++ b/doc/examples/transcoding.c@@ -391,7 +391,7 @@ static int init_filters(void)
if (ifmt_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO)- filter_spec = "null"; /* passthrough (dummy) filter for video */+ filter_spec = "movie=input2.mp4[in2];[in2]setpts=PTS+2/TB[out2];[in][out2]overlay=x=20:y=120:enable='between(t,2,4)':shortest=1"; /* passthrough (dummy) filter for video */ else filter_spec = "anull"; /* passthrough (dummy) filter for audio */ ret = init_filter(&filter_ctx[i], stream_ctx[i].dec_ctx,StevenLiu:ffmpeg StevenLiu$下面看一下生成出來的視頻的效果：
如果不設置 PTS 的話，效果如下：
今天就介紹這些。