[FFmpeg-user] FFmpeg Not Transcoding In Real Time

[Gabriel Balaich]

Hello all,

been setting up a recording build for a friend to resemble mine, but I
can't seem to get things to work in real-time despite sufficient (as far as
I can tell) hardware.

System specs: 8600K, GTX 1050ti, 16Gb RAM, 1tb 860 EVO

Test command:
ffmpeg -y -hide_banner -thread_queue_size 9999 -indexmem 9999
-guess_layout_max 0 -f dshow -rtbufsize 2147.48M `
-video_size 1920x1080 -framerate 60 `
-i
video="@device_pnp_\\?\usb#vid_07ca&pid_0570&mi_00#7&3886ab1a&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global":audio="@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{0A494693-5F33-4304-88D7-394757E09648}"
`
-thread_queue_size 9999 -indexmem 9999 -guess_layout_max 0 -f dshow
-rtbufsize 2147.48M `
-video_size 1920x1080 -framerate 60 `
-i
video="@device_pnp_\\?\usb#vid_07ca&pid_0570&mi_00#7&24df76f&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global":audio="@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{0E6F0DEF-2B29-4117-8D30-13F01160AC5B}"
`
-map 0:0,0:1 -map 0:1 -c:v h264_nvenc -r 60 -rc-lookahead 120 -forced-idr 1
-strict_gop 1 -sc_threshold 0 -flags +cgop `
-force_key_frames "expr:gte(t,n_forced*2)" -preset: llhp -pix_fmt nv12 -b:v
250M -minrate 250M -maxrate 250M `
-bufsize 250M -c:a aac -ar 44100 -b:a 384k -ac 2 -vsync 1 -ss 00:00:00.000 `
-max_muxing_queue_size 9999 -f segment -segment_time 600 -segment_wrap 9
-reset_timestamps 1 `
-segment_format_options max_delay=0
C:\Users\Jordan\Videos\FFmpeg\Left\Left%02d.ts `
-map 1:0,1:1 -map 1:1 -c:v h264_nvenc -r 60 -rc-lookahead 120 -forced-idr 1
-strict_gop 1 -sc_threshold 0 -flags +cgop `
-force_key_frames "expr:gte(t,n_forced*2)" -preset: llhp -pix_fmt nv12 -b:v
250M -minrate 250M -maxrate 250M `
-bufsize 250M -c:a aac -ar 44100 -b:a 384k -ac 2 -vsync 1 -ss 00:00:00.000 `
-max_muxing_queue_size 9999 -f segment -segment_time 600 -segment_wrap 9
-reset_timestamps 1 `
-segment_format_options max_delay=0
C:\Users\Jordan\Videos\FFmpeg\Right\Right%02d.ts

For one reason or another this command is not transcoding video in
real-time, which is a big issue when you're trying to record not simply
convert a file. When I omit one of the two outputs or if I halve the
resolution of each input / output everything works in real-time. This would
make me believe there is a bottleneck in the system somewhere but when
monitoring everything in task manager nothing is even close to capping out
(GPU encoder, cpu, ram, and SSD below 30% usage).

Furthermore when I try recording both streams in one 4K60 video via OBS
things work perfectly fine, as in real-time. So I don't understand how
transcoding two 1080p60 streams in FFmpeg would be anymore intensive than
one 4K60 stream in OBS... When looking at the GPUs encoding chip usage it
shows about 30% for the two 1080p60 streams on FFmpeg and 80% usage for
single 4K60 stream in OBS, yet the OBS recording is real-time while the
FFmpeg recording is at .6x speed.

Lastly I run a very similar command at home but instead of two 1080p60
streams I'm doing 2 4K60 streams, a 1080p60 stream, and another 2 audio
streams without issue. I do have a GTX 1080 though... however when
monitoring usage for the 1050 TI being used in this situation nothing
points to a GPU bottlecap.

Any insight would be greatly appreciated, been going at this for hours now
with no clue as to what could be causing this.

Full console output from a "minute" (according to the console, took 2
minutes of real time to get to that point) of recording:
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\Users\Jordan>
PS C:\Users\Jordan> ffmpeg -y -thread_queue_size 9999 -indexmem 9999
-guess_layout_max 0 -f dshow -rtbufsize 2147.48M `

-video_size 1920x1080 -framerate 60 `
-i

video="@device_pnp_\\?\usb#vid_07ca&pid_0570&mi_00#7&3886ab1a&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global":audio="@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{0A494693-5F33-4304-88D7-394757E09648}"
`

-thread_queue_size 9999 -indexmem 9999 -guess_layout_max 0 -f dshow

-rtbufsize 2147.48M `

-video_size 1920x1080 -framerate 60 `
-i

video="@device_pnp_\\?\usb#vid_07ca&pid_0570&mi_00#7&24df76f&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global":audio="@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{0E6F0DEF-2B29-4117-8D30-13F01160AC5B}"
`

-map 0:0,0:1 -map 0:1 -c:v h264_nvenc -r 60 -rc-lookahead 120

-forced-idr 1 -strict_gop 1 -sc_threshold 0 -flags +cgop `

-force_key_frames "expr:gte(t,n_forced*2)" -preset: llhp -pix_fmt nv12

-b:v 250M -minrate 250M -maxrate 250M `

-bufsize 250M -c:a aac -ar 44100 -b:a 384k -ac 2 -vsync 1 -ss

00:00:00.000 `

-max_muxing_queue_size 9999 -f segment -segment_time 600 -segment_wrap 9

-reset_timestamps 1 `

-segment_format_options max_delay=0

C:\Users\Jordan\Videos\FFmpeg\Left\Left%02d.ts `

-map 1:0,1:1 -map 1:1 -c:v h264_nvenc -r 60 -rc-lookahead 120

-forced-idr 1 -strict_gop 1 -sc_threshold 0 -flags +cgop `

-force_key_frames "expr:gte(t,n_forced*2)" -preset: llhp -pix_fmt nv12

-b:v 250M -minrate 250M -maxrate 250M `

-bufsize 250M -c:a aac -ar 44100 -b:a 384k -ac 2 -vsync 1 -ss

00:00:00.000 `

-max_muxing_queue_size 9999 -f segment -segment_time 600 -segment_wrap 9

-reset_timestamps 1 `

-segment_format_options max_delay=0

C:\Users\Jordan\Videos\FFmpeg\Right\Right%02d.ts
ffmpeg version N-92573-gc047901012 Copyright (c) 2000-2018 the FFmpeg
developers
built with gcc 8.2.1 (GCC) 20181201
configuration: --enable-gpl --enable-version3 --enable-sdl2
--enable-fontconfig --enable-gnutls --enable-iconv --enable-libass
--enable-libbluray --enable-libfreetype --enable-libmp3lame
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg
--enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr
--enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack
--enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2
--enable-libzimg --enable-lzma --enable-zlib --enable-gmp
--enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc
--enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom
--enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid
--enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2
--enable-avisynth --enable-libopenmpt
libavutil 56. 24.101 / 56. 24.101
libavcodec 58. 41.100 / 58. 41.100
libavformat 58. 23.102 / 58. 23.102
libavdevice 58. 6.101 / 58. 6.101
libavfilter 7. 46.101 / 7. 46.101
libswscale 5. 4.100 / 5. 4.100
libswresample 3. 4.100 / 3. 4.100
libpostproc 55. 4.100 / 55. 4.100
[mjpeg @ 000001a6974dc580] Found EOI before any SOF, ignoring
[mjpeg @ 000001a6974dc580] No JPEG data found in image
Input #0, dshow, from
'video=@device_pnp_\\?\usb#vid_07ca&pid_0570&mi_00#7&3886ab1a&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global:audio=@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{0A494693-5F33-4304-88D7-394757E09648}':
Duration: N/A, start: 1009.409000, bitrate: N/A
Stream #0:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), yuvj422p(pc,
bt470bg/unknown/unknown), 1920x1080, 60 fps, 60 tbr, 10000k tbn, 10000k tbc
Stream #0:1: Audio: pcm_s16le, 44100 Hz, 2 channels, s16, 1411 kb/s
[mjpeg @ 000001a697590e80] Found EOI before any SOF, ignoring
[mjpeg @ 000001a697590e80] No JPEG data found in image
Input #1, dshow, from
'video=@device_pnp_\\?\usb#vid_07ca&pid_0570&mi_00#7&24df76f&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global:audio=@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\wave_{0E6F0DEF-2B29-4117-8D30-13F01160AC5B}':
Duration: N/A, start: 1009.922000, bitrate: N/A
Stream #1:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), yuvj422p(pc,
bt470bg/unknown/unknown), 1920x1080, 60 fps, 60 tbr, 10000k tbn, 10000k tbc
Stream #1:1: Audio: pcm_s16le, 44100 Hz, 2 channels, s16, 1411 kb/s
Stream mapping:
Stream #0:0 -> #0:0 [sync #0:1] (mjpeg (native) -> h264 (h264_nvenc))
Stream #0:1 -> #0:1 (pcm_s16le (native) -> aac (native))
Stream #1:0 -> #1:0 [sync #1:1] (mjpeg (native) -> h264 (h264_nvenc))
Stream #1:1 -> #1:1 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
[mjpeg @ 000001a6975927c0] Found EOI before any SOF, ignoring
[mjpeg @ 000001a6975927c0] No JPEG data found in image
Error while decoding stream #0:0: Invalid data found when processing input
[swscaler @ 000001a6a174aa00] deprecated pixel format used, make sure you
did set range correctly
[segment @ 000001a6a0cc8180] Opening
'C:\Users\Jordan\Videos\FFmpeg\Left\Left00.ts' for writing/A
Output #0, segment, to 'C:\Users\Jordan\Videos\FFmpeg\Left\Left%02d.ts':
Metadata:
encoder : Lavf58.23.102
Stream #0:0: Video: h264 (h264_nvenc) (Main), nv12(progressive),
1920x1080, q=-1--1, 250000 kb/s, 60 fps, 90k tbn, 60 tbc
Metadata:
encoder : Lavc58.41.100 h264_nvenc
Side data:
cpb: bitrate max/min/avg: 250000000/0/250000000 buffer size:
250000000 vbv_delay: -1
Stream #0:1: Audio: aac (LC), 44100 Hz, stereo, fltp, 384 kb/s
Metadata:
encoder : Lavc58.41.100 aac
[mjpeg @ 000001a6a0b89d00] Found EOI before any SOF, ignoring
[mjpeg @ 000001a6a0b89d00] No JPEG data found in image
Error while decoding stream #1:0: Invalid data found when processing input
[swscaler @ 000001a6a1623a00] deprecated pixel format used, make sure you
did set range correctly
[segment @ 000001a6a1885140] Opening
'C:\Users\Jordan\Videos\FFmpeg\Right\Right00.ts' for writing
Output #1, segment, to 'C:\Users\Jordan\Videos\FFmpeg\Right\Right%02d.ts':
Metadata:
encoder : Lavf58.23.102
Stream #1:0: Video: h264 (h264_nvenc) (Main), nv12(progressive),
1920x1080, q=-1--1, 250000 kb/s, 60 fps, 90k tbn, 60 tbc
Metadata:
encoder : Lavc58.41.100 h264_nvenc
Side data:
cpb: bitrate max/min/avg: 250000000/0/250000000 buffer size:
250000000 vbv_delay: -1
Stream #1:1: Audio: aac (LC), 44100 Hz, stereo, fltp, 384 kb/s
Metadata:
encoder : Lavc58.41.100 aac
frame= 3939 fps= 31 q=13.0 Lq=13.0 size=N/A time=00:01:05.63 bitrate=N/A
dup=4 drop=0 speed=0.512x
video:1155166kB audio:6138kB subtitle:0kB other streams:0kB global
headers:0kB muxing overhead: unknown
[aac @ 000001a6a1882d00] Qavg: 377.927
[aac @ 000001a6a1882880] Qavg: 378.411
PS C:\Users\Jordan>
_______________________________________________
ffmpeg-user mailing list
ffmpeg-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-***@ffmpeg.org with sub

[Gabriel Balaich]

Post by Gabriel Balaich
Furthermore when I try recording both streams in one 4K60 video via OBS
things work perfectly fine, as in real-time. So I don't understand how
transcoding two 1080p60 streams in FFmpeg would be anymore intensive than
one 4K60 stream in OBS... When looking at the GPUs encoding chip usage it
shows about 30% for the two 1080p60 streams on FFmpeg and 80% usage for
single 4K60 stream in OBS, yet the OBS recording is real-time while the
FFmpeg recording is at .6x speed.
Lastly I run a very similar command at home but instead of two 1080p60
streams I'm doing 2 4K60 streams, a 1080p60 stream, and another 2 audio
streams without issue. I do have a GTX 1080 though... however when
monitoring usage for the 1050 TI being used in this situation nothing
points to a GPU bottlecap.

Update: we swapped the 1050ti for a GTX 1080 to no avail. It seems fairly
obvious at this point that it is the CPU causing problems. However this
still makes little sense, as stated I use a 6800K and do much more that
what I'm trying to do on this build. The 8600K should have higher single
core performance... However, the 6800K has Hyper Threading wile the 8600K
does not, is it possible that FFmpeg better utilizes a Hyper Threaded
processor?

It looks to be only hitting 1-2 threads, anyway to force it to use more?
-threads doesn't seem to do anything regardless of where I insert it into
the command.

Lastly would it be possible to utilize hardware accelerated decode via GPU
to alleviate the issue? When I try to instert -hwaccel cuvid -v:c
h264_cuvid before the inputs like tutorials would suggest the recording
immediately errors out saying "Cannot set format options for [capture
card]". And when I remove -v:c h264_cuvid and just use -hwaccel cuvid I get
and endless string of errors stating "yuvj422p is incompatible", I'm
assuming that is the capture cards native pixel format. When I try to set
the capture cards pixel format to NV12 it errors out again with "Cannot set
format options for [capture card]", not sure what other pixel formats GPU
acceleration supports but I assume I'm just out of luck with this capture
cards limited pixel format options.

Once again any insight would be much appreciated.
_______________________________________________
ffmpeg-user mailing list
ffmpeg-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-***@ffmpeg.org with

[James Girotti]

Post by Gabriel Balaich

Post by Gabriel Balaich
Furthermore when I try recording both streams in one 4K60 video via OBS
things work perfectly fine, as in real-time. So I don't understand how
transcoding two 1080p60 streams in FFmpeg would be anymore intensive than
one 4K60 stream in OBS... When looking at the GPUs encoding chip usage it
shows about 30% for the two 1080p60 streams on FFmpeg and 80% usage for
single 4K60 stream in OBS, yet the OBS recording is real-time while the
FFmpeg recording is at .6x speed.

This makes me think it's something with your commandline. I presume OBS
does not call the ffmpeg binary, but rather uses the APIs. So it might not
be possible to tell exactly what settings it uses.

Post by Gabriel Balaich

Post by Gabriel Balaich
Lastly I run a very similar command at home but instead of two 1080p60
streams I'm doing 2 4K60 streams, a 1080p60 stream, and another 2 audio
streams without issue. I do have a GTX 1080 though... however when
monitoring usage for the 1050 TI being used in this situation nothing
points to a GPU bottlecap.

Theoretically, the 1050Ti should be able to encode streams up to 8K (that's
what I've gathered from Nvidia's docs).

Post by Gabriel Balaich
Update: we swapped the 1050ti for a GTX 1080 to no avail. It seems fairly
obvious at this point that it is the CPU causing problems. However this
still makes little sense, as stated I use a 6800K and do much more that
what I'm trying to do on this build. The 8600K should have higher single
core performance... However, the 6800K has Hyper Threading wile the 8600K
does not, is it possible that FFmpeg better utilizes a Hyper Threaded
processor?
It looks to be only hitting 1-2 threads, anyway to force it to use more?
-threads doesn't seem to do anything regardless of where I insert it into
the command.

The GPU encoder is "single-thread", so it is the bottleneck as far as
multi-threading.

Post by Gabriel Balaich
Lastly would it be possible to utilize hardware accelerated decode via GPU
to alleviate the issue? When I try to instert -hwaccel cuvid -v:c
h264_cuvid before the inputs like tutorials would suggest the recording
immediately errors out saying "Cannot set format options for [capture
card]". And when I remove -v:c h264_cuvid and just use -hwaccel cuvid I get
and endless string of errors stating "yuvj422p is incompatible", I'm
assuming that is the capture cards native pixel format. When I try to set
the capture cards pixel format to NV12 it errors out again with "Cannot set
format options for [capture card]", not sure what other pixel formats GPU
acceleration supports but I assume I'm just out of luck with this capture
cards limited pixel format options.

No, your input is MJPEG. You cannot use "-hwaccel cuvid -c:v h264_cuvid".
That would be if your input was h264. There is no hw decoder for MJPEG.
That's probably okay though and may be better (I don't want to get into
that part). It's highly likely that your CPU should be able to decode fast
enough.

Post by Gabriel Balaich
Once again any insight would be much appreciated.

It's hard for me to understand your complete commandline, there are a lot
of options. It seems to me that some of your options are not optimum, but
they may be the settings that you want/need, which is fine that's up to you
to decide (I don't quite understand why you'd want to output in NV12, but
that's your business). I suggest that you drastically simplify your
commandline. It will be easier to troubleshoot. You might also try saving
the input directly to a file without any transcoding. Then use that file to
test with at first to see how many FPS you can get. Something like:

ffmpeg -i "whatever_your_input_is" -map 0 -c copy -t 60 input1.ts

ffmpeg -i input1.ts -c:v h264_nvenc output1.ts

If that performs well, then try two inputs. If that performs well, then try
the "live" input, etc.

Best of Luck,
-J
_______________________________________________
ffmpeg-user mailing list
ffmpeg-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-***@ffmpeg.org with subject "unsubscribe

[Gabriel Balaich]

Post by James Girotti
It's hard for me to understand your complete commandline, there are a lot
of options. It seems to me that some of your options are not optimum, but
they may be the settings that you want/need, which is fine that's up to you
to decide (I don't quite understand why you'd want to output in NV12, but
that's your business). I suggest that you drastically simplify your
commandline. It will be easier to troubleshoot. You might also try saving
the input directly to a file without any transcoding. Then use that file to
ffmpeg -i "whatever_your_input_is" -map 0 -c copy -t 60 input1.ts
ffmpeg -i input1.ts -c:v h264_nvenc output1.ts
If that performs well, then try two inputs. If that performs well, then try
the "live" input, etc.

Thanks for the reply!

Basically every option in the command is put in place to keep the two parts
syncronized (A/V) with themselves and each other during segmentation. I run
basically the same command on my computer but with more sources, it took
months and months to figure out how to keep everything synchronized.

That being said I definitely should have simplified the command before
asking for help, I guess my thought process was I need this specific
command working not a simpler one, but that makes it extremely hard to
troubleshoot. I eventually simplified things down to this:

ffmpeg -y -hide_banner -thread_queue_size 9999 -f dshow -rtbufsize 2147.48M
-video_size 1920x1080 -framerate 60 `
-i
video="@device_pnp_\\?\usb#vid_07ca&pid_0570&mi_00#7&3886ab1a&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global"
`
-thread_queue_size 9999 -f dshow -rtbufsize 2147.48M -video_size 1920x1080
-framerate 60 `
-i
video="@device_pnp_\\?\usb#vid_07ca&pid_0570&mi_00#7&24df76f&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global"
`
-map 0 -c:v h264_nvenc -r 60 -preset: llhp -pix_fmt yuv420p -b:v 1M
-minrate 1M -maxrate 1M -bufsize 1M `
C:\Users\Jordan\Videos\FFmpeg\Left\Left.ts `
-map 1 -c:v h264_nvenc -r 60 -preset: llhp -pix_fmt yuv420p -b:v 1M
-minrate 1M -maxrate 1M -bufsize 1M `
C:\Users\Jordan\Videos\FFmpeg\Right\Right.ts

And the same issues persisted, also my color format was nv12 at the time
because I was just trying to test things, oops.

I went on to swap out the CPU, RAM, SSD, and nothing fixed the issue
despite the same command working on my PC with the same software and
drivers... However eventually I pulled one of the caputre cards from his
system and put it in mine and was able to replicate the issue, this was not
reproducible with any of my other capture cards and I have 4 of them.

The issue seems to be caused by Avermedia's stream engine bloatware. The
card actually shows as two devices, one without stream engine in the name
and one with, if you select the one with the issues are alleviated but
still present. In retrospect bloatware / unconventional firmware easily
explains the extremely hard to diagnose issues. This still doesn't quite
explain why they work perfectly in OBS but not FFmpeg, I mean the stream
engine software does seem to be pretty clearly built for OBS... But
honestly at this point I don't care, I've spent way too much time trying to
get these cards to work while knowing other ones work right out the gate.

We decided to just put a new order in for a couple Elgato HD60 Pros which I
can confirm do not exhibit the same behavior. We originally purchase the
Avermedia Live Gamer HD 2s because they were on deal for $100 and had
pretty much identical features to the Elgatos on paper.
_______________________________________________
ffmpeg-user mailing list
ffmpeg-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-***@ffmpeg.org with subject "unsubscri