[FFmpeg-user] Mixing two audio streams in one video

[Dainius (GreatEmerald)]

I am trying to capture a video in which I need to have one video
stream, that is, screen capture, and two audio streams, namely system
sounds and the microphone, using OpenSUSE Linux 11.3 through the
command line. If my command line looks like this (testing with sound
only at the moment):

/usr/bin/ffmpeg \
-xerror \
-f alsa \
-ac 2 \
-i hw:1,0 \
-f alsa \
-i hw:0,0 \
-acodec pcm_s16le \
-y /home/dainius/temp/test.wav

Only the first stream gets recorded. It reports that both streams are
being captured, but only the first one is being recorded and output to
file:

[alsa @ 0x6376d0] Estimating duration from bitrate, this may be inaccurate
Input #0, alsa, from 'hw:1,0':
Duration: N/A, start: 7886.368401, bitrate: N/A
Stream #0.0: Audio: pcm_s16le, 44100 Hz, 2 channels, s16, 1411 kb/s
[alsa @ 0x638950] Estimating duration from bitrate, this may be inaccurate
Input #1, alsa, from 'hw:0,0':
Duration: N/A, start: 7886.381408, bitrate: N/A
Stream #1.0: Audio: pcm_s16le, 44100 Hz, 2 channels, s16, 1411 kb/s
Output #0, wav, to '/home/dainius/temp/test.wav':
Metadata:
encoder : Lavf52.83.0
Stream #0.0: Audio: pcm_s16le, 44100 Hz, 2 channels, s16, 1411 kb/s
Stream mapping:
Stream #0.0 -> #0.0

As you can tell, I use ALSA as the sound source (hw:1,0 is the
microphone - and yes, it it indeed a stereo microphone - while hw:0,0
is system audio). Is there a way to mix both these streams together
into one? Or maybe output the streams into different files, and then
mux them together later?

[James Darnley]

Post by Dainius (GreatEmerald)
As you can tell, I use ALSA as the sound source (hw:1,0 is the
microphone - and yes, it it indeed a stereo microphone - while hw:0,0
is system audio). Is there a way to mix both these streams together
into one? Or maybe output the streams into different files, and then
mux them together later?

If you want to mix two input streams, use sox. If you just want to
record the two streams, use a container that will support more than
one stream and use -newaudio

[Dainius (GreatEmerald)]

Ah, I see. How do you construct the sox pipe exactly? First pipe part
is ffmpeg with input options, then sox with mixing options, and then
ffmpeg again with output options?

[James Darnley]

Post by Dainius (GreatEmerald)
Ah, I see. How do you construct the sox pipe exactly? First pipe part
is ffmpeg with input options, then sox with mixing options, and then
ffmpeg again with output options?

You could do ffmpeg | sox | ffmpeg but I don't know if the sox pipe
format supports more than one stream. A far simpler way would be to
record in sox, mix it there, then pipe out to ffmpeg.

[Dainius (GreatEmerald)]

I see, so use SoX as an input for FFmpeg. So the command should be
something like "sox <record and mix here> - | ffmpeg -f yuv4mpegpipe
-i - <video input and output here>" right? But how do you record two
streams from different devices in SoX itself? I'm finding very little
information on this subject. It seems that soxmix is the one I should
use, but I can't find its syntax anywhere. But if I'm getting this
right, the command line for me should be something like "soxmix -t
alsa hw:1,0 hw:0,0 - |" right?

[James Darnley]

Post by Dainius (GreatEmerald)
I see, so use SoX as an input for FFmpeg. So the command should be
something like "sox <record and mix here> - | ffmpeg -f yuv4mpegpipe
-i - <video input and output here>" right?

No. sox INPUT1 INPUT2 -p FILTERS | ffmpeg -f sox -i - ...

Post by Dainius (GreatEmerald)
But how do you record two
streams from different devices in SoX itself? I'm finding very little
information on this subject. It seems that soxmix is the one I should
use, but I can't find its syntax anywhere. But if I'm getting this
right, the command line for me should be something like "soxmix -t
alsa hw:1,0 hw:0,0 - |" right?

I don't know about that one, you would have to read the docs about
alsa use through it.

[Dainius (GreatEmerald)]

All right, it works, thanks a lot! Just for the record, this is how I set it up:

sox -t alsa hw:0,0 -t alsa hw:1,0 -m -p gain | /usr/bin/ffmpeg -f
x11grab -xerror -r 25 -s 808x628 -i :0.0+0,408 -vcodec flv -qmax 5 -f
sox -i - -acodec pcm_s16le -y /home/dainius/temp/test.avi

The SoX command mixes ALSA hw:0,0 and hw:1,0, applies gain, then pipes
it to FFmpeg.