Discussion:
[FFmpeg-user] Dropping 1 frame every n seconds
Steve
2015-05-31 09:46:18 UTC
Permalink
I would like to try and test changing the tempo of the audio in a video without losing sync and without having to blend or merge frames.

At 25fps, dropping 1 frame would allow the the audio to be pitched/time stretched by 4% and dropping 1 frame every 2 seconds would allow for 2%

I understand how to modify the audio with atempo but am not sure of the best method to drop one frame every second or every other second - is it possible with a standard filter command?
Paul B Mahol
2015-05-31 12:38:02 UTC
Permalink
Post by Steve
I would like to try and test changing the tempo of the audio in a video
without losing sync and without having to blend or merge frames.
Post by Steve
At 25fps, dropping 1 frame would allow the the audio to be pitched/time
stretched by 4% and dropping 1 frame every 2 seconds would allow for 2%
Post by Steve
I understand how to modify the audio with atempo but am not sure of the
best method to drop one frame every second or every other second - is it
possible with a standard filter command?

See select and framestep filter documentation.
Post by Steve
_______________________________________________
ffmpeg-user mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Moritz Barsnick
2015-05-31 22:18:47 UTC
Permalink
Post by Steve
Post by Steve
At 25fps, dropping 1 frame would allow the the audio to be pitched/time
stretched by 4% and dropping 1 frame every 2 seconds would allow for 2%
See select and framestep filter documentation.
(I may be wrong, but:)

This isn't complete, the video won't play faster after dropping the
frames, due to the timestamps of the retained frames. I think that
after dropping one in N frames, you need to increase the speed using
the setpts filter with a factor of (N-1)/N.

setpts=24/25*PTS

Just figuring this out from the top of my head, it's late. ;-)

Moritz
Steve
2015-05-31 22:51:54 UTC
Permalink
Post by Moritz Barsnick
Post by Steve
Post by Steve
At 25fps, dropping 1 frame would allow the the audio to be pitched/time
stretched by 4% and dropping 1 frame every 2 seconds would allow for 2%
See select and framestep filter documentation.
(I may be wrong, but:)
This isn't complete, the video won't play faster after dropping the
frames, due to the timestamps of the retained frames. I think that
after dropping one in N frames, you need to increase the speed using
the setpts filter with a factor of (N-1)/N.
setpts=24/25*PTS
Just figuring this out from the top of my head, it's late. ;-)
Hi Moritz,

Sorry I probably should have explained better, I don’t just want to drop the frame and leave it blank or adjust the frame rate from 25 to 24fps. Essentially I want to remove it, then bring the following frames forward one frame to fill it’s place. So the 25fps rate is maintained, but every nth second, one frame is dropped and the adjacent frames brought forward by one frame. I hope this will allow the picture to keep in sync with the sound once the audio is speeded up.

I guess I could a/ export to an image sequence b/ delete the relevant frames c/ renumber the remaining frames to be sequential and then d/ re-encode it at 25fps with the modified audio but I wonder if that approach is overkill and whether a filter can be configured to do it?

One other thing I forgot to add, the source is all i-frame only so no problem with dropping or moving p or b frames

Many Thanks

Steve
Post by Moritz Barsnick
Moritz
_______________________________________________
ffmpeg-user mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Moritz Barsnick
2015-06-01 13:07:30 UTC
Permalink
Hi Steve,
Post by Steve
Sorry I probably should have explained better, I don’t just want to
drop the frame and leave it blank or adjust the frame rate from 25 to
24fps. Essentially I want to remove it, then bring the following
frames forward one frame to fill it’s place. So the 25fps rate is
maintained, but every nth second, one frame is dropped and the
adjacent frames brought forward by one frame. I hope this will allow
the picture to keep in sync with the sound once the audio is speeded
up.
I think I understood quite well from the fact that you wish to speed up
your video and audio by 2 or 4 percent. But please be more precise,
even if the answers can be generic. What are you actually trying to
achieve? What's the purpose?

May I guess this: For whatever reason, you want to speed up your video,
e.g. by two or four percent. You want video and audio to keep in sync,
and you want to retain the original frame rate. Right?

General examples are given here:
https://trac.ffmpeg.org/wiki/How%20to%20speed%20up%20/%20slow%20down%20a%20video

The first thing to note - which those examples hint at - it that ffmpeg
apparently retains the frame rate automatically, if not told otherwise
by "-r".

Why I chose to suggest:
setpts=24/25*PTS
is because you were suggesting to drop 1 in 25 frames. I was only
showing how to speed up the remaining 24 frames, by achieving 25/24ths
of the rate. Now, mathematically speaking, if you speed up audio by 4%,
you would have to choose 26/25ths of the speed, dropping one in 26
frames! ;-) Important math for sync.

So you probably want:
setpts=25/26*PTS

ffmpeg automatically (in my experiments) tries to retain the rate of 25
fps, and does this by dropping or adding frames as necessary. (Doc for
"-r" says: "As an output option, duplicate or drop input frames to
achieve constant output frame rate fps.") What I don't know is
_exactly_ which frames ffmpeg drops. I assume those of which the PTS is
too far off 1/fps. But the results should be fine for you. I
experimented with a 25fps video with atempo=2.0 and setpts=0.5*PTS, and
the result was fine, i.e. had 25fps, was in sync, was double speed, and
had half the amount of frames of the original video.
Post by Steve
I guess I could a/ export to an image sequence b/ delete the relevant
frames c/ renumber the remaining frames to be sequential and then d/
re-encode it at 25fps with the modified audio but I wonder if that
approach is overkill and whether a filter can be configured to do it?
That's what you would do if you dropped frames with the select filter
and let setpts calculate new consecutive PTS for you:

select=mod(n\,26),setpts=N/(25*TB)

(Untested.)
The video would look jumpy once per second (missing frame), but PTS
would be exactly 1/25 second apart for each frame. I don't know whether
that's important for any formats or players.
Post by Steve
One other thing I forgot to add, the source is all i-frame only so no
problem with dropping or moving p or b frames
No, frame dropping and/or recalculation of PTS means re-encoding
anyway, so the original structure is not taken into consideration.

HTH,
Moritz

P.S.: Question to the developers: Is PTS really to do with frame
encoding, or could it also be a bitstream filter, and thereby
applicable with stream copy?
Steve
2015-06-01 13:27:09 UTC
Permalink
Post by Moritz Barsnick
Hi Steve,
Post by Steve
Sorry I probably should have explained better, I don’t just want to
drop the frame and leave it blank or adjust the frame rate from 25 to
24fps. Essentially I want to remove it, then bring the following
frames forward one frame to fill it’s place. So the 25fps rate is
maintained, but every nth second, one frame is dropped and the
adjacent frames brought forward by one frame. I hope this will allow
the picture to keep in sync with the sound once the audio is speeded
up.
I think I understood quite well from the fact that you wish to speed up
your video and audio by 2 or 4 percent. But please be more precise,
even if the answers can be generic. What are you actually trying to
achieve? What's the purpose?
May I guess this: For whatever reason, you want to speed up your video,
e.g. by two or four percent. You want video and audio to keep in sync,
and you want to retain the original frame rate. Right?
https://trac.ffmpeg.org/wiki/How%20to%20speed%20up%20/%20slow%20down%20a%20video
The first thing to note - which those examples hint at - it that ffmpeg
apparently retains the frame rate automatically, if not told otherwise
by "-r".
setpts=24/25*PTS
is because you were suggesting to drop 1 in 25 frames. I was only
showing how to speed up the remaining 24 frames, by achieving 25/24ths
of the rate. Now, mathematically speaking, if you speed up audio by 4%,
you would have to choose 26/25ths of the speed, dropping one in 26
frames! ;-) Important math for sync.
setpts=25/26*PTS
ffmpeg automatically (in my experiments) tries to retain the rate of 25
fps, and does this by dropping or adding frames as necessary. (Doc for
"-r" says: "As an output option, duplicate or drop input frames to
achieve constant output frame rate fps.") What I don't know is
_exactly_ which frames ffmpeg drops. I assume those of which the PTS is
too far off 1/fps. But the results should be fine for you. I
experimented with a 25fps video with atempo=2.0 and setpts=0.5*PTS, and
the result was fine, i.e. had 25fps, was in sync, was double speed, and
had half the amount of frames of the original video.
Post by Steve
I guess I could a/ export to an image sequence b/ delete the relevant
frames c/ renumber the remaining frames to be sequential and then d/
re-encode it at 25fps with the modified audio but I wonder if that
approach is overkill and whether a filter can be configured to do it?
That's what you would do if you dropped frames with the select filter
select=mod(n\,26),setpts=N/(25*TB)
(Untested.)
The video would look jumpy once per second (missing frame), but PTS
would be exactly 1/25 second apart for each frame. I don't know whether
that's important for any formats or players.
Post by Steve
One other thing I forgot to add, the source is all i-frame only so no
problem with dropping or moving p or b frames
No, frame dropping and/or recalculation of PTS means re-encoding
anyway, so the original structure is not taken into consideration.
HTH,
Moritz
Hi Moritz,

Thank you for the detailed answer, I think that should work, I’ll experiment and report back.

As for the application - it’s music videos, we want to pitch the audio to speed them up slightly as radio do with audio tracks.

Many Thanks

Steve
Post by Moritz Barsnick
P.S.: Question to the developers: Is PTS really to do with frame
encoding, or could it also be a bitstream filter, and thereby
applicable with stream copy?
_______________________________________________
ffmpeg-user mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Moritz Barsnick
2015-06-01 14:45:15 UTC
Permalink
Post by Steve
Thank you for the detailed answer, I think that should work, I’ll
experiment and report back.
Yes, please report back. You're welcome. :-)
Post by Steve
As for the application - it’s music videos, we want to pitch the
audio to speed them up slightly as radio do with audio tracks.
If I were a musician and considered myself an artist, I'd try to kill
you for doing this. ;-) (Radio does this? Wow. But then again, they do
a lot of things to kill sound anyway.)

Cheers,
Moritz

Continue reading on narkive:
Loading...