[FFmpeg-user] Scene detection

[Fori Pepe]

Hi,

I would like to generate images from videos at every scene changes.
My present solution is:
- ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
- lav2yuv -S list.el destination_video.avi
It generates a scene list text file with time codes that I use later to creating images.

I was wondering maybe there is an easier solution.
I have browsed the net to get a better solution (ffmpeg, sc_threshold etc.), but I couldn't find any other just my present lav2yus solution.

Is it possible to use ffmpeg to create images on scene changes?
Or is it possible to use ffmpeg to create a text file with the time codes of scene changes?

If you have any idea, I would be very happy.
Thanks in advance.

[Bouke]

---- Original Message -----
From: "Fori Pepe" <foripepe at yahoo.com>
To: <ffmpeg-user at mplayerhq.hu>
Sent: Thursday, October 01, 2009 5:25 PM
Subject: [FFmpeg-user] Scene detection

Post by Fori Pepe
Hi,
I would like to generate images from videos at every scene changes.
- ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
- lav2yuv -S list.el destination_video.avi
It generates a scene list text file with time codes that I use later to creating images.
I was wondering maybe there is an easier solution.
I have browsed the net to get a better solution (ffmpeg, sc_threshold
etc.), but I couldn't find any other just my present lav2yus solution.
Is it possible to use ffmpeg to create images on scene changes?
Or is it possible to use ffmpeg to create a text file with the time codes of scene changes?
If you have any idea, I would be very happy.

No idea with FFmpeg, but a couple of years ago i had a similar need, and
i've created this:
http://www.videotoolshed.com/?page=products&pID=13

Bouke

Post by Fori Pepe
Thanks in advance.
_______________________________________________
ffmpeg-user mailing list
ffmpeg-user at mplayerhq.hu
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[Stefano Sabatini]

Post by Fori Pepe
Hi,
I would like to generate images from videos at every scene changes.
- ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
- lav2yuv -S list.el destination_video.avi
It generates a scene list text file with time codes that I use later to creating images.
I was wondering maybe there is an easier solution.
I have browsed the net to get a better solution (ffmpeg, sc_threshold etc.), but I couldn't find any other just my present lav2yus solution.
Is it possible to use ffmpeg to create images on scene changes?
Or is it possible to use ffmpeg to create a text file with the time codes of scene changes?
If you have any idea, I would be very happy.

It should be not too hard to implement a scene detection filter in
libavfilter.

Regards.

[hvidal]

Post by Stefano Sabatini

Post by Fori Pepe
Hi,
I would like to generate images from videos at every scene changes.
- ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
- lav2yuv -S list.el destination_video.avi
It generates a scene list text file with time codes that I use later to creating images.
I was wondering maybe there is an easier solution.
I have browsed the net to get a better solution (ffmpeg, sc_threshold etc.), but I couldn't find any other just my present lav2yus solution.
Is it possible to use ffmpeg to create images on scene changes?
Or is it possible to use ffmpeg to create a text file with the time codes of scene changes?
If you have any idea, I would be very happy.

It should be not too hard to implement a scene detection filter in
libavfilter.

Perhaps a clarification is in order: when FP says
'at every scene change,' is the idea to generate a
time code or an event or an automatic cut when the
video 'content' changes? Or is this somehow encoded in
the video stream as control/time data?

Pardon the dumb question, but what is the goal? Is this
an artificial vision kind of problem or a 'see-the-marker-
and-cut/edit' problem?

h

[Stefano Sabatini]

Post by hvidal

Post by Stefano Sabatini

Post by Fori Pepe
Hi,
I would like to generate images from videos at every scene changes.
- ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
- lav2yuv -S list.el destination_video.avi
It generates a scene list text file with time codes that I use later to creating images.
I was wondering maybe there is an easier solution.
I have browsed the net to get a better solution (ffmpeg, sc_threshold etc.), but I couldn't find any other just my present lav2yus solution.
Is it possible to use ffmpeg to create images on scene changes?
Or is it possible to use ffmpeg to create a text file with the time codes of scene changes?
If you have any idea, I would be very happy.

It should be not too hard to implement a scene detection filter in
libavfilter.

Perhaps a clarification is in order: when FP says
'at every scene change,' is the idea to generate a
time code or an event or an automatic cut when the
video 'content' changes? Or is this somehow encoded in
the video stream as control/time data?

Cut-scenes are usually not hardcoded in the stream at least not
directly, but some encoders may decide for example to use an I-frame
when they detect a scene change, for improve the overall compression
ratio.

Post by hvidal
Pardon the dumb question, but what is the goal? Is this
an artificial vision kind of problem or a 'see-the-marker-
and-cut/edit' problem?

You could store the various first frames of each scene in order to get
a sort of photographic storyline of a movie, or you could store the
timestamps somewhere in order to use that to automatically cut every
movie 'shot' with a video editor.

Both options should be possible with FFmpeg+libavfilter.

Regards.

[Leonardo Boselli]

Il 2 Oct 2009 alle 18:05 Bouke immise in rete

Post by Bouke

Is it possible to use ffmpeg to create images on scene changes? Or
is it possible to use ffmpeg to create a text file with the time
codes of scene changes?
If you have any idea, I would be very happy.

No idea with FFmpeg, but a couple of years ago i had a similar need,
http://www.videotoolshed.com/?page=products&pID=13

If you have a DV file whith timecode could joust have a piece of
software that split the file on each timecode jump ??? --
Leonardo Boselli
Nucleo Informatico e Telematico del Dipartimento Ingegneria Civile
Universita` di Firenze , V. S. Marta 3 - I-50139 Firenze
tel [phone] cell [phone] fax [phone]
http://www.dicea.unifi.it/~leo

[Developer]

I would think outputting an ".idx" file, with the TS location for each
scene change as well as a specific >out message line would be a better
solution?

this would allow multiple scenarios for use, as everyone has different
needs and could apply the solution to their need easily with just having
this information.

(1) grab a still 5 seconds > TS value
(2) splice video at exact spot
(3) tag as chapter in MP4

One could monitor the output of ffmpeg for SCENECHANGE event with the
Timestamp, or one could look for the idx file as it being written.
Allows for both inline and out of process manipulation then?

Just 2 thoughts

Shawn

Post by Leonardo Boselli
Il 2 Oct 2009 alle 18:05 Bouke immise in rete

Post by Bouke

Is it possible to use ffmpeg to create images on scene changes? Or
is it possible to use ffmpeg to create a text file with the time
codes of scene changes?
If you have any idea, I would be very happy.

No idea with FFmpeg, but a couple of years ago i had a similar need,
http://www.videotoolshed.com/?page=products&pID=13

If you have a DV file whith timecode could joust have a piece of
software that split the file on each timecode jump ??? --
Leonardo Boselli
Nucleo Informatico e Telematico del Dipartimento Ingegneria Civile
Universita` di Firenze , V. S. Marta 3 - I-50139 Firenze
tel [phone] cell [phone] fax [phone]
http://www.dicea.unifi.it/~leo
_______________________________________________
ffmpeg-user mailing list
ffmpeg-user at mplayerhq.hu
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[Bouke]

----- Original Message -----
From: "Leonardo Boselli" <leo at dicea.unifi.it>
To: "Bouke" <bouke at editb.nl>; "FFmpeg user questions and RTFMs"
<ffmpeg-user at mplayerhq.hu>
Sent: Saturday, October 03, 2009 4:27 PM
Subject: Re: [FFmpeg-user] Scene detection

Post by Leonardo Boselli
Il 2 Oct 2009 alle 18:05 Bouke immise in rete

Post by Bouke

Is it possible to use ffmpeg to create images on scene changes? Or
is it possible to use ffmpeg to create a text file with the time
codes of scene changes?
If you have any idea, I would be very happy.

No idea with FFmpeg, but a couple of years ago i had a similar need,
http://www.videotoolshed.com/?page=products&pID=13

If you have a DV file whith timecode could joust have a piece of
software that split the file on each timecode jump ??? --

Not sure if this is possible with FFmpeg, but i could make something that
splits the file on each shot break.
Not sure what you mean with Timecode. QT timecode cannot have breaks. I'm
not sure how the timecode is stored in a DV file, nor if it is possible with
that.
Or are you talking about LTC on an audio channel?

Bouke

Post by Leonardo Boselli
Leonardo Boselli
Nucleo Informatico e Telematico del Dipartimento Ingegneria Civile
Universita` di Firenze , V. S. Marta 3 - I-50139 Firenze
tel [phone] cell [phone] fax [phone]
http://www.dicea.unifi.it/~leo
_______________________________________________
ffmpeg-user mailing list
ffmpeg-user at mplayerhq.hu
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[Fori Pepe]

Hi,

Thanks the responses for everybody.
What I really want is ffmpeg detect the scene changes (using I-frame?), and ffmpeg create an image on these points, or just gives me a list about these time codes.

It would be the most comfortable to use it like this:
ffmpeg -i test.mpg --detect_scene_changes scenes.txt

Do you think is it possible?

Post by Stefano Sabatini
Cut-scenes are usually not hardcoded in the stream at least not
directly, but some encoders may decide for example to use an I-frame
when they detect a scene change, for improve the overall compression
ratio.
You could store the various first frames of each scene in order to get
a sort of photographic storyline of a movie, or you could store the
timestamps somewhere in order to use that to automatically cut every
movie 'shot' with a video editor.
Both options should be possible with FFmpeg+libavfilter.
Hi,
I would like to generate images from videos at every scene changes.
- ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
- lav2yuv -S list.el destination_video.avi
It generates a scene list text file with time codes that I use later to
creating images.
I was wondering maybe there is an easier solution.
I have browsed the net to get a better solution (ffmpeg, sc_threshold
etc.), but I couldn't find any other just my present lav2yus solution.
Is it possible to use ffmpeg to create images on scene changes?
Or is it possible to use ffmpeg to create a text file with the time codes
of scene changes?
If you have any idea, I would be very happy.

[Stefano Sabatini]

Post by Fori Pepe
Hi,
Thanks the responses for everybody.
What I really want is ffmpeg detect the scene changes (using
I-frame?), and ffmpeg create an image on these points, or just gives
me a list about these time codes.

Using I-frames+sc_threshold is not a reliable way to detect scene
changes, and it only works with codec supporting I-frames (for example
it wouldn't work with rawvideo).

Anyway you can use ffprobe -show_frames to detect the I-frames and
their timestamps in a multimedia stream.

Post by Fori Pepe
ffmpeg -i test.mpg --detect_scene_changes scenes.txt
Do you think is it possible?

The fake syntax for a lavfi filter (BIG WARNING, libavfilter is not
still properly integrated into FFmpeg, check the soc repo) would be
something of the kind:

ffmpeg -i in.avi -vfilters "[IN] split [SC_DETECT] [OUT], [SC_DETECT] sc_detect=PARAMS, process_sc_frames=PARAMS" /dev/null

As you see you would need two filters, one detecting scene changes and
outputting the detected scene change frames to the output, and a
filter to furtherly processes the information of the frames, for
example it could print to a file the timestamp of the file or to
render as an image file the processed frames.

The sc_detect filter implementation is the tricky part, but
a naive implementation shouldn't be too hard to accomplish.

Regards.

[lovelove]

Hi, can someone please help a desperate soul like me?

I am in the same situation as Fori Pepe

I would like to export/extract images from videos at every scene changes.
Is it possible to use ffmpeg to create images on scene changes?
ffmpeg -i test.mpg --detect_scene_changes newscene[00-99].png

... i.e. export first frame of each new scene as an image. This effectively
generates a storyboard.
It also would be an enormous help in identifying added and deleted scenes
when comparing 2 videos.

ffmpeg -i in.avi -vfilters "[IN] split [SC_DETECT] [OUT], [SC_DETECT]
sc_detect=PARAMS, process_sc_frames=PARAMS" /dev/null
As you see you would need two filters, one detecting scene changes and
outputting the detected scene change frames to the output, and a
filter to furtherly processes the information of the frames, for
example it could print to a file the timestamp of the file or to
render as an image file the processed frames.
The sc_detect filter implementation is the tricky part, but
a naive implementation shouldn't be too hard to accomplish.

I am really a beginner, so I don't quite know what to do with that. All I
know is I have been looking for >1 year on the www, from top to bottom and
could not find anything.

I would be so thankful to anybody of you here, if you could tell me what to
do.
with my best regards, Christian Davideck


--
View this message in context: http://ffmpeg-users.933282.n4.nabble.com/Scene-detection-tp941845p3724931.html
Sent from the FFmpeg-users mailing list archive at Nabble.com.

[Mark Himsley]

Post by lovelove
Hi, can someone please help a desperate soul like me?
I am in the same situation as Fori Pepe

I would like to export/extract images from videos at every scene changes.
Is it possible to use ffmpeg to create images on scene changes?
ffmpeg -i test.mpg --detect_scene_changes newscene[00-99].png

... i.e. export first frame of each new scene as an image. This effectively
generates a storyboard.
It also would be an enormous help in identifying added and deleted scenes
when comparing 2 videos.

ffmpeg -i in.avi -vfilters "[IN] split [SC_DETECT] [OUT], [SC_DETECT]
sc_detect=PARAMS, process_sc_frames=PARAMS" /dev/null
As you see you would need two filters, one detecting scene changes and
outputting the detected scene change frames to the output, and a
filter to furtherly processes the information of the frames, for
example it could print to a file the timestamp of the file or to
render as an image file the processed frames.
The sc_detect filter implementation is the tricky part, but
a naive implementation shouldn't be too hard to accomplish.

I wonder if any code from the BBC R&D's open source (LGPL) Video Shot
Change Detector could be used?

http://www.bbc.co.uk/opensource/projects/shot_change/

[lovelove]

Post by Mark Himsley
I wonder if any code from the BBC R&D's open source (LGPL) Video Shot
Change Detector could be used?
http://www.bbc.co.uk/opensource/projects/shot_change/

The netiquette says no smalltalk, still I want to say how grateful I am for
your reply. Hope that's ok.

Here is other code I found for scene change detection:
(being coding illiterate I don't know how userful it is)

http://www.mathworks.com/products/computer-vision/demos.html?file=/products/demos/shipping/vision/videoscenechange.html
http://www.catenary.com/howto/motion.html
http://forum.doom9.org/showthread.php?p=268534#post268534

(of course I did NOT expect anyone to do coding work)

proof-of-concept:

--
View this message in context: http://ffmpeg-users.933282.n4.nabble.com/Scene-detection-tp941845p3725015.html
Sent from the FFmpeg-users mailing list archive at Nabble.com.

[Dave Rice]

Post by lovelove
Hi, can someone please help a desperate soul like me?
I am in the same situation as Fori Pepe

I would like to export/extract images from videos at every scene changes.
Is it possible to use ffmpeg to create images on scene changes?
ffmpeg -i test.mpg --detect_scene_changes newscene[00-99].png

... i.e. export first frame of each new scene as an image. This effectively
generates a storyboard.
It also would be an enormous help in identifying added and deleted scenes
when comparing 2 videos.

I don't know how to do this in ffmpeg along, but you could pipe yuv4mpegpipe output from ffmpeg to yuvdiff. See: http://www.silicontrip.net/~mark/view.php/lavtools/20100906-yuvdiff

For instance:
ffmpeg -i input_file.mov -pix_fmt yuv420p -f yuv4mpegpipe - | yuvdiff -g > frame_difference_values.txt

You'll get a two column output of frame number and a number representing the difference between the current frame and the prior. Then determine a threshold for what is a scene cut, parse the output to get the frame numbers of cuts, then use those to cut the desired frames out.

Dave Rice
avpreserve.com

[Peter B.]

Post by Dave Rice
ffmpeg -i input_file.mov -pix_fmt yuv420p -f yuv4mpegpipe - |
yuvdiff -g > frame_difference_values.txt
You'll get a two column output of frame number and a number
representing the difference between the current frame and the prior.
Then determine a threshold for what is a scene cut, parse the output
to get the frame numbers of cuts, then use those to cut the desired
frames out.

For getting a list of detected scene cut offsets, I can also suggest
using "shotdetect" by Johan Mathe:
http://shotdetect.nonutc.fr/

(NOTE: The current release is quite old and contains some issues, but
I've fixed some of them and Johan agreed to merge them back upstream -
there's no official release containing those changes, but if you need
them, let me know)

...you could take its output XML with the frame offset numbers, and
use a XSLT stylesheet to generate the necessary ffmpeg commands which
actually perform the cuts. However, that's just some implementation
idea from the top of my head :)

Or is there any way to feed some sort of "edit list" to ffmpeg?

Regards,
Peter B.

[Stefano Sabatini]

Post by Peter B.

Post by Dave Rice
ffmpeg -i input_file.mov -pix_fmt yuv420p -f yuv4mpegpipe - |
yuvdiff -g > frame_difference_values.txt
You'll get a two column output of frame number and a number
representing the difference between the current frame and the
prior. Then determine a threshold for what is a scene cut, parse
the output to get the frame numbers of cuts, then use those to cut
the desired frames out.

For getting a list of detected scene cut offsets, I can also suggest
http://shotdetect.nonutc.fr/
(NOTE: The current release is quite old and contains some issues,
but I've fixed some of them and Johan agreed to merge them back
upstream - there's no official release containing those changes, but
if you need them, let me know)
...you could take its output XML with the frame offset numbers, and
use a XSLT stylesheet to generate the necessary ffmpeg commands
which actually perform the cuts. However, that's just some
implementation idea from the top of my head :)
Or is there any way to feed some sort of "edit list" to ffmpeg?

Scene-detection looks like a much requested feature, please file a
feature request on the issue tracker. I'll hope we'll be able to find
some mechanisms for allowing users to (micro?)fund and vote on the
issues/features they want to be fixed.

On the technical side: this has been already discussed on
ffmpeg-devel, an underkill solution may be as simple as tweaking the
select filter and adopt a simple pixel-per-pixel average difference as
metric.

For images with moving objects that's not a proper solution, and you
need motion estimation code, which is already integrated in
libavcodec, so the ideal solution would be to refactor the ME code in
libavcodec and move it to libavutil. By doing this you may also
improve the code itself, and thus improve the codecs using it at the
same time.

[Peter B.]

Post by Stefano Sabatini
On the technical side: this has been already discussed on
ffmpeg-devel, an underkill solution may be as simple as tweaking the
select filter and adopt a simple pixel-per-pixel average difference as
metric.
For images with moving objects that's not a proper solution, and you
need motion estimation code, which is already integrated in
libavcodec, so the ideal solution would be to refactor the ME code in
libavcodec and move it to libavutil. By doing this you may also
improve the code itself, and thus improve the codecs using it at the
same time.

For ideas/details about how to implement a scene-cut detection, I
would suggest looking at the code of shotdetect (Johan Mathe is really
nice, and the license is LGPL by the way).
His approach is actually really simple and straightforward, and we've
now been using it in our long-term archive solution for quite a while
and could provide information based on hands-on experience about what
to expect from that straight-forwards algorithm.

I'm mentioning this, because I think Johan's algorithm is merely a
variation of a "pixel-per-pixel average difference" and it works
surprisingly well - on really different kinds of content.

Pb

[lovelove]

Mark Himsley, Dave Rice, Peter B. and Stefano Sabatini: Thank you all for
your replies and useful suggestions. Sorry for the delay, I got a bit
side-tracked, but here is my follow-up post in the scene change detection
thread.

Post by Stefano Sabatini
Scene-detection looks like a much requested feature, please file a
feature request on the issue tracker.

I have written ticket#442: http://ffmpeg.org/trac/ffmpeg/ticket/442 and also
included your ideas and links. Sorry I can't do more, my coding skills
aren't even enough for "hello world" greetings :-(

Post by Stefano Sabatini
On the technical side: this has been already discussed on
ffmpeg-devel, an underkill solution may be as simple as tweaking the
select filter and adopt a simple pixel-per-pixel average difference as
metric.

I think this would work only with motionless scenes, but could fail on e.g.
a continuous shot of a crowded street. Or spinning elements (
Loading Image... ). But I think you

Post by Stefano Sabatini
For images with moving objects that's not a proper solution, and you need
motion estimation code, which is already integrated in libavcodec, so the
ideal solution would be to refactor the ME code in libavcodec and move it
to libavutil. By doing this you may also improve the code itself, and thus
improve the codecs using it at the same time.

What about the -sc_threshold parameter in FFmpeg? Isn't this actually the
implementation of what we are talking here?

Post by Stefano Sabatini
For ideas/details about how to implement a scene-cut detection, I
would suggest looking at the code of shotdetect (Johan Mathe is really
nice, and the license is LGPL by the way).
His approach is actually really simple and straightforward, and we've
now been using it in our long-term archive solution for quite a while
and could provide information based on hands-on experience about what
to expect from that straight-forwards algorithm.

good call!
In ticket#442 I also listed a couple of other open source examples which can
possibly be re-used.



--
View this message in context: http://ffmpeg-users.933282.n4.nabble.com/Scene-detection-tp941845p3795513.html
Sent from the FFmpeg-users mailing list archive at Nabble.com.

[lovelove]

Post by lovelove

Post by Stefano Sabatini
an underkill solution may be as simple as tweaking the
select filter and adopt a simple pixel-per-pixel average difference as
metric.

I think this would work only with motionless scenes, but could fail on
e.g. a continuous shot of a crowded street. Or spinning elements (
http://www.wheeloffortuneinfo.com/WheelAnimated-White.gif ).

An alternative to the pixel-per-pixel average difference would be to detect
sudden histogram changes (if that's any easier to implement). This shouldn't
be fooled by the 2 examples I gave above, I think.


--
View this message in context: http://ffmpeg-users.933282.n4.nabble.com/Scene-detection-tp941845p3795646.html
Sent from the FFmpeg-users mailing list archive at Nabble.com.

[Roger Pack]

Post by Stefano Sabatini

Post by Peter B.
Or is there any way to feed some sort of "edit list" to ffmpeg?

you can specify start and stop times. EDL support would be a nice
feature at some point :)

Post by Stefano Sabatini
Scene-detection looks like a much requested feature, please file a
feature request on the issue tracker. I'll hope we'll be able to find
some mechanisms for allowing users to (micro?)fund and vote on the
issues/features they want to be fixed.

I've been interested in creating something like this recently.
Contact me offline to discuss it and requirements :)

re: scene detection

You could split your video to jpeg's then use imagemagick's "diff"
tool metric, and detect frames that have large diff's (sorry if this
was already mentioned previously).

Cheers!
-roger-

[atigian]

Post by Roger Pack
EDL support would be a nice
feature at some point :)

Using Powershell I've been able to create a batch file that generates
thumbnails for every shot change in the EDL file:
ffmpeg -i videos\your_video_file.mov -f image2 -ss 00:00:00.000 -f image2 -s
146x82 -vframes 1 images\00000000.jpg
ffmpeg -i videos\your_video_file.mov -f image2 -ss 00:00:42.000 -f image2 -s
146x82 -vframes 1 images\00004200.jpg

I explain the whole conversion process and how to sync the thumbnails with a
video player in this link:
http://www.videoproductionslondon.com/blog/edl-to-html-with-thumbnails
http://www.videoproductionslondon.com/blog/edl-to-html-with-thumbnails

When I don't have an EDL, my approach is to do the scene detection during
the encoding using keyint, min-keyint and scenecut, thus inserting a key
frame in every scene change. Once I've encoded the file I just extract all
key frames:
ffmpeg -vf select="eq(pict_type\,PICT_TYPE_I)" -i myvideo.mp4 -vsync 2 -s
73x41 -f image2 thumbnails-%02d.jpeg

I also create a text file with the timecodes of the key frames by adding:
-loglevel debug 2>&1 | grep "pict_type:I -> select:1" | cut -d " " -f 6 - >
keyframe-timecodes.txt

And I use this file to sync the thumbnails of the scene changes with a video
player:
http://www.videoproductionslondon.com/blog/scene-change-detection-during-encoding-key-frame-extraction-code
http://www.videoproductionslondon.com/blog/scene-change-detection-during-encoding-key-frame-extraction-code

--
View this message in context: http://ffmpeg-users.933282.n4.nabble.com/Scene-detection-tp941845p4431928.html
Sent from the FFmpeg-users mailing list archive at Nabble.com.

[Boštjan Strojan]

Post by atigian
When I don't have an EDL, my approach is to do the scene detection during
the encoding using keyint, min-keyint and scenecut, thus inserting a key
frame in every scene change. Once I've encoded the file I just extract all
ffmpeg -vf select="eq(pict_type\,PICT_TYPE_I)" -i myvideo.mp4 -vsync 2 -s
73x41 -f image2 thumbnails-%02d.jpeg
-loglevel debug 2>&1 | grep "pict_type:I -> select:1" | cut -d " " -f 6 - >
keyframe-timecodes.txt
And I use this file to sync the thumbnails of the scene changes with a video
http://www.videoproductionslondon.com/blog/scene-change-detection-during-encoding-key-frame-extraction-code
http://www.videoproductionslondon.com/blog/scene-change-detection-during-encoding-key-frame-extraction-code

Thanks, much appreciated.

b.

[Roger Pack]

Post by Stefano Sabatini
Scene-detection looks like a much requested feature, please file a
feature request on the issue tracker. I'll hope we'll be able to find
some mechanisms for allowing users to (micro?)fund and vote on the
issues/features they want to be fixed.

A bit unrelated, but I'd be willing to donate say $100 toward a
"motion detection" filter (you specify rectangles and intensity
thresholds, if any is triggered it allows the frame through, otherwise
it drops it).
Thanks!
-roger-

[lovelove]

Post by Peter B.
For getting a list of detected scene cut offsets, I can also suggest
http://shotdetect.nonutc.fr/
(NOTE: The current release is quite old and contains some issues, but
I've fixed some of them and Johan agreed to merge them back upstream -
there's no official release containing those changes, but if you need
them, let me know)

Thank you Peter. I tested shotdetect today and it really has some nice
concepts. The http://shotdetect.nonutc.fr/ graphs are really cool. Problem
is, almost all input videos failed. They were either outright refused or
failed to produce any meaningful output. For example, just try any video
from youtube, they all fail. I got only a single video to work.

It's really a pity because I that's the only command line tool with scene
change detection I found and I really quite badly wanted this to work ...

If you want to send me your improved release, I would of course be happy to
test it and give feedback. I'd have some comments/suggestions, but I think
they are secondary to the input issue.


--
View this message in context: http://ffmpeg-users.933282.n4.nabble.com/Scene-detection-tp941845p3801047.html
Sent from the FFmpeg-users mailing list archive at Nabble.com.

[Peter B.]

Post by lovelove
Thank you Peter. I tested shotdetect today and it really has some nice
concepts. The http://shotdetect.nonutc.fr/ graphs are really cool. Problem
is, almost all input videos failed. They were either outright refused or
failed to produce any meaningful output. For example, just try any video
from youtube, they all fail. I got only a single video to work.

Sorry, sorry, sorry for my late reply!
I was on vacation and didn't overlooked your feedback. :(

Here's the patch for improving the current version of shotdetect
(1.0.86) [1]

If you've been using the official shotdetect binary, I presume the
problems with the videos are, because it's compiled against a very old
version of ffmpeg (pre v.0.5).
My patch updates the code base to compile against the current version of
ffmpeg, which should fix the issues you're having with not being able to
open videos.

Additionally, I've added time-markers in the graphs, so they're more
useful to read :)

The patched version has now been in production use on Debian Lenny and
Squeeze systems at the Austrian Mediathek [2] for over a year now.


Regards,
Pb


== References:
[1]
http://download.das-werkstatt.com/pb/contribs/patches/shotdetect-1.0.86_to_1.0.86dva1.patch
[2] http://dva-profession.mediathek.at/

[Roger Pack]

Post by Peter B.
Here's the patch for improving the current version of shotdetect
(1.0.86) [1]

Have you submitted them upstream I presume? :)

Post by Peter B.
Additionally, I've added time-markers in the graphs, so they're more
useful to read :)

Also I assume libx264 uses something like this to decide on its
i-frame's (without having to do 2-pass)... ?

[Peter B.]

Post by Roger Pack

Post by Peter B.
Here's the patch for improving the current version of shotdetect
(1.0.86) [1]

Have you submitted them upstream I presume? :)

Yes, I did :)

I'm directly collaborating with the original author, Johan Mathe.
I'm always committing my changes back upstream, but the new version
hasn't been released yet - therefore the patch.


Pb

[Roger Pack]

Post by Roger Pack
Also I assume libx264 uses something like this to decide on its
i-frame's (without having to do 2-pass)... ?

It apperas that libx264 does some scene cut detection internally, to
look for i-frames,
viz vlc's man page:

--sout-x264-scenecut=<integer [-1 .. 100]>
Extra I-frames aggressivity
Scene-cut detection. Controls how aggressively to insert extra
I-frames. With small values of scenecut, the codec often has to force
an I-frame when it would exceed keyint. Good values of scenecut may
find a better location for the I-frame. Large values use more
I-frames than necessary, thus wasting bits. -1 disables scene-cut
detection, so I-frames are inserted only every other keyint frames,
which probably leads to ugly encoding artifacts. Range 1 to 100.

I wonder why 2 pass is so much better then...

[Peter B.]

Post by Roger Pack
It apperas that libx264 does some scene cut detection internally, to
look for i-frames,
I wonder why 2 pass is so much better then...

I think because 2-pass is not only about when to put i-frames, but
rather how to distribute the available encoding bandwidth among the
video in an optimized way.

Example:
2 scenes in a video:
1) several minutes of almost no motion: low bandwidth requirement
2) a short scene with lots of motion: high bandwidth requirement

With 2-pass encoding, the codec is able to decide to save some bits in
the low-motion sequences in order to use them later on in high-motion
scenes.

At least this is how I understood its purpose...

Pb

[gxr]

Hi,

Does anybody had any success in compiling shotdetect with the latest ffmpeg
version.
I am trying to compile in mac osx. I have used the patches posted in this
thread as well as another one found on the net but with no success. Then
tweaked the code to solve some error cpp issues related to some changes in
ffmpeg and at the end managed to succeed but I get the following error when
using the executable:

shotdetect -i example.avi -o ./shots -s 60
Shot log :: Processing movie.
./shots/motion.csv
Using network protocols without global network initialization. Please use
avformat_network_init(), this will become mandatory later.
Shot log :: 1568
Shot log :: 1968
Shot log :: 2168
...
Shot log :: 74407
Shot log :: 74607
Shot log :: 74808
Abort trap: 6

Can somebody help?
If you have succeeded in compiling shotdetect what version of ffmpeg did you
use?

-George



--
View this message in context: http://ffmpeg-users.933282.n4.nabble.com/Scene-detection-tp941845p4399695.html
Sent from the FFmpeg-users mailing list archive at Nabble.com.

[Peter B.]

Post by gxr
Does anybody had any success in compiling shotdetect with the latest ffmpeg
version.

I'm using a patched, self-compiled shotdetect version for our video
archiving tool. Daily.
However, it's been compiled with FFmpeg v0.5.2 (so far away from recent)

Post by gxr
I have used the patches posted in this
thread as well as another one found on the net but with no success.

I am not sure which thread you're referring to, as I could only find
your current message in the mailing list archive [1]

I've written and released a patch [2] which makes it possible to
compile to currently available "latest" shotdetect version (1.0.86)
[3]. My patch has already been applied upstream, but the new version
hasn't been released yet.

As I do require shotdetect to build against FFmpeg, I'll have to make
this work anyway - unfortunately, my schedule is quite full, so I
can't tell when I'll find the time to do so.

After applying the patch (1.0.86-1.0.86dva1), could you please post
which compiler errors you're getting?

Yours,
Pb

== References:
[1] https://lists.ffmpeg.org/pipermail/ffmpeg-user/2012-February/005085.html
[2]
http://dva-profession.svn.sourceforge.net/viewvc/dva-profession/tags/REL-1.0/misc/
[3] http://shotdetect.nonutc.fr/releases/latest.tar.gz

[Clément Bœsch]

Post by Peter B.

Post by gxr
Does anybody had any success in compiling shotdetect with the latest ffmpeg
version.

I'm using a patched, self-compiled shotdetect version for our video
archiving tool. Daily.
However, it's been compiled with FFmpeg v0.5.2 (so far away from recent)

As a side note, if someone is motivated, maybe
https://ffmpeg.org/trac/ffmpeg/ticket/442 could be solved by porting
shotdetect in libavfilter; this will ensure the feature will always be
available.

I don't think it will require that much work.

[...]

[Peter B.]

Post by Clément Bœsch
As a side note, if someone is motivated, maybe
https://ffmpeg.org/trac/ffmpeg/ticket/442 could be solved by porting
shotdetect in libavfilter; this will ensure the feature will always be
available.

I absolutely agree.
I've generally been thinking of porting shotdetect's functionality to
libavfilter in the long term to ensure its availability.

Shotdetect does not only provide a scene-detection feature, but also
returns graph images which are incredibly useful for quality-checks of
video material.

We (the Video-archive team at the Austrian Mediathek) will definitely
need a way of sustaining and possibly improving the features currently
provided by shotdetect.

However, this is currently non high-priority, so it's not scheduled
yet - but it will. Sooner or later...

Regards,
Pb

[Peter B.]

Post by gxr
Does anybody had any success in compiling shotdetect with the latest ffmpeg
version.

I'm using a patched, self-compiled shotdetect version for our video
archiving tool. Daily.
However, it's been compiled with FFmpeg v0.5.2 (so far away from recent)

Post by gxr
I have used the patches posted in this
thread as well as another one found on the net but with no success.

I am not sure which thread you're referring to, as I could only find
your current message in the mailing list archive [1]

I've written and released a patch [2] which makes it possible to
compile to currently available "latest" shotdetect version (1.0.86)
[3]. My patch has already been applied upstream, but the new version
hasn't been released yet.

As I do require shotdetect to build against FFmpeg, I'll have to make
this work anyway - unfortunately, my schedule is quite full, so I
can't tell when I'll find the time to do so.

After applying the patch (1.0.86-1.0.86dva1), could you please post
which compiler errors you're getting?

Yours,
Pb

== References:
[1] https://lists.ffmpeg.org/pipermail/ffmpeg-user/2012-February/005085.html
[2]
http://dva-profession.svn.sourceforge.net/viewvc/dva-profession/tags/REL-1.0/misc/
[3] http://shotdetect.nonutc.fr/releases/latest.tar.gz