[FFmpeg-user] FFMPEG introduces color shift to DNxHD MOVs

Discussion:

Nathan Rusch

2012-07-06 16:58:59 UTC

Hello all,

In trying to troubleshoot another issue, I’ve run across a subtle, consistent color shift being introduced by FFMPEG to DNxHD-encoded MOVs. Thus, they fail to match both DNxHD MOVs generated with the Avid codec, as well as both MOVs’ common source material (an 8-bit RGB TIFF sequence).

This is on Windows 7 using the following:
- Avid codecs 2.3.7 installed from http://avid.force.com/pkb/articles/en_US/download/en423319?popup=true&NewLang=en&DocType=1450
- ffmpeg version N-42069-g3233ad4 (git-3233ad4) 32-bit binary from http://ffmpeg.zeranoe.com/builds/
- Quicktime 7.7.2 (1680.56)

Encoder command:
ffmpeg -y -sameq -r 24 -pix_fmt rgb24 -i C:\Some\Path\input_sequence.%04d.tif -vcodec dnxhd -b:v 115M -r 23.976 -qmax 51 -pix_fmt yuv422p -f mov C:\Some\Other\Path\output_file.mov

I have derived a color matrix that can be applied either as a pre-correction to the input material or as a post-correction to the FFMPEG output to get the result to match an Avid-encoded file, and this raises a few questions:

1) Has anyone encountered this issue in the past? I have searched the available archives of this list, as well as the general Googleverse, but to no avail.
2) Has anyone ever done tests comparing outputs from FFMPEG with those generated by the Avid codec? On the one hand, it seems like perhaps not, or this would likely be documented, but on the other hand, I don’t see how an encoder could have been implemented in FFMPEG without someone at least doing a side-by-side comparison with the target format...
3) Does anyone know where this comes from? I’ve heard whispers that the Avid codec uses some kind of a proprietary color transform when encoding DNxHDs, and thus expects decoded files to be in this same state, but so far I haven’t found anything conclusive.
4) If this is a known issue, is there a reason FFMPEG is not attempting to align its outputs to those of the Avid codec, given the relatively simple task of deriving the proper correction matrix?
5) Is it possible to have FFMPEG apply an arbitrary color matrix as an intermediate step in its encoding process?

Thank you all for any information you can provide,

-Nathan

Mike Scheutzow

2012-07-06 17:21:12 UTC

Permalink

Post by Nathan Rusch
Hello all,
In trying to troubleshoot another issue, I’ve run across a subtle,
consistent color shift being introduced by FFMPEG to DNxHD-encoded
MOVs. Thus, they fail to match both DNxHD MOVs generated with the
Avid codec, as well as both MOVs’ common source material (an 8-bit
RGB TIFF sequence).

ffmpeg always assumes the BT.601 colorspace when converting from
RGB->YUV and from YUV->RGB. A patch to fix this is welcome, but no one
has ever cared enough to provide it.

So if you do:

ffmpeg: RGB -> [BT.601 matrix] -> YUV

other tool: YUV -> [BT.709 matrix] -> RGB

then you get a color shift like you describe.

Workaround: feed ffmpeg with a file containing YUV that's been properly
converted.

Mike Scheutzow

Nathan Rusch

2012-07-07 01:18:34 UTC

Permalink

Thanks for the information Mike.

This definitely clears up the cause of the issue. It's unfortunate that a
manual preprocessing step is required to achieve 1:1 RGB reconstruction when
compared to the source material or the Avid codec, but at least it's a known
constant that can be accounted for. Perhaps I will explore a manual patch in
the future.

Cheers,

-Nathan

-----Original Message-----
From: Mike Scheutzow
Sent: Friday, July 06, 2012 10:21 AM
To: FFmpeg user questions and RTFMs
Subject: Re: [FFmpeg-user] FFMPEG introduces color shift to DNxHD MOVs

Nathan Rusch

2012-07-07 03:52:36 UTC

Permalink

I wanted to follow up on this for posterity (in case anyone else comes down
this same road).

I was able to get the colors to match 1:1 to the source by adding a
colormatrix operation to the conversion process via libavfilter. The flag
and value added are:

-vf colormatrix=bt601:bt709

Thanks again for getting me moving in the right direction Mike.

-Nathan

-----Original Message-----
From: Mike Scheutzow
Sent: Friday, July 06, 2012 10:21 AM
To: FFmpeg user questions and RTFMs
Subject: Re: [FFmpeg-user] FFMPEG introduces color shift to DNxHD MOVs

James Zhang

2012-07-07 04:25:50 UTC

Permalink

unsubscribe

Post by Nathan Rusch
I wanted to follow up on this for posterity (in case anyone else comes
down this same road).
I was able to get the colors to match 1:1 to the source by adding a
colormatrix operation to the conversion process via libavfilter. The flag
-vf colormatrix=bt601:bt709
Thanks again for getting me moving in the right direction Mike.
-Nathan
-----Original Message----- From: Mike Scheutzow
Sent: Friday, July 06, 2012 10:21 AM
To: FFmpeg user questions and RTFMs
Subject: Re: [FFmpeg-user] FFMPEG introduces color shift to DNxHD MOVs

ffmpeg always assumes the BT.601 colorspace when converting from
RGB->YUV and from YUV->RGB. A patch to fix this is welcome, but no one
has ever cared enough to provide it.
ffmpeg: RGB -> [BT.601 matrix] -> YUV
other tool: YUV -> [BT.709 matrix] -> RGB
then you get a color shift like you describe.
Workaround: feed ffmpeg with a file containing YUV that's been properly
converted.
Mike Scheutzow
______________________________**_________________
ffmpeg-user mailing list
http://ffmpeg.org/mailman/**listinfo/ffmpeg-user<http://ffmpeg.org/mailman/listinfo/ffmpeg-user>
______________________________**_________________
ffmpeg-user mailing list
http://ffmpeg.org/mailman/**listinfo/ffmpeg-user<http://ffmpeg.org/mailman/listinfo/ffmpeg-user>

Mark Himsley

2012-07-08 11:06:21 UTC

Permalink

Post by Nathan Rusch
-----Original Message----- From: Mike Scheutzow
Sent: Friday, July 06, 2012 10:21 AM
To: FFmpeg user questions and RTFMs
Subject: Re: [FFmpeg-user] FFMPEG introduces color shift to DNxHD MOVs

Reformatted so your top-posting is converted into in-line threadding.

I'd like to post, for posterity, that this will reduce the fidelity of
your video.

RGB -> YUV (BT.601) is a lossy transformation (in the sense that you
cannot go RGB -> YUV (BT.601) -> RGB and expect to get out exactly what
you put in)

AND

YUV (BT.601) -> YUV (BT.709) is a lossy transformation (in the sense
that you cannot go YUV (BT.601) -> YUV (BT.709) -> YUV (BT.601) and
expect to get out exactly what you put in)

Concatenating two lossy transforms is very bad. I understand that this
is currently your only course of action.

I think a worthwhile addition to the scale filter (which does the format
conversion) would be to add a choice of colourspace, much like the use
of "format" forces the scale filter to change colour format. Tim
Nicholson has been discussing something similar.

Perhaps (this is just off the top of my head, and I'm willing to be shot
down) something like: "-vf scale=0:0:rgbyuv=bt709,format=yuv422p" may
tell the scale filter to convert to yuv422p and if an RGB -> YUV
conversion is needed then use the RGB -> BT.709 conversion.

--
Mark

Robert Krüger

2012-07-08 13:48:35 UTC

Permalink

Hi,

Post by Mark Himsley

Post by Nathan Rusch

Reformatted so your top-posting is converted into in-line threadding.

I'd like to post, for posterity, that this will reduce the fidelity of your
video.
RGB -> YUV (BT.601) is a lossy transformation (in the sense that you cannot
go RGB -> YUV (BT.601) -> RGB and expect to get out exactly what you put in)
AND
YUV (BT.601) -> YUV (BT.709) is a lossy transformation (in the sense that
you cannot go YUV (BT.601) -> YUV (BT.709) -> YUV (BT.601) and expect to get
out exactly what you put in)
Concatenating two lossy transforms is very bad. I understand that this is
currently your only course of action.
I think a worthwhile addition to the scale filter (which does the format
conversion) would be to add a choice of colourspace, much like the use of
"format" forces the scale filter to change colour format. Tim Nicholson has
been discussing something similar.
Perhaps (this is just off the top of my head, and I'm willing to be shot
down) something like: "-vf scale=0:0:rgbyuv=bt709,format=yuv422p" may tell
the scale filter to convert to yuv422p and if an RGB -> YUV conversion is
needed then use the RGB -> BT.709 conversion.

wouldn't it be more flexible if one could specify something like
source and target colorspace because then you could also convert and
scale e.g. bt601 YUV to bt709 YUV or vice versa more like "-vf
scale=0:0:src_colorspace=bt709,format=yuv422p,colorspace=bt601"?

The source colorspace should ideally be set by default to the value of
the source stream's AVCodecContext.colorspace if the information is
there. I do not know the libavfilter API well enough to know if there
is a way to transport the colorspace information through the rest of
the chain (did not see it at first glance), so in the ffmpeg case the
corresponding AVCodecContext fields can be filled so the codecs which
support that information in their bitstream e.g. h264 and mpeg2 or
some container formats can make the corresponding entries when
encoding/muxing but this is getting more of a dev posting.

Tim Nicholson

2012-07-09 07:25:27 UTC

Permalink

Post by Mark Himsley
[...]
I'd like to post, for posterity, that this will reduce the fidelity of
your video.
RGB -> YUV (BT.601) is a lossy transformation (in the sense that you
cannot go RGB -> YUV (BT.601) -> RGB and expect to get out exactly what
you put in)

Well lossy because of subsampling and then add in rounding errors...

Post by Mark Himsley
AND
YUV (BT.601) -> YUV (BT.709) is a lossy transformation (in the sense
that you cannot go YUV (BT.601) -> YUV (BT.709) -> YUV (BT.601) and
expect to get out exactly what you put in)

Not so much lossy, but more down to rounding errors really IMHO. And
also wasteful of cpu cycles with all that floating point matrix arithmetic.

Post by Mark Himsley
Concatenating two lossy transforms is very bad. I understand that this
is currently your only course of action.
I think a worthwhile addition to the scale filter (which does the format
conversion) would be to add a choice of colourspace, much like the use
of "format" forces the scale filter to change colour format. Tim
Nicholson has been discussing something similar.

libswscale currently supports 8 colourspaces, and the colormatrix filter
only 4. By default libswscale uses "ITU-R Rec. 624-4 System B, G" which
amounts to 601 coefficients. However it is not clear (to me at least) if
any process ever uses anything other than the default. There are a
number of #defines of SWS_CS_* in libswscale/swscale.h which never seem
to get used anywhere (well grepping didn't reveal anything)

Post by Mark Himsley
Perhaps (this is just off the top of my head, and I'm willing to be shot
down) something like: "-vf scale=0:0:rgbyuv=bt709,format=yuv422p" may
tell the scale filter to convert to yuv422p and if an RGB -> YUV
conversion is needed then use the RGB -> BT.709 conversion.

And fix color_range as well?

--
Tim