[FFmpeg-user] RGB-YUV color shift

[George L.P. Fitz]

Hello,

I am seeing a slight color shift when using ffmpeg to encode to yuv based codecs like prores and v210 from an rgb source. The resulting file has slightly more saturation and gamma than the original- it doesn't look way different, but there is enough of a difference to be noticed. I'm assuming this has to do with the rgb->yuv colorspace conversion, since encoding from a yuv source to a yuv codec shows no perceivable color difference. Has anyone run across this and if so, found a way to keep the color shift negligible?

Here's a sample command:

ffmpeg -f rawvideo -pix_fmt rgb24 -s 1920x1080 -r 23.976 -i c3yoz4vuuy.video.raw -vcodec prores -profile 3 test.mov

Thanks for any help you can provide

-George

[Francois Visagie]

-----Original Message-----
Sent: 18 October 2012 01:03
Subject: [FFmpeg-user] RGB-YUV color shift
Hello,
I am seeing a slight color shift when using ffmpeg to encode to yuv based
codecs like prores and v210 from an rgb source. The resulting file has

slightly

more saturation and gamma than the original- it doesn't look way

different,

but there is enough of a difference to be noticed. I'm assuming this has

to do

with the rgb->yuv colorspace conversion, since encoding from a yuv source
to a yuv codec shows no perceivable color difference.

Correct.

Has anyone run across
this and if so, found a way to keep the color shift negligible?

I'm not sure which of these might work for you, but in principle:
* avoid colourspace conversion (I think not possible for you?)
* try to specify different conversion mapping, i.e. [0..255] ->
[0..255] instead of the [0..255] -> [16..255] which might be the case
currently.

Unfortunately this is the extent of my knowledge on the subject but
hopefully enough to point you towards something to investigate. Or enough to
keep you busy until the experts arrive ;-).

ffmpeg -f rawvideo -pix_fmt rgb24 -s 1920x1080 -r 23.976 -i
c3yoz4vuuy.video.raw -vcodec prores -profile 3 test.mov
Thanks for any help you can provide
-George
_______________________________________________
ffmpeg-user mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

[Tim Nicholson]

Post by George L.P. Fitz
Hello,
I am seeing a slight color shift when using ffmpeg to encode to yuv based codecs like prores and v210 from an rgb source. The resulting file has slightly more saturation and gamma than the original- it doesn't look way different, but there is enough of a difference to be noticed. I'm assuming this has to do with the rgb->yuv colorspace conversion, since encoding from a yuv source to a yuv codec shows no perceivable color difference. Has anyone run across this and if so, found a way to keep the color shift negligible?

Although ffmpeg supports a number of colourspace definitions. Third
party applications that use the libs make use of these choices but not
currently ffmpeg*, it only ever uses its SWS_CS_DEFAULT which is set to
BT601/BT470BG/SMPTE170m etc etc. IF you are working with HD pictures you
really want BT709/SMPTE240.

So either you have to convert RGB->yuv the use the
"colormatrix" filter, which may lead to some concatenation/rounding
errors, or change the ffmpeg default to your required colourspace and
recompile.

Post by George L.P. Fitz
ffmpeg -f rawvideo -pix_fmt rgb24 -s 1920x1080 -r 23.976 -i c3yoz4vuuy.video.raw -vcodec prores -profile 3 test.mov
Thanks for any help you can provide
-George
[..]

* There is an advanced AVCodec option of "-colorspace" which ought to
allow setting of the required values (1 for 709 & 5 for 601) but the
last time I played with it it made no difference to the output....

[George L.P. Fitz]

Post by Tim Nicholson

Post by George L.P. Fitz
Hello,
I am seeing a slight color shift when using ffmpeg to encode to yuv based codecs like prores and v210 from an rgb source. The resulting file has slightly more saturation and gamma than the original- it doesn't look way different, but there is enough of a difference to be noticed. I'm assuming this has to do with the rgb->yuv colorspace conversion, since encoding from a yuv source to a yuv codec shows no perceivable color difference. Has anyone run across this and if so, found a way to keep the color shift negligible?

Although ffmpeg supports a number of colourspace definitions. Third
party applications that use the libs make use of these choices but not
currently ffmpeg*, it only ever uses its SWS_CS_DEFAULT which is set to
BT601/BT470BG/SMPTE170m etc etc. IF you are working with HD pictures you
really want BT709/SMPTE240.
So either you have to convert RGB->yuv the use the
"colormatrix" filter, which may lead to some concatenation/rounding
errors, or change the ffmpeg default to your required colourspace and
recompile.

Tim, I tried changing the ffmpeg default since for this particular project I'm working on, all the source clips will always be in BT709:

In the source tree, I modified ffmpeg/libswscale/swscale.h

BEFORE:
#define SWS_CS_ITU709 1
#define SWS_CS_FCC 4
#define SWS_CS_ITU601 5
#define SWS_CS_ITU624 5
#define SWS_CS_SMPTE170M 5
#define SWS_CS_SMPTE240M 7
#define SWS_CS_DEFAULT 5

AFTER:
#define SWS_CS_ITU709 1
#define SWS_CS_FCC 4
#define SWS_CS_ITU601 5
#define SWS_CS_ITU624 5
#define SWS_CS_SMPTE170M 5
#define SWS_CS_SMPTE240M 7
#define SWS_CS_DEFAULT 1

Then I recompiled and tested. However I'm still seeing the same color shift. As far as you know, should that change I made have worked?

Otherwise, I took your advice on the colormatrix filter. I used -vf colormatrix=bt601:bt709 and that gives a much better result. So that seems to be a workable solution, at least for me.

Thanks very much for the help....

Post by Tim Nicholson

Post by George L.P. Fitz
ffmpeg -f rawvideo -pix_fmt rgb24 -s 1920x1080 -r 23.976 -i c3yoz4vuuy.video.raw -vcodec prores -profile 3 test.mov
Thanks for any help you can provide
-George
[..]

* There is an advanced AVCodec option of "-colorspace" which ought to
allow setting of the required values (1 for 709 & 5 for 601) but the
last time I played with it it made no difference to the output....
--
Tim

[Carl Eugen Hoyos]

Post by George L.P. Fitz
#define SWS_CS_ITU709 1
#define SWS_CS_FCC 4
#define SWS_CS_ITU601 5
#define SWS_CS_ITU624 5
#define SWS_CS_SMPTE170M 5
#define SWS_CS_SMPTE240M 7
#define SWS_CS_DEFAULT 5
#define SWS_CS_ITU709 1
#define SWS_CS_FCC 4
#define SWS_CS_ITU601 5
#define SWS_CS_ITU624 5
#define SWS_CS_SMPTE170M 5
#define SWS_CS_SMPTE240M 7
#define SWS_CS_DEFAULT 1

Is your original problem only reproducible when encoding
to prores or also if you encode with a yuv420p encoder?
(For example -vcodec mpeg4 -qscale 2)

I wonder if the reason that changing the default has no
effect is that it does not work for a 10bit colourspace.

Carl Eugen

[George L.P. Fitz]

Post by Carl Eugen Hoyos

Post by George L.P. Fitz
#define SWS_CS_ITU709 1
#define SWS_CS_FCC 4
#define SWS_CS_ITU601 5
#define SWS_CS_ITU624 5
#define SWS_CS_SMPTE170M 5
#define SWS_CS_SMPTE240M 7
#define SWS_CS_DEFAULT 5
#define SWS_CS_ITU709 1
#define SWS_CS_FCC 4
#define SWS_CS_ITU601 5
#define SWS_CS_ITU624 5
#define SWS_CS_SMPTE170M 5
#define SWS_CS_SMPTE240M 7
#define SWS_CS_DEFAULT 1

Is your original problem only reproducible when encoding
to prores or also if you encode with a yuv420p encoder?
(For example -vcodec mpeg4 -qscale 2)

I had previously seen the same color shift when encoding to v210 and x264. I haven't tried mpeg4 yet. I'll try that Monday and report back.

Post by Carl Eugen Hoyos
I wonder if the reason that changing the default has no
effect is that it does not work for a 10bit colourspace.
Carl Eugen
_______________________________________________
ffmpeg-user mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

[George L.P. Fitz]

Post by Carl Eugen Hoyos

Post by George L.P. Fitz
#define SWS_CS_ITU709 1
#define SWS_CS_FCC 4
#define SWS_CS_ITU601 5
#define SWS_CS_ITU624 5
#define SWS_CS_SMPTE170M 5
#define SWS_CS_SMPTE240M 7
#define SWS_CS_DEFAULT 5
#define SWS_CS_ITU709 1
#define SWS_CS_FCC 4
#define SWS_CS_ITU601 5
#define SWS_CS_ITU624 5
#define SWS_CS_SMPTE170M 5
#define SWS_CS_SMPTE240M 7
#define SWS_CS_DEFAULT 1

Is your original problem only reproducible when encoding
to prores or also if you encode with a yuv420p encoder?
(For example -vcodec mpeg4 -qscale 2)
I wonder if the reason that changing the default has no
effect is that it does not work for a 10bit colourspace.

I tried encoding to an 8 bit codec (mpeg4 as you suggested):

ffmpeg -f rawvideo -pix_fmt rgb24 -s 1920x1080 -r 23.976 -i c3yoz4vuuy.video.raw -vcodec mpeg4 -qscale 2 test_recompile.mp4

The resulting mpeg4 has the exact same color shift issue. So it doesn't appear to be a 10bit vs 8bit issue. So even though I changed SWS_CS_DEFAULT to be 1 and then recompiled, I can't tell where that has had any impact on this color shift issue whatsoever.

Post by Carl Eugen Hoyos
Carl Eugen
_______________________________________________
ffmpeg-user mailing list
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

[Tim Nicholson]

Post by George L.P. Fitz

Post by Carl Eugen Hoyos

Post by George L.P. Fitz
#define SWS_CS_ITU709 1
#define SWS_CS_FCC 4
#define SWS_CS_ITU601 5
#define SWS_CS_ITU624 5
#define SWS_CS_SMPTE170M 5
#define SWS_CS_SMPTE240M 7
#define SWS_CS_DEFAULT 5
#define SWS_CS_ITU709 1
#define SWS_CS_FCC 4
#define SWS_CS_ITU601 5
#define SWS_CS_ITU624 5
#define SWS_CS_SMPTE170M 5
#define SWS_CS_SMPTE240M 7
#define SWS_CS_DEFAULT 1

Is your original problem only reproducible when encoding
to prores or also if you encode with a yuv420p encoder?
(For example -vcodec mpeg4 -qscale 2)
I wonder if the reason that changing the default has no
effect is that it does not work for a 10bit colourspace.

ffmpeg -f rawvideo -pix_fmt rgb24 -s 1920x1080 -r 23.976 -i c3yoz4vuuy.video.raw -vcodec mpeg4 -qscale 2 test_recompile.mp4
The resulting mpeg4 has the exact same color shift issue. So it doesn't appear to be a 10bit vs 8bit issue. So even though I changed SWS_CS_DEFAULT to be 1 and then recompiled, I can't tell where that has had any impact on this color shift issue whatsoever.

I have a vague recollection that there are a number of different
colourspace definitions.

you have:-
AVColorPrimaries
AVColorTransferCharacteristic
AVColorSpace

Which are all enum with the same valuer for each colourspace as well as
the SWS variants, so it may be you need to poke the code somewhere else
as well.

[Chris Bouchard]

Post by Carl Eugen Hoyos
Is your original problem only reproducible when encoding
to prores or also if you encode with a yuv420p encoder?
(For example -vcodec mpeg4 -qscale 2)

I have found the same colour shift, and that using
-vf colormatrix=bt601:bt709 will fix the colours to
be almost 100% accurate for conversions from RGB to YUV.
Let's say 98% accurate; most people wouldn't notice.

However there is definitely something else going on with ffmpeg's
pixel format conversions: specifically
when converting to yuv420p. I have noticed a very slight green tint
being applied when you convert
pixel formats from e.g. yuv422p to yuv420p.

NO COLOUR SHIFT:
-pix_fmt yuv422p -vcodec libx264

SUBTLE GREEN TINT:
-pix_fmt yuv420p -vcodec libx264


Whatever options you use a grey image will go slightly green when ffmpeg
converts it to 420. The problem seems to get worse if you scale
the video down as well as convert pixel formats, so it could be
an issue in the scaler. Definitely something dodgy with the pixel format
conversion to 420.

It's interesting that YouTube doesn't exhibit this problem,
so their engineers have somehow got around it. But other
video sites like Vimeo and my ffmpeg encodes
do have this issue...

Any thoughts from experts would be very welcome!

[Carl Eugen Hoyos]

This post might be inappropriate. Click to display it.

[Bob Maple]

Post by Chris Bouchard
I have found the same colour shift, and that using
-vf colormatrix=bt601:bt709 will fix the colours to
be almost 100% accurate for conversions from RGB to YUV.
Let's say 98% accurate; most people wouldn't notice.

Could I ask how you are testing this? I am getting quite the opposite
result converting a TGA codec Quicktime to ProRes or h264 using that
colormatrix conversion, the results are very different from the original.

Bob

[chris bouchard]

Post by Bob Maple
Could I ask how you are testing this? I am getting quite the opposite
result converting a TGA codec Quicktime to ProRes or h264 using that
colormatrix conversion, the results are very different from the original.
Bob

I was using an RGB image sequence source - not a quicktime source,
so that might already be
assumed to be 709...

In the end I found that the default scaler was causing the slight green tint.

If you add this flag:

-sws_flags lanczos+accurate_rnd

this fixed the green tint I was experiencing with the default down scaling.
Note that lanczos alone will go green without the +accurate_rnd option.

[Vincent Olivier]

Post by chris bouchard
I was using an RGB image sequence source - not a quicktime source,
so that might already be
assumed to be 709...
In the end I found that the default scaler was causing the slight green tint.
-sws_flags lanczos+accurate_rnd
this fixed the green tint I was experiencing with the default down scaling.
Note that lanczos alone will go green without the +accurate_rnd option.

Sweet baby jesus!!! That's exactly what I'm looking for (I think). Does it mean, in the case of a 422 to 444 rescaling, that the "accurate lanczos" only "adds" the interpolated lines to the chroma channels and leaves the luma channel and the original chroma samples as-is?

Because if it does, that's what I was trying to make sure, and if it doesn't, well, the quest goes on! And also, what DOES it mean? :D

Vincent