[FFmpeg-user] dpx image sequence to ffv1 mkv issues

[Camara Caamaño, Xavier]

Dear list,

I have some issues related to the wrapping of a dpx imatge sequence into ffv1 in mkv container.

I get an error when I try to wrap a 6K 16bit image sequence: Invalid minimum required packet size
I know I'm pushing a lot, but these are the original scanned images of a 35mm negative film we get in our film archive's scanner.

This is my command output:

ffmpeg -f image2 -framerate 24 -start_number 0000000 -i "
C:\Temp\6K dpx 16bit\%06d.dpx" -strict -2 -c:v ffv1 -level 3 -g 1 -slicecrc 1 -
slices 16 C:\Temp\test.mkv
ffmpeg version N-82095-g5867234 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 5.4.0 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dx
va2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-lib
ebur128 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --ena
ble-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfree
type --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enab
le-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-lib
openh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschr
oedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheor
a --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvo
rbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --ena
ble-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --e
nable-decklink --enable-zlib
libavutil 55. 33.100 / 55. 33.100
libavcodec 57. 63.103 / 57. 63.103
libavformat 57. 53.100 / 57. 53.100
libavdevice 57. 0.103 / 57. 0.103
libavfilter 6. 64.100 / 6. 64.100
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 2.100 / 2. 2.100
libpostproc 54. 0.100 / 54. 0.100
Input #0, image2, from 'C:\Temp\6K dpx 16bit\%06d.dpx':
Duration: 00:00:04.21, start: 0.000000, bitrate: N/A
Stream #0:0: Video: dpx, rgb48be, 6192x4668, 24 tbr, 24 tbn, 24 tbc
[ffv1 @ 0326f8e0] bits_per_raw_sample > 8, forcing range coder
Output #0, matroska, to 'C:\Temp\test.mkv':
Metadata:
encoder : Lavf57.53.100
Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), rgb48le, 6192x4668, q=2-31, 20
0 kb/s, 24 fps, 1k tbn, 24 tbc
Metadata:
encoder : Lavc57.63.103 ffv1
Stream mapping:
Stream #0:0 -> #0:0 (dpx (native) -> ffv1 (native))
Press [q] to stop, [?] for help
[ffv1 @ 0326f8e0] Invalid minimum required packet size 4277846272 (max allowed i
s 2147483615)
Video encoding failed
Conversion failed!

I ran this under windows 7 with a windows build of ffmpeg.
However, I just tried the same cmd under Macos (ffmpeg brewed) and it worked, but with a warning:
Cannot allocate worst case packet size, the encoding could fail.

This is the output under Macos:
$ ffmpeg -f image2 -framerate 24 -start_number 0000000 -i "/Volumes/EDICIO-1/6K dpx 16bit/%06d.dpx" -strict -2 -c:v ffv1 -level 3 -g 1 -slicecrc 1 -slices 16 "/Volumes/EDICIO-1/6K dpx 16bit/temp.mkv"
ffmpeg version 3.3.2 Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.3.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-libfreetype --enable-libmp3lame --enable-librubberband --enable-libtesseract --enable-libx264 --enable-libx265 --enable-libxvid --enable-opencl --disable-lzma --enable-libopenjpeg --disable-decoder=jpeg2000 --extra-cflags=-I/usr/local/Cellar/openjpeg/2.1.2_1/include/openjpeg-2.1 --enable-vda
libavutil 55. 58.100 / 55. 58.100
libavcodec 57. 89.100 / 57. 89.100
libavformat 57. 71.100 / 57. 71.100
libavdevice 57. 6.100 / 57. 6.100
libavfilter 6. 82.100 / 6. 82.100
libavresample 3. 5. 0 / 3. 5. 0
libswscale 4. 6.100 / 4. 6.100
libswresample 2. 7.100 / 2. 7.100
libpostproc 54. 5.100 / 54. 5.100
Input #0, image2, from '/Volumes/EDICIO-1/6K dpx 16bit/%06d.dpx':
Duration: 00:00:04.21, start: 0.000000, bitrate: N/A
Stream #0:0: Video: dpx, rgb48be, 6192x4668, 24 tbr, 24 tbn, 24 tbc
Stream mapping:
Stream #0:0 -> #0:0 (dpx (native) -> ffv1 (native))
Press [q] to stop, [?] for help
[ffv1 @ 0x7f8053815a00] bits_per_raw_sample > 8, forcing range coder
Output #0, matroska, to '/Volumes/EDICIO-1/6K dpx 16bit/temp.mkv':
Metadata:
encoder : Lavf57.71.100
Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), rgb48le, 6192x4668, q=2-31, 200 kb/s, 24 fps, 1k tbn, 24 tbc
Metadata:
encoder : Lavc57.89.100 ffv1
[ffv1 @ 0x7f8053815a00] Cannot allocate worst case packet size, the encoding could fail
Last message repeated 1 times
frame= 2 fps=1.2 q=-0.0 size= 45588kB time=00:00:00.04 bitrate=8684965.0kbi[ffv1 @ 0x7f8053815a00] Cannot allocate worst case packet size, the encoding could fail
...(It keeps warning for every frame)...
frame= 101 fps=0.2 q=-0.0 size=10661728kB time=00:00:04.16 bitrate=20955104.3kbframe= 101 fps=0.2 q=-0.0 size=10661728kB time=00:00:04.16 bitrate=20955104.3kbframe= 101 fps=0.2 q=-0.0 Lsize=10792501kB time=00:00:04.16 bitrate=21212131.9kbits/s speed=0.00658x
video:10792495kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000055%
EDICIO-1:~ EDICIO1$

I don't really understand what the error and warning mean, can anybody enlighten me?
I uploaded a dpx file at:
https://wetransfer.com/downloads/bdcf070cade355e680cb504e46c1e97820171106090602/e88690d48775c2cca865dee6c4e133f220171106090602/e9c08d

Regards,

Xavi Camara
Xavier Càmara
Centre de Conservació i Restauració
[cid:***@01D356E8.93E5F900]
Carretera BV-1274 Km.1 Tel. [phone] Ext. 15905
08225 Terrassa Email: ***@gencat.cat<mailto:***@gencat.cat>

[Kieran O Leary]

Hi,

On Mon, Nov 6, 2017 at 9:18 AM, Camara Caamaño, Xavier
<***@gencat.cat> wrote:
> Dear list,
>
> I have some issues related to the wrapping of a dpx imatge sequence into ffv1 in mkv container.
>
> I get an error when I try to wrap a 6K 16bit image sequence: Invalid minimum required packet size
> I know I'm pushing a lot, but these are the original scanned images of a 35mm negative film we get in our film archive's scanner.
>
I did some synthetic testing using ffmpeg generated files using
lavfi/testsrc and was able to reproduce the issue, so it's not related
to your input file or DPX in general.

./ffmpeg -f lavfi -i testsrc=size=6192x4668 -vframes 1 -c:v ffv1
-level 3 -f null -

ffmpeg version N-88563-gd68a557 Copyright (c) 2000-2017 the FFmpeg developers

built with Apple LLVM version 8.0.0 (clang-800.0.42.1)

configuration:

libavutil 56. 0.100 / 56. 0.100

libavcodec 58. 1.100 / 58. 1.100

libavformat 58. 1.100 / 58. 1.100

libavdevice 58. 0.100 / 58. 0.100

libavfilter 7. 0.101 / 7. 0.101

libswscale 5. 0.101 / 5. 0.101

libswresample 3. 0.101 / 3. 0.101

Input #0, lavfi, from 'testsrc=size=6192x4668':

Duration: N/A, start: 0.000000, bitrate: N/A

Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24,
6192x4668 [SAR 1:1 DAR 516:389], 25 tbr, 25 tbn, 25 tbc

Stream mapping:

Stream #0:0 -> #0:0 (rawvideo (native) -> ffv1 (native))

Press [q] to stop, [?] for help

Output #0, null, to 'pipe:':

Metadata:

encoder : Lavf58.1.100

Stream #0:0: Video: ffv1, bgr0, 6192x4668 [SAR 1:1 DAR 516:389],
q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc

Metadata:

encoder : Lavc58.1.100 ffv1

[ffv1 @ 0x7ff574001800] Cannot allocate worst case packet size, the
encoding could fail

frame= 1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
speed=0.185x

video:160kB audio:0kB subtitle:0kB other streams:0kB global
headers:0kB muxing overhead: unknown




> This is my command output:
>
> ffmpeg -f image2 -framerate 24 -start_number 0000000 -i "
> C:\Temp\6K dpx 16bit\%06d.dpx" -strict -2 -c:v ffv1 -level 3 -g 1 -slicecrc 1 -
> slices 16 C:\Temp\test.mkv
> ffmpeg version N-82095-g5867234 Copyright (c) 2000-2016 the FFmpeg developers
> built with gcc 5.4.0 (GCC)


This version looks old, which probably explains why your more recent
OSX version shows a different error.
[...]

> I don't really understand what the error and warning mean, can anybody enlighten me?

Neither do I really, seems to be something to do with frame size, as I
started experimenting with reducing the input frame size and found the
error went away when I got to 3224*4500. Adding a pixel to either of
those numbers results in the error.

./ffmpeg -f lavfi -i testsrc=size=3224x4500 -vframes 1 -c:v ffv1
-level 3 -f null -

ffmpeg version N-88563-gd68a557 Copyright (c) 2000-2017 the FFmpeg developers

built with Apple LLVM version 8.0.0 (clang-800.0.42.1)

configuration:

libavutil 56. 0.100 / 56. 0.100

libavcodec 58. 1.100 / 58. 1.100

libavformat 58. 1.100 / 58. 1.100

libavdevice 58. 0.100 / 58. 0.100

libavfilter 7. 0.101 / 7. 0.101

libswscale 5. 0.101 / 5. 0.101

libswresample 3. 0.101 / 3. 0.101

Input #0, lavfi, from 'testsrc=size=3224x4500':

Duration: N/A, start: 0.000000, bitrate: N/A

Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24,
3224x4500 [SAR 1:1 DAR 806:1125], 25 tbr, 25 tbn, 25 tbc

Stream mapping:

Stream #0:0 -> #0:0 (rawvideo (native) -> ffv1 (native))

Press [q] to stop, [?] for help

Output #0, null, to 'pipe:':

Metadata:

encoder : Lavf58.1.100

Stream #0:0: Video: ffv1, bgr0, 3224x4500 [SAR 1:1 DAR 806:1125],
q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc

Metadata:

encoder : Lavc58.1.100 ffv1

frame= 1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
speed=0.288x

video:147kB audio:0kB subtitle:0kB other streams:0kB global
headers:0kB muxing overhead: unknown

> I uploaded a dpx file at:
> https://wetransfer.com/downloads/bdcf070cade355e680cb504e46c1e97820171106090602/e88690d48775c2cca865dee6c4e133f220171106090602/e9c08d

Dear god 6K 16-bit DPX is HUGE.

Best,

Kieran.
_______________________________________________
ffmpeg-user mailing list
ffmpeg-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-***@ffmpeg.org with subject "unsubscri

[Kieran O'Leary]

Hi,

Just in terms of the issue:
That error message was introduced in December 2006 here: https://github.com/FFmpeg/FFmpeg/commit/38a7834bbb24ef62466b076715e0add60e1d6962

in order to fix this issue with 5k encodes:
https://trac.ffmpeg.org/ticket/6005

which refers to a max packet size of 2147483615

So I'm not sure what needs to be done here to fix this, unless it means that there is a frame size limit.

How is this issue affecting your encodes, is the process of going from DPX -> FFV1 - > DPX lossless in terms of framemd5s?

-Kieran
------
This email has been scanned for spam and malware by The Email Laundry.

_______________________________________________
ffmpeg-user mailing list
ffmpeg-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-***@ffmpeg.org with subject "un

[Camara Caamaño, Xavier]

Hi,


I updated the Windows build for ffmpeg and at least the error doesn't show up. I still get the warning of:

Cannot allocate worst case packet size, the encoding could fail.



However, after wrapping the 6k 16bit dpx sequence into ffv1.mkv and de-wrapping again into dpx, the checksums
are the same, so it's a good thing.

The last issue I came up to was the DPX version. My original DPX file is DPX version 2, but after de-wrapping MediaInfo shows they are DPX version 1. Checksums are identical, however.

Regards,

Xavi

_______________________________________________
ffmpeg-user mailing list
ffmpeg-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-***@ffmp

[Carl Eugen Hoyos]

2017-11-06 13:21 GMT+01:00 Camara Caamaño, Xavier <***@gencat.cat>:

> I updated the Windows build for ffmpeg and at least the error doesn't show up.

Yes.

> I still get the warning of:
>
> Cannot allocate worst case packet size, the encoding could fail.

This is expected for high-res input, you will get an error if
encoding fails (afair).

> However, after wrapping the 6k 16bit dpx sequence into ffv1.mkv and
> de-wrapping again into dpx, the checksums are the same, so it's a
> good thing.
>
> The last issue I came up to was the DPX version. My original DPX file is
> DPX version 2

FFmpeg only supports writing dpx version 1 files.
Why is that an issue for you?

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
ffmpeg-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-***@ffmpeg.org with su

[Camara Caamaño, Xavier]

Thanks Carl and Kieran,

The version is not an issue at all, just curious because I saw in this post:
https://ffmpeg.org/pipermail/ffmpeg-user/2015-March/025630.html
that the recreated DPX files were also version 2.

Regards,

Xavi

-----Missatge original-----
De: ffmpeg-user [mailto:ffmpeg-user-***@ffmpeg.org] En nom de Carl Eugen Hoyos
Enviat: dilluns, 6 de novembre de 2017 13:41
Per a: FFmpeg user questions <ffmpeg-***@ffmpeg.org>
Tema: Re: [FFmpeg-user] dpx image sequence to ffv1 mkv issues

2017-11-06 13:21 GMT+01:00 Camara Caamaño, Xavier <***@gencat.cat>:

> I updated the Windows build for ffmpeg and at least the error doesn't show up.

Yes.

> I still get the warning of:
>
> Cannot allocate worst case packet size, the encoding could fail.

This is expected for high-res input, you will get an error if encoding fails (afair).

> However, after wrapping the 6k 16bit dpx sequence into ffv1.mkv and
> de-wrapping again into dpx, the checksums are the same, so it's a good
> thing.
>
> The last issue I came up to was the DPX version. My original DPX file
> is DPX version 2

FFmpeg only supports writing dpx version 1 files.
Why is that an issue for you?

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
ffmpeg-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email ffmpeg-user-***@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-user mailing list
ffmpeg-***@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-***@ffmpeg.org with