FFMpeg color offset #60947
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
11 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#60947
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
System Information
Operating system: Win7x64, Ubuntu18x64
Blender Version
2.80 beta
Short description of error
Files encoded through FFMpeg.h264 in Blender show up with a consistent color space offset (warmer and different gamma) in other software/players/viewers, but are viewed "as saved" if opened in Blender. At the same time h264 files produced in other software are open in Blender with no noticeable color shift, but re-rendering them through Blender introduces it.
Same content rendered into other formats, like PNG, does not have such effect.
At first I thought those are my codecs on a windows machine, but the output/behavior is the same on Ubuntu.
Given that the h264 files produced elsewhere are open in Blender without matching "negative" color offset (at least one I could see), could it be that FFMpeg in Blenders is set up to default to some particular color space for encoding?
Exact steps for others to reproduce the error
cc-test.zip
Open, render animation, compare to what is in the source image in any other software.
Edit:
Trying to reproduce the issue on command line ffmpeg it was found present unless some flags are used.
Flags:
are needed for other programs to read colors identical to png.
Plus these:
are needed to make Blender recognize the converted colors again properly.
Added subscriber: @KonstantinsVisnevskis
#98280 was marked as duplicate of this issue
Added subscriber: @ZedDB
Yes, blender is set to use the "Filmic" view transform. You can set the colorspace and view transform in the Render tab -> Color Management.
Is it fine if you set it to "Default" instead?
It is already set to sRGB/Default in the attached file. I was talking about the color space used by the FFMpeg encoder inside Blender (FFMpeg in converters like Handbrake seems not to have this issue). The difference is less perceivable than that between Filmic and sRGB, but it is quite strong in terms of color correction.
On an unrelated side note, given Filmic was mentioned - applying Filmic color space transform to scene rendering by default may be a good strategy for many cases, but it probably will be a source of headache and many complaints for anything involving video editing as any imported file already has color transforms applied to it, which will damage external footage and will accumulate on re-rendering even on that made in Blender, unless paid attention to turn off.
It seems there is an option for sequencer to change view transform separately, which would solve that, but it seems to have no effect. Could that be a bug or am I missing the idea behind?
Added subscriber: @iss
@iss is this something for you?
It wasn't clear from description, if this applies to renders from video sequencer(VSE bug) or all renders(FFMPEG bug).
Just to be on the same page - is the color difference of black background between PNG and video file symptom of a problem, we are talking about?
If this is the case:
When "bad output" loaded in blender, value of black is around 0.004 for RGB A=1
To me it looks like broken rendering
Now I am thinking that good question is why is this "bad output" displayed "correctly" in blender
Either way, to be fair I am quite familiar with VSE core, but other than that I have just a little knowledge.
I would pass this to somebody experienced with FFMPEG or if this turns out to be VSE problem, I am OK to work on it.
I wouldn't be probably able to localize the problem efficiently.
Tried rendering 3D scene, not VSE.
Rendered 3D scene as mp4, viewed in:
Rendered as raw avi, viewed in:
Rendered 3D scene as png, viewed in:
This all might suggest that it's a matter of some obscure binary choice up to each application, however
for example files rendered in HandBrake video converter (using FFMpeg, AFAIK) in the same mp4+h264 format are opened correctly in any of AfterEffects, Firefox, VLC or Blender.
Yet same content re-coded with Blender is opened properly only in Blender and Firefox.
Attached - a screenshot showing color difference by cutting away overlay image with a diagonal rectangle (may seem minor, but is way more prominent in some situations), source renders, blend file.
cc-test5.blend
mp4.mp4
Also - I might've written that too long previously, but what should ColorManagement->Sequencer option do? Logically it seems it should give an option not to have Filmic or other transform applied to sequencer when it is set for the 3D scene, but it seems not to have any effect at all.
Does this also happen with the 2.79 build from https://builder.blender.org/download/ ? Or is this just in 2.8?
It does happen in 2.79 on windows/linux too, however I was wrong - Converting in Handbrake (FFMpeg) directly from PNG does produce the color shift too. Converting from MOV+h264 that was created from the same PNG in AfterEffects however does not. Which suggests that there is a problem and that it is solvable, but it's unlikely to be a Blender's bug.
Googling leaves impression that some people have solved at least some color space problems converting with FFMpeg by passing some additional arguments. I'll try digging, but is there any info on how Blender communicates with FFMpeg? Are frames transferred directly or via files? Debug mode doesn't say anything upon saving animation.
Blender uses the ffmpeg C API to encode. So frames are encoded from memory not from disk.
Found it! Adding this option to ffmpeg seems to produce result with no color shifting:
And according to same stackoverflow post these ought to add proper metadata flags. Although not sure and didn't find difference in ffmpeg commandline output:
Edit: Ok, not so fast. Now it produces coolor shift in Blender, and those flags seem to dictate how exactly...
Ok, so this:
does the conversion for other programs to read colors identical to png.
And this:
makes Blender recognize the converted colors properly.
Attached - source png, command line ffmpeg + options converted mp4.
Sans slight luminance variation due to noise smoothing, they look identical in Blender, Natron, VLC vs MSpreview, After Effects.
Now there is slight color variation in Firefox and DJV imaging viewer though.
0007.mp4
Also Blender rendered mp4 for reference:
Blender-wo.mp4
Thank you for reporting and investigating this.
I will assign this to Sergey - don't know anybody else with knowledge of FFMPEG
This still occurs in official 2.8 - rendering test ffmpeg produces color shifted image, compared to source png.
It may not be a big deal for many, but it is lowering the quality of production more than it should for those that don't notice and creates an impression of FFmpeg being dangerous/useless for those that do.
I mean - details are expected to be lost in compression, but colors are much harder to control for, thus anything potentially messing those is a "stay-away".
Added subscriber: @Sergey
What's the point of not fixing this given the solution is found? Doesn't FFmpeg API allow flags like command line version?
Thousands of people render ffmpegs every day not suspecting the colors get screwed, and those that do probably think (as I did before), that ffmpeg is just not good for anything and have to reseort to external converters.
Either developers don't have time or are not aware of this issue or don't consider it significant enough to put what they are currently doing to the side and work on this issue.
Personally I am aware, but don't have time currently
It's bit more complicated but similar in nature.
There are people(including me) who want this to be resolved, among with many more issues.
It's question of priorities and available resources.
Thanks for the reply.
Added subscriber: @PrettyFireNOI7
Added subscriber: @mirh
I believe you got the thing inverted actually.
The programs actually read the colors identical to png, but they have then to take quite literally a guess about how to map them to screen (since ffmpeg drops any colour information that could be precious)
A long standing convention in the field is just to assume undefined SD content is bt.601, while undefined HD is bt.709 (the precise cutoff between these two points being kind of up in the air for the specific implementation).
Forcing conversion to bt709 at the source may thus conform to these "implicit expectations" for a 4K render, but if you were to target 320x240 resolution you'd be SOL (conversely, it should work out of the box as of now).
The right fix (pending upstream fixing their problem) is actually to force tag the bt.601 color matrix to the output files.
p.s. take note browsers are quite the can of worms , and you better use some legit video player to discuss this issue
@mirh
Browsers are cans of worms by their nature. Mentioned those just for context.
Didn't know/notice that. Thanks for a tip.
My observation and suggested fix are purely empirical. That is, for example (at the time of testing and without checking resolution dependence you just suggested):
I first noticed the color shift first after downscaling a composition from AE in Blender and rendering it again in AE, which introduced more and more color shift with each re-rendering cycle. Had to stop outputting non-image-sequence video from Blender since.
After effects, just like "top notch" video players, probably has this heuristics that chooses the color space based on the HD resolution and thinks it's rec.709 even though ffmpeg targets 601 for everything by default.
Other rendering software may actually properly flag the video stream, or even if they don't they may dynamically convert the color space based on the output dimensions (especially if they target professional scenarios that may closely tie with TV work)
This is also what
out_color_matrixdoes, "matching the conventions" expected for unflagged videos.But since the video is still untagged (and ffmpeg also doesn't have this questionable inputs guess game from the last century) when imported it will be assumed to be rec.601, in turn shifting colors again.
(standalone ffmpeg of course produces the same results of blender that uses it)
The proper solution AFAICT is using
-bsf:v h264_metadata=matrix_coefficients=6(or 5) to tag the exported video.(this is H264-specific clearly, but similar parameters should also exist for other codecs, if you cannot wait until ffmpeg fixes it itself)
What are the downsides of the flags I posted? They do seem to work in standalone. Are those h264 exclusive also? On the other hand it seems h264 is de-facto standard for light on hardware/size video.
Well, putting aside it just doesn't feel right to begin with to let such important information unspecified, as I was saying it's all a precarious matter of resolutions.
Just try to render some teeny-weeny video, and you'll see it's the bt.709 color matrix now to be screwing it.
Added subscriber: @ValZapod
No, that is never a correct solution unless you need to change with -c copy and even then hevc_metadata is quite buggy and loses layers.
The only player that supports BT.709 transfer is mpv with --target-trc=gamma2.2 and of course MacOS/iOS.
You just did not tag the output as BT.601 since that is the default. FFmpeg always defaults to BT.601 even for untagged HD content.
sRGB transfer is not the same as BT.709. If you are recording screen it is sRGB. You can tag -color_trc iec61966_2_1
I think there is no any bugs here.
Yes, there is still the usual bug that affects every program ever on the face of the planet.
https://trac.ffmpeg.org/ticket/9167
I reckon "solution" was perhaps overselling it.. it's more of a best effort workaround if any, but none the less until ffmpeg is fixed that should do it.
(also, I don't think many people here are expert in 2D video production.. don't assume anything to be obvious)
"I think there is no any bugs here."
Ahh... the immortal argument of who's in the wrong when all but one person drive on the opposite side of the road to what is said in the law.
I too love arguing about technicalities, except when the consequences are doing real damage to real people.
Added subscribers: @Leroy-Xie, @EAW, @OmarEmaraDev, @badbunny_uk
Added subscriber: @DaveDeer
Is there a way to fix this problem? I'm exporting a png sequence via VSE using MPEG-4/H.264 setup and I get shifted colors. What's the best way to avoid this?
Just in case it is helpful to anyone, here is a link to the Academy Software Foundation's current Encoding Guidelines that were made public a few months ago.
https://academysoftwarefoundation.github.io/EncodingGuidelines/
The issue is slightly mentioned here.. And I'll grant I had forgotten about the libswscale improvements.
They are kinda wrong though.
Ffmpeg isn't doing any conversion by itself, unless told otherwise.
It's the video player that assumes untagged full hd material is bt601, which is hardly "unlikely".
"person drive on the opposite side of the road to what is said in the law."
Stable API is much more important than law.
@EAW Thanks, will check it out.