This repository has been archived on 2023-10-09. You can view files and clone it, but cannot push or open issues or pull requests.
Files
blender-archive/source/blender/compositor/operations/COM_PlaneTrackWarpImageOperation.cpp

124 lines
3.9 KiB
C++
Raw Normal View History

Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
/*
* Copyright 2013, Blender Foundation.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software Foundation,
* Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*
* Contributor:
* Sergey Sharybin
*/
#include "COM_PlaneTrackWarpImageOperation.h"
#include "COM_ReadBufferOperation.h"
#include "MEM_guardedalloc.h"
#include "BLI_listbase.h"
#include "BLI_math.h"
#include "BLI_math_color.h"
extern "C" {
# include "BLI_jitter.h"
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
# include "BKE_movieclip.h"
# include "BKE_node.h"
# include "BKE_tracking.h"
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
}
Fix for EWA (elliptical weighted average) sampling in the compositor. EWA sampling is designed for downsampling images, i.e. scaling down the size of input image pixels, which happens regularly in compositing. While the standard sampling methods (linear, cubic) work reasonably well for linear transformations, they don't yield good results in non-linear cases like perspective projection or arbitrary displacement. EWA sampling is comparable to mipmapping, but avoids problems with discontinuities. To work correctly the EWA algorithm needs partial derivatives of the mapping functions which convert output pixel coordinates back into the input image space (2x2 Jacobian matrix). With these derivatives the EWA algorithm projects ellipses into the input space and accumulates colors over their area. This calculation was not done correctly in the compositor, only the derivatives du/dx and dv/dy were calculation, basically this means it only worked for non-rotated input images. The patch introduces full derivative calculations du/dx, du/dy, dv/dx, dv/dy for the 3 nodes which use EWA sampling currently: PlaneTrackWarp, MapUV and Displace. In addition the calculation of ellipsis area and axis-aligned bounding boxes has been fixed. For the MapUV and Displace nodes the derivatives have to be estimated by evaluating the UV/displacement inputs with 1-pixel offsets, which can still have problems on discontinuities and sub-pixel variations. These potential problems can only be alleviated by more radical design changes in the compositor functions, which are out of scope for now. Basically the values passed to the UV/Displacement inputs would need to be associated with their 1st order derivatives, which requires a general approach to derivatives in all nodes.
2013-12-04 16:05:56 +01:00
BLI_INLINE void warpCoord(float x, float y, float matrix[3][3], float uv[2], float deriv[2][2])
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
{
float vec[3] = {x, y, 1.0f};
mul_m3_v3(matrix, vec);
Fix for EWA (elliptical weighted average) sampling in the compositor. EWA sampling is designed for downsampling images, i.e. scaling down the size of input image pixels, which happens regularly in compositing. While the standard sampling methods (linear, cubic) work reasonably well for linear transformations, they don't yield good results in non-linear cases like perspective projection or arbitrary displacement. EWA sampling is comparable to mipmapping, but avoids problems with discontinuities. To work correctly the EWA algorithm needs partial derivatives of the mapping functions which convert output pixel coordinates back into the input image space (2x2 Jacobian matrix). With these derivatives the EWA algorithm projects ellipses into the input space and accumulates colors over their area. This calculation was not done correctly in the compositor, only the derivatives du/dx and dv/dy were calculation, basically this means it only worked for non-rotated input images. The patch introduces full derivative calculations du/dx, du/dy, dv/dx, dv/dy for the 3 nodes which use EWA sampling currently: PlaneTrackWarp, MapUV and Displace. In addition the calculation of ellipsis area and axis-aligned bounding boxes has been fixed. For the MapUV and Displace nodes the derivatives have to be estimated by evaluating the UV/displacement inputs with 1-pixel offsets, which can still have problems on discontinuities and sub-pixel variations. These potential problems can only be alleviated by more radical design changes in the compositor functions, which are out of scope for now. Basically the values passed to the UV/Displacement inputs would need to be associated with their 1st order derivatives, which requires a general approach to derivatives in all nodes.
2013-12-04 16:05:56 +01:00
uv[0] = vec[0] / vec[2];
uv[1] = vec[1] / vec[2];
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
Fix for EWA (elliptical weighted average) sampling in the compositor. EWA sampling is designed for downsampling images, i.e. scaling down the size of input image pixels, which happens regularly in compositing. While the standard sampling methods (linear, cubic) work reasonably well for linear transformations, they don't yield good results in non-linear cases like perspective projection or arbitrary displacement. EWA sampling is comparable to mipmapping, but avoids problems with discontinuities. To work correctly the EWA algorithm needs partial derivatives of the mapping functions which convert output pixel coordinates back into the input image space (2x2 Jacobian matrix). With these derivatives the EWA algorithm projects ellipses into the input space and accumulates colors over their area. This calculation was not done correctly in the compositor, only the derivatives du/dx and dv/dy were calculation, basically this means it only worked for non-rotated input images. The patch introduces full derivative calculations du/dx, du/dy, dv/dx, dv/dy for the 3 nodes which use EWA sampling currently: PlaneTrackWarp, MapUV and Displace. In addition the calculation of ellipsis area and axis-aligned bounding boxes has been fixed. For the MapUV and Displace nodes the derivatives have to be estimated by evaluating the UV/displacement inputs with 1-pixel offsets, which can still have problems on discontinuities and sub-pixel variations. These potential problems can only be alleviated by more radical design changes in the compositor functions, which are out of scope for now. Basically the values passed to the UV/Displacement inputs would need to be associated with their 1st order derivatives, which requires a general approach to derivatives in all nodes.
2013-12-04 16:05:56 +01:00
deriv[0][0] = (matrix[0][0] - matrix[0][2] * uv[0]) / vec[2];
deriv[1][0] = (matrix[0][1] - matrix[0][2] * uv[1]) / vec[2];
deriv[0][1] = (matrix[1][0] - matrix[1][2] * uv[0]) / vec[2];
deriv[1][1] = (matrix[1][1] - matrix[1][2] * uv[1]) / vec[2];
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
}
PlaneTrackWarpImageOperation::PlaneTrackWarpImageOperation() : PlaneTrackCommonOperation()
{
this->addInputSocket(COM_DT_COLOR, COM_SC_NO_RESIZE);
this->addOutputSocket(COM_DT_COLOR);
this->m_pixelReader = NULL;
this->setComplex(true);
}
void PlaneTrackWarpImageOperation::initExecution()
{
PlaneTrackCommonOperation::initExecution();
this->m_pixelReader = this->getInputSocketReader(0);
const int width = this->m_pixelReader->getWidth();
const int height = this->m_pixelReader->getHeight();
float frame_corners[4][2] = {{0.0f, 0.0f},
{(float) width, 0.0f},
{(float) width, (float) height},
{0.0f, (float) height}};
BKE_tracking_homography_between_two_quads(this->m_frameSpaceCorners,
frame_corners,
this->m_perspectiveMatrix);
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
}
void PlaneTrackWarpImageOperation::deinitExecution()
{
this->m_pixelReader = NULL;
}
Fix for EWA (elliptical weighted average) sampling in the compositor. EWA sampling is designed for downsampling images, i.e. scaling down the size of input image pixels, which happens regularly in compositing. While the standard sampling methods (linear, cubic) work reasonably well for linear transformations, they don't yield good results in non-linear cases like perspective projection or arbitrary displacement. EWA sampling is comparable to mipmapping, but avoids problems with discontinuities. To work correctly the EWA algorithm needs partial derivatives of the mapping functions which convert output pixel coordinates back into the input image space (2x2 Jacobian matrix). With these derivatives the EWA algorithm projects ellipses into the input space and accumulates colors over their area. This calculation was not done correctly in the compositor, only the derivatives du/dx and dv/dy were calculation, basically this means it only worked for non-rotated input images. The patch introduces full derivative calculations du/dx, du/dy, dv/dx, dv/dy for the 3 nodes which use EWA sampling currently: PlaneTrackWarp, MapUV and Displace. In addition the calculation of ellipsis area and axis-aligned bounding boxes has been fixed. For the MapUV and Displace nodes the derivatives have to be estimated by evaluating the UV/displacement inputs with 1-pixel offsets, which can still have problems on discontinuities and sub-pixel variations. These potential problems can only be alleviated by more radical design changes in the compositor functions, which are out of scope for now. Basically the values passed to the UV/Displacement inputs would need to be associated with their 1st order derivatives, which requires a general approach to derivatives in all nodes.
2013-12-04 16:05:56 +01:00
void PlaneTrackWarpImageOperation::executePixelSampled(float output[4], float x, float y, PixelSampler sampler)
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
{
Fix for EWA (elliptical weighted average) sampling in the compositor. EWA sampling is designed for downsampling images, i.e. scaling down the size of input image pixels, which happens regularly in compositing. While the standard sampling methods (linear, cubic) work reasonably well for linear transformations, they don't yield good results in non-linear cases like perspective projection or arbitrary displacement. EWA sampling is comparable to mipmapping, but avoids problems with discontinuities. To work correctly the EWA algorithm needs partial derivatives of the mapping functions which convert output pixel coordinates back into the input image space (2x2 Jacobian matrix). With these derivatives the EWA algorithm projects ellipses into the input space and accumulates colors over their area. This calculation was not done correctly in the compositor, only the derivatives du/dx and dv/dy were calculation, basically this means it only worked for non-rotated input images. The patch introduces full derivative calculations du/dx, du/dy, dv/dx, dv/dy for the 3 nodes which use EWA sampling currently: PlaneTrackWarp, MapUV and Displace. In addition the calculation of ellipsis area and axis-aligned bounding boxes has been fixed. For the MapUV and Displace nodes the derivatives have to be estimated by evaluating the UV/displacement inputs with 1-pixel offsets, which can still have problems on discontinuities and sub-pixel variations. These potential problems can only be alleviated by more radical design changes in the compositor functions, which are out of scope for now. Basically the values passed to the UV/Displacement inputs would need to be associated with their 1st order derivatives, which requires a general approach to derivatives in all nodes.
2013-12-04 16:05:56 +01:00
float xy[2] = {x, y};
float uv[2];
float deriv[2][2];
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
Fix for EWA (elliptical weighted average) sampling in the compositor. EWA sampling is designed for downsampling images, i.e. scaling down the size of input image pixels, which happens regularly in compositing. While the standard sampling methods (linear, cubic) work reasonably well for linear transformations, they don't yield good results in non-linear cases like perspective projection or arbitrary displacement. EWA sampling is comparable to mipmapping, but avoids problems with discontinuities. To work correctly the EWA algorithm needs partial derivatives of the mapping functions which convert output pixel coordinates back into the input image space (2x2 Jacobian matrix). With these derivatives the EWA algorithm projects ellipses into the input space and accumulates colors over their area. This calculation was not done correctly in the compositor, only the derivatives du/dx and dv/dy were calculation, basically this means it only worked for non-rotated input images. The patch introduces full derivative calculations du/dx, du/dy, dv/dx, dv/dy for the 3 nodes which use EWA sampling currently: PlaneTrackWarp, MapUV and Displace. In addition the calculation of ellipsis area and axis-aligned bounding boxes has been fixed. For the MapUV and Displace nodes the derivatives have to be estimated by evaluating the UV/displacement inputs with 1-pixel offsets, which can still have problems on discontinuities and sub-pixel variations. These potential problems can only be alleviated by more radical design changes in the compositor functions, which are out of scope for now. Basically the values passed to the UV/Displacement inputs would need to be associated with their 1st order derivatives, which requires a general approach to derivatives in all nodes.
2013-12-04 16:05:56 +01:00
pixelTransform(xy, uv, deriv);
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
Fix for EWA (elliptical weighted average) sampling in the compositor. EWA sampling is designed for downsampling images, i.e. scaling down the size of input image pixels, which happens regularly in compositing. While the standard sampling methods (linear, cubic) work reasonably well for linear transformations, they don't yield good results in non-linear cases like perspective projection or arbitrary displacement. EWA sampling is comparable to mipmapping, but avoids problems with discontinuities. To work correctly the EWA algorithm needs partial derivatives of the mapping functions which convert output pixel coordinates back into the input image space (2x2 Jacobian matrix). With these derivatives the EWA algorithm projects ellipses into the input space and accumulates colors over their area. This calculation was not done correctly in the compositor, only the derivatives du/dx and dv/dy were calculation, basically this means it only worked for non-rotated input images. The patch introduces full derivative calculations du/dx, du/dy, dv/dx, dv/dy for the 3 nodes which use EWA sampling currently: PlaneTrackWarp, MapUV and Displace. In addition the calculation of ellipsis area and axis-aligned bounding boxes has been fixed. For the MapUV and Displace nodes the derivatives have to be estimated by evaluating the UV/displacement inputs with 1-pixel offsets, which can still have problems on discontinuities and sub-pixel variations. These potential problems can only be alleviated by more radical design changes in the compositor functions, which are out of scope for now. Basically the values passed to the UV/Displacement inputs would need to be associated with their 1st order derivatives, which requires a general approach to derivatives in all nodes.
2013-12-04 16:05:56 +01:00
m_pixelReader->readFiltered(output, uv[0], uv[1], deriv[0], deriv[1], COM_PS_BILINEAR);
}
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
Fix for EWA (elliptical weighted average) sampling in the compositor. EWA sampling is designed for downsampling images, i.e. scaling down the size of input image pixels, which happens regularly in compositing. While the standard sampling methods (linear, cubic) work reasonably well for linear transformations, they don't yield good results in non-linear cases like perspective projection or arbitrary displacement. EWA sampling is comparable to mipmapping, but avoids problems with discontinuities. To work correctly the EWA algorithm needs partial derivatives of the mapping functions which convert output pixel coordinates back into the input image space (2x2 Jacobian matrix). With these derivatives the EWA algorithm projects ellipses into the input space and accumulates colors over their area. This calculation was not done correctly in the compositor, only the derivatives du/dx and dv/dy were calculation, basically this means it only worked for non-rotated input images. The patch introduces full derivative calculations du/dx, du/dy, dv/dx, dv/dy for the 3 nodes which use EWA sampling currently: PlaneTrackWarp, MapUV and Displace. In addition the calculation of ellipsis area and axis-aligned bounding boxes has been fixed. For the MapUV and Displace nodes the derivatives have to be estimated by evaluating the UV/displacement inputs with 1-pixel offsets, which can still have problems on discontinuities and sub-pixel variations. These potential problems can only be alleviated by more radical design changes in the compositor functions, which are out of scope for now. Basically the values passed to the UV/Displacement inputs would need to be associated with their 1st order derivatives, which requires a general approach to derivatives in all nodes.
2013-12-04 16:05:56 +01:00
void PlaneTrackWarpImageOperation::pixelTransform(const float xy[2], float r_uv[2], float r_deriv[2][2])
{
warpCoord(xy[0], xy[1], m_perspectiveMatrix, r_uv, r_deriv);
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
}
bool PlaneTrackWarpImageOperation::determineDependingAreaOfInterest(rcti *input, ReadBufferOperation *readOperation, rcti *output)
{
float UVs[4][2];
Fix for EWA (elliptical weighted average) sampling in the compositor. EWA sampling is designed for downsampling images, i.e. scaling down the size of input image pixels, which happens regularly in compositing. While the standard sampling methods (linear, cubic) work reasonably well for linear transformations, they don't yield good results in non-linear cases like perspective projection or arbitrary displacement. EWA sampling is comparable to mipmapping, but avoids problems with discontinuities. To work correctly the EWA algorithm needs partial derivatives of the mapping functions which convert output pixel coordinates back into the input image space (2x2 Jacobian matrix). With these derivatives the EWA algorithm projects ellipses into the input space and accumulates colors over their area. This calculation was not done correctly in the compositor, only the derivatives du/dx and dv/dy were calculation, basically this means it only worked for non-rotated input images. The patch introduces full derivative calculations du/dx, du/dy, dv/dx, dv/dy for the 3 nodes which use EWA sampling currently: PlaneTrackWarp, MapUV and Displace. In addition the calculation of ellipsis area and axis-aligned bounding boxes has been fixed. For the MapUV and Displace nodes the derivatives have to be estimated by evaluating the UV/displacement inputs with 1-pixel offsets, which can still have problems on discontinuities and sub-pixel variations. These potential problems can only be alleviated by more radical design changes in the compositor functions, which are out of scope for now. Basically the values passed to the UV/Displacement inputs would need to be associated with their 1st order derivatives, which requires a general approach to derivatives in all nodes.
2013-12-04 16:05:56 +01:00
float deriv[2][2];
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
/* TODO(sergey): figure out proper way to do this. */
Fix for EWA (elliptical weighted average) sampling in the compositor. EWA sampling is designed for downsampling images, i.e. scaling down the size of input image pixels, which happens regularly in compositing. While the standard sampling methods (linear, cubic) work reasonably well for linear transformations, they don't yield good results in non-linear cases like perspective projection or arbitrary displacement. EWA sampling is comparable to mipmapping, but avoids problems with discontinuities. To work correctly the EWA algorithm needs partial derivatives of the mapping functions which convert output pixel coordinates back into the input image space (2x2 Jacobian matrix). With these derivatives the EWA algorithm projects ellipses into the input space and accumulates colors over their area. This calculation was not done correctly in the compositor, only the derivatives du/dx and dv/dy were calculation, basically this means it only worked for non-rotated input images. The patch introduces full derivative calculations du/dx, du/dy, dv/dx, dv/dy for the 3 nodes which use EWA sampling currently: PlaneTrackWarp, MapUV and Displace. In addition the calculation of ellipsis area and axis-aligned bounding boxes has been fixed. For the MapUV and Displace nodes the derivatives have to be estimated by evaluating the UV/displacement inputs with 1-pixel offsets, which can still have problems on discontinuities and sub-pixel variations. These potential problems can only be alleviated by more radical design changes in the compositor functions, which are out of scope for now. Basically the values passed to the UV/Displacement inputs would need to be associated with their 1st order derivatives, which requires a general approach to derivatives in all nodes.
2013-12-04 16:05:56 +01:00
warpCoord(input->xmin - 2, input->ymin - 2, this->m_perspectiveMatrix, UVs[0], deriv);
warpCoord(input->xmax + 2, input->ymin - 2, this->m_perspectiveMatrix, UVs[1], deriv);
warpCoord(input->xmax + 2, input->ymax + 2, this->m_perspectiveMatrix, UVs[2], deriv);
warpCoord(input->xmin - 2, input->ymax + 2, this->m_perspectiveMatrix, UVs[3], deriv);
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
float min[2], max[2];
INIT_MINMAX2(min, max);
for (int i = 0; i < 4; i++) {
minmax_v2v2_v2(min, max, UVs[i]);
}
rcti newInput;
newInput.xmin = min[0] - 1;
newInput.ymin = min[1] - 1;
newInput.xmax = max[0] + 1;
newInput.ymax = max[1] + 1;
Merge plane track feature from tomato branch This commit includes all the changes made for plane tracker in tomato branch. Movie clip editor changes: - Artist might create a plane track out of multiple point tracks which belongs to the same track (minimum amount of point tracks is 4, maximum is not actually limited). When new plane track is added, it's getting "tracked" across all point tracks, which makes it stick to the same plane point tracks belong to. - After plane track was added, it need to be manually adjusted in a way it covers feature one might to mask/replace. General transform tools (G, R, S) or sliding corners with a mouse could be sued for this. Plane corner which corresponds to left bottom image corner has got X/Y axis on it (red is for X axis, green for Y). - Re-adjusting plane corners makes plane to be "re-tracked" for the frames sequence between current frame and next and previous keyframes. - Kayframes might be removed from the plane, using Shit-X (Marker Delete) operator. However, currently manual re-adjustment or "re-track" trigger is needed. Compositor changes: - Added new node called Plane Track Deform. - User selects which plane track to use (for this he need to select movie clip datablock, object and track names). - Node gets an image input, which need to be warped into the plane. - Node outputs: * Input image warped into the plane. * Plane, rasterized to a mask. Masking changes: - Mask points might be parented to a plane track, which makes this point deforming in a way as if it belongs to the tracked plane. Some video tutorials are available: - Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4 - Artist video: https://vimeo.com/71727578 This is mine and Keir's holiday code project :)
2013-08-16 09:46:30 +00:00
return NodeOperation::determineDependingAreaOfInterest(&newInput, readOperation, output);
}