Issue was caused by tacks map merge re-allocating the tracks and this
didn't update plane tracks.
Ideally tracks_map_merge shall not re-allocate tracks, but for now
just update plane tracks. Shouldn't be too much slow anyway and could
always be tweaked without affecting any artists.
This commit includes all the changes made for plane tracker
in tomato branch.
Movie clip editor changes:
- Artist might create a plane track out of multiple point
tracks which belongs to the same track (minimum amount of
point tracks is 4, maximum is not actually limited).
When new plane track is added, it's getting "tracked"
across all point tracks, which makes it stick to the same
plane point tracks belong to.
- After plane track was added, it need to be manually adjusted
in a way it covers feature one might to mask/replace.
General transform tools (G, R, S) or sliding corners with
a mouse could be sued for this. Plane corner which
corresponds to left bottom image corner has got X/Y axis
on it (red is for X axis, green for Y).
- Re-adjusting plane corners makes plane to be "re-tracked"
for the frames sequence between current frame and next
and previous keyframes.
- Kayframes might be removed from the plane, using Shit-X
(Marker Delete) operator. However, currently manual
re-adjustment or "re-track" trigger is needed.
Compositor changes:
- Added new node called Plane Track Deform.
- User selects which plane track to use (for this he need
to select movie clip datablock, object and track names).
- Node gets an image input, which need to be warped into
the plane.
- Node outputs:
* Input image warped into the plane.
* Plane, rasterized to a mask.
Masking changes:
- Mask points might be parented to a plane track, which
makes this point deforming in a way as if it belongs
to the tracked plane.
Some video tutorials are available:
- Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4
- Artist video: https://vimeo.com/71727578
This is mine and Keir's holiday code project :)
Clean up inconsistencies in the libmv C API:
- All type identifiers are libmv_TypeName
- All function identifiers libmv_functionName
- Prefer libmv_nounVerb function names (e.g. libmv_featuresDestroy)
- Match Blender code formatting rather than Google
- Spelling corrections
Code review: https://codereview.appspot.com/11494044/
Hopefully it's more readable now. Took me a while to remmeber
all the stuff going on here while was looking into possibility
of implementing some feature here.
Implements an automatic keyframe selection algorithm which uses
couple of approaches to find out best keyframes candidates:
- First, slightly modifier Pollefeys's criteria is used, which
limits correspondence ration from 80% to 100%. This allows to
reject keyframe candidate early without doing heavy math in
cases there're not much common features with first keyframe.
- Second step is based on Geometric Robust Information Criteria
(aka GRIC), which checks whether features motion between
candidate keyframes is better defined by homography or
fundamental matrices.
To be a good keyframe candidate, fundamental matrix need to
define motion better than homography (in this case F-GRIC will
be smaller than H-GRIC).
This two criteria are well described in this paper:
http://www.cs.ait.ac.th/~mdailey/papers/Tahir-KeyFrame.pdf
- Final step is based on estimating reconstruction error of
a full-scene solution using candidate keyframes. This part
is based on the following paper:
ftp://ftp.tnt.uni-hannover.de/pub/papers/2004/ECCV2004-TTHBAW.pdf
This step requires reconstruction using candidate keyframes
and obtaining covariance matrix of 3D points positions.
Reconstruction was done pretty much straightforward using
other simple pipeline routines, and for covariance estimation
pseudo-inverse of Hessian is used, which is in this case
(J^T * J)+, where + denotes pseudo-inverse.
Jacobian matrix is estimating using Ceres evaluate API.
This is also crucial to get rid of possible gauge ambiguity,
which is in our case made by zero-ing 7 (by gauge freedoms
number) eigen values in pseudo-inverse.
There're still room for improving and optimizing the code,
but we need some point to start with anyway :)
Thanks to Keir Mierle and Sameer Agarwal who assisted a lot
to make this feature working.
- Added const modifiers where it makes sense and
helps keep code safe.
- Reshuffled argument to match <inputs>,<outputs>
convention on parameters.
- Pass values to ApplyRadialDistortionCameraIntrinsics
by a constant reference.
This will save lots of CPU ticks passing relatively
heavy jet objects to this function when running
bundle adjustment.
This means when you've got reconstructed scene assigned to a
3d camera (via camera solver constraint) and applies scale on
this camera from Ctrl-A menu, scale will be applied on the
reconstructed scene and reset camera size to identity.
This is very useful feature for scene orientation, when you'll
just scale camera by S in the viewport to match bundles
some points in the space, and then you'll easiy make camera
have identity scale (which is needed for nice working moblur
and other things mentioning by Sebastian :) without loosing
scale of bundles themselves.
Behavior of apply scale for cameras without clip assigned
to them does not change at all.
Makes code in tracking.cc much easier to understand and modify,
without worring to breck compulation with Libmv disabled.
It is still possible compilation will break due to libmv-capi
changes, but that's not happening so much often.
This operator will run a tracker from previous
keyframe to current frame for all selected markers.
Current markers positions are considering initial
position guess which could be updated by a tracker
for better match.
Useful in cases when feature disappears from the
frame and then appears again. Usage in this case
is the following:
- When feature point re-appeared on frame, manully
place marker on it.
- Use Refine Markers operation (which is in Track
panel) to allow tracker to find a better match.
Depending on direction of tracking use either
Forwards or Backwards refining. It's easy: if
trackign happens forwards, use Refine Frowards,
otherwise use Refine Backwards :)
Additional changes:
- Cleaned up sources to reduce mess in some
big functions.
- Removed unused function from libmv c-api.
- Made functions naming more consistent.
- Use bool for internal stuff in tracking.c.
Shall be no functional changes :)
Made it so reconstructed scene always scaled in a way
that variance of camera centers is unity.
This solves "issues" when different keyframes will
give the same reprojection error but will give scenes
with different.scale, which could easily have been
considered as a bad keyframe combination.
This change is essential for automatic keyframe
selection algorithm to work reliable for user.
Also removed unneeded image buffer scaling, it was only needed
for "early output" if there was no rotation. That is no longer
supported since it used to pixelate result a lot and interpolation
is always used now.
Saves quite a few of memory and CPU cycles.
- Nearest interpolation was always used when there's
no rotation for stabilization. Was a failure of
optimization heuristic.
- Made 2d stabilization frame acquiring threaded.
This function is only used for display and sequencer
which will only benefit of threads here.
- Fixed bug introduced in r48749 which lead to
re-making stable frame on every redraw.
This made preview working but that broke internals
of tracking.
Namely, BlurredImageAndDerivativesChannels is giving
much more blurred image because it was assuming pixel
center is an integer position.
Guess other parts of libmv used to suffer because of
this issue.
Now pixel centering happens in blender side, and
libmv assumes integer position is a pixel center.
This commit implements multi-threaded calculation of frames
when building proxies. Both scaling and undistortion steps
are now threaded.
Frames and proxy resolution are still handled one-by-one,
saving files after every single step. So if HDD is not so
fast, this commit could have not so much benefit.
Internal changes:
- Added IMB_scaleImBuf_threaded which scales given image
buffer in multiple threads and uses bilinear filtering.
- libmv's camera intrinsics now have SetThreads() method
which is used to specify how many OpenMP threads to use
for buffer distortion/undistortion.
And yeah, this code is using OpenMP for threading.
- Reshuffled a bit libmv-capi calls and added function
BKE_tracking_distortion_set_threads to specify number
of threads used by intrinscis.
- Dopesheet need to be updated when adding or switching
between objects.
- After removing object it shall also be tagged for update,
otherwise crash will likely happen.
from Lawrence D'Oliveiro (ldo)
More use of bool type, necessitating adding inclusion of BLI_utildefines.h, or moving it up in the inclusion order if it was already included, in various places
- storage.c: make some variables only used in bli_builddir local to that
- storage.c: BLI_file_descriptor_size should allow 0 as a valid file descriptor
- path_util.c: make pointers to non-reentrant storage returned from folder routines const, necessitating making variables holding these returned pointers const elsewhere as well
- path_util.c: BLI_string_to_utf8 closes iconv context in case of conversion error
- blf_lang.c: fill_locales routine now has its own "languages" local variable to construct paths (was stealing internal storage belonging to BLI_get_folder before)
Issue was caused by the way how pattern sampling happens in case of
anchored display: track offset is applying on search buffer which
means offset is rounding to an integer. Fractional pat of offset was
completely ignoring which lead to jumps in pattern buffer.
This was only a visualization issue in track preview widget.
Several major things are done in this commit:
- First of all, logic of modal solver was changed.
We do not rely on only minimizer to take care of
guessing rotation for frame, but we're using
analytical rotation computation for point clouds
to obtain initial rotation.
Then this rotation is being refined using Ceres
minimizer and now instead of minimizing average
distance between points of point of two clouds,
minimization of reprojection error of point
cloud onto frame happens.
This gives quite a bit of precision improvement.
- Second bigger improvement here is using bundle
adjustment for a result of first step when we're
only estimating rotation between neighbor images
and reprojecting markers.
This averages error across the image sequence
avoiding error accumulation. Also, this will
tweak bundles themselves a bit for better match.
- And last bigger improvement here is support of
camera intrinsics refirenment.
This allowed to significantly improve solution
for real-life footage and results after such
refining are much more usable than it were before.
Thanks to Keir for the help and code review.
This commits adds extra refirenment entry in the menu which is
"K1, K2" and which will apparently refine only this distortion
coefficients.
This would be useful in cases when you know for sure focal length
(which could be obtained from lens, EXIF and so) but not sure
about how good you manual calibration is.
Be careful tho, there're no internal constraints on this
coefficients so distortion model could just screw up into insane
values.
- Fill in image dimension for camera intrinsics used for
solution refirement.
- Retrieve K3 from camera intrinsics after refining.
Shall be no functional changes, just makes things clear
and robust for further improvements.
structure instead of passing all the parameters to every function.
Makes it much easier to tweak distortion model.
---
svn merge -r52854:52855 ^/branches/soc-2011-tomato
- Moved keyframes and refirement flags into reconstruction options structure
- Moved distortion coefficients and other camera intrinsics into own structure
- Cleaned up reconstruction functions in libmv c-api
---
svn merge -r52853:52854 ^/branches/soc-2011-tomato
Highlight background depending on number of tracks existing on frame.
This is not so much mathematically accurate displaying where things
shall be improved, but it's nice feedback about which frames better
be reviewed.
Bad frames are tracks < 8, highlighted with red.
OK-ish frame are 8 <= tracks < 16, highlighted with yellow.
Could be some artifacts with color region start/end, this is a bit
unclear what exactly expected to be highlighted -- frames are
displayed as dots, but in fact they're quite noticeable segments.
---
svn merge -r54572:54573 ^/branches/soc-2011-tomato
Calculate dependent area based on distortion model rather than
using 15% of image resolution for this.
Some assumptions here:
- We're assuming real-life camera calibration is used here
- Maximal undistortion delta would be achieved on frame boundary
- Distortion fully goes inside frame
This makes it possible to approximate margin for distortion by
checking undistortion delta across frame boundary and use it
for dependent area of interest.
We do not use any formula-based equation here because we're likely
support other distortion models and in that case it'll be stupid
to try detecting formula here.
- Moved keyframes and refirement flags into reconstruction options structure
- Moved distortion coefficients and other camera intrinsics into own structure
- Cleaned up reconstruction functions in libmv c-api
