Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ceres nonlinear refactor #248

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

Alpus
Copy link

@Alpus Alpus commented Feb 6, 2019

Add ModelFitter class that encapsulates ceres fitting logic. Using it the user can build needed cost function. Also, it supports a few cameras fitting. Pull request changes ceres example too.

@patrikhuber patrikhuber self-requested a review February 6, 2019 18:13
Copy link
Owner

@patrikhuber patrikhuber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(removed)

Copy link
Owner

@patrikhuber patrikhuber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

Thanks a lot for the PR! It is quite a large PR. :-) I haven't reviewed all of it yet, but I've left some comments, after a first glance over it.

In general I think it would be good to base the PR off the devel branch, remove all the "zero-changes" and unneeded reformatting changes, this would make the PR quite a bit smaller and easier to grasp what has been changed, so it's more manageable to review. I've put more review comments inline.

Thanks!

.gitignore Outdated
.idea/
cmake-build-*

# Testing data files
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should put examples/data in .gitignore, as there's already the example images there (and if there are changes to those, we don't want to ignore them). If users have more test data, they should put it somewhere else.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Will change.

# Mac specific files
*.DS_Store

# IntelliJ IDEA files
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the IntelliJ files, is it not possible to do out-of-source builds what that IDE? We prefer to recommend that users to out-of-source builds, instead of putting build directories into the source directory and then adding those build directories to .gitignore.
Of course if it's unavoidable IDE project files, we could add it here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clion IDE creates a hidden folder .idea/ in the root of the project. Authors of this IDE recommend to add it in .gitignore (https://intellij-support.jetbrains.com/hc/en-us/articles/206544839).

What is about cmake-build-* - it is just build folder and yes, it seems reasonable to remove it from here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done



#ifndef EOS_CERES_EXAMPLE_USE_PERSPECTIVE
#define EOS_CERES_EXAMPLE_USE_PERSPECTIVE true
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to put these defines as variables in main instead? Perhaps as constexpr, so they would be compile-time too? (It should be supported by C++14 I think?)

Copy link
Author

@Alpus Alpus Feb 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was to have an option to pass these arguments to compiler. What do you think about it?

}
};

struct CliArguments {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd like to keep the structure of the command-line argument handling like it is in the other examples now. It might in principle be a good idea to refactor those things into methods like you did, but I think particularly the introduction of a new type (CliArguments) and even an exception make the code much less trivial and harder to read for beginners. The example is really meant as a "simple example", and even if the main gets a bit long, I think it's easier to reason about than introducing new types/functions.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was made to decompose a too big main function. Do you think it is better to put all this code together? Maybe it is better to leave this function, but pass all that it returns as in-out arguments? What do you think? It will avoid new struct creation and will save code decomposition.

}

template<typename LandmarkType>
void draw_landmarks(Mat &image, const IndexedLandmarkCollection<LandmarkType> &landmarks,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you could please use the .clang-format file from the repo for formatting :-) (e.g. in this particular case, the style is to put the ampersand next to the type)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, just didn't notice .clang-format file.

/**
* Block shapes fitting
*/
void block_shapes_fitting() {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With "block" here you mean "set constant"? I think "block" could be confused with the term "parameter block" from Ceres. Perhaps name it set_shape_coefficients_constant()?
(Same applies to the similar functions below)

Copy link
Author

@Alpus Alpus Feb 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

* * @return std::array with 3 or 4 (for perspective projection) parameters.
*/
auto find_blendshapes(const morphablemodel::Blendshapes* const blendshapes = nullptr) const {
if (blendshapes == nullptr) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not really sure I understand the purpose of this method. Why do we need a pointer that can be null, and could we not use MorphableModel::has_separate_expression_model() and eos::cpp17::holds_alternative<...>(...) for the variant?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. I will change it.

@@ -78,7 +78,7 @@ struct ModelContour

// We store r/l separately because we currently only fit to the contour facing the camera.
// Also if we were to fit to the whole contour: Be careful not to just fit to the closest. The
// "invisible" ones behind might be closer on an e.g 90 angle. Store CNT for left/right side separately?
// "invisible" ones behind might be closer on an e.g 90° angle. Store CNT for left/right side separately?
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what's the change on this line - maybe it's a non-ASCII degree sign (°)? It would be great if you could remove all "zero-changes", it would make the PR a bit shorter.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

@@ -177,8 +177,9 @@ class MorphableModel
core::Mesh mesh;
if (has_texture_coordinates())
{
mesh = sample_to_mesh(shape, color, shape_model.get_triangle_list(),
color_model.get_triangle_list(), texture_coordinates);
mesh =
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be best to base the PR off the devel branch, then it wouldn't be necessary to cherry-pick this commit.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not a problem to base on develop branch but it can't be compiled. So I can't check that everything OK with my work.

Error:

.../include/eos/render/texture_extraction.hpp:609:68: error: no viable constructor or deduction guide for deduction of template arguments of 'tvec4'
        auto clip_coords = projection_matrix * view_model_matrix * glm::tvec4(vtx.x(), vtx.y(), vtx.z(), 1.0f);

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just don't want to make unrelated to ceres approach changes.

@@ -193,6 +193,14 @@ class PcaModel
return draw_sample(coeffs_float);
};

template<std::size_t N>
Eigen::VectorXf draw_sample(std::array<double, N> coefficients) const
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure the use case is large enough for this to be in the PcaModel file. I would move it for now to the place where it's used (i.e. app or Ceres fitting).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can move it from here if you want, but there are many such functions in this file. As far as I understand the compiler will remove it from final code if the function will not be used in the program. Do you think it is really needed?

@Alpus
Copy link
Author

Alpus commented Feb 8, 2019

I can push new changes, but I can't build devel "as is". So I can't test my new code :-). Besides, I have some problems with color cost function fitting because I have no texture. I didn't change it before, but to support new eos::core::Image I need to do it.

GCC compiler raise an error when it try to compile braces initializer of IndexLandmark
struct. This commit add explicit definition of intializers for this struct.
@Alpus Alpus force-pushed the ceres_nonlinear_refactor branch from 3fc3fb5 to 183fbf3 Compare February 18, 2019 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants