Fix: --crop-bottom
for equirect data
#3525
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I am trying to process an equirect video using
--crop-bottom=0.2
to remove myself from the bottom portion of the frames.Example frame:
However, I get training images that include my body:
I was able to identify an error in this function:
nerfstudio/nerfstudio/process_data/equirect_utils.py
Lines 213 to 230 in 6b60855
bound_arr
is initially the array[-45, 0, 45]
, which means that it will generate perspective training images using 120 deg FoV cameras pointed at altitudes -45, 0, and 45 degrees. However, with--crop-bottom=0.2
we end up withbound_arr == [-57.75, -25.5, -6.0]
at the end of the function, which is not what we want.Doing the math a little bit, I discovered that it is likely that the implementations for
_crop_top()
and_crop_bottom()
were accidentally swapped, as once i made this change, I ended up withbound_arr == [6.0, 25.5, 57.75]
which is more correct. Indeed, a +6 degree altitude is the minimum value for excluding the bottom 20% of the equirect for a 120 deg FoV perspective camera. This is the new training image created:Additional notes:
--crop-factor 0.2 0 0 0
(i.e. crop top 20%) now correctly givesbound_arr == [-57.75, -25.5, -6.0]
--crop-factor 0.1 0.1 0 0
(i.e. crop top and bottom 10%) givesbound_arr == [-22.3125, -4.125, 12.0]
, where all altitudes should actually be between -12 and 12 degrees. But that is work for another PR (which I am happy to do!)