Missing key/value in FineMatching

[spacycoder]

Hi, great work!

Quick question, the docstring for the forward function in "FineMatching" says it should update "expec_f" but there doesn't seem to be anything that sets this key. Is the comment wrong or is this an error?

https://github.com/zju3dv/EfficientLoFTR/blob/a2ef8b0096df2dce9154fc8e48a53ff60cadf68b/src/loftr/utils/fine_matching.py#L30C1-L31C1

[boyagesmile]

“expect_f" is update in https://github.com/zju3dv/EfficientLoFTR/blob/a2ef8b0096df2dce9154fc8e48a53ff60cadf68b/src/losses/loftr_loss.py
line 203

        if 'expec_f' not in data:
            sim_matrix_f, m_ids, i_ids, j_ids_di, j_ids_dj = data['sim_matrix_ff'], data['m_ids_f'], data['i_ids_f'], data['j_ids_f_di'], data['j_ids_f_dj']
            del data['sim_matrix_ff'], data['m_ids_f'], data['i_ids_f'], data['j_ids_f_di'], data['j_ids_f_dj']
            delta = create_meshgrid(3, 3, True, sim_matrix_f.device).to(torch.long) # [1, 3, 3, 2]
            m_ids = m_ids[...,None,None].expand(-1, 3, 3)
            i_ids = i_ids[...,None,None].expand(-1, 3, 3)
            # Note that j_ids_di & j_ids_dj in (i, j) format while delta in (x, y) format
            j_ids_di = j_ids_di[...,None,None].expand(-1, 3, 3) + delta[None, ..., 1]
            j_ids_dj = j_ids_dj[...,None,None].expand(-1, 3, 3) + delta[None, ..., 0]

            sim_matrix_f = sim_matrix_f.reshape(-1, self.local_regressw*self.local_regressw, self.local_regressw+2, self.local_regressw+2) # [M, WW, W+2, W+2]
            sim_matrix_f = sim_matrix_f[m_ids, i_ids, j_ids_di, j_ids_dj]
            sim_matrix_f = sim_matrix_f.reshape(-1, 9)

            sim_matrix_f = F.softmax(sim_matrix_f / self.local_regress_temperature, dim=-1)
            heatmap = sim_matrix_f.reshape(-1, 3, 3)
            
            # compute coordinates from heatmap
            coords_normalized = dsnt.spatial_expectation2d(heatmap[None], True)[0]
            data.update({'expec_f': coords_normalized})

[wyf2020]

Thank you for the question and the reply. There seems to be a typo in the comment here. If it's for testing, the forward function will not output 'expec_f'. If it's for training, 'expect_f' will be calculated later in the loss function.