Regarding your analysis on the existing recurrent fusion in eqn. (5)

[d1024choi]

Hi. Thanks for sharing your great work here! While reading your paper, some parts of your analysis made me confused. So it would be very appreciated if you kindly answer my questions below.

(1)
In section, 3.1, you mentioned that under the existing recurrent fusion, only current and previous BEV features affect the parameters of the fusion module, W. In contrast, since the parallel fusion uses all previous BEV features, it is affected by the long-term historical information.

However, $\bar{B}{i-1}$ in eqn. (3) is calculated from its predecessor $\bar{B}{i-2}$, which means the current BEV feature $\bar{B}{i}$ ends up being affected by $\bar{B}{i-2}$. Of course, the gradients from $\bar{B}{i-2}$ will not directly affect W, but they will indirectly affect W through $\bar{B}{i-1}$. Do I understand correctly?

(2)
In your RecurrentBEV, only the parameters of TA-GRU are affected by the gradients of the historical BEV features? If I understand correctly, how to you stop the gradients from propagating to Image Encoder/View Transformation? Simple detach operation works?

Thank you for your answer in advance and I am looking forward to hearing from you!!
Have a nice day :)

[lucifer443]

Q1: This is an optimization problem. In the beginning of the training process，the quality of $\bar{B}{i-1}$ is very low，if the $\bar{B}{i-2}$ can not directly affect W, the the quality of $\bar{B}{i-1}$ feature can not improve directly in the training, this will lead the fusion module use more current feature which hinder the utilize of long-term information.
Q2: In the one hand, if not detach the GPU will out of memory. In the other hand, we think the image feature extraction is independent in time series.