The image is a detailed, academic poster titled "GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction", created by Xiao Chen, Quanyi Li, Tai Wang, Tianfan Xue, and Jiangmiao Pang from OpenRobotLab, Shanghai AI Laboratory, and MMLab, The Chinese University of Hong Kong. The poster presents a comprehensive study on GenNBV, an RL-based end-to-end NBV policy designed for active 3D reconstruction, providing visual data and detailed methodology. Sections of the poster include: 1. **Motivation**: Discusses the shortcomings of existing NBV policies such as limited action spaces and small object handling, and positions GenNBV's innovations including free action space, collision avoidance, and generalizable state embeddings. 2. **Methodology**: Outlines the process of GenNBV covering three main steps: extracting state embeddings, predicting the next viewpoint, and agent capturing novel observations. This section emphasizes the importance of probabilistic occupancy grid for indicating occupancy and exploration. 3. **Overview**: Highlights the advantages of GenNBV, such as its ability for free-space exploration, cross-dataset generalization, adaptation to various geometries, and self-occlusion detail capture. 4. **Visualization Comparison**: Displays visual examples comparing GenNBV to other policies like Scan-RL, showing GenNBV’s superior ability in capturing more self-occluded areas and building high-fidelity reconstructions. 5. **Generalizability Evaluation**: Provides an analysis of GenNBV's performance across various datasets, indicating its effectiveness in unfamiliar environments. 6. **Ablation Study**: Examines the individual contributions of different components of the GenNBV system, featuring detailed numerical results and performance metrics. 7. **Next-Best-View Capturing**: Indicates key findings and metrics highlighting the superior performance and generalizability of GenNBV. The poster also includes QR codes for accessing further resources such as the project page and published papers, a contact email, and emphasizes three key takeaways about the generalizability of NBV policies, essential elements of RL-based policies, and generalization potentials across datasets and categories. Text transcribed from the image: GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction Xiao Chen 1,2 Quanyi Li 1 Tai Wang1 Tianfan Xue2,1* Jiangmiao Pang1* 1 OpenRobotLab, Shanghai AI Laboratory 2 MMLab, The Chinese University of Hong Kong * Motivation Existing NBV policies * Limited action space (≤ 3 DoF) * No need to consider collision * Per-scene optimized representations * Indirect heuristic criteria * Small objects GenNBV (Ours) * Free action space (5 DoF) * Collision avoidance * Generalizable state embeddings * Evaluation metrics as reward * Large objects and scenes * Overview * GenNBV is the first RL-based end-to-end NBV policy, which allows free-space exploration and cross-dataset generalization. * GenNBV is guided by informative and generalizable state embedding. * GenNBV extend previous limited 3 DoF action to 5 DoF free space, that allow adapt to any geometry and capture self-occlusion details. * GenNBV can be generalized cross-dataset and cross-category w/o finetuning. Project Page: gennbv.github.io Contact: cx123@ie.cuhk.edu.hk * Next-Best-View Capturing Hemisphere (CR: 89.8%) Scan-RL (CR: 90.6%) GenNBV (CR: 96.8%) * Methodology Step 1: Height Step 2: Grid cell Historical Observations Hybrid Scene Representation Pose State π Semantic NP Geometric Rep π Attention Action a_t π Policy π Inference Reward Calculation * Pipeline: ① extract state embedding from observations; ② NBV policy predicts next best viewpoint; ③ agent captures novel observations; ④ Repeat ①-③. * Probabilistic occupancy grid indicates the occupancy and exploration process. Closed-set Generalization Results Mesh AUC: F1-IoU Covering Ratio SceneNN-20 Map 71.85% 74.91% 80.47% Binary-3D Grid 52.98% 51.88% 68.99% House 53-Scene 50.98% 48.14% 60.05% * Reward function: Coverage Ratio (CR), CRt = CRt-1 + (NCt/NS), where NC = NSc∫0 texplored, NS == 100% Occupied Free Unknown Evaluation Binary Grid Probabilistic 3D Grid The effectiveness of probabilistic 3D grid * Generalizability Evaluation * GenNBV is trained on hundreds of houses from Houses3K training set Evaluation Details * Visualization Comparison Scan-RL GenNBV (Ours) (a) Houses3K GenNBV captures more self-occlusion unseen objects and thus build high (b) Houses3K GenNBV generalizes well to unseen * Ablation Study Representation Category Hybrid Probabilistic 3D Grid Binary 3D Grid Mesh 2D Map Evaluation Metrics Hausdorff SceneNN GeoHausdoff * Takeaways: 1) Evaluating of NBV policies: RL-based > Info Gain-based > Heuristic 2) Key elements of RL-based policies: free action space and informative scene rep. 3) Further generalization: cross-dataset -> cross-category -> indoor/outdoor scenes