Beyond Bayer: Task-Optimal Sensor Co-Design for Robust Autonomous-Driving Segmentation

arXiv:2606.24096v1 Announce Type: new Abstract: Robust perception underpins autonomous driving, and most recent progress comes from scaling the model-larger backbones, foundation models, and cooperative multi-agent fusion. We pursue a complementary, upstream question: what should the camera itself measure? Using a differentiable RAW-to-task pipeline, we decompose which sensor degrees of freedom benefit dense prediction. Learning the spectral colour-filter-array (CFA) weights is the dominant leve...