Can NeRFs See Without Cameras?

NeurIPS 2025
Chaitanya Amballa1, Sattwik Basu1*, Yu-Lin Wei1*, Zhijian Yang2, Mehmet Ergezer2, Romit Roy Choudhury1,2
1 University of Illinois Urbana-Champaign; 2 Amazon
* Equal contribution
đź“„Paper arXiv

1. How is EchoNeRF different from vanilla NeRFs?

Left: vanilla NeRF—cameras capture light from an object (e.g., a bulldozer). Right: EchoNeRF—phones capture RF multipath echoes from a house.

In NeRFs (left), the sensors are cameras that record light (as RGB images) coming from an object of interest (bulldozer). From these multi-view RGB images, (optical) NeRFs learns to synthesize novel views of the object and reconstruct its shape. Similarly, in EchoNeRF (right), the sensors are phones that measure RF signals (as signal strength) present in an environment (the house), except the measurements are now made inside the environment as opposed to outside. Given a few such RF measurements, EchoNeRF learns to infer the spatial layout of the house. While NeRFs use cameras to see objects, EchoNeRF uses RF signals to see environments. How does EchoNeRF achieve this?

2. EchoNeRF: Generalizing NeRFs to multipath signals

Overview of Camera NeRFs vs EchoNeRF

A camera pixel (left) primarily captures light arriving directly from the object in front of it—along the line-of-sight (LoS) path (see [1]). In contrast, an RF sensor (right) such as a WiFi receiver collects signals that include multiple reflections, or “echoes,” from walls and other objects in the surrounding environment. EchoNeRF is designed to learn from such multipath signals, enabling it to “see” the environment and infer spatial structure from sparse WiFi measurements.

3. What applications can EchoNeRF solve?

Floorplan application

While EchoNeRF can be viewed as a general framework, as an application, we show how it can be used to solve a floorplan estimation problem in the above figure. (a) User's device measures wireless signal power. (b) Transmitted signals arrive at the device along a line of sight (LoS) path and after reflections from surrounding walls. (c) Our LoS model estimates crude walls. (d) Our first-order reflection model refines the inner and outer walls. (e) Together, the floorplan is estimated which can then be used to predict signals at new locations.

4. How do the voxels' opacity change over time?

Few visualizations are shown below, especially GIF animations that are difficult to include in the paper. We show EchoNeRF's two-stage learning process below and compare with the ground truth floorplan.

Ground Truth

GT 1
GT 2
GT 3
GT 4

Stage 1

Stage 1 - 1
Stage 1 - 2
Stage 1 - 3
Stage 1 - 3

Stage 2

Stage 2 - 1
Stage 2 - 2
Stage 2 - 3
Stage 2 - 3

5. Can we improve EchoNeRF's output?

We can use several post-processing steps to improve the output of EchoNeRF. Below, we show the outputs of EchoNeRF and the results of applying post-processing — through traditional image processing and via diffusion models [3] .

Output images from EchoNeRF

Raw 1
Raw 2
Raw 3
Raw 4

Post-processed images (image processing)

The estimated floorplan images are first binarized and then refined by applying morphological closing to fill small white gaps, followed by dilation to expand wall thickness. A median blur is used to smooth the edges, and finally contour detection is applied to remove small black regions based on area.

Post 1
Post 2
Post 3
Post 4

Post-processed images (diffusion models)

We can also use generative priors to improve the output of EchoNeRF. We first train a diffusion model (DDPM [3] ) on the ZInD dataset [4], which contains clean floorplans. Once trained, we perform denoising only over the final steps of the diffusion process, starting from the EchoNeRF's output inserted at time step \( t = 100 \). The final outputs after denoising are shown below.

Post 1
Post 2
Post 3
Post 4

We also show the intermediate steps visualizing the denoising trajectory. The first column shows the diffusion process at time step \( t = 100 \), which is the output of EchoNeRF with added noise. The last column shows the final denoised output.

Diffusion Trajectory 2
Diffusion Trajectory 3
Diffusion Trajectory 5
Diffusion Trajectory 8
Diffusion Trajectory 8

6. What (other) forward tasks can EchoNeRF solve?

To demonstrate that EchoNeRF solves the true inverse problem, we perform the following tasks:

  • Channel Impulse Response (CIR) Prediction: We randomly select a Tx-Rx pair inside the floorplan and generate the CIRs using the NVIDIA's Sionna Simulator [5]. This is done for both the ground truth and post-processed floorplans.
  • Basic Ray tracing: We simulate higher-order reflections (up to order 3) using the Sionna engine [5]. Ray tracing visuals below show Tx-Rx locations and echo paths for both ground truth and predicted floorplans.

a.1 CIR on Ground Truth Floorplans

RIR GT 1
RIR GT 2
RIR GT 3
RIR GT 4

a.2 CIR on Post-Processed Floorplans

RIR PP 1
RIR PP 2
RIR PP 3
RIR PP 4

b.1 Ray tracing on Ground Truth Floorplans

We also show the ray tracing results by generating higher order reflections (until order 3) using Sionna [5] using the ground truth floorplans. Tx and Rx are shown in the image as red and green stars, respectively.

RIR GT 1
RIR GT 2
RIR GT 3
RIR GT 4

b.2 Ray tracing on Post-Processed Floorplans

We repeat the above steps for the post-processed floorplans, and show the results below.

RIR PP 1
RIR PP 2
RIR PP 3
RIR PP 4

References

  1. B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis," ECCV, 2020. [arXiv]
  2. X. Zhao, Z. An, Q. Pan, and L. Yang, "NeRF2: Neural Radio-Frequency Radiance Fields," Proceedings of the 29th Annual International Conference on Mobile Computing and Networking (MobiCom), 2023. [arXiv]
  3. J. Ho, A. Jain, and P. Abbeel, "Denoising Diffusion Probabilistic Models," Advances in Neural Information Processing Systems (NeurIPS), 2020. [arXiv]
  4. S. Cruz, W. Hutchcroft, Y. Li, R. Martin-Brualla, D. B. Goldman, M. W. Turek, and S. Izadi, "Zillow Indoor Dataset: Annotated Floor Plans with 360° Panoramas and 3D Room Layouts," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. [PDF]
  5. J. Hoydis, S. Cammerer, F. Ait Aoudia, A. Vem, N. Binder, G. Marcus, and A. Keller, "Sionna: An Open-Source Library for Next-Generation Physical Layer Research," arXiv preprint arXiv:2203.11854, 2022. [arXiv]

BibTeX

    @inproceedings{echo-nerf-2025,
      title     = {Can NeRFs See Without Cameras?},
      author    = {Amballa, Chaitanya and Basu, Sattwik and Wei, Yu-Lin and Yang, Zhijian and Ergezer, Mehmet and Choudhury, Romit Roy},
      booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
      year      = {2025},
      url       = {https://arxiv.org/pdf/2505.22441},
      eprint    = {2505.22441},
      archivePrefix = {arXiv}
    }