Presented by: Jonathan Katz
UCSD/Department of Urology

Introduction

Recent developments in Neural Radiance Field Processing (NERF) have leveraged the power of neural networks to quickly reconstruct 3D spaces from 2D images. Our objective was to utilize this technology to 3D render video recordings of diagnostic cystoscopies and test their fidelity.

Materials

With IRB approval we recorded two diagnostic cystoscopies (one with an Ambu disposable flexible cystoscope and the other with a Wolf digital cystoscope). We converted the videos to images and then curated the images to choose approximately 100 images, which minimized blur and spanned a large segment of the bladder. We then utilized the NVIDIA Instant Neural Graphics Primitives (iNGP), a NeRF algorithm that uses multiresolution hash encoding with a compact neural network for significantly faster convergence, to reconstruct the bladder and render novel, unseen views. We computed the structural index similarity (SSIM) and Peak signal to noise ratio (PSNR) to assess the quality and fidelity of the 3D rendering.   

Results

,

Both videos were able to be utilized for 3D rendering using iNGP. The rendering derived from the Wolf cystoscopy had a PSNR=29.8 [min=27.2, max=32.6] and SSIM=0.89. Similarly, the rendering derived from the Ambu cystoscopy had a PSNR=31.3  [min=27.1, max=35.1] and SSIM=0.90, static sample images below (Image 1).

Conclusion

Independent of cystoscopy equipment, both 3D renderings achieved reasonable fidelity. Major limitations to widespread adoption of this technology include the need for a curator to select representative and high-quality images from the initial cystoscopy video recording and the relatively small segments of bladder successfully rendered. Nonetheless, we feel that with further refinement this technology can be scaled to create 3D renderings of cystoscopies that will enable evaluation of both completeness and quality of the cystoscopy. Furthermore, this technology would be able to facilitate the comparison of cystoscopies performed in the same patient over time.   

Funding

None

Co-Authors

Jamie Finegan,
UCSD/Department of Urology

Jingpei Lu,
UCSD/Department of Computer Science

Shan Lin,
UCSD/Department of Computer Science

Michael Yip,
UCSD/Department of Computer Science

Roger Sur,
UCSD/Department of Urology