Curvature-Aware Training for Coordinate Networks

Paper

Shin-Fang Chng*¹, Hemanth Saratchandran*¹, Sameera Ramasinghe², Lachlan MacDonald¹, Simon Lucey¹, ¹Australian Institute for Machine Learning (AIML), University of Adelaide, ²Amazon
^*denotes equal contribution

Abstract

Coordinate networks are widely used in computer vision due to their ability to represent signals as compressed, continuous entities. However, training these networks with first-order optimizers can be slow, hindering their use in real-time applications. Recent works have opted for shallow voxel-based representations to achieve faster training, but this sacrifices memory efficiency. This work proposes a solution that leverages second-order optimization methods to significantly reduce training times for coordinate networks while maintaining their compressibility. Experiments demonstrate the effectiveness of this approach on various signal modalities, such as audio, images, videos, shape and neural radiance fields (NeRF).

Code

We provide a demo IPython notebook as a simple reference for the core idea.

TODOs

• provide demo
• python scripts for other modalities
• hessian analysis

Citation

@article{saratchandran2023curvature,
  title={Curvature-Aware Training for Coordinate Networks},
  author={Saratchandran, Hemanth and Chng, Shin-Fang and Ramasinghe, Sameera and MacDonald, Lachlan and Lucey, Simon},
  journal={arXiv preprint arXiv:2305.08552},
  year={2023}
}