README.md

---
language:
- en
---
# Hypernerf Practice

[Huggingface Link](https://huggingface.co/xieyizheng/hypernerf_practice)

This README contains overview on applyiing several custom scenes to reproduce [the hypernerf project](https://github.com/google/hypernerf).
- hand
- chess
- laptop
- dvd
- tomato-mark


## hand

most successful one, trained intensively, I cropped out wobbly artifacts from top

interpolation: <br>
![Image 1](gif/hand-interpolate.gif)<br>
novelview:<br>
![Image 2](gif/hand-novelview.gif)

PSNR: <br>
coarse metrics:  metric/psnr≈31.7468 <br>
fine metrics:  metric/psnr≈32.5462

## chess

minimal movement, trained intensively, lighting was varying a lot during capturing, causing flickering

interpolation: <br>
![Image 7](gif/chess-interpolate.gif) <br>
novelview: <br>
![Image 8](gif/chess-novelview.gif) <br>

PSNR: <br>
coarse metrics:  metric/psnr≈28.4936 <br>
fine metrics:  metric/psnr≈30.5115

## laptop

trained intensively, the thin screen edge failed, far field not so great. more wobbly artifacts, I assume caused by either camera registration error or hypernerf's natural artifact.

interpolation: <br>
![Image 3](gif/laptop-interpolate.gif) <br>
novelview: <br>
![Image 6](gif/laptop-novelview.gif)

PSNR: <br>
coarse metrics:  metric/psnr≈23.8846 <br>
fine metrics:  metric/psnr≈25.1817

## dvd

the floating dvd logo: not so great, raw capture is not well set. too short, and "time" has been "associated" with viewing angle by the model.

novelview:
![Image 4](gif/dvd-interpolate-novelview.gif)

PSNR: <br>
coarse metrics:  metric/psnr≈27.1150  <br>
fine metrics:  metric/psnr≈28.6814 



## tomato-mark

wobbly artifacts, I assume caused by either camera registration error or hypernerf's natural artifact. This is present in the hand scene and laptop scene as well

interpolation: <br>
![Image 5](gif/tomato-mark-interpolate.gif) <br>
PSNR: <br>
coarse metrics:  metric/psnr≈27.5819  <br>
fine metrics:  metric/psnr≈27.9282 

The Wobbly artifact mentioned earlier: <br>
![Image 9](gif/hand-artifact.gif) <br>

# reproduction steps

Due to deprecation, the hypernerf project doesn't work out of the box. <br>
I had to modify dependencies, then minor changes at rendering part. <br>
For this reproduction, I only used the notebooks. <br>
The provided instructions below have been tested to work. <br>
[Click to reproduction steps](reproduction-instruction.md)

# Dataset&checkpoint Download:
All in one:<br>https://huggingface.co/datasets/xieyizheng/hypernerf_custom_scenes/resolve/main/custom_scenes_all.zip

Separately:<br>https://huggingface.co/datasets/xieyizheng/hypernerf_custom_scenes/tree/main