Commit
·
80c3093
1
Parent(s):
ea793a2
Update README.md
Browse files
README.md
CHANGED
|
@@ -14,6 +14,74 @@ https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-
|
|
| 14 |
We will also upload our project on the Hugging Face Hub Repository.
|
| 15 |
[https://huggingface.co/flax-community/putting-nerf-on-a-diet/](https://huggingface.co/flax-community/putting-nerf-on-a-diet/)
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
## Demo
|
| 18 |
[https://huggingface.co/spaces/flax-community/DietNerf-Demo](https://huggingface.co/spaces/flax-community/DietNerf-Demo)
|
| 19 |
|
|
@@ -26,5 +94,29 @@ We will also upload our project on the Hugging Face Hub Repository.
|
|
| 26 |
| Cloud TPU Team | Leader : [Alex Lau](https://github.com/riven314), Members : [JaeYoung Chung](https://github.com/robot0321), [Sunghyun Kim](https://github.com/MrBananaHuman), [Aswin Pyakurel](https://github.com/masapasa) |
|
| 27 |
| Project Managing | [Stella Yang](https://github.com/codestella) To Watch Our Project Progress, Please Check [Our Project Notion](https://www.notion.so/Putting-NeRF-on-a-Diet-e0caecea0c2b40c3996c83205baf870d) |
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
## License
|
| 30 |
[Apache License 2.0](https://github.com/codestella/putting-nerf-on-a-diet/blob/main/LICENSE)
|
|
|
|
| 14 |
We will also upload our project on the Hugging Face Hub Repository.
|
| 15 |
[https://huggingface.co/flax-community/putting-nerf-on-a-diet/](https://huggingface.co/flax-community/putting-nerf-on-a-diet/)
|
| 16 |
|
| 17 |
+
Our JAX/Flax implementation currently supports:
|
| 18 |
+
|
| 19 |
+
<table class="tg">
|
| 20 |
+
<thead>
|
| 21 |
+
<tr>
|
| 22 |
+
<th class="tg-0lax"><span style="font-weight:bold">Platform</span></th>
|
| 23 |
+
<th class="tg-0lax" colspan="2"><span style="font-weight:bold">Single-Host GPU</span></th>
|
| 24 |
+
<th class="tg-0lax" colspan="2"><span style="font-weight:bold">Multi-Device TPU</span></th>
|
| 25 |
+
</tr>
|
| 26 |
+
</thead>
|
| 27 |
+
<tbody>
|
| 28 |
+
<tr>
|
| 29 |
+
<td class="tg-0lax"><span style="font-weight:bold">Type</span></td>
|
| 30 |
+
<td class="tg-0lax">Single-Device</td>
|
| 31 |
+
<td class="tg-0lax">Multi-Device</td>
|
| 32 |
+
<td class="tg-0lax">Single-Host</td>
|
| 33 |
+
<td class="tg-0lax">Multi-Host</td>
|
| 34 |
+
</tr>
|
| 35 |
+
<tr>
|
| 36 |
+
<td class="tg-0lax"><span style="font-weight:bold">Training</span></td>
|
| 37 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
| 38 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
| 39 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
| 40 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
| 41 |
+
</tr>
|
| 42 |
+
<tr>
|
| 43 |
+
<td class="tg-0lax"><span style="font-weight:bold">Evaluation</span></td>
|
| 44 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
| 45 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
| 46 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
| 47 |
+
<td class="tg-0lax"><img src="http://storage.googleapis.com/gresearch/jaxnerf/check.png" alt="Supported" width=18px height=18px></td>
|
| 48 |
+
</tr>
|
| 49 |
+
</tbody>
|
| 50 |
+
</table>
|
| 51 |
+
|
| 52 |
+
## Installation
|
| 53 |
+
|
| 54 |
+
```
|
| 55 |
+
# Clone the repo
|
| 56 |
+
svn export https://github.com/google-research/google-research/trunk/jaxnerf
|
| 57 |
+
# Create a conda environment, note you can use python 3.6-3.8 as
|
| 58 |
+
# one of the dependencies (TensorFlow) hasn't supported python 3.9 yet.
|
| 59 |
+
conda create --name jaxnerf python=3.6.12; conda activate jaxnerf
|
| 60 |
+
# Prepare pip
|
| 61 |
+
conda install pip; pip install --upgrade pip
|
| 62 |
+
# Install requirements
|
| 63 |
+
pip install -r jaxnerf/requirements.txt
|
| 64 |
+
# [Optional] Install GPU and TPU support for Jax
|
| 65 |
+
# Remember to change cuda101 to your CUDA version, e.g. cuda110 for CUDA 11.0.
|
| 66 |
+
pip install --upgrade jax jaxlib==0.1.57+cuda101 -f https://storage.googleapis.com/jax-releases/jax_releases.html
|
| 67 |
+
# install flax and flax-transformer
|
| 68 |
+
pip install flax transformer[flax]
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
+
## Dataset
|
| 72 |
+
Download the datasets from the [NeRF official Google Drive](https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1).
|
| 73 |
+
Please download the `nerf_synthetic.zip` and unzip them
|
| 74 |
+
in the place you like. Let's assume they are placed under `/tmp/jaxnerf/data/`.
|
| 75 |
+
|
| 76 |
+
## How to use
|
| 77 |
+
```
|
| 78 |
+
python -m train \
|
| 79 |
+
--data_dir=/PATH/TO/YOUR/SCENE/DATA \ % e.g., nerf_synthetic/lego
|
| 80 |
+
--train_dir=/PATH/TO/THE/PLACE/YOU/WANT/TO/SAVE/CHECKPOINTS \
|
| 81 |
+
--config=configs/CONFIG_YOU_LIKE
|
| 82 |
+
```
|
| 83 |
+
you can toggle the semantic loss by “use_semantic_loss” in cofiguration files.
|
| 84 |
+
|
| 85 |
## Demo
|
| 86 |
[https://huggingface.co/spaces/flax-community/DietNerf-Demo](https://huggingface.co/spaces/flax-community/DietNerf-Demo)
|
| 87 |
|
|
|
|
| 94 |
| Cloud TPU Team | Leader : [Alex Lau](https://github.com/riven314), Members : [JaeYoung Chung](https://github.com/robot0321), [Sunghyun Kim](https://github.com/MrBananaHuman), [Aswin Pyakurel](https://github.com/masapasa) |
|
| 95 |
| Project Managing | [Stella Yang](https://github.com/codestella) To Watch Our Project Progress, Please Check [Our Project Notion](https://www.notion.so/Putting-NeRF-on-a-Diet-e0caecea0c2b40c3996c83205baf870d) |
|
| 96 |
|
| 97 |
+
## References
|
| 98 |
+
This project is based on “JAX-NeRF”.
|
| 99 |
+
```
|
| 100 |
+
@software{jaxnerf2020github,
|
| 101 |
+
author = {Boyang Deng and Jonathan T. Barron and Pratul P. Srinivasan},
|
| 102 |
+
title = {{JaxNeRF}: an efficient {JAX} implementation of {NeRF}},
|
| 103 |
+
url = {https://github.com/google-research/google-research/tree/master/jaxnerf},
|
| 104 |
+
version = {0.0},
|
| 105 |
+
year = {2020},
|
| 106 |
+
}
|
| 107 |
+
```
|
| 108 |
+
|
| 109 |
+
This project is based on “JAX-NeRF”.
|
| 110 |
+
```
|
| 111 |
+
@misc{jain2021putting,
|
| 112 |
+
title={Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis},
|
| 113 |
+
author={Ajay Jain and Matthew Tancik and Pieter Abbeel},
|
| 114 |
+
year={2021},
|
| 115 |
+
eprint={2104.00677},
|
| 116 |
+
archivePrefix={arXiv},
|
| 117 |
+
primaryClass={cs.CV}
|
| 118 |
+
}
|
| 119 |
+
```
|
| 120 |
+
|
| 121 |
## License
|
| 122 |
[Apache License 2.0](https://github.com/codestella/putting-nerf-on-a-diet/blob/main/LICENSE)
|