GMPI (Generative Multiplane Images) is a multi-plane image generation framework that enables 2D GANs to have 3D perception capabilities.
The resulting output is called Generative Multiplanar Imagery (GMPI) and emphasizes that its rendering is not only of high quality but also guarantees a consistent view, which sets GMPI apart from many previous works. Importantly, the number of alpha maps can be dynamically adjusted and varied between training and inference, alleviating memory issues and enabling fast training of GMPI at 1024 2 resolution in less than half a day.
Project Page | paper
environment settings
This code was tested on Ubuntu 18.04 with CUDA 10.2.
conda env create -f environment.yml
Use pretrained checkpoints
download checkpoint
cd /path/to/this/repo
export GMPI_ROOT=$PWDPlease downloadpre-training checkpointand put them in${GMPI_ROOT}/ckpts
The structure should be:
.
+-- ckpts
| +-- gmpi_pretrained
| | +-- FFHQ256
| | +-- FFHQ512
| | +-- FFHQ1024
| | +-- AFHQCat
| | +-- MetFaces
Use the following variables for illustration.
# This can be FFHQ256, FFHQ512, FFHQ1024, AFHQCat, or MetFaces
export DATASET_NAME=FFHQ1024
export OUTPUT_DIR=${GMPI_ROOT}/ckpts/gmpi_pretrained/${DATASET_NAME}
# Set this to your favourate seed
export SEED=589
# - When psi = 1.0 there is no truncation, which is used for quantitative results in the paper.
# - To obtain better qualitative results, use psi < 1.0.
export TRUNCATION_PSI=1.0render a single image
The following command renders the image${OUTPUT_DIR}/rendered.png,as well as:
mpi_alpha.png: alpha map for all planes,mpi_rgb.png: All planes use the same RGB texture,mpi_rgba.png: RGB-alpha image for all planes.
conda activate gmpi && \
export PYTHONPATH=${GMPI_ROOT}:${GMPI_ROOT}/gmpi/models:$PYTHONPATH && \
python ${GMPI_ROOT}/gmpi/eval/vis/render_video.py \
--ckpt_path ${OUTPUT_DIR}/generator.pth \
--output_dir ${OUTPUT_DIR} \
--seeds ${SEED} \
--nplanes 96 \
--truncation_psi ${TRUNCATION_PSI} \
--exp_config ${OUTPUT_DIR}/config.pth \
--render_single_image 1Notice:nplanes = 96Used in papers to report quantitative and qualitative results, but GMPI is capable of producing high-quality results even with 32 planes.nplanesUse a lower value (such as 32) if you encounter CUDA out-of-memory errors.
#GMPI #Homepage #Documentation #Downloads #Multiplane #Image #Generation #Framework #News Fast Delivery