This repository contains the official implementation of the following paper:

Hierarchy Flow For High-Fidelity Image-to-Image Translation
https://arxiv.org/abs/2308.06909

Abstract: Image-to-image (I2I) translation comprises a wide spectrum of tasks. Here we divide this problem into three levels: strong-fidelity translation, normal-fidelity translation, and weak-fidelity translation, indicating the extent to which the content of the original image is preserved. Although existing methods achieve good performance in weak-fidelity translation, they fail to fully preserve the content in both strong- and normal-fidelity tasks, e.g. sim2real, style transfer and low-level vision. In this work, we propose Hierarchy Flow, a novel flow-based model to achieve better content preservation during translation. Specifically, 1) we first unveil the drawbacks of standard flow-based models when applied to I2I translation. 2) Next, we propose a new design, namely hierarchical coupling for reversible feature transformation and multi-scale modeling, to constitute Hierarchy Flow. 3) Finally, we present a dedicated aligned-style loss for a better trade-off between content preservation and stylization during translation. Extensive experiments on a wide range of I2I translation benchmarks demonstrate that our approach achieves state-of-the-art performance, with convincing advantages in both strong- and normal-fidelity tasks.

conda create -n lcl python=3.10 conda activate lcl conda install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 cudatoolkit=10.2 -c pytorch pip install -r requirements.txt

Please download the GTA5 dataset and Cityscapes dataset (leftImg8bit) first.

Modify the configs/GTA2Cityscapes/config.yaml

#change the source_root and target_root for train and test respectively train: source_list: 'datasets/GTA/train.txt' target_list: 'datasets/Cityscapes/train.txt' source_root: '{YOUR PATH TO GTA}' target_root: '{YOUR PATH TO CITYSCAPES}' test: source_list: 'datasets/GTA/test.txt' target_list: 'datasets/Cityscapes/test.txt' source_root: '{YOUR PATH TO GTA}' target_root: '{YOUR PATH TO CITYSCAPES}'

Training

#FOR SLURM bash scripts/GTA2CITY/train.sh partition GPU_NUM #NO SLURM CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 main.py --config configs/config.yaml

Test

#FOR SLURM bash scripts/GTA2CITY/eval.sh partition GPU_NUM {ckpt_path} #NO SLURM CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 main.py --config configs/config.yaml --load_path {ckpt_path}

1. Release the Code.
2. Add GTA2Cityscapes exp.
3. Add COCO2Wikiart exp.
4. Add more exp.

@article{fan2023hierarchy, title={Hierarchy Flow For High-Fidelity Image-to-Image Translation}, author={Fan, Weichen and Chen, Jinghuan and Liu, Ziwei}, journal={arXiv preprint arXiv:2308.06909}, year={2023} }

Under construction...