Publications

Also find my full publication list at Google Scholar.


Preprint

4D Diffusion for Dynamic Protein Structure Prediction with Reference Guided Motion Alignment [paper]
Kaihui Cheng*, Ce Liu*, Qingkun Su, Jun Wang, Liwei Zhang, Yining Tang, Yao Yao, Siyu Zhu*, Yuan Qi*
arXiv preprint 2408.12419


SlingBAG: Sliding Ball Adaptive Growth Algorithm with Differentiable Radiation Enables Super-efficient Iterative 3D Photoacoustic Image Reconstruction [paper]
Shuang Li*, Yibing Wang*, Jian Gao*, Chulhong Kim, Seongwook Choi, Yu Zhang, Qian Chen, Yao Yao*, Changhui Li*
arXiv preprint 2407.11781


Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation [paper] [project]
Mingwang Xu*, Hui Li*, Qingkun Su*, Hanlin Shang, Liwei Zhang, Ce Liu, Jingdong Wang, Yao Yao, Siyu Zhu*
arXiv preprint 2406.08801


Mani-GS: Gaussian Splatting Manipulation with Triangular Mesh [paper] [project]
Xiangjun Gao*, Xiaoyu Li*, Yiyu Zhuang, Qi Zhang, Wenbo Hu, Chaopeng Zhang*, Yao Yao*, Ying Shan, Long Quan
arXiv preprint 2405.17811


2024

Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer [paper] [project]
Shuang Wu*, Youtian Lin*, Feihu Zhang, Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, Yao Yao*
Conference on Neural Information Processing Systems (NeurIPS) 2024


Relightable 3D Gaussian: Real-time Point Cloud Relighting with BRDF Decomposition and Ray Tracing [paper] [project]
Jian Gao*, Chun Gu*, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang*, Yao Yao*
European Conference on Computer Vision (ECCV) 2024


STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians [paper] [project]
Yifei Zeng*, Yanqin Jiang*, Siyu Zhu, Yuanxun Lu, Youtian Lin, Hao Zhu, Weiming Hu, Xun Cao, Yao Yao*
European Conference on Computer Vision (ECCV) 2024


Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance [paper] [project]
Shenhao Zhu*, Junming Leo Chen*, Zuozhuo Dai, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu*, Siyu Zhu*
European Conference on Computer Vision (ECCV) 2024


High-Fidelity Free-View Synthesis of Emotional 3D Talking Head
Qianyun He, Xinya Ji, Yicheng Gong, Yuanxun Lu, Zhengyu Diao, Linjia Huang, Yao Yao, Siyu Zhu, Zhan Ma, Songcen Xu, Xiaofei Wu, Zixiao Zhang, Xun Cao, Hao Zhu*
European Conference on Computer Vision (ECCV) 2024


High-Fidelity Free-View Synthesis of Emotional 3D Talking Head
Yuxiao He, Yiyu Zhuang, Yanwen Wang, Yao Yao, Siyu Zhu, Xiaoyu Li, Qi Zhang, Xun Cao, Hao Zhu*
European Conference on Computer Vision (ECCV) 2024


Stereo Risk: A Continuous Modeling Approach to Stereo Matching [paper]
Ce Liu*, Suryansh Kumar*, Shuhang Gu, Radu Timofte, Yao Yao*, Luc Van Gool
International Conference on Machine Learning (ICML) 2024 (oral)


GaussianPro: 3D Gaussian Splatting with Progressive Propagation [paper] [project]
Kai Cheng*, Xiaoxiao Long*, Kaizhi Yang, Yao Yao, Wei Yin, Yuexin Ma, Wenping Wang, Xuejin Chen*
International Conference on Machine Learning (ICML) 2024


Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle [paper] [project]
Youtian Lin, Zuozhuo Dai, Siyu Zhu, Yao Yao*
Computer Vision and Pattern Recognition (CVPR) 2024 (highlight)


Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusion [paper] [project]
Yuanxun Lu, Jingyang Zhang, Shiwei Li, Tian Fang, David McKinnon, Yanghai Tsin, Long Quan, Xun Cao, Yao Yao*
Computer Vision and Pattern Recognition (CVPR) 2024


Consistent4D: Consistent 360 Dynamic Object Generation from Monocular Video [paper] [project]
Yanqin Jiang, Li Zhang, Jin Gao, Weiming Hu, Yao Yao*
International Conference on Learning Representations (ICLR) 2024


JointNet: Extending Text-to-Image Diffusion for Dense Distribution Modeling [paper] [project]
Jingyang Zhang, Shiwei Li, Yuanxun Lu, Tian Fang, David McKinnon, Yanghai Tsin, Long Quan, Yao Yao*
International Conference on Learning Representations (ICLR) 2024


2023

NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimation [paper] [project]
Jingyang Zhang, Yao Yao*, Shiwei Li, Jingbo Liu, Tian Fang, David McKinnon, Yanghai Tsin, Long Quan
International Conference on Computer Vision (ICCV) 2023


Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detail [paper] [project]
Yiyu Zhuang, Qi Zhang, Ying Feng, Hao Zhu, Yao Yao, Xiaoyu Li, Yan-Pei Cao, Ying Shan, Xun Cao
Siggraph Asia 2023


AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidance [paper] [project]
Zuozhuo Dai, Zhenghao Zhang, Yao Yao, Bingxue Qiu, Siyu Zhu, Long Qin, Weizhi Wang
arXiv 2311.12886


AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation [paper] [project]
Yifei Zeng, Yuanxun Lu, Xinya Ji, Yao Yao, Hao Zhu*, Xun Cao
arXiv 2306.09864


2022

Vis-MVSNet: Visibility-Aware Multi-view Stereo Network [paper] [code]
Jingyang Zhang, Shiwei Li, Zixin Luo, Tian Fang, Yao Yao*
International Journal of Computer Vision (IJCV) 2022 (invited paper)


NeILF: Neural Incident Light Field for Physically-based Material Estimation [paper] [code] [data]
Yao Yao, Jingyang Zhang, Jingbo Liu, Yihang Qu, Tian Fang, David McKinnon, Yanghai Tsin, Long Quan
European Conference on Computer Vision (ECCV) 2022


Critical Regularizations for Neural Surface Reconstruction in the Wild [paper]
Jingyang Zhang, Yao Yao*, Shiwei Li, Tian Fang, David McKinnon, Yanghai Tsin, Long Quan
Computer Vision and Pattern Recognition (CVPR) 2022


2021

Learning Signed Distance Field for Multi-view Surface Reconstruction [paper] [code]
Jingyang Zhang, Yao Yao*, Long Quan
International Conference on Computer Vision (ICCV) 2021 (oral)


2020

Visibility-aware Multi-view Stereo Network [paper] [code]
Jingyang Zhang, Yao Yao*, Shiwei Li, Zixin Luo, Tian Fang
British Machine Vision Conference (BMVC) 2020 (oral)


Learning Stereo Matchability in Disparity Regression Networks [paper] [code]
Jingyang Zhang, Yao Yao*, Zixin Luo, Shiwei Li, Tianwei Shen, Tian Fang, Long Quan
International Conference on Pattern Recognition (ICPR) 2020 (best student paper award)


BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks [paper] [dataset]
Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, Long Quan
Computer Vision and Pattern Recognition (CVPR) 2020


ASLFeat: Learning Local Features of Accurate Shape and Localization [paper] [code]
Zixin Luo, Lei Zhou, Xuyang Bai, Hongkai Chen, Jiahui Zhang, Yao Yao, Shiwei Li, Tian Fang, Long Quan
Computer Vision and Pattern Recognition (CVPR) 2020


Learning Temporal Camera Relocalization using Kalman Filtering [paper] [code]
Lei Zhou, Zixin Luo, Tianwei Shen, Jiahui Zhang, Mingmin Zhen, Yao Yao, Tian Fang, Long Quan
Computer Vision and Pattern Recognition (CVPR) 2020 (oral)


2019

Self-Supervised Learning of Depth and Motion Under Photometric Inconsistency [paper] [code]
Tianwei Shen, Lei Zhou, Zixin Luo, Yao Yao, Shiwei Li, Jiahui Zhang, Tian Fang, Long Quan
International Conference on Computer Vision Workshops (ICCVW) 2019


Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference [paper] [code]
Yao Yao, Zixin Luo, Shiwei Li, Tianwei Shen, Tian Fang, Long Quan
Computer Vision and Pattern Recognition (CVPR) 2019


Cross-atlas Convolution for Parameterization Invariant Learning on Textured Mesh Surface [paper]
Shiwei Li, Zixin Luo, Mingmin Zhen, Yao Yao, Tianwei Shen, Tian Fang, Long Quan
Computer Vision and Pattern Recognition (CVPR) 2019


ContextDesc: Local Descriptor Augmentation with Cross-Modality Context [paper] [code]
Zixin Luo, Tianwei Shen, Lei Zhou, Jiahui Zhang, Yao Yao, Shiwei Li, Tian Fang, Long Quan
Computer Vision and Pattern Recognition (CVPR) 2019 (oral)


2018

MVSNet: Depth Inference for Unstructured Multi-view Stereo [paper] [supp] [code]
Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, Long Quan
European Conference on Computer Vision (ECCV) 2018 (oral)


GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints [paper] [code]
Zixin Luo, Tianwei Shen, Lei Zhou, Siyu Zhu, Runze Zhang, Yao Yao, Tian Fang, Long Quan
European Conference on Computer Vision (ECCV) 2018


Reconstructing Thin Structures of Manifold Surfaces by Integrating Spatial Curves [paper] [supp]
Shiwei Li, Yao Yao, Tian Fang, Long Quan
Computer Vision and Pattern Recognition (CVPR) 2018


2017

Reletive Camera Refinement for Accurate Dense Reconstruction [paper]
Yao Yao, Shiwei Li, Siyu Zhu, Hanyu Deng, Tian Fang, Long Quan
International Conference on 3D Vision (3DV) 2017 (spotlight oral)


2014

Revised depth map estimation for multi-view stereo [paper]
Yao Yao, Hao Zhu, Yongming Nie, Xiaoli Ji, Xun Cao
International Conference on 3D Imaging (IC3D) 2014 (oral)