Dr Shanxin Yuan

Lecturer in Digital Environment

School of Electronic Engineering and Computer Science
Queen Mary University of London

shanxin.yuan@qmul.ac.uk

shanxinyuan.github.io/

Centre for Multimodal AI

Research
Publications

Research

3D Computer Vision, Reconstruction and Rendering, Multimodal Learning, Low-level Vision, Music Understanding

Interests

The research focuses on computer vision and machine learning, with a strong philosophy of transferring research to significant real-world applications. I work on several lines of research, especially 3D digital humans and computational photography. The recent topics include hand/head/body pose estimation and reconstruction, neural rendering for deformable objects, pose retargeting, immersive gaming, music understanding, and fashion AI. My previous research has been successfully shipped to several products that are being used by millions of people worldwide.

Publications

Publications of specific relevance to the Centre for Multimodal AI

2024

ST-ITO: Controlling audio effects for style transfer with inference-time optimization
Steinmetz C, Singh S, Comunit� M, Ibnyahya I, Yuan S, Benetos E and Reiss J
25th International Society for Music Information Retrieval Conference (ISMIR) San Francisco, CA, USA 10 Nov 2024 - 14 Nov 2024.

DOI 10.48550/arxiv.2410.21233

QMRO

10-11-2024

Non-local degradation modeling for spatially adaptive single image super-resolution
Zhang Q, Zheng B, Li Z, Liu Y, Zhu Z, Slabaugh G and Yuan S
Neural Networks, Elsevier vol. 175, 106293-106293.

DOI 10.1016/j.neunet.2024.106293

QMRO

10-04-2024

GoLDFormer: A global–local deformable window transformer for efficient image restoration
Chen Q, Zheng B, Yan C, Zhu Z, Wang T, Slabaugh G and Yuan S
Journal of Visual Communication and Image Representation, Elsevier vol. 100, 104117-104117.

DOI 10.1016/j.jvcir.2024.104117

QMRO

01-04-2024

Graph neural networks in vision-language image understanding: a survey
Senior H, Slabaugh G, Yuan S and Rossi L
The Visual Computer, Springer Science and Business Media Llc

DOI 10.1007/s00371-024-03343-0

QMRO

29-03-2024

Video Demoiring With Deep Temporal Color Embedding and Video-Image Invertible Consistency
Liu L, An J, Yuan S, Zhou W, Li H, Wang Y and Tian Q
Ieee Transactions On Multimedia, Institute of Electrical and Electronics Engineers (Ieee) vol. 26, 7386-7397.

DOI 10.1109/tmm.2024.3366765

26-02-2024

Wavelet-based network for high dynamic range imaging
Dai T, Li W, Cao X, Liu J, Jia X, Leonardis A, Yan Y and Yuan S
Computer Vision and Image Understanding, Elsevier vol. 238

DOI 10.1016/j.cviu.2023.103881

01-01-2024

2023

Depth-guided deep filtering network for efficient single image bokeh rendering
Chen Q, Zheng B, Zhou X, Huang A, Sun Y, Chen C, Yan C and Yuan S
Neural Computing and Applications, Springer vol. 35 (28), 20869-20887.

DOI 10.1007/s00521-023-08852-y

QMRO

26-07-2023

Low-Light Video Enhancement with Synthetic Event Guidance
Liu L, An J, Liu J, Yuan S, Chen X, Zhou W, Li H, Wang YF and Tian Q
Proceedings of the AAAI Conference on Artificial Intelligence. vol. 37 (2), 1692-1700.

DOI 10.1609/aaai.v37i2.25257

26-06-2023

NeRFVS: Neural Radiance Fields for Free View Synthesis via Geometry Scaffolds
Yang C, Li P, Zhou Z, Yuan S, Liu B, Yang X, Qiu W and Shen W
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). vol. 00, 16549-16558.

DOI 10.1109/cvpr52729.2023.01588

24-06-2023

Improving Dynamic HDR Imaging with Fusion Transformer
Chen R, Zheng B, Zhang H, Chen Q, Yan C, Slabaugh G and Yuan S
Annual AAAI Conference on Artificial Intelligence. vol. 37 (1), 340-349.

DOI 10.1609/aaai.v37i1.25107

QMRO

07-02-2023

2022

DomainPlus: Cross Transform Domain Learning towards High Dynamic Range Imaging
Zheng B, Pan X, Zhang H, Zhou X, Slabaugh G, Yan C and Yuan S
Proceedings of the 30th ACM International Conference on Multimedia., 1954-1963.

DOI 10.1145/3503161.3547823

10-10-2022

Learning Frequency Domain Priors for Image Demoireing
Zheng B, Yuan S, Yan C, Tian X, Zhang J, Sun Y, Liu L, Leonardis A and Slabaugh G
Ieee Transactions On Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers (Ieee) vol. 44 (11), 7705-7717.

DOI 10.1109/tpami.2021.3115139

QMRO

04-10-2022

SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-trained Siamese Transformers
Liu L, Yuan S, Liu J, Guo X, Yan Y and Tian Q
Proceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022. vol. 36, 1747-1755.
30-06-2022

Constrained Predictive Filters for Single Image Bokeh Rendering
Zheng B, Chen Q, Yuan S, Zhou X, Zhang H, Zhang J, Yan C and Slabaugh G
Ieee Transactions On Computational Imaging, Institute of Electrical and Electronics Engineers (Ieee) vol. 8, 346-357.

DOI 10.1109/tci.2022.3171417

QMRO

03-05-2022

Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment
Hu X, Li X, Busam B, Zhou Y, Leonardis A and Yuan S
BMVC 2022 - 33rd British Machine Vision Conference Proceedings.
01-01-2022

TAPE: Task-Agnostic Prior Embedding for Image Restoration
Liu L, Xie L, Zhang X, Yuan S, Chen X, Zhou W, Li H and Tian Q
Lecture Notes in Computer Science. vol. 13678, 447-464.

DOI 10.1007/978-3-031-19797-0_26

01-01-2022

2021

CBREN: Convolutional Neural Networks for Constant Bit Rate Video Quality Enhancement
Zhao H, Zheng B, Yuan S, Zhang H, Yan C, Li L and Slabaugh G
Ieee Transactions On Circuits and Systems For Video Technology, Institute of Electrical and Electronics Engineers (Ieee) vol. 32 (7), 4138-4149.

DOI 10.1109/tcsvt.2021.3123621

QMRO

27-10-2021

2020

NTIRE 2020 Challenge on Image Demoireing: Methods and Results
Yuan S, Timofte R, Leonardis A, Slabaugh G, Luo X, Zhang J, Qu Y, Hong M, Xie Y, Li C, Xu D, Chu Y, Sun Q, Liu S, Zong Z, Nan N, , Kim S, Nam H, Kim J, et al.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). vol. 00, 1882-1893.

DOI 10.1109/cvprw50498.2020.00238

19-06-2020

Video Super-resolution with Temporal Group Attention
Isobe T, Li S, Jia X, Yuan S, Slabaugh G, Xu C, Li Y-L, Wang S and Tian Q
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). vol. 00, 8005-8014.

DOI 10.1109/cvpr42600.2020.00803

13-06-2020

Image Demoireing with Learnable Bandpass Filters
Zheng B, Yuan S, Slabaugh G and Leonardis A
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). vol. 00, 3633-3642.

DOI 10.1109/cvpr42600.2020.00369

13-06-2020

Self-adaptively learning to Demoiré from focused and defocused image pairs
Liu L, Yuan S, Liu J, Bao L, Slabaugh G and Tian Q
Advances in Neural Information Processing Systems. vol. 2020-December
01-01-2020

Wavelet-Based Dual-Branch Network for Image Demoiréing
Liu L, Liu J, Yuan S, Slabaugh G, Leonardis A, Zhou W and Tian Q
Lecture Notes in Computer Science. vol. 12358, 86-102.

DOI 10.1007/978-3-030-58601-0_6

01-01-2020

2019

AIM 2019 Challenge on Image Demoireing: Dataset and Study
Yuan S, Timofte R, Slabaugh G and Leonardis A
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). vol. 00, 3526-3533.

DOI 10.1109/iccvw.2019.00437

27-10-2019

3D Hand Pose Estimation from RGB Using Privileged Learning with Depth Data
Yuan S, Stenger B and Kim T-K
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). vol. 00, 2866-2873.

DOI 10.1109/iccvw.2019.00348

27-10-2019

AIM 2019 Challenge on Image Demoireing: Methods and Results
Yuan S, Timofte R, Slabaugh G, Leonardis A, Zheng B, Ye X, Tian X, Chen Y, Cheng X, Fu Z, Yang J, Hong M, Lin W, Yang W, Qu Y, Shin H-K, Kim J-Y, Ko S-J, Dong H, Guo Y, et al.
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). vol. 00, 3534-3545.

DOI 10.1109/iccvw.2019.00438

27-10-2019

2018

Opening the Black Box: Hierarchical Sampling Optimization for Hand Pose Estimation
Tang D, Ye Q, Yuan S, Taylor J, Kohli P, Keskin C, Kim T-K and Shotton J
Ieee Transactions On Pattern Analysis and Machine Intelligence, Institute of Electrical and Electronics Engineers (Ieee) vol. 41 (9), 2161-2175.

DOI 10.1109/tpami.2018.2847688

15-06-2018

First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations
Garcia-Hernando G, Yuan S, Baek S and Kim T-K
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition., 409-419.

DOI 10.1109/cvpr.2018.00050

01-06-2018

Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
Yuan S, Hernando GG, Stenger B, Moon G, Chang JY, Lee KM, Molchanov P, Kautz J, Honari S, Ge L, Yuan J, Chen X, Wang G, Yang F, Akiyama K, Wu Y, Wan Q, Madadi M, Escalera S, Li S, et al.
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition., 2636-2645.

DOI 10.1109/cvpr.2018.00279

01-06-2018

2017

BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis
Yuan S, Ye Q, Stenger B, Jain S and Kim T-K
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)., 2605-2613.

DOI 10.1109/cvpr.2017.279

01-07-2017

2016

Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation
Ye Q, Yuan S and Kim T-K
Lecture Notes in Computer Science. vol. 9912, 346-361.

DOI 10.1007/978-3-319-46484-8_21

01-01-2016