Sihan XU (Si.X)

Sihan XU (Si.X)

University of Michigan, Ann Arbor

Welcome to my homepage! I’m a undergraduate student of Computer Science at the University of Michigan and an Undergraduate Research Assistant at SLED lab and VisionX. Before transferring to the University of Michigan, I’m a student at the University of Nottingham, pursuing B.S. CS with AI. I’m interested in unsupervised/self-supervised learning and generative models, also interested in doing stupid things. Feedbacks are always welcome~

My research journey began with a strong focus on generative models in computer vision, particularly in generative adversarial networks (GANs) and diffusion models. Currently, my work has expanded to encompass a broader range of topics, including computer vision, natural language processing, 3D models and robotics.

I think I will focus on the following topics from first principles and Occam’s razor in the future:

  • Multimodal Representation Learning, especially in unsupervised methods.

  • Causal Generative Model, not only just about autoregressive models or diffusion models.

  • Machine Learning, something more than current learning approaches.

I am actively looking for PhD position to start in 2025 Fall.

News

  • 2024.11:   One paper get accepted by TMLR.
  • 2024.09:   One paper get accepted by NeurIPS2024.
  • 2024.02:   InfEdit to appear in CVPR 2024.
  • 2023.10:   CycleNet to appear in NeurIPS 2023, and the preprint is available.
  • 2022.11:   The preprint version and demos of ACE are available.

Publications

Preprint
sym

SAB3R: Semantic-Augmented Backbone in 3D Reconstruction

Xuweiyi Chen*, Tian Xia*, Sihan XU, Jianing Yang, Joyce Chai, Zezhou Cheng

Coming soon.

TMLR
sym

[TMLR]UniCtrl: Improving the Spatiotemporal Consistency of Text-to-Video Diffusion Models via Training-Free Unified Attention Control

Tian Xia*, Xuweiyi Chen*, Sihan XU†

†Correspondence

[Project][Arxiv][Demo][Code]

NeurIPS 2024
sym

[NeurIPS2024]Multi-Object Hallucination in Vision-Language Models

Xuweiyi Chen*, Ziqiao Ma*, Xuejun Zhang*, Sihan XU, Shengyi Qian, Jianing Yang, David F. Fouhey, Joyce Chai

[Project][Arxiv][Code][Dataset]

CVPR 2024
sym

[CVPR2024]Inversion-Free Image Editing with Natural Language

Sihan XU*, Yidong Huan*, Jiayi Pan, Ziqiao Ma, Joyce Chai

*First author

[Project][Arxiv][Demo][Code]

[UMich CSE News Coverage]

[🏆Top2 at GenAI-Arena Leaderboard]

NeurIPS 2023
sym

[NeurIPS2023]CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulation

Sihan XU*, Ziqiao Ma*, Yidong Huang, Honglak Lee, Joyce Chai

*First author

[Project][Arxiv][Code]

[UMich CSE News Coverage]

Michigan AI Symposium 2022
sym

ACE: Zero-Shot Image to Image Translation via Pretrained Auto-Contrastive-Encoder

Sihan XU*†, Zelong Jiang*, Ruisi Liu*, Kaikai Yang, Zhijie Huang

*First author and †Correspondence

[Arxiv][Demo]

Projects

  • PixAI.art Anime Video Model

    Large scale and state of the art anime video model (at that time).

  • Rethink the Noise Prior of Initialization Gap in Video Diffusion Models

    Tian Xia, Yinuo Yang, Sihan XU

    EECS 442 Cource Project

  • Transferwiki - Founder

    TransferWiki is a platform created to assist students from mainland China who are planning to transfer to universities abroad, particularly in the United States, Canada, and the United Kingdom. This platform addresses the challenges and information asymmetry faced by students during the transfer process.

Experience

Talks

  • 2023.12, Open Vocabulary Image Processing via Diffusion Models @ SLED.

Honors and Awards

  • University Honors.
  • Silver (TOP 4%) at Google Smartphone Decimeter Challenge Competition.
  • TOP 1 at UoN Hackathon.

Educations

  • 2022.09 - 2025.06, B.S Computer Science with AI, University of Michigan (Pursuing Honor Degree)
  • 2020.09 - 2022.06, BSc Hons Computer Science with AI, University of Nottingham (First Class Honor)

Service

  • Conference reviewer: CVPR, EMNLP, ICLR

sihanxu