Precognition Lab @ HKUST (Guangzhou)
Our research lab, the Precognition Lab (智能感知与预测实验室), is interested in building human-level Embodied AI systems that can effectively perceive, reason and interact with the real world for the good of humans. Here is an up-to-date research roadmap.
Our lab's computing resources include 32 RTX 3090/4090 GPUs and a cluster of 24 A6000 GPUs with a 100TB NAS. See this post. And we have multiple mobile platforms with robot arms and dex hands:

Check out our lab's cool publications and demos.
Our lab has over 10K followers on social media： [Rong's 知乎] [Yujin's 知乎] [Junwei's 知乎] [Junwei's 小红书] [Junwei's LinkedIn]
Media Coverage
News
Lab Members
Publications
* indicates corresponding authors.
    Preprint.
  1. GLOVER: Generalizable Open-Vocabulary Affordance Reasoning for Task-Oriented Grasping
    Teli Ma, Zifan Wang, Jiaming Zhou, Mengmeng Wang, Junwei Liang*
    [Paper] [Project page] [Code]
  2. SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Grounding
    Rong Li, Shijie Li, Lingdong Kong, Xulei Yang, Junwei Liang*
    [Paper] [Project page] [Code]
  3. From Cognition to Precognition: A Future-Aware Framework for Social Navigation
    Zeying Gong, Tianshuai Hu, Ronghe Qiu, Junwei Liang*
    [Paper] [Project page]
  4. Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation
    Jiaming Zhou, Teli Ma, Kun-Yu Lin, Ronghe Qiu, Zifan Wang, Junwei Liang*
    [Paper] [Project page]
  5. Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Maps
    Dicong Qiu, Wenzong Ma, Zhenfu Pan, Hui Xiong, Junwei Liang*
    [Paper] [Project page] [Video]
  6. Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation
    Teli Ma, Jiaming Zhou, Zifan Wang, Ronghe Qiu, Junwei Liang*
    CoRL 2024
    [Paper] [Project page] [Video]
  7. Prioritized Semantic Learning for Zero-shot Instance Navigation
    Xinyu Sun, Lizhao Liu, Hongyan Zhi, Ronghe Qiu, Junwei Liang*
    ECCV 2024
    [Paper] [Dataset/Code/Model]
  8. Dragtraffic: Interactive and Controllable Traffic Scene Generation for Autonomous Driving
    Sheng WANG, Ge SUN, Fulong MA, Tianshuai HU, Qiang QIN, Yongkang SONG, Lei ZHU, Junwei Liang*
    IROS 2024
    [Paper] [Project page] [Code]
  9. Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Models
    Xiaoyu Zhu, Hao Zhou, Pengfei Xing, Long Zhao, Hao Xu, Junwei Liang, Alexander Hauptmann, Ting Liu, Andrew Gallagher
    ECCV 2024
    [Paper] [Project Page]
  10. An Examination of the Compositionality of Large Generative Vision-Language Models
    Teli Ma, Rong Li, Junwei Liang*
    NAACL 2024
    [Paper] [Project Page] [Dataset] [知乎]
  11. FinTextQA: A Dataset for Long-form Financial Question Answering
    Jian Chen, Peilin Zhou, Yining Hua, Yingxin Loh, Kehui Chen, Ziyuan Li, Bing Zhu*, Junwei Liang*
    ACL 2024
    [Paper] [Code/Model]
  12. VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting
    Yujin Tang, Peijie Dong, Zhenheng Tang, Xiaowen Chu, Junwei Liang*
    CVPR 2024 Precognition Workshop
    [Paper] [Project Page/Code/Model]
  13. PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series Forecasting
    Zeying Gong, Yujin Tang, Junwei Liang*
    IJCAI 2024 Workshop: DATA SCIENCE MEETS OPTIMISATION
    [Paper] [Project Page/Code/Model]
  14. PostRainBench: A comprehensive benchmark and a new model for precipitation forecasting
    Yujin Tang, Jiaming Zhou, Xiang Pan, Zeying Gong, Junwei Liang*
    ICLR 2024 Workshop: Tackling Climate Change with Machine Learning (Spotlight paper)
    [Paper] [Project Page/Code/Model]
  15. TFNet: Exploiting Temporal Cues for Fast and Accurate LiDAR Semantic Segmentation
    Rong Li, ShiJie Li, Xieyuanli Chen, Teli Ma, Wang Hao, Juergen Gall, Junwei Liang*
    CVPR 2024 Workshop on Autonomous Driving
    [Paper]
  16. STMT: A Spatial-Temporal Mesh Transformer for MoCap-Based Action Recognition
    Xiaoyu Zhu, Po-Yao Huang, Junwei Liang, Celso M de Melo, Alexander G Hauptmann
    CVPR 2023
    [Paper] [Project Page/Code/Model]
  17. Multi-dataset Training of Transformers for Robust Action Recognition
    Junwei Liang, Enwei Zhang, Jun Zhang, Chunhua Shen
    NeurIPS 2022 (Spotlight paper, 3.7% acceptance rate, 384/10411)
    [Paper] [Project Page/Code/Model]
  18. The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction
    Junwei Liang, Lu Jiang, Kevin Murphy, Ting Yu, Alexander Hauptmann
    CVPR 2020  
    [Paper] [BibTex] [Demo Video] [Project Page/Code/Model] [blog] [知乎] [Google Research] [读芯术学术报告] [Invited presentation at ICPR'20 pattern forecasting workshop]
  19. Peeking into the Future: Predicting Future Person Activities and Locations in Videos
    Junwei Liang, Lu Jiang, Juan Carlos Niebles, Alexander Hauptmann, Li Fei-Fei
    CVPR 2019 (Translated and reported by multiple Chinese media (量子位 & 机器之心, 02/13/2019), with 30k+ views in a week.)
    #1 Tensorflow-based code on PaperWithCode in Trajectory Prediction task.
    [Paper] [BibTex] [Demo Video] [Project Page/Code/Model] [Google Research]