Our research lab, the Precognition Lab (智能感知与预测实验室), is interested in building human-level Embodied AI systems that can effectively perceive, reason and interact with the real world for the good of humans.
Here is an up-to-date research roadmap.
Our lab's computing resources include 32 RTX 3090/4090 GPUs and a cluster of 24 A6000 GPUs with a 100TB NAS. See this post.
And we have multiple mobile platforms with robot arms and dex hands:
Check out our lab's cool publications and demos.
Our lab has over 10K followers on social media:
[Rong's 知乎]
[Yujin's 知乎]
[Junwei's 知乎]
[Junwei's 小红书]
[Junwei's LinkedIn]
- 10/2024 广州市委书记郭永航到香港科技大学(广州)调研,雅可比机器人进行Demo演示 [广州新闻联播]
- 10/2024 国内外高校具身智能实验室盘点(香港、新加坡篇) [具身智能之心]
- 09/2024 “万亿”具身智能的师徒“江湖” [硅星人Pro]
- 07/2024 在世界人工智能大会发表“面向通用服务的具身智能”演讲 [上海WAIC] [联汇科技]
- 06/2024 在香港科技大学(广州)第二届INNOTECH展示机器狗和灵巧手Demo [香港科技大学(广州)第二届INNOTECH创科嘉年华再创辉煌]
- 06/2024 三沙卫视采访 [打造大湾区科创品牌盛会 香港科技大学(广州)创科嘉年华举办]
- 05/2024 WAIC · 云帆奖五周年:AI 青年,执掌未来十年的钥匙 [全球高校人工智能学术联盟]
- 04/2024 大模型“越狱” 如何监管开发者 [应邀 广州日报 采访] [HKUST(GZ) under the spotlight (April-May 2024, Issue 1) (Link)]
- 04/2024 他的代码在NASA上天,在港科广落地 [by HKUST(GZ)] [INFO Hub] [AI Thrust]
- 02/2024 华人CMU校友回国创业,自研具身智能机器人,致力于开放场景的商业化落地 [by DeepTech深科技]
- 12/2023 雅可比机器人获得2023新一代人工智能(深圳)创业大赛三等奖 [by 163 news, 雅可比机器人]
- 10/2023 Patch才是时序预测的王道? [by 圆圆的算法笔记, kaggle竞赛宝典] [Paper]
- 10/2023 TFNet:利用时间线索实现快速且精确的激光雷达语义分割 [by 自动驾驶专栏] [Paper]
- 06/2023 Honorable Mention at the International Robot Manipulation Competition, Robothon (2023机器人马拉松挑战赛) [by 香港科技大学(广州)公众号] [Robothon 2023]
- 11/2024 Presented "Towards General Service Embodied AI" at ARTS 2024. [自主机器人技术研讨会]
- 11/2024 My first PhD student, Xiaoyu Zhu, has successfully defended her thesis and will graduate from CMU. Congrats to Xiaoyu! [Learning Generalizable Visual Representations Towards Novel Viewpoints, Scenes and Vocabularies]
- 10/2024 Presented "Towards General Service Embodied AI" at Huawei and CCF-YOCSEF seminar.
- 09/2024 Presented "Towards General Service Embodied AI" at CCF/CSIG GAMES Seminar. [第九届计算机图形学与混合现实研讨会]
- 09/2024 One paper accepted at CoRL 2024.
- 09/2024 One paper accepted at NeurIPS 2024.
- 08/2024 2 papers accepted at IROS 2024.
- 09/2024 Presented "Towards General Service Embodied AI" at CAA's Seminar. [CAA中国自动化学会云讲座]
- 08/2024 Presented "Towards General Service Embodied AI" at CAAI's Embodied AI Seminar. [CAAI中国人工智能学会具身智能青年学者研讨会第五期] [Video Recording]
- 07/2024 Two papers accepted at ECCV 2024.
- 05/2024 1 paper accepted at NAACL 2024. 1 paper accepted at ACL 2024. Main conference.
- 02/2024 Serve as a panelist at the VALSE Embodied AI webinar. [VALSE] [bilibili]
- 02/2024 Co-organizing the The 6th workshop on Precognition: Seeing through the Future @CVPR 2024. [Call For Papers] [知乎] [小红书]
- 12/2023 Keynote speech at the CEII2023 Workshop [Schedule]
- 10/2023 Co-organizing the Open-world Visual Perception Workshop (“开放世界下的视觉感知和增强”主题论坛) @PRCV 2023 [Schedule]
- 01/2023 Co-organizing the The 5th workshop on Precognition: Seeing through the Future @CVPR 2023. [Call For Papers] [知乎]
- Teli Ma [Github] [Google Scholar] [知乎]
- Jiaming Zhou [Github] [Google Scholar]
- Zifan Wang [Github] [Google Scholar]
- Dicong Qiu [Github] [Google Scholar]
- Ronghe Qiu [Github] [Google Scholar]
- Zeying Gong [Github] [Google Scholar]
- Rong Li [知乎] [Github] [Google Scholar]
- Tianshuai Hu [Github] [Google Scholar]
- Jiayi Liu [Github]
- Xiaoyu Zhu (Ph.D. student @CMU, co-advising) [Github] [Twitter] [Google Scholar]
- Jinhui Ye [Github] [Google Scholar]
- Sheng Wang [Google Scholar] [Research Gate]
- Jian Chen (Now with HSBC and pursuing a PhD) [Google Scholar] [Github]
- Yujin Tang (Now with Ming-Hsuan Yang) [知乎] [Github]
- Xinyu Sun (Now at DJI) [Github] [Google Scholar]
* indicates corresponding authors.
-
Preprint.
-
GLOVER: Generalizable Open-Vocabulary Affordance Reasoning for Task-Oriented Grasping
Teli Ma, Zifan Wang, Jiaming Zhou, Mengmeng Wang, Junwei Liang*
-
From Cognition to Precognition: A Future-Aware Framework for Social Navigation
Zeying Gong, Tianshuai Hu, Ronghe Qiu, Junwei Liang*[Paper] [Project page]
-
Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation
Jiaming Zhou, Teli Ma, Kun-Yu Lin, Ronghe Qiu, Zifan Wang, Junwei Liang*[Paper] [Project page]
-
Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Maps
Dicong Qiu, Wenzong Ma, Zhenfu Pan, Hui Xiong, Junwei Liang*
-
Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation
Teli Ma, Jiaming Zhou, Zifan Wang, Ronghe Qiu, Junwei Liang*CoRL 2024
-
Prioritized Semantic Learning for Zero-shot Instance Navigation
Xinyu Sun, Lizhao Liu, Hongyan Zhi, Ronghe Qiu, Junwei Liang*ECCV 2024
-
Dragtraffic: Interactive and Controllable Traffic Scene Generation for Autonomous Driving
Sheng WANG, Ge SUN, Fulong MA, Tianshuai HU, Qiang QIN, Yongkang SONG, Lei ZHU, Junwei Liang*IROS 2024
-
Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Models
Xiaoyu Zhu, Hao Zhou, Pengfei Xing, Long Zhao, Hao Xu, Junwei Liang, Alexander Hauptmann, Ting Liu, Andrew GallagherECCV 2024[Paper] [Project Page]
-
An Examination of the Compositionality of Large Generative Vision-Language Models
Teli Ma, Rong Li, Junwei Liang*NAACL 2024
-
FinTextQA: A Dataset for Long-form Financial Question Answering
Jian Chen, Peilin Zhou, Yining Hua, Yingxin Loh, Kehui Chen, Ziyuan Li, Bing Zhu*, Junwei Liang*ACL 2024[Paper] [Code/Model]
-
VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting
Yujin Tang, Peijie Dong, Zhenheng Tang, Xiaowen Chu, Junwei Liang*CVPR 2024 Precognition Workshop
-
PatchMixer: A Patch-Mixing Architecture for Long-Term Time Series Forecasting
Zeying Gong, Yujin Tang, Junwei Liang*IJCAI 2024 Workshop: DATA SCIENCE MEETS OPTIMISATION
-
PostRainBench: A comprehensive benchmark and a new model for precipitation forecasting
Yujin Tang, Jiaming Zhou, Xiang Pan, Zeying Gong, Junwei Liang*ICLR 2024 Workshop: Tackling Climate Change with Machine Learning (Spotlight paper)
-
TFNet: Exploiting Temporal Cues for Fast and Accurate LiDAR Semantic Segmentation
Rong Li, ShiJie Li, Xieyuanli Chen, Teli Ma, Wang Hao, Juergen Gall, Junwei Liang*CVPR 2024 Workshop on Autonomous Driving[Paper]
-
STMT: A Spatial-Temporal Mesh Transformer for MoCap-Based Action Recognition
Xiaoyu Zhu, Po-Yao Huang, Junwei Liang, Celso M de Melo, Alexander G HauptmannCVPR 2023
- Multi-dataset Training of Transformers for Robust Action Recognition
Junwei Liang, Enwei Zhang, Jun Zhang, Chunhua ShenNeurIPS 2022 (Spotlight paper, 3.7% acceptance rate, 384/10411)- The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction
Junwei Liang, Lu Jiang, Kevin Murphy, Ting Yu, Alexander HauptmannCVPR 2020- Peeking into the Future: Predicting Future Person Activities and Locations in Videos
Junwei Liang, Lu Jiang, Juan Carlos Niebles, Alexander Hauptmann, Li Fei-FeiCVPR 2019 (Translated and reported by multiple Chinese media (量子位 & 机器之心, 02/13/2019), with 30k+ views in a week.)#1 Tensorflow-based code on PaperWithCode in Trajectory Prediction task. - Multi-dataset Training of Transformers for Robust Action Recognition