Skip to content

Latest commit

 

History

History
37 lines (33 loc) · 3.4 KB

paper_Kanzhi_Cheng.md

File metadata and controls

37 lines (33 loc) · 3.4 KB

Kanzhi Cheng's Papers

  • OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis

    • Qiushi Sun, Kanzhi Cheng, Zichen Ding, Chuanyang Jin, Yian Wang, Fangzhi Xu, Zhenyu Wu, Chengyou Jia, Liheng Chen, Zhoumianze Liu, Ben Kao, Guohao Li, Junxian He, Yu Qiao, Zhiyong Wu
    • 🏛️ Institutions: Shanghai AI Lab, HKU, Johns Hopkins University, SJTU, Oxford, HKUST
    • 📅 Date: Dec 27, 2024
    • 📑 Publisher: arXiv
    • 💻 Env: [GUI]
    • 🔑 Key: [model], [dataset], [trajectory data], [reward model], [e2e model], [data synthesis], [OS-Genesis]
    • 📖 TLDR: This paper introduces OS-Genesis, an interaction-driven pipeline that automates the construction of high-quality and diverse GUI agent trajectory data without human supervision. By employing reverse task synthesis and a trajectory reward model, OS-Genesis enables effective end-to-end training of GUI agents, significantly enhancing their performance on online benchmarks.

  • OS-ATLAS: A Foundation Action Model For Generalist GUI Agents

    • Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, Yu Qiao
    • 🏛️ Institutions: Shanghai AI Lab, SJTU, HKU, MIT
    • 📅 Date: October 30, 2024
    • 📑 Publisher: arXiv
    • 💻 Env: [GUI]
    • 🔑 Key: [model], [dataset], [benchmark], [OS-Atlas]
    • 📖 TLDR: This paper introduces OS-Atlas, a foundational GUI action model designed to enhance GUI grounding and out-of-distribution tasks. The authors developed a toolkit to synthesize multi-platform GUI grounding data, resulting in a cross-platform corpus of over 13 million GUI elements. OS-Atlas demonstrates significant performance improvements across six benchmarks spanning mobile, desktop, and web platforms.

  • SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents

    • Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, Zhiyong Wu
    • 🏛️ Institutions: Nanjing University, Shanghai AI Lab
    • 📅 Date: January 19, 2024
    • 📑 Publisher: ACL 2024
    • 💻 Env: [GUI]
    • 🔑 Key: [model], [benchmark], [GUI grounding], [visual grounding]
    • 📖 TLDR: TBD.

  • Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language Models

    • Fangzhi Xu, Qiushi Sun, Kanzhi Cheng, Jun Liu, Yu Qiao, Zhiyong Wu
    • 🏛️ Institutions: Xi'an Jiaotong University, Shanghai AI Lab, HKU, Nanjing University
    • 📅 Date: November 2023
    • 📑 Publisher: arXiv
    • 💻 Env: [GUI (evaluated on web, math reasoning, and logic reasoning environments)]
    • 🔑 Key: [framework], [dataset], [neural-symbolic self-training], [online exploration], [self-refinement]
    • 📖 TLDR: This paper introduces ENVISIONS, a neural-symbolic self-training framework designed to improve large language models (LLMs) by enabling self-training through interaction with a symbolic environment. The framework addresses symbolic data scarcity and enhances LLMs' symbolic reasoning proficiency by iteratively exploring, refining, and learning from symbolic tasks without reinforcement learning. Extensive evaluations across web navigation, math, and logical reasoning tasks highlight ENVISIONS as a promising approach for enhancing LLM symbolic processing.