Skip to content

Latest commit

 

History

History
86 lines (60 loc) · 8.11 KB

README.md

File metadata and controls

86 lines (60 loc) · 8.11 KB

Seminar: Machine Learning for Language and Vision

News

May 9, 2023. Our grading standard is online.

Apr 26, 2023. Our tentative schedule is online. Please contact us ASAP if you have any questions.

Apr 17, 2023. Our kickoff meeting is scheduled for April 18, 2023, from 12:15 to 13:45, at C7 3 - Seminarraum 1.12. As the demand for attendance is exceedingly high, we have added more papers to our list. We kindly request that you attend the first two meetings to secure your spot. In the event that you are unable to attend but would still like to participate, please send us an email and we will arrange remote attendance for you.

Mar 27, 2023. If you are interested, please send Xudong Hong an email to register for this seminar.

Schedule

Date Topic Paper Title Presenter
May 9, 2023 Contrastive Pre-training - Image Learning transferable visual models from natural language supervision Raj Mohan Tumarada
May 9, 2023 Contrastive Pre-training - Image Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation Larisa Ivanova
May 16, 2023 Seq2seq Pre-training - Image Simple Visual Language Model Pretraining with Weak Supervision Julian Schlenker
May 16, 2023 Seq2seq Pre-training - Image Flava: A foundational language and vision alignment model Mehrad Zamani
May 23, 2023 Pre-training - Video Merlot: Multimodal neural script knowledge models Yage Zhang
May 23, 2023 Pre-training - Video Merlot reserve: Neural script knowledge through vision and language and sound -
May 30, 2023 Multitask Learning Unifying Vision-and-Language Tasks via Text Generation Nitish Juttu
May 30, 2023 Multitask Learning Unit: Multimodal multitask learning with a unified transformer Zixuan Liu
June 6, 2023 Multitask Learning Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework Jakob Gürtler
June 6, 2023 Parameter Efficiency - Prompting An empirical study of gpt-3 for few-shot knowledge-based vqa Raphael Maximilian Stephan Maser
June 13, 2023 Parameter Efficiency - Prompting Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language Karen Li
June 13, 2023 Parameter Efficiency - Prompt Tuning Multimodal few-shot learning with frozen language models Muhammad Anas Tahir
June 20, 2023 Parameter Efficiency - Prompt Tuning Transitional adaptation of pretrained models for visual storytelling Mahnoor Shahid
June 20, 2023 Parameter Efficiency - Prefix-Tuning Hyperpelt: Unified parameter-efficient language model tuning for both language and vision-and-language tasks Sijie Wu
June 27, 2023 Parameter Efficiency - Prefix-Tuning Visual Prompt Tuning Prathvish Mithare
June 27, 2023 Parameter Efficiency - Adapters Vl-adapter: Parameter-efficient transfer learning for vision-and-language tasks Muhammed Saeed
July 4, 2023 Parameter Efficiency - Adapters LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning Rajkumar Anilkumar Vaghashiya
July 4, 2023 Generative Model - Text-to-Image High-resolution image synthesis with latent diffusion models Shreyash Arya
July 11, 2023 Generative Model - GPT GPT-4, GPT-4 Technical Report Abdul Rafay
July 11, 2023 Generative Model - GPT Visual ChatGPT: Talking, Drawing, and Editing with Visual Foundation Models -
July 18, 2023 Reinforcement Learning No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling -
July 18, 2023 Reinforcement Learning What Makes a Good Story? Designing Composite Rewards for Visual Storytelling -
July 25, 2023 Summary Xudong Hong and Ruitao Feng

Introduction

Please find it here.

Topics

Please find them here.

Grading

  • 10% draft presentation (due each Wednesday)
  • 10% questions about the papers (due each Friday)
  • 35% final talk
  • 5% Attendance of all the talks and giving feedback
  • 5% Discussion during the talk with the others
  • 35% Term paper on your understanding of the paper. 5 pages, using ACL 2023 template
  • (optional) 10% Demo

Representations

  1. [Kickoff Meeting](./Seminar ML for Language and Vision_kickoff.pdf)
  2. [Introduction Meeting](./Seminar ML for Language and Vision_introduction.pdf)

Requirement

Students should have a basic understanding of deep learning, natural language processing and computer vision concepts. Students are expected to actively engage in discussions and critically analyze the papers presented during the seminar. They are also encouraged to share their own insights and perspectives on the topics covered.

Discussion Format

We will have a group discussion on each paper, where participants need to first present the papers. Then others can share their thoughts and insights on the research.

Date and Time

every Tue 12:15-13:45

kick-off meeting on Apr 18, 2023 Location: Gebäude C7 3 - Seminarraum 1.12

Contact

If you have any questions or concerns, please contact us via email. We look forward to seeing you at the discussion!

Xudong Hong: [email protected]

Ruitao Feng: [email protected]

(The following is under construction. Please stay tuned. )