-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathindex.html
258 lines (229 loc) · 20.1 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
<!DOCTYPE html>
<html lang="en">
<title>CSCW User AI Auditing 2023</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="https://unpkg.com/tachyons/css/tachyons.min.css">
<link rel="stylesheet" href="./style.css">
<body class="sans-serif">
<header class="sans-serif">
<div class="cover bg-left bg-center-l accent-bg">
<div class="pb4 pb4-m pb4-l">
<nav class="dt w-100 mw8 center">
<div class="dtc w6 v-mid pa1">
<a href="" class="f4 fw5 dib no-underline grow white pa2 grow">
CSCW User AI Auditing '23
</a>
</div>
<div class="dtc v-mid tr pa3">
<a class="f6 fw4 hover-white no-underline white-70 dn dib-ns pv2 ph3" href="#cfp">Call for
Participation</a>
<a class="f6 fw4 hover-white no-underline white-70 dn dib-l pv2 ph3" href="#info">Key Information</a>
<a class="f6 fw4 hover-white no-underline white-70 dn dib-l pv2 ph3" href="#agenda">Agenda</a>
</div>
</nav>
<div class="tc-l mt4 mt5-m mt6-l ph5 w-75 center">
<h1 class="f2 f1-l fw6 white-90 mb0 lh-title">Supporting User Engagement in Testing, Auditing, and
Contesting AI</h1>
<h2 class="fw3 f3 white-80 mt3 mb4">CSCW 2023 Workshop<br>Sunday, October 15, 2023, 10am-5pm</h2>
<a class="f6 link grow br3 ba bw1 ph3 pv2 mb2 dib white" href="https://forms.gle/rqdFDNNy95V7TugR9" target="_blank">→ Submission Form (deadline 9/20/23)</a>
</div>
</div>
</div>
</header>
<div class="text-section-padded">
<article class="cf ph3 ph5-ns pv5" id="overview">
<header class="fn fl-ns w-30-ns pr4-ns">
<h1 class="f3 lh-title fw5 mb3 mt0 pt3 bt bw2 accent-txt">
Overview
</h1>
</header>
<div class="fn fl-ns w-70-ns">
<p class="f5 lh-copy mt0-ns">
In recent years, there has been a growing interest in <b>involving end users directly in testing, auditing, and contesting AI systems</b>. The involvement of end users from diverse backgrounds can be essential to overcome AI developers' blind spots and to surface issues that would otherwise go undetected prior to causing real-world harm. Emerging bodies of work in CSCW and HCI have begun to explore ways to <b>engage end-users in testing and auditing AI systems</b>, and to empower users to <b>contest erroneous AI outputs</b>. However, we know little about how to support effective user engagement.
</p>
<p class="f5 lh-copy">
In this one-day workshop at <a href="https://cscw.acm.org/2023/" target="_blank" class="accent-txt">CSCW 2023</a>, we will bring together <b>researchers and practitioners from academia, industry, and non-profit organizations</b> to share ongoing efforts related to this workshop's theme. Central to our discussions will be the challenges encountered in developing tools and processes to support user involvement, strategies to incentivize involvement, the asymmetric power dynamic between AI developers and end users, and the role of regulation in enhancing the accountability of AI developers and ameliorating potential burdens towards end-users. Overall, we hope the workshop outcome could orient the future of user engagement in building more responsible AI.
</p>
</div>
</article>
<article class="cf ph3 ph5-ns pv5" id="cfp">
<header class="fn fl-ns w-30-ns pr4-ns">
<h1 class="f3 lh-title fw5 mb3 mt0 pt3 bt bw2 accent-txt">
Call for Participation
</h1>
</header>
<div class="fn fl-ns w-60-ns">
<p class="f5 lh-copy mt0-ns">
We welcome participants who work on related areas in supporting user engagement in testing, auditing, and contesting AI. Interested participants will be asked to contribute a brief statement of interest to the workshop. Submissions can take several forms:
<ol>
<li class="pv1"><b>Position paper or Paper Draft</b> discussing or contributing to one or more themes highlighted in this proposal. Paper drafts may be under submission, and there are no page limits.</li>
<li class="pv1"><b>Video or audio demo of an interactive system</b> that is relevant to user-engagement in AI testing, auditing, and contesting. Submissions should fall within 3-5 minutes in length.</li>
<li class="pv1"><b>"Encore" submission</b> of a highly-relevant conference or journal paper.</li>
<li class="pv1"><b>Statement of research interest</b> for attending the workshop. Submissions should be in ACM single column format and no longer than 1 page, excluding references.
</li>
</ol>
</p>
<p class="f5 lh-copy">
Each submission will be reviewed by 1-2 organizers and accepted based on quality of the submission and diversity of perspectives to allow for a meaningful exchange of knowledge between a broad range of stakeholders.
</p>
<p>
<a class="f6 link grow br3 ba bw1 ph3 pv2 mb2 dib accent-txt" href="https://forms.gle/rqdFDNNy95V7TugR9" target="_blank">→ Submission Form (deadline 9/20/23)</a>
</p>
</div>
</article>
<article class="cf ph3 ph5-ns pv5" id="info">
<header class="fn fl-ns w-30-ns pr4-ns">
<h1 class="f3 lh-title fw5 mb3 mt0 pt3 bt bw2 accent-txt">
Key Information
</h1>
</header>
<div class="fn fl-ns w-70-ns">
<p><b>Submission deadline</b>: Wednesday, September 20, 2023, 11:59pm AoE (deadline extended!) <s>Friday, September 15, 2023, 11:59pm AoE</s></p>
<p><b>Notification of acceptance</b>: Monday, September 25, 2023</p>
<p><b>Workshop date</b>: Sunday, October 15, 2023, 10:00am-5:00pm</p>
<p><b>Workshop location</b>: Boundary Waters C, <a href="https://cscw.acm.org/2023/index.php/venue/" target="_blank" class="accent-txt">Hyatt Regency Minneapolis</a></p>
</div>
</article>
<article class="cf ph3 ph5-ns pv5" id="agenda">
<header class="fn fl-ns w-30-ns pr4-ns">
<h1 class="f3 lh-title fw5 mb3 mt0 pt3 bt bw2 accent-txt">
Agenda
</h1>
</header>
<div class="fn fl-ns w-70-ns">
<p class="f5 lh-copy mt0-ns">
The primary goal of this one-day, in-person workshop is to bring together researchers and AI practitioners from academia, industry, and non-profits to share their ongoing efforts around engaging end users in testing, auditing, and contesting AI systems.
<ul>
<li class="pv1"><b>Welcome and Introduction</b> <i>(10:00-10:15am)</i>: Organizers will welcome the participants, present the topic, and outline the format of the workshop session.</li>
<li class="pv1"><b>Coffee break</b> <i>(10:15-10:30am)</i>: Coffee and pastries are available at the hotel lobby provided by the CSCW organizers.</li>
<li class="pv1"><b>Author Presentation Group A</b> <i>(10:30-11:40am)</i>: Authors in Group A will briefly introduce their work. After the short presentation (5 - 6 min), each participant will have a short Q&A session from the audiences.</li>
<li class="pv1"><b>Panel A</b> <i>(11:40-12:00pm)</i>: One of the organizers will lead a panel discussion featuring authors from Group A, providing them with an extended opportunity to share their work and interests.</li>
<li class="pv1"><b>Lunch break</b> <i>(12:00-1:20pm)</i></li>
<li class="pv1"><b>Author Presentation Group B</b> <i>(1:20-2:30pm)</i>: Authors in Group B will briefly introduce their work. After the short presentation (5 - 6 min), each participant will have a short Q&A session from the audiences.</li>
<li class="pv1"><b>Coffee break</b> <i>(2:30-3:00pm)</i>: Coffee and pastries are available at the hotel lobby provided by the CSCW organizers.</li>
<li class="pv1"><b>Panel B</b> <i>(3:00-3:20pm)</i>: One of the organizers will lead a panel discussion featuring authors from Group B, providing them with an extended opportunity to share their work and interests.</li>
<li class="pv1"><b>Workshop Activity</b> <i>(3:20-4:20pm)</i>: We will discuss a set of questions in 4 groups, each facilitated by the organizers. Through this workshop activity, we aim to first develop a shared vision (or visions) for the preferred future of AI system testing, auditing, and contesting. We hope to identify and understand challenges in engaging end users effectively, and co-create and brainstorm potential solutions to address these challenges.</li>
<li class="pv1"><b>Workshop Report</b> <i>(4:20-4:50pm)</i>: Participants from each breakout group report back and discuss with the larger group.</li>
<li class="pv1"><b>Endnote</b> <i>(4:50-5:00pm)</i></li>
<li class="pv1"><b>(Optional) Dinner social event</b> <i>(starting 6:30pm)</i></li>
</ul>
</p>
</div>
</article>
<article class="cf ph3 ph5-ns pv5" id="agenda">
<header class="fn fl-ns w-30-ns pr4-ns">
<h1 class="f3 lh-title fw5 mb3 mt0 pt3 bt bw2 accent-txt">
Accepted Work
</h1>
</header>
<div class="fn fl-ns w-70-ns">
<p class="f5 lh-copy mt0-ns">
<h4>Workshop papers</h4>
<ul>
<li class="pv1"><b>Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making.</b> <i>Min Hun Lee and Chong Jun Chew.</i> Singapore Management University.</li>
<li class="pv1"><b>Answerability by Design in Sociotechnical Systems.</b> <i>Dilara Keküllüoğlu, Louise Hatherall, Nayha Sethi, Tillmann Vierkant, Shannon Vallor, Nadin Kokciyan, Michael Rovatsos.</i> The University of Edinburgh. [<a href="./media/papers/Answerability_by_Design.pdf" target="_blank">link</a>]</li>
<li class="pv1"><b>Has Wizard of Oz Testing Passed Its Use By Date?</b> <i>James Simpson.</i> Macquarie University. [<a href="./media/papers/Has_Wizard_of_Oz_Testing_Passed.pdf" target="_blank">link</a>]</li>
<li class="pv1"><b>GPTutor: an open-source AI pair programming tool alternative to Copilot.</b> <i>Eason Chen, Ray Huang, Bo Shen Liang, Damien Chen, Pierce Huang.</i> Carnegie Mellon University. [<a href="./media/papers/GPTutor.pdf" target="_blank">link</a>]</li>
<li class="pv1"><b>Enhancing User Engagement in AI Auditing Through Gamification and Storytelling.</b> <i>Weirui Peng, Mengyi Wei, Kyrie Zhixuan Zhou.</i> Columbia University, Technical University of Munich, University of Illinois at Urbana-Champaign. [<a href="./media/papers/Enhancing_User_Engagement_in_AI_Auditing.pdf" target="_blank">link</a>]</li>
<li class="pv1"><b>Uncovering Effective Steering Strategies in Code-Generating LLMs with Socratic Feedback.</b> <i>Zheng Zhang, Alex C. Williams, Jonathan Buck, Xiaopeng Li, Matthew Lease, Li Erran Li.</i> University of Notre Dame, AWS AI [<a href="./media/papers/Human_feedback_for_LLM_debugging.pdf" target="_blank">link</a>]</li>
<li class="pv1"><b>The Potential of Diverse Youth as Stakeholders in Identifying and Mitigating Algorithmic Bias for a Future of Fairer AI.</b> <i>Jaemarie Solyst, Ellia Yang, Shixian Xie, Amy Ogan, Jessica Hammer, Motahhare Eslami.</i> Carnegie Mellon University. [<a href="./media/papers/Potential_of_Diverse_Youth.pdf" target="_blank">link</a>]</li>
<li class="pv1"><b>Re-imagining Fairness in Machine Learning: A Framework for Building in Socio-cultural and Contextual Awareness</b> <i>Corey Jackson, Tallal Ahmed, Devansh Saxena.</i> University of Wisconsin - Madison, Carnegie Mellon University. [<a href="./media/papers/Re-imagining_Fairness_in_Machine_Learning.pdf" target="_blank">link</a>]</li>
<li class="pv1"><b>Auditing Personalized Recommendation Algorithms Through Spontaneous Click-Based User Interaction</b> <i>Qunfang Wu, Zitong Huang, Yaqi Zhang, Chung-Chin Eugene Liu.</i> University of North Carolina at Chapel Hill, Syracuse University. [<a href="./media/papers/Auditing_Personalized_Recommendation_Algorithms.pdf" target="_blank">link</a>]</li>
<li class="pv1"><b>A Democratic Platform for Engaging with Disabled Community in Generative AI Development.</b> <i>Deepak Giri and Erin Brady.</i> Indiana University. [<a href="./media/papers/A_Democratic_Platform_for_Engaging.pdf" target="_blank">link</a>]</li>
<li class="pv1"><b>Patient-Facing Machine Learning for Prenatal Stress Reduction in the United States: A Co-design Toolkit.</b> <i>Mara Ulloa, Negar Kamali, Glenn J Fernandes, Elizabeth Soyemi, Miranda Beltzer, Niharika Gopinath Menon, Nabil Alshurafa, Maia Jacobs.</i> Northwestern University.</li>
<li class="pv1"><b>Social Network Timeline Bias and Amplification.</b> <i>Nathan Bartley.</i> University of Southern California. [<a href="./media/papers/Social_Network_Timeline_Bias_and_Amplification.pdf" target="_blank">link</a>]</li>
<li class="pv1"><b>(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court.</b> <i>Angela Jin et al.</i> University of California, Berkeley.</li>
</ul>
<h4>Statements of Interest</h4>
<ul>
<li class="pv1"><b>Creator-driven Auditing of YouTube Algorithms.</b> <i>Yoonseo Choi and Juho Kim.</i> KAIST.</li>
<li class="pv1"><b>Challenges and opportunities for user-engagement in testing and contesting AI: Early findings from Latin America.</b> <i>Claudia Lopez.</i> Universidad Técnica Federico Santa Maria.</li>
<li class="pv1"><b>Curating AI: Trust as the User's Compass.</b> <i>Ruyuan Wan.</i> University of Notre Dame.</li>
<li class="pv1"><b>Centering user perspectives in an interdisciplinary AI training data quality framework. Experiences from the KITQAR project.</b> <i>Lou Therese Brandner and Simon David Hirsbrunner.</i> University of Tübingen.</li>
<li class="pv1"><b>Engaging Users in Auditing Generative AI Systems by Crowdsourcing Prompts.</b> <i>Howard Han.</i> Carnegie Mellon University.</li>
</ul>
</p>
</div>
</article>
<article class="cf ph3 ph5-ns pv5" id="info">
<header class="fn fl-ns w-30-ns pr4-ns">
<h1 class="f3 lh-title fw5 mb3 mt0 pt3 bt bw2 accent-txt">
Organizers
</h1>
</header>
<div class="fn fl-ns w-70-ns">
<section class="cf w-100 pa2-ns">
<div class="w-100 measure-wide">
<div class="fl w-third pa2">
<div class="aspect-ratio aspect-ratio--1x1">
<img src="./media/deng.jpg"
class="db bg-center cover aspect-ratio--object br-100" />
</div>
<a href="https://www.wesleydeng.com/" class="ph2 ph0-ns pb3 link db" target="_blank">
<p class="f5-ns mb0 fw6 black-90">Wesley Hanwen Deng</p>
<p class="f6 fw4 mt2 black-60">Carnegie Mellon University</p>
</a>
</div>
<div class="fl w-third pa2">
<div class="aspect-ratio aspect-ratio--1x1">
<img src="./media/lam.jpg"
class="db bg-center cover aspect-ratio--object br-100" />
</div>
<a href="https://michelle123lam.github.io/" class="ph2 ph0-ns pb3 link db" target="_blank">
<h3 class="f5 f4-ns mb0 black-90"></h3>
<p class="f5-ns mb0 fw6 black-90">Michelle S. Lam</p>
<p class="f6 fw4 mt2 black-60">Stanford University</p>
</a>
</div>
<div class="fl w-third pa2">
<div class="aspect-ratio aspect-ratio--1x1">
<img src="./media/cabrera.png"
class="db bg-center cover aspect-ratio--object br-100" />
</div>
<a href="https://cabreraalex.com/" class="ph2 ph0-ns pb3 link db" target="_blank">
<p class="f5-ns mb0 fw6 black-90">Ángel Alexander Cabrera</p>
<p class="f6 fw4 mt2 black-60">Carnegie Mellon University</p>
</a>
</div>
</div>
<div class="w-100 measure-wide">
<div class="fl w-third pa2">
<div class="aspect-ratio aspect-ratio--1x1">
<img src="./media/metaxa.jpg"
class="db bg-center cover aspect-ratio--object br-100" />
</div>
<a href="https://metaxa.net/" class="ph2 ph0-ns pb3 link db" target="_blank">
<p class="f5-ns mb0 fw6 black-90">Danaë Metaxa</p>
<p class="f6 fw4 mt2 black-60">University of Pennsylvania</p>
</a>
</div>
<div class="fl w-third pa2">
<div class="aspect-ratio aspect-ratio--1x1">
<img src="./media/eslami.jpg"
class="db bg-center cover aspect-ratio--object br-100" />
</div>
<a href="https://www.motahhare.com/" class="ph2 ph0-ns pb3 link db" target="_blank">
<p class="f5-ns mb0 fw6 black-90">Motahhare Eslami</p>
<p class="f6 fw4 mt2 black-60">Carnegie Mellon University</p>
</a>
</div>
<div class="fl w-third pa2">
<div class="aspect-ratio aspect-ratio--1x1">
<img src="./media/holstein.png"
class="db bg-center cover aspect-ratio--object br-100" />
</div>
<a href="https://www.thecoalalab.com/kenholstein" class="ph2 ph0-ns pb3 link db" target="_blank">
<p class="f5-ns mb0 fw6 black-90">Kenneth Holstein</p>
<p class="f6 fw4 mt2 black-60">Carnegie Mellon University</p>
</a>
</div>
</div>
</section>
</div>
</article>
</div>
<footer class="pv4 ph3 ph5-m ph6-l mid-gray">
<small class="f6 db tc">CSCW 2023.</small>
</footer>
</body>
</html>