-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
260 lines (228 loc) · 11.9 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="">
<meta name="author" content="">
<link rel="icon" href="../../favicon.ico">
<title>Human(e) AI: Age Adaptation</title>
<link href="css/bootstrap.min.css" rel="stylesheet">
<link href="css/signin.css" rel="stylesheet">
<script src="js/ie-emulation-modes-warning.js"></script>
<link rel="stylesheet" href="style.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bulma@0.9.4/css/bulma.min.css">
</head>
<body>
<section class="section">
<div class="container">
<div class="columns">
<div class="column">
<h1 class="title">Exploring Age Adaptation of Conversational Systems</h1>
<br>
<h2 class="subtitle">About</h2>
<p>What if conversational agents would be able to adapt in the way they interact with humans? Unfortunately, state-of-the-art AI systems currently lack sophisticated adaptation capabilities. Our long-term goal is to develop artificial agents that can adapt to individuals/user groups at any level (age, expertise, language style, etc.) and that are perceived as trustable by users.</p>
<br>
<p>In the following project, we focused on users of different age groups, and asked:</p>
<br>
<ul>
<li> <b>•</b> <i>Can some degree of adaptation be achieved by training a state-of-the-art system with data targeted to specific age groups? </i></li>
<li> <b>•</b> <i>Does this lead to differences in the perceived degree of anthropomorphism, social presence, trust, appreciation, and comprehensibility of the message? </i></li>
</ul>
<br>
<p>Below you can find an open-source demo that showcases the results of the project regarding the first of these research questions.</p>
<br>
<p>This project is a collaboration between the <a href="https://dmg-illc.github.io/dmg/">Dialogue Modelling Group</a> at the Institute for Logic, Language and Computation (ILLC) and the Persuasive Communication group at the <a href="https://ascor.uva.nl/">Amsterdam School of Communication Research (ASCoR)</a> at the University of Amsterdam.</p>
</div>
<div class="column">
<!--add image here-->
<img src="ages.gif" alt="image" width="800" height="800">
<br>
<p style="text-align: right;"><i>Illustrative image generated with <a href="https://openai.com/dall-e-2/">DALL·E 2</a></i></p>
</div>
</div>
<br>
<h2 class="subtitle">System adaptation</h2>
<p>Try it yourself. The state-of-the-art language model would first receive a prompt with corresponding criteria to generate an adapted output. Select a combinations from the options below to see what kind of adaptation different systems produced:</p>
<div id="prompts" class="row">
<div class="column column-left">
<h3 class="action-step">Prompt</h3>
</div>
<div class="column column-right">
<div id="prompts-btns">
<button id="prompt-button-1" class="prompt-button">Hey.</button>
<button id="prompt-button-2" class="prompt-button">Hello, tell me about your latest holiday.</button>
<button id="prompt-button-3" class="prompt-button">Hi, how's it going?</button>
<button id="prompt-button-4" class="prompt-button">Can we talk?</button>
<button id="prompt-button-5" class="prompt-button">Good weather we're having.</button>
</div>
</div>
</div>
<div id='adaptation' hidden>
<div class="row-above">
<div class="column arrow">
<h3 class="action-step">↓</h3>
</div>
<div class="column not-sure">
<button id="not-sure-button-1" class="not-sure_button">Show details +</button>
</div>
<div>
<p hidden class="step-explanation" id="step-expl-1">
This option allows you to choose whether the resulting output will be adapted towards a particular age group or not.
<br>
<br>
In the non-adaptive setting, the model takes a prompt and generates an output by sampling from the distribution of words that are most likely to follow the prompt.
<br>
<br>
In the age adapted setting, the model is trying to generate text that is controlled for age.</p>
</div>
</div>
<div class="row">
<div class="column column-left">
<h3 class="action-step">Adaptation</h3>
</div>
<div class="column column-right">
<div id="adaptation-btns">
<button id="adaptation-button-1" class="adaptation-button">Adapted</button>
<button id="adaptation-button-2" class="adaptation-button">Non-adapted</button>
</div>
</div>
</div>
</div>
<div id="model" hidden>
<div class="row-above">
<div class="column arrow">
<h3 class="action-step">↓</h3>
</div>
<div class="column not-sure">
<button id="not-sure-button-2" class="not-sure_button">Show details +</button>
</div>
<div>
<p hidden class="step-explanation" id="step-expl-2">
The Bag of Words method relies on lists of words that are representative of each age group’s language. The words are automatically extracted from the full version of the dataset via a frequency-based approach. In particular, for each age group, the words are ordered by frequency and the most frequent words are kept. The words that are in both age groups are removed. This leaves a list of words that are specific to each age group.
<br>
<br>
The discriminator model is a classifier that is trained to distinguish between two groups. In this case, the classifier is used to distinguish between two groups. The classifier is trained using a pre-trained language model. The discriminator parameters that are then used in this age-adaptive setting come from the training epoch with the highest test accuracy.
</p>
</div>
</div>
<div class="row">
<div class="column column-left">
<h3 class="action-step">Model</h3>
</div>
<div class="column column-right">
<div id="model-btns">
<button id="model-button-1" class="model-button">Bag of Words</button>
<button id="model-button-2" class="model-button">Discriminator</button>
</div>
</div>
</div>
</div>
<div id="style" hidden>
<div class="row-above">
<div class="column arrow">
<h3 class="action-step">↓</h3>
</div>
<div class="column not-sure">
<button id="not-sure-button-3" class="not-sure_button">Show details +</button>
</div>
<div>
<p hidden class="step-explanation" id="step-expl-3">This option allows you to adapt the example model output towards a younger or an older age group. The age adaption is evaluated using a classifier model. You can see the evaluation of the classifier model along with the resulting output.</p>
</div>
</div>
<div class="row">
<div class="column column-left">
<h3 class="action-step">Style</h3>
</div>
<div class="column column-right">
<div id="style-btns">
<button id="style-button-1" class="style-button">Younger</button>
<button id="style-button-2" class="style-button">Older</button>
</div>
</div>
</div>
</div>
<div id="output" hidden>
<div class="row-above">
<div class="column arrow">
<h3 class="action-step">↓</h3>
</div>
<div class="column not-sure">
<button id="not-sure-button-4" class="not-sure_button">Show details +</button>
</div>
<div>
<p hidden class="step-explanation" id="step-expl-4">The model output is an example of what the selected model produced, given the particular criteria selection you have made. You can see how well our classifer evaluated the output below.</p>
</div>
</div>
<div class="row">
<div class="column column-left">
<h3 class="action-step">Model output</h3>
</div>
<div class="column column-right">
<p id="model-output-prompt">Prompt</p>
<p id="model-output">model output</p>
</div>
</div>
<div id="output-stats">
<br>
<p id="output-auto-res-1">80% model probability toward ‘younger’;</p>
<p id="output-auto-res-2">20% model probability toward ‘older‘;</p>
</div>
</div>
<br>
<br>
<br>
<h2 class="subtitle">Test yourself</h2>
<p>To test whether adaptations produced by the language models are detectable, we paired outputs from different models together and asked human evaluators to judge which one sounded more like coming from an older or a younger speaker. In total, we collected 9,000 judgments from 467 different participants.
<br>
<br>
How would you do yourself? Try to guess the age group of the following examples and see if your guesses agree with our human evaluators.
</p>
<br>
<h3 class="action-step" id="older_younger_question">Which one of the two outputs sounds like a text that could be written by a younger speaker?</h3>
<div id="age-comp">
<div class="row">
<div class="column column-left selection">
<button id="selection-btn-1" class="selection-button">Younger</button>
</div>
<div class="column column-right selection">
<button id="selection-btn-2" class="selection-button">Older</button>
</div>
</div>
</div>
<div class="column not-sure">
<button id="next-button" class="not-sure_button">Next example →</button>
</div>
<div id="comp-results" hidden>
<br>
<p id="hum-res-1">80% of our participands agrees with you</p>
<p id="auto-res-1">80% our automatic evaluation agrees with you</p>
<p id="auto-res-2">20% our automatic evaluation agrees with you</p>
</div>
<br>
<br>
<br>
<div class="columns">
<div class="column" style="padding-right: 5rem;">
<h2 class="subtitle">Find out more</h2>
<br>
<p>The work received funding from the University of Amsterdam’s Research Priority Area Human(e) AI and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 819455).</p>
<br>
<br>
<a href="papers/GEM_paper.pdf"><button id="paper_button">Read the full paper (GEM submission)</button></a>
<br>
<a href="https://ceur-ws.org/Vol-3033/paper47.pdf"><button id="paper_button">Read the full paper (CLiC-it submission)</button></a>
</div>
<div class="column">
<br>
</div>
</div>
</container>
<br>
<br>
<br>
</section>
<script src="app.js"></script>
</body>
</html>