forked from pages-themes/minimal
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathqt_annotation_phase2_gen.html
217 lines (160 loc) · 13.5 KB
/
qt_annotation_phase2_gen.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<script>window.NREUM||(NREUM={});NREUM.info={"beacon":"bam-cell.nr-data.net","errorBeacon":"bam-cell.nr-data.net","licenseKey":"b6a3395450","applicationID":"1491523","transactionName":"dVgIFxEMD1RTREteClRESQYHChdXRGkURgBAXgMU","queueTime":1,"applicationTime":116,"agent":""}</script>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="robots" content="noindex">
<meta name="csrf-param" content="authenticity_token" />
<meta name="csrf-token" content="6nalJbOzvoiM2KyXdRpLJFIrARtnvA2kn6Yiij1hh4eVMsbYrJSiqeTSBKoNiX8ImVh33VFfdiY4kZVe+pxWww==" />
<!--[if lt IE 9]>
<script src="http://html5shim.googlecode.com/svn/trunk/html5.js" type="text/javascript"></script>
<![endif]-->
<link rel="stylesheet" media="all" href="https://d1epz76kxf2sd6.cloudfront.net/assets/core_ui/application-c73e1b3a28e6f2975cd7e313a71201b4cf50ad73da96baf6a23af86e385b48e7.css" /><link rel="stylesheet" media="all" href="https://d1epz76kxf2sd6.cloudfront.net/assets/application-652661d23dce3126f8a1debe63a9b24fa4dd9be1d52bf6645e3e5920aea8b607.css" /><link rel="stylesheet" media="print" href="https://d1epz76kxf2sd6.cloudfront.net/assets/core_ui/print-24e9db670e5dceaeb10108d7dd38bafe26c86f1912401dbd40f47b1551a7183e.css" />
<link rel="stylesheet" media="all" href="https://d1epz76kxf2sd6.cloudfront.net/assets/builder_worker-e9adc9e201c14ee4b24d515633bd9e416ba3813ad3fc604689b81ad9444532e4.css" />
<style type="text/css" media="screen">
body {
background-color: #CCCCCC;
color: #000000;
width: 1200px;
}
#summary-inner {
margin-left: -3px;
}
.row-fluid {
margin-bottom: 10pt;
}
.table {
margin-top:7px;
}
.table thead tr {
border-bottom: none;
}
.list > tr {
border-left: 2px solid white;
}
h1 {
color: #000000;
margin-bottom: 15pt;
}
h2.legend {
position: absolute;
left: -999em;
}
h2:first-of-type {
margin-top: 8px;
}
.radios div {
width: 30%;
float:left;
}
.cml_row label {
font-size: 12pt;
font-weight: bold;
}
.well {
padding: 10px;
min-height: 100px;
}
.summary {
padding: 0;
margin: auto;
}
#summary_pos {
color: blue;
margin-bottom: 10pt;
}
#summary_neg {
color: red;
}
.words {
color: green;
font-size: 24pt;
font-weight: bold;
}
.red-words {
color: red !important;
}
.affix {
top: 0px;
position:fixed;
z-index: 9999;
background-color: white;
}
.highlighted {
background-color:silver !important;
}
.keyword {
background-color:#fff4d1;
}
.check {
vertical-align: middle !important;
text-align: center !important;
}
.check-head {
text-align: center !important;
}
.green-strike {
color: #336600;
text-decoration: line-through;
}
.red-strike {
color: #990000;
text-decoration: line-through;
}
.group-section {
margin: 15px;
line-height: 1.8;
}
.aspect_service {
border-left: 2px solid #191970;
}
.aspect_cleanliness {
border-left: 2px solid #006400;
}
.aspect_rooms {
border-left: 2px solid #ffd700;
}
.aspect_food {
border-left: 2px solid #00ff00;
}
.aspect_location {
border-left: 2px solid #00ffff;
}
.aspect_building {
border-left: 2px solid #ff00ff;
}
.aspect_other {
border-left: 2px solid #ffb6c1;
}
</style>
</head>
<body>
<h1 class="job-title">Phase 2: Opinion Summary Collection (General)</h1>
<div id="assignment-instructions">
<div class="collapse in well">
<h2 data-empty="true"><span style="color: rgb(65, 168, 95);">[Please read carefully before attempting the task!]</span></h2>
<h2 data-empty="true">I. General Instructions:</h2>
<ul><li>Attempt HITs if you are a <strong>native speaker of English</strong>.</li><li>We are happy to <strong>receive feedback</strong> and improve this job accordingly. Feel free to send your comments to: s.angelidis [at] ed [dot] ac [dot] uk or use figure-eight's messaging.</li><li>Your responses are <strong>confidential</strong>. Any publications based on these will not include your specific responses, but rather aggregate information from many individuals. We will not ask any information that can be used to identify who you are.</li><li><strong><span style="color: rgb(184, 49, 47);">Important:</span></strong> This task is, to some extent, subjective, so we do not use test questions to evaluate the quality of answers. <strong>However</strong>, we monitor answers to identify workers who don't follow our basic instructions or give random answers. Such workers will be <strong>rejected and banned</strong> from this and future tasks.<br /><br /></li></ul>
<h2>II. The Goal: Summarizing Customer Opinions about Hotels</h2>
<p>The goal of this job is to generate <strong>high-level</strong> summaries of the opinions expressed in a set of review sentences about a particular hotel. The review sentences presented to you have been previously identified as <strong>representative</strong> and <strong>useful</strong>. </p>
<p>At each page, you will see sentences about the hotel which have been grouped according to the <strong>aspect</strong> of the customers' experience they discuss, e.g., the quality of the <em><span style="color: rgb(41, 105, 176);"><u><strong>rooms</strong></u></span></em>, or the <em><span style="color: rgb(41, 105, 176);"><u><strong>cleanliness</strong></u></span></em> of the hotel, or the <em><span style="color: rgb(41, 105, 176);"><u><strong>location</strong></u></span> </em>of the hotel, etc. </p>
<p>You are asked to summarize the opinions at a high level, by producing 1 summary sentence for every aspect. The full summary must not exceed the upper limit of 100 words. We are not looking for summaries that cover all expressed opinions in detail. Instead, only the most common and useful opinions should be mentioned.<br /><br /></p>
<h2>III. The Task </h2>
<p>More specifically, your task is to:</p>
<ul><li>Read the review sentences that have been pre-selected and grouped into aspects for you.</li><li>For each aspect, try to express the prevailing opinions of the reviewers in <strong>a single sentence</strong>. Add this sentence to the text box.</li><li>There are <u>one</u><u> </u><u>exception</u> for the "1-sentence-per-aspect" limit:<ul><li>The <em>"other" </em>aspect, which you will encounter in some pages. This includes sentences that don't fit the rest of the aspects. Since these sentences might be diverse in content, you are allowed to write <u>up to 3 sentences</u> for it, if needed.</li></ul></li><li>Avoid including too many details in the summary. We enforce the limits of 1 sentence per aspect and 100 total words to make sure your summaries only cover the opinions at a high level.</li><li>If contradicting opinions exist, use your judgment and summarize the prevailing ones.</li><li>Your summaries should be <u><strong>simple</strong></u>, <strong>grammatical</strong> and <strong>coherent</strong>.</li><li><span style="color: rgb(184, 49, 47);"><strong>Important: </strong>At times you will see <strong>irrelevant sentences</strong> which <u>do not relate to the specified aspect</u>. Please <strong>ignore</strong> these.</span></li></ul>
<p>A few important notes regarding your summaries:</p>
<ul><li><strong>Note #1:</strong> You <strong><span style="color: rgb(44, 130, 201);">should</span></strong> try to express opinions which relate to the specified aspect<strong> at a high level</strong>.<br /><br />For example, the following sentences:<table><tbody><tr><td style="background-color: rgb(239, 239, 239);">The staff were only too happy to talk about the city of new orleans, with tips on where to go, best resturants etc.. We booked a swamp tour through the hotel which was great.<br /></td><td><br /></td><td><br /></td></tr><tr><td align="top" style="background-color: rgb(239, 239, 239);">Also all of the staff were effective and friendly and the rooms were very clean.</td><td><br /></td><td><br /></td></tr><tr><td align="top" style="background-color: rgb(239, 239, 239);">Each time the front desk staff have been helpful and pleasant.</td><td><br /></td><td><br /></td></tr><tr><td align="top" style="background-color: rgb(239, 239, 239);">The Good: The service really is as good as everyone says.</td><td><br /></td><td><br /></td></tr><tr><td align="top" style="background-color: rgb(239, 239, 239);">To begin with, the staff was very friendly and helpful.</td><td><br /></td><td><br /></td></tr><tr><td align="top" style="background-color: rgb(239, 239, 239);">The staff at the hotel are super nice and attentive.</td><td><br /></td><td><br /></td></tr><tr><td align="top" style="background-color: rgb(239, 239, 239);">The staff was VERY accommodating and friendly.</td><td><br /></td><td><br /></td></tr><tr><td align="top" style="background-color: rgb(239, 239, 239);">Front desk was friendly and helpful.</td></tr></tbody></table><br />can be summarized as:<br /><em>"The staff were friendly and helpful."</em><br /><br /></li><li><strong>Note #2:</strong> If opinions in the pre-selected sentences directly contradict each other (e.g., "the rooms were large" and "the rooms were very comfortable" <strong>vs. </strong>"the rooms were smaller than I would like"), you should use your judgement and write a sentence that expresses the most common opinion. E.g., in this case: <em>"The rooms were quite comfortable".</em><br /><br /></li><li><strong>Note #3:</strong> Your sentences <u>should not</u> be written in first person.<br />For example, you <strong><span style="color: rgb(44, 130, 201);">should</span></strong> write:<br />"The beds were really comfortable."<br />but you <strong><span style="color: rgb(226, 80, 65);">should not</span></strong> write:<br />"I found the beds really comfortable."<br /><br /></li><li><strong>Note #3:</strong> Your summaries <strong><span style="color: rgb(226, 80, 65);">should not</span></strong> include references to the reviewers, like:<br />"Many reviewers found the beds comfortable" or "One person didn't like the bed"</li></ul>
<h2><br />III. The Interface </h2>
<p>For this task, we have prepared an interface that, hopefully, helps you generate a summary as quickly as possible. We have attempted to make it easy for you to:</p>
<ul><li>Identify the main keywords discussed in the pre-selected sentences, and sort the sentences based on these.</li><li>Discard irrelevant sentences.</li><li>Keep track of the sentences that you have used so far to generate the summary.</li></ul>
<p>Below is a sequence of screenshots highlighting the main interface features. <strong><span style="color: rgb(209, 72, 65);">An instruction video follows right after.</span></strong> Please spend a few minutes going over these to familiarize yourself with the task :)</p>
<p><br /><strong>1) Initially, the interface will look like below:</strong><img class="fr-dib" src="https://s3.amazonaws.com/crowdflower-make-cloud/images%2F1579088443237-untitled" style="width: 1174px; height: 1608.18px;" /></p>
<p><br /><br /><strong>2) You can double-click on a keyword and all sentences that contain it will be grouped together at the top of the list:</strong><img class="fr-dib" src="https://s3.amazonaws.com/crowdflower-make-cloud/images%2F1579089667845-untitled" style="width: 1181px; height: 775.145px;" /></p>
<p><br /><br /><strong>3) The same can be achieved by double-clicking any word that gets highlighted when you hover the pointer over it:</strong><img class="fr-dib" src="https://s3.amazonaws.com/crowdflower-make-cloud/images%2F1579089968209-untitled" style="width: 1184px; height: 753.256px;" /><br /><strong>4) Using these tools, you can start writing your summary. Remember: <u>you must write 1 sentences per aspect</u>! You can use the <span style="color: rgb(65, 168, 95);">"Already included?"</span> check-boxed to strike out sentences you've already considered (or decided not to use). Also, the <span style="color: rgb(184, 49, 47);">"Is irrelevant?"</span> check-boxes can help discard sentences that don't discuss the specified aspect. <u>You DON'T have to use these check-boxes</u>, they are only provided for your convenience.</strong><img class="fr-dib" src="https://s3.amazonaws.com/crowdflower-make-cloud/images%2F1579090869901-untitled" style="width: 1175px; height: 772.92px;" /><br /><strong>5) After writing up the sentence of an aspect, move on to the next one! How much detail you provide in your sentences depends on your judgment (some details may seem more important than others), and on how much space you have (i.e., words left).</strong><strong><img class="fr-dib" src="https://s3.amazonaws.com/crowdflower-make-cloud/images%2F1579091519364-untitled" style="width: 1176px; height: 750.851px;" /></strong><br /><strong>6) At the end, you have produced a high-level summary of opinions :) There are many ways to write such a summary, so don't stress about "getting it right". Just make sure that the important opinions are included, without getting into too much details.<img class="fr-dib" src="https://s3.amazonaws.com/crowdflower-make-cloud/images%2F1579092158530-untitled" style="width: 1169px; height: 699.754px;" /></strong></p>
<h2>IV. Video Demonstration</h2>
<p><span class="fr-video fr-fvc fr-dvi fr-draggable" contenteditable="false"><iframe src="https://www.youtube.com/embed/VQqbCmk6jzY" frameborder="0" allowfullscreen="" style="width: 657px; height: 463px;"></iframe></span></p>
</div>
</div>
</body>
</html>