Assessment configuration: keyword reference

Keyword reference for the exam YAML file

This document lists the configuration options for an exam YAML file. For the basics of the YAML syntax, follow this link.

Keywords

An exam YAML configuration includes:

• Global keywords that configure the exam basic data:

  Keyword	Description
  [question ID]	A number representing a question of the exam.
  course code	Official exam course code.
  alternative codes	Official alternative codes for the exam course.
  course name	Official name of the exam course.
  begins	Date and time when the exam will begin.
  duration	Official exam duration, not including file upload time.
  upload time	Additional time given to upload answers for the exam.
  extensions	List of students who have extensions, with corresponding extra time.
  rubric	Set of instructions or rules for the student to undertake the exam.
  labelled subparts	Flag to show/hide labels for question subparts.

• Questions configured with question keywords:

  Keyword	Description
  [part ID]	A letter representing a part of the current question.
  title	The question title.
  instructions	General instructions for the student about the question.
  show part weights	Flag to show/hide information about part weights for the question.

• Parts of a question, configured with part keywords:

  Keyword	Description
  [section ID]	A Roman number representing a section of the current part.
  instructions	General instructions for the student about the part.
  show subpart weights	Flag to show/hide information about subpart weights for the part.

• Sections of a part (also called subparts), configured with section keywords:

  Keyword	Description
  instructions	General instructions for the student about the section.
  maximum mark	Maximum mark which can be assigned to the section.
  tasks	List of tasks to be undertaken by the student to complete the section.

• Tasks of a section, configured with task keywords:

  Keyword	Description
  instructions	The task's instructions.
  type	The type of task.
  lines	Optional number of lines (default 5) for essay and code tasks.
  choices	The list of choices for tasks which include a choice.
  answer	The right answer for the task (used for auto-marking procedures).

Global keywords

[question ID]

This is a number, starting from 1, identifying a question of the exam.

Keyword type: Global keyword.

Example of [question ID]:

1:
  title: Reasoning About Haskell Programs
  instructions: for the next tasks, consider the Haskell program below. [...]
  ...

Related topics:

• List of all question keywords.
• The [part ID] keyword.
• The [section ID] keyword.

course code

This indicates the official exam course code.

Keyword type: Global keyword.

Possible inputs: a string with the course code.

Example of course code:

course code: 70055

Additional details: Alternative codes can also be provided if needed, using the alternative codes keyword.

alternative codes

This lists all the alternative official exam course codes, apart from the main one already specified using course code.

Keyword type: Global keyword.

Possible inputs: a list of strings with alternative course codes.

Example of alternative codes:

alternative codes:
  - 67890
  - abcde
  - xy123

Related topics: The course code keyword.

course name

This indicates the official exam course name.

Keyword type: Global keyword.

Possible inputs: a string with the course name.

Example of course name:

course name: Python Programming

Related topics: The course code keyword.

begins

This indicates the official start date and time of the exam.

Keyword type: Global keyword.

Possible inputs:

A date and time for the beginning of the exam. The date and time must be in the UTC time zone.

In particular, it must have the following format:

yyyy-mm-ddThh:mm:ssZ

where T separates the date from the time of the day, and Z is the UTC time zone.

Example of begins:

Exam that begins on 10/01/2022 at 10am UTC time (or GMT).

begins: 2022-01-10T10:00:00Z

Exam that begins on 10/01/2022 at 2pm BST time (summer time, note the one hour difference with UTC).

begins: 2022-01-10T13:00:00Z

Additional details: The formate used for date and time is a particular case of ISO 8601 format. But note that the other ISO 8601 formats are currently unsupported. In particular, the time zone must always be UTC, which means that in summer the BST time zone cannot be used.

Related topics:

• The duration keyword.
• The upload time keyword.
• The ISO 8601 format.

duration

This indicates the official duration of the exam.

Keyword type: Global keyword.

Possible inputs: The duration of the exam in minutes.

Example of duration:

duration: 130

Additional details:

• The duration should not include the upload time. The duration of the upload time must be specified using keyword upload time.
• Currently we only support a duration expressed in minutes (but this may change in the future). Therefore, for longer assessments like courseworks, the overall duration must always be converted in minutes (e.g. 2 days corresponds to 22460=2880 minutes).

Related topics:

• The begins keyword.
• The upload time keyword.

upload time

This indicates the additional time given to upload answers, usually for exams where file upload is used.

Keyword type: Global keyword.

Possible inputs: The duration of upload time in minutes.

Example of upload time:

upload time: 15

Additional details:

• Currently we only support the upload time expressed in minutes (but this may change in the future).

Related topics:

• The begins keyword.
• The duration keyword.

extensions

This keyword identifies all students who have time extensions, with corresponding extra time.

Keyword type: Global keyword.

Possible inputs: pairs of student usernames and corresponding extra time expressed in minutes and ending with the minutes keyword.

Example of extensions:

extensions:
  ac4014: 37 minutes
  ip914: 24 minutes
  jsbailey: 10 minutes

Related topics:

• The begins keyword.
• The duration keyword.
• The extensions keyword.

rubric

This keyword is used to provide a set of instructions or rules for the student to undertake the exam.

Keyword type: Global keyword.

Possible inputs: The rubric subkeys:

• rubric:instructions
• rubric:questions to answer

rubric:instructions

This contains the general instructions which will be shown to the student to correctly undertake the assessment.

Keyword type: Global keyword.

Possible inputs: A text with the instructions for the student.

Example of rubric:instructions:

rubric:
  instructions: |
    * Refer to the pdf question paper for instructions and full questions.
    * You can make changes to your answers as many times as you like before the deadline; only your final submission will be marked.

rubric:questions to answer

This is the number of questions the students is required to answer for the assessment (it can be less than the total number of questions).

Keyword type: Global keyword.

Possible inputs: A positive integer.

Example of rubric:questions to answer:

rubric:
  questions to answer: 3

labelled subparts

This style option indicates whether labels (Roman numbers) for subparts should be shown or hidden to the student.

Keyword type: Global keyword.

Possible inputs:

• true or false. If keyword is not used, then the default assumption is false.

Example of labelled subparts:

labelled subparts: true

Question keywords

[part ID]

This is a single letter, starting from a, identifying a part of the current question.

Keyword type: Question keyword.

Example of [part ID]:

1:
  title: Reasoning About Haskell Programs
  a:
    instructions: For the next tasks, consider the Haskell program below.
...

Related topics:

• List of all part keywords.
• The [question ID] keyword.

title

This is the title of the current question.

Keyword type: Question keyword.

Possible inputs: A string with the question title.

Example of title:

2:
  title: Reasoning About Haskell Programs

Related topics:

• The instructions keyword.

instructions

This includes the instructions for the student to work on the current question.

Keyword type: Question keyword.

Possible inputs: A text with the instructions for the current question.

Example of instructions:

3:
  instructions: For the next tasks, consider the Haskell program below.

Related topics:

• The title keyword.

show part weights

This style option indicates whether weights of each part of the current question should be shown to the student.

Keyword type: Question keyword.

Possible inputs:

• true or false. If keyword is not used, then the default assumption is true.

Example of show part weights:

4:
  show part weights: true

Related topics:

• The show subpart weights keyword.

Part keywords

[section ID]

This is a Roman number, starting from i, identifying a section (or subpart) of the current part.

Keyword type: Part keyword.

Example of [section ID]:

1:
  a:
    i:
      instructions: For the next tasks, consider the Haskell program below.
...

Related topics:

• List of all section keywords.
• The [question ID] keyword.
• The [part ID] keyword.

instructions

This includes the instructions for the student to work on the current part.

Keyword type: Part keyword.

Possible inputs: A text with the instructions for the current part.

Example of instructions:

1:
  a:
    instructions: For the next tasks, consider the Haskell program below.

Related topics:

• The instructions question keyword.

show subpart weights

This style option indicates whether weights of each section (or subpart) of the current part should be shown to the student.

Keyword type: Question keyword.

Possible inputs:

• true or false. If keyword is not used, then the default assumption is false.

Example of show subpart weights:

1:
  a:
    show subpart weights: true

Related topics:

• The show part weights keyword.

Section keywords

instructions

This includes the instructions for the student to work on the current section.

Keyword type: Section keyword.

Possible inputs: A text with the instructions for the current section.

Example of instructions:

1:
  a:
    i:
      instructions: For the next tasks, consider the Haskell program below.

Related topics:

• The instructions question keyword.
• The instructions part keyword.

maximum mark

This is the maximum mark which can be assigned to the student for the current section.

Keyword type: Section keyword.

Possible inputs: A positive integer.

Example of maximum mark:

1:
  a:
    i:
      maximum mark: 8

tasks

This is a list of all the tasks the student can undertake in the current section.

Keyword type: Section keyword.

Possible inputs: A list of task items, each described by task keywords.

Example of tasks:

1:
  a:
    i:
      tasks:
      - instructions: ...
        type: ...
      - type: ...
      - type: ...
      ...

Related topics:

• List of all task keywords.

Task keywords

type

This required keyword represents the task type.

Keyword type: Task keyword.

Possible inputs:

All the possible task types:

• essay
• code
• multiple choice select one
• multiple choice select several
• integer from [N] to [M]
• flag

The possible inputs for type are:

essay

This represents a text area for an answer in plain text. The optional tasks:[task]:lines keyword is used to specify the height of the text area in terms of number of lines (default is 5).

Example of type:essay:

tasks:
  - instructions: "What's the **best** lightsaber EVER, and why have you just thought of Darth Maul's double lightsaber?"
    type: essay
    lines: 2

code

This represents a text area to type code or pseudo-code to complete coding tasks. The code will be rendered as monospaced to properly support indentation. The optional tasks:[task]:lines keyword is used to specify the height of the text area in terms of number of lines (default is 5).

Example of type:code:

tasks:
  - instructions: "Write the QuickSort algorithm in pseudo-code below."
    type: code
    lines: 10

multiple choice select one

This represents a multiple choice question where the students has to select only one answer among the options provided. The options are represented using the choices keyword.

Note: This task can also support auto-marking, by using the answer keyword.

Example of type:multiple choice select one:

      tasks:
        - instructions: "Which of the following animals is a feline?"
          type: multiple choice select one
          choices:
            ...
          answer:
            ...

Related topics:

• The choices keyword.
• The answer keyword.

multiple choice select several

This represents a multiple choice question where the students can select multiple answers among the options provided. The options are represented using the choices keyword.

Note: This task can also support auto-marking, by using the answer keyword.

Example of type:multiple choice select several:

      tasks:
        - instructions: "Which of the following animals are felines?"
          type: multiple choice select several
          choices:
            ...
          answer:
            ...

Related topics:

• The choices keyword.
• The answer keyword.

integer from [N] to [M]

This represents a task where the student is required to provide an integer number, e.g. as a result of a calculation or the execution of an algorithm. Placeholders N and M are integer values representing an optional lower bound and an optional upper bound respectively.

Example of type:integer from [N] to [M]:

      tasks:
        - instructions: "What is the smallest prime number between 400 and 500?"
          type: integer from 400 to 500

flag

This represents a numeric flag.

Note: this task-type is as of today used exclusively by 60015 (Networks and Web Security).

Example of type:flag:

tasks:
  - instructions: "Report the flag you found in the malicious code:"
    type: flag

lines

This is the optional value indicating the number of rows to inject in the text area field for the rendering of type:essay and type:code tasks. When not specified, the default value for the number of lines is 5.

Example of lines:

tasks:
  - instructions: "What's the **best** lightsaber EVER, and why have you just thought of Darth Maul's double lightsaber?"
    type: essay
    lines: 2

choices

This is list of options presented to the student, if the current task includes a choice between them.

Keyword type: Task keyword.

Possible inputs: list pairs of lowercase letter (as option ID) and the text to be shown to the student for that option.

Note: in order to use auto-marking it is also possible to indicate how many marks are awarded (or deducted) for each answer, by using the answer keyword.

Example of choices:

      tasks:
        - instructions: "Which of the following animals is a feline?"
          type: multiple choice select one
          choices:
            - a: "Dog"
            - b: "Elephant"
            - c: "Cat"
            - d: "Crocodile"
            - e: "Bee"

Related topics:

• The multiple choice select one task type.
• The multiple choice select several task type.
• The answer keyword.

instructions

Optional keyword representing the specific instructions for the task:

Example of instructions:

      tasks:
        - instructions: "Which of the following animals is a feline?"
          type: essay

answer

In case auto-marking, this optional keyword is used to indicate the amount of marks awarded (or deducted, in case of negative score) when selecting an option in multiple choice questions.

Keyword type: Task keyword.

Possible inputs: Dictionary where keys are unique option IDs (defined using the choices keyword) and values are scores, i.e. numbers indicating the amount of marks awarded for that option (or deducted, if the score is negative).

Notes:

• Marks can also be non-integer.
• It is not compulsory to map each option to a score. The unmapped options will have default score 0.
• The sum of all the scores must not exceed the maximum mark for the current section, indicated using the maximum mark section keyword.

Example of answer:

      tasks:
        - instructions: "Which of the following animals is a feline?"
          type: multiple choice select one
          choices:
            - a: "Dog"
            - b: "Elephant"
            - c: "Cat"
            - d: "Crocodile"
            - e: "Bee"
          answer:
            c: 4
            d: 1.5
            e": -2

Related topics:

• The multiple choice select one task type.
• The multiple choice select several task type.
• The choices keyword.