Questionnaires

Questionnaires

Article Information
Category:
Last modified:
20.Jun.2022
Article UID:
32319
Cite This Page SKYbrary Aviation Safety. (June 20, 2022). Questionnaires. Retrieved June 4, 2026 from https://skybrary.aero/articles/questionnaires UID: 32319
COPY
How to cite a SKYbrary article

Description

Questionnaires are familiar to most people currently alive. The proper design of questionnaires is however, considerably less well-known. Space does not permit adequate treatment of questionnaire design - see references for full details.

Discussion

The time honoured questionnaire remains one of the cheapest methods of accumulating relevant data on simulations. It is cheap, versatile, and appears to provide objective evidence.

Some points should be born in mind:

  • Some controllers may give the answer they think the group as a whole will give rather than their personal view.
  • Controllers generally are less likely to do this than many other occupational groups.
  • Some controllers may give the answer they think their management wants, although controllers are even less likely to do this than other groups.
  • Non-native English speakers may misunderstand idiomatic English phrases.
  • Controllers may simply not fill in long questionnaires or apparently stupid questions.
  • Advise the controllers to read through the questionnaire before starting to reply.
  • Give an unequivocal example of how you wish them to reply.

References

  • EEC Report #275 Appendix 5 shows an example of the presentation of a small number of responses to a detailed questionnaire.
    • Where a questionnaire has been written using a word-processing package, the stored version can often be presented with the numbers giving each response inserted, and free-hand comments transcribed in a different typeface.
  • The best, most concise and relevant information is to be found in Chapter 3 (Subjective Assessment) by Murray Sinclair, in Wilson and Corlett (1990). In particular, Figure 3.9 on P75 summarises the overall process of data collection via questionnaire, from the sponsor, via the questionnaire designer, to the respondent and back. Sinclair also defines the classical Lickert Scale, where a statement is made, and the subject responds by marking a point on a 5-point scale, usually presented as a horizontal line, with strongly agree at one end and strongly disagree at the other.
  • The CD-ROM "HUMAN FACTORS IN THE DESIGN AND EVALUATION OF AIR TRAFFIC CONTROL SYSTEMS" published by the Volpe Centre in 1996 provides some guidelines for the design of questionnaires, particularly in ATC.
  • Questionnaire Design, Interviewing and Attitude Measurement (Oppenheim, 1966+1992) provides more general guidelines. (The second edition 1992, is considerably updated from the 1966 version, and should be used).
Generics
Type of method Subjective
Questionnaires are explicitly designed to extract subjective opinions.
Target of method Stress, Strain
Questionnaires filled in by controllers present their perception of how the simulation affected them, which reflects strain, but much of the information derived will also reflect the intensity of the traffic – which is stress
Time Scale of method Days, Weeks
Only very short (single page) questionnaires will be tolerated after each exercise. Depending on the organisation, questionnaires may be presented after each week, or at the end of the simulation.
Portability of method Yes
Questionnaires can be used almost anywhere.
Observer Effect No
Controllers are generally less susceptible to outside influences when filling in a questionnaire than when de-briefing or in direct interviews.
Context of studies
Laboratory studies Use
 
Simulation studies Use
 
Field studies Use
 
Potential problems with the method
Failure risk None
If too many, or too lengthy and apparently irrelevant questions are asked, controllers may not bother to fill in the questionnaire.
Bias risk None
Care must be taken to avoid ‘leading questions’, or unconscious bias.
Ethical problems None
Provided confidentiality is observed, there should be no significant ethical problems with questionnaires.
Costs of the method
Staff Cost Low
Distribution and collection of questionnaires normally requires no measurement staff, and is often better left to the project leader.
Set-up Cost Low
Designing a questionnaire requires more than a simple list of questions. Texts and computer packages for the design of questionnaires are available and should be used.
Running Cost Low
Simulation staff can carry out distribution and retrieval of questionnaires, incurring no extra cost.
Analysis Cost Moderate
The cost of questionnaire analysis varies according to the depth required. Where there are relatively few questionnaires, manual techniques can be used, assisted by a word-processor. A professional analysis package is preferable where substantial numbers (more than 30 questionnaires) are involved. Free-form comments usually need to be examined and transcribed manually.
Analysis data
Analysis Speed Moderate
Depending, again, on the degree of detail, questionnaire responses can be summarised in a few hours or days. Proper use of a computer package will accelerate this considerably.
Data Automation Yes
Some questionnaires, particularly if developed using a modern computer package, can be automatically read and inserted into a data bank. At present, this is not done at EEC, since the cost and constraints of the use of a package are not considered justifiable.
Analysis Automation Yes
Where data has been automatically or manually transcribed into a data bank, automatic analysis is easy. Computer packages provide most of the necessary tools. Free-form answers cannot practically be analysed automatically.
Status Established
Established - routinely used.

SKYbrary Partners:

Safety knowledge contributed by: