9.1 The method
The method used in this master thesis has been satisfying. There were several aspects
contributing to choosing what method to use, but the most important was the
opportunity to perform a lot of work in the users right context, at their place of work,
and the contextual design method was therefore naturally given. One aspect contributing
to the choice of creating prototypes was that there is no similar system with an
Operations Monitor existing today, and to reduce the cognitive load of the users, trying
to imagine one, it had to be visualized. It was also the best way to communicate the ideas
to the people at Carmen, the members of the Descartes team. The greatest benefit from
creating prototypes as a design method is that it facilitates the cooperation between
designer and user, creating a greater understanding for each other’s problems and assets.
It was important to be able to create and to demonstrate scenarios, for the
designers to understand the type of tasks there is and for the users to recognize their
work. Discussions concerning what functions are needed to solve a problem came up
during tests, and without these discussions important aspects related to reality would
probably not have been considered. It is important to discuss how the result will be used,
since it is difficult to predict how the future users will actually use the system and how it
will be experienced.
A certain degree of criticism has been involved when evaluating the results from
tests and workshops in accordance with literature and previous work, since the
prototypes and recommendations evolved from these are nothing but guidelines and
experiments. A weakness with this method is that the scenarios are typical and realistic
problems, but never enough since each task for a controller is always unique, and this
complexity affects the result. The time constraint is always an important factor, and that
is also a weakness to this method, it is difficult to perform realistic tests in the right
context with all the complexity concerning the work in the operations control of an
airline company.
In the design of the prototype, the users have been involved for test purpose. The
designers have decided all decisions concerning the design, but the users have been
consulted along the way. It would of course have been optimal to have crew controllers
cooperating in all stages of the design process according to participatory design, ensuring
all details to be optimal. Since this was not possible, the design process became iterative,
with tests performed along the way. Users were involved when creating the foundation
for the work, demonstrating their way of working today, as well as in testing our general
ideas, the information presented and the interaction. The creation of all the graphical
details was entirely the designers’ choice; there was no possibility to focus on this too
during the tests.
It is a great advantage of having a prototype to communicate concept and ideas,
but one has to be aware of that it can also limit the tests. The focus tends to be directed
to the details of the appearance, making it harder for the test person to see what is
missing and what to change. When testing the third prototype, letting the test persons be
aware of that it was not graphically completed, the focus were more directed towards the
functionality, interaction, concept and what was missing.
User studies have been conducted to study how the users interact with the systems or
prototypes available in different stages of the design process. At first the purpose was to
study how the controllers use the available systems today, to gather requirements and
information. Later on user studies was conducted to evaluate the work, the reactions, the
ideas, and to investigate whether the prototype was enough for achieving the goals we
had set up in advance.
The contextual inquiry worked very well for observations as well as for testing.
Problems with conducting this kind of interviews became very obvious, since the users
are all experts on what they do, it is their every day tasks, and they summarize their
performance down to nothing. This is why the observations are important, while
studying when the work is actually performed; the complexity of the problems as well as
of the work procedure becomes obvious. Details, problems and hidden structures would
not have come to our knowledge if we were not to observe the work in the right context.
Performing the interviews in the control room while they were working, gave us a lot of
useful information about to what extent they take advantage of divided attention within
the entire room, while focusing on a task. The fact that these interviews and observations
were performed at two different airline companies contributed to even more concrete
and detailed information. Unfortunately the days for the visits have been really calm days,
without any major disturbances. This provided more time to discuss and interview, but
less knowledge of how the work is performed, how problems are actually solved, which
has been a great lack throughout the work.
Sometimes problems occurred with keeping focus during the interviews, even
though there was a pre-decided goal. To keep focus turned out to be difficult, since the
environment and the problems were totally unfamiliar, especially in the beginning. As
soon as we started to collect the information, it was realized that the unfocused parts of
the interviews contributed with a lot of important facts too, since this was the
controllers’ personal opinions and stories.
The number of performed tests was satisfying, even though more tests would
have contributed to even more aspects and opinions to consider. The selection of test
persons was satisfying in one sense; they were a great difference in ages and therefore
experience. To become as company independent as possible, tests would have to have
been performed in several cultures, meaning in other parts of the world. More aspects
would probably have come up if all test persons were crew controllers, even though this
actual variation in professions contributed to a wider range of subjects were to be
discussed.
Is it possible to test and evaluate a graphical user interface separated from the rest of the
system? This question has been vital during the entire design process, and several
answers are to be found. It is possible when performing tests with test persons that are
familiar with the Descartes concept; the focus is much more on just the Operations
Monitor, its concept and functions. When performing tests with users that have not
heard of the type of decision support system that is Descartes, then it is almost
impossible to separate them. In that case, it is preferable to test only the traditional way
of solving problems, not involving the solvers or the new way of working. This way, the
purpose of the monitor would be to support the users in their existing systems and just
be an add on, which is not quite correct, and still the trap of the general idea of an user
interface. When using a system, the user does not make a distinction between the
interface and the underlying functionality. How is it possible to measure the usability of a
system if that underlying interaction is not satisfying, or even existing?
How the results in this thesis could be used in the future has to be discussed. The
analysis is a good basis for future work concerning the crew controllers, as well as the
recommendations. Specific details in the design should not be considered as optimal in
any sense, since this was not the main purpose of the thesis. Since this concept of an
Operations Monitor has been abstracted from the rest of the Descartes in many levels,
then it is recommended to create new analysis based upon how the work will change the
routines for the users, what reorganizations are meant to be, and so forth, if the
Operations Monitor is to be a part of a larger system.
It was good practice to really have to understand the users work in detail, since it
provided the possibility to practice our capability to make them express exactly what they
are doing, every single step, even though it might be totally obvious and natural to them.
It is not easy to make someone aware of actions that they have in their backbone, to
make them verbally express their silent knowledge, especially when you are not familiar
with the environment yourself. This might be complicated, but never the less necessary.
Bạn đang xem 9. - AIRLINE CREW CONTROL OPERATIONS MONITOR: PART 2