This page provides additional resources that enable replication of our work published at EmpiRE2015. See the main page for our work on empirical validation of security risk assessment methods and other experiments.

The goal of the experiment is to compare the effect of using domain-general versus domain-specific catalogs of threats and security controls on security risk assessment's actual effectiveness and perception.

The participants of the experiment were 18 MSc students from different universities in Europe participating to EIT ICT Labs, a partnership between universities, research center and companies that promotes innovation in education and research. The participants worked in group of two. The groups were randomly assigned to two types of security catalogues: five groups used domain-specific catalogues and four groups used domain-general catalogues.

To conduct security risk assessment the groups used SESAR ATM Security Risk Assessment Method (SecRAM). As an instance of domain-specific catalogues we used the security catalogue developed by EUROCONTROL which come with SecRAM method. As an instance of domain-general catalogues we chose the threats and security controls catalogues of the BSI IT-Grundschutz standard.

The materials related to SecRAM method and EUROCONTROL EATM catalogues are confidential.

As application scenario to be used by the participants, we chose a new operational concept which is emerging in the ATM named Remotely Operated Tower (ROT). The participants conducted security risk assessment using SecRAM with the assigned catalogues.

ROT tutorial

The experiment was conducted as part of the Winter School. The participants were given a tutorial on SESAR SecRAM method of the duration of 8 hours spanned over 2 days. The tutorial was divided into different parts. Each part consisted of 45 minutes of introduction of a couple of steps of the method, followed by 45 minutes of application of the steps and 15 minutes of presentation and discussion of the results with the expert. Once trained on the application scenario and the method, the participants had at least 6 hours in the class to revise the security risk assessment. After the application phase participants delivered their final reports documenting the conducted security risk assessment of the ROT.

During the experiment we distributed among participants two type of questionnaires:

• Pre-task questionnaire to collect some information about participants and their background:Background Questionnaire.
• Post-task questionnaire to collect participants' perception of the method and catalogues: version for domain-specific catalogues and domain-general catalogues.

Data collected during the experiment are available upon request.