Table of Contents

The Role of Catalogues of Threats and Security Controls in Security Risk Assessment: An Empirical Study with ATM Professionals

This page provides additional resources that enable replication of our work published at REFSQ2015. See the main page for our work on empirical validation of security risk assessment methods and other experiments.

Goals

The goal of the experiment is to compare the effect of using domain-general versus domain-specific catalogs of threats and security controls on security risk assessment's actual effectiveness and perception.

Context of the Experiment

Subjects

The participants of the experiment were 15 professionals from severalATMItalian companies. The participants worked individually. The participants were divided into three groups and assigned to three different treatments. They were asked to apply individually the same method, namely SESAR SecRAM, with the support of domain-specific catalogues (EUROCONTROL ATM), domain-general catalogues (BSI IT-Grundschutz) or without any catalogues.

Method and Catalogues

To conduct security risk assessment the groups used SESAR ATM Security Risk Assessment Method (SecRAM). As an instance of domain-specific catalogues we used the security catalogue developed by EUROCONTROL which come with SecRAM method. As an instance of domain-general catalogues we chose the threats and security controls catalogues of the BSI IT-Grundschutz standard.

The materials related to SecRAM method and EUROCONTROL EATM catalogues are confidential.

Application Scenario

As application scenario to be used by the participants, we chose a new operational concept which is emerging in the ATM named Remotely Operated Tower (ROT).

Task

The experiment was conducted as part of the SESAR SecRAM workshop organized by EMFASE project. The participants were given a tutorial on the method of the duration of 8 hours spanned over 2 days. The tutorial was divided into different parts. Each part consisted of 30 minutes of introduction of a couple of steps of the method, followed by 45 minutes of application of the steps and 15 minutes of presentation and discussion of the results with the expert. After the application phase participants delivered their final reports documenting the conducted security risk assessment of the ROT.

Measurements

During the experiment we distributed among participants two type of questionnaires:

Data collected during the experiment are available upon request.