EMFASE (Empirical Framework for Security Design and Economic Trade-Off) is funded by SESAR Joint Undertaking (WPE Call for Tender) and is managed by Eurocontrol.
Evaluation and validation methodologies are integral parts of Air Traffic Management (ATM). They are well understood for safety, environmental and other business cases – for which operational validation guidelines exist which are well defined and widely used. In contrast, the effectiveness of risk assessment practices for security, as well as comparative evaluation of such practices, is largely uncharted territory. We don't know to what degree the practices and their activities provide security and whether or not they give return on investment. Furthermore, we currently don't know how to evaluate or compare security practices; there are no accepted metrics to decide that activity X works better than activity Y in a given setting. This becomes even more true in an uncertain and rapidly changing environment with changing demands by users and changing threats.
The question is: How can SESAR stakeholders know that their methods for ensuring security in the complex ATM domain really work? Would additional expensive security analysis and measures be worth the cost?
One cannot simply use proven techniques from safety and just replace “safety” with “security”: safety risk analysis assumes a game against Nature (including involuntary human errors), while security risks are a game against Man. Nature might not deliberately trigger two faults; while Man can. On the opposite side, Nature is never running short of budget or motivation, while Man does. The purpose of this project is to provide ways of evaluating and comparing risk assessment methods for security in ATM, especially in relation to human factors. The goal is to provide relevant stakeholders with the means to select the risk assessment methods are best suited for the task at hand, for example security assessment in relation to introduction of a particular new system by taking into account the specific aspect of security.
The only way to know the actual effectiveness of a risk assessment activity is to empirically investigate it. In this project we will therefore subject risk assessment methods to scientific empirical methods. It is obviously unfeasible to investigate all existing methods, so a selection of methods to investigate will be made. While the project will evaluate this selection of existing methods, the overall framework (concepts, terminology, study designs and metrics) that must be developed to do this evaluation will be of a general nature so as to enable later replications and comparable studies.
University of Trento (Coordinator, Italy), SINTEF, DeepBlue and University of Southampton.
Please check SVN Repository (Restricted Access)
Below is the list of experiments and related activities.
Trento's results are also reported in Empirical Validation of Risk and Security Methodologies.
In part (*) means confidential documents are distributed