User Tools

Site Tools


seceng-course-exp-2012

This is an old revision of the document!


An Experimental Comparison of Two Risk-Based Security Methods

An experiment by Katsyarina Labunets, Fabio Massacci, Federica Paci, Le Minh Sang Tran.

This page provides additional resources that enable replication of our work published at ESEM 2013.

Goals

The goal of the experiment was to evaluate and compare two types of risk-driven methods, namely, visual methods (CORAS) and textual methods (SREP) with respect to their effectiveness in identifying threats and security requirements, and the participants’ perception of the two methods.

Context of the Experiment

Subjects

The experiment involved 28 participants: 16 students of the master in Computer Science and 12 students of the EIT ICT LAB master in Security and Privacy. They were divided into 16 groups using a randomized block design.

Methods

The methods evaluated were CORAS (visual method) and SREP (textual method).

Case Study

The participants applied the methods to a Smart Grid application scenario.

Task

The experiment was conducted as part of the Security Engineering course. Here, you can find the summary of the Tasks to be accomplished in the experiment.

Measurements

Results

  • Methods' effectiveness

Results show that visual method is more effective in identifying threats than textual method. This is confirmed if we consider the number of threats identified with visual and textual methods across the task assigned to the groups. Instead, with respect to number of security requirements, textual method is slightly more effective than the visual one in identifying security requirements.

  • Methods' perception
  • Qualitative explanation

Additional Material

  • For additional information on the experimental design please see the Experimental Protocol.
  • For privacy reasons, at the beginning of the experiment a Consent Form was administered to participants.
  • Participants' results have been assessed by methods and domain experts (see Evaluation Score Sheet).
seceng-course-exp-2012.1404834259.txt.gz · Last modified: 2021/01/29 10:58 (external edit)