User Tools
seceng-course-exp-2012
Differences
This shows you the differences between two versions of the page.
|
seceng-course-exp-2012 [2014/07/08 17:34] federica.paci@unitn.it [Measurements] |
seceng-course-exp-2012 [2021/01/29 10:58] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== An Experimental Comparison of Two Risk-Based Security Methods ====== | ||
| - | An experiment by Katsyarina Labunets, Fabio Massacci, Federica Paci, Le Minh Sang Tran. | ||
| - | |||
| - | This page provides additional resources that enable replication of our work published at {{:research_activities:experiments:2013-seceng:labunets-esem-2013-accepted.pdf|ESEM 2013}}. | ||
| - | ===== Goals ===== | ||
| - | The goal of the experiment was to evaluate and compare two types of risk-driven methods, namely, visual methods (CORAS) and textual methods (SREP) with respect to their //effectiveness// in identifying threats and security requirements, and the //participants’ perception// of the two methods. | ||
| - | ===== Context of the Experiment ===== | ||
| - | |||
| - | ==== Subjects ==== | ||
| - | The experiment involved 28 participants: 16 students of the master in Computer Science and 12 students of the EIT ICT LAB master in Security and Privacy. They were divided into 16 groups using a randomized block design. | ||
| - | |||
| - | ==== Methods ==== | ||
| - | The methods evaluated were {{:research_activities:experiments:2013-seceng:lecture-06-riskwithcoras.pptx|CORAS}} (visual method) and {{:research_activities:experiments:2013-seceng:srep_tutorial.pdf|SREP}} | ||
| - | (textual method). | ||
| - | |||
| - | |||
| - | ==== Case Study ==== | ||
| - | The participants applied the methods to a {{:research_activities:experiments:2013-seceng:lecture-03-casestudy.pptx|Smart Grid application scenario}}. | ||
| - | |||
| - | |||
| - | ==== Task ==== | ||
| - | The experiment was conducted as part of the Security Engineering course. Here, you can find the summary of the {{:research_activities:experiments:2013-seceng:experiment-agenda.pdf|Tasks}} to be accomplished in the experiment. | ||
| - | ===== Measurements ===== | ||
| - | |||
| - | |||
| - | * {{:research_activities:experiments:2013-seceng:q1_-_background.docx|Background Questionnaire}} - collect participants demographic data. | ||
| - | * {{:research_activities:experiments:2013-seceng:seceng2013-post-task-questionnaire.pdf|Post-Task Questionnaire}} - assess participants’ perception of visual and textual methods. | ||
| - | * {{:research_activities:experiments:2013-seceng:interview_guide.docx|Interview Guide}} - collect participants' opinion on advantages and disadvantages of visual and textual methods. | ||
| - | * {{:research_activities:experiments:2013-seceng:securityengineering2013.docx|Final Report}} - document methods' application. | ||
| - | |||
| - | |||
| - | ===== Results ===== | ||
| - | The main findings are that the visual method yields to identify more threats than textual one, while the textual one is slightly better to identify security requirements. The difference in the number of threats identified with the two methods is statistically significant and participants’ interviews suggests that this is due to the difference in the artifacts used to model threats. | ||
| - | The visual method uses diagrams to represent threats while the textual method uses tables: diagrams help brainstorming on threats and thus yield participants to identify more threats. On the contrary, the difference in the number of security requirements identified with the two methods is not statistically significant. | ||
| - | The textual method identified a slightly higher number of security requirements but this is not statistically significant. A possible explanation emerging from the interviews is that process supported by the textual method | ||
| - | offers a systematic approach to identify security requirements. | ||
| - | In addition, the visual method’s overall perception and intention to use are higher than for the textual method. | ||
| - | ===== Additional Material ===== | ||
| - | * For additional information on the experimental design please see the {{:research_activities:experiments:2013-seceng:experiment-description.pdf|Experimental Protocol}}. | ||
| - | * For privacy reasons, at the beginning of the experiment a {{:research_activities:experiments:2013-seceng:consent-form-security-engineering.docx|Consent Form}} was administered to participants. | ||
| - | * Participants' results have been assessed by methods and domain experts (see {{:research_activities:experiments:2013-seceng:evaluation_sheet.xlsx|Evaluation Score Sheet}}). | ||
| - | | ||
seceng-course-exp-2012.txt · Last modified: 2021/01/29 10:58 (external edit)
Page Tools
