Differences

This shows you the differences between two versions of the page.

--- emfase [2016/03/22 09:21]
elisa.chiarani@unitn.it [Project presentation]
+++ emfase [2021/01/29 11:58]
@@ Line 1: / Line 1: @@
-====== The EMFASE Project ======
-EMFASE (//Empirical Framework for Security Design and Economic Trade-Off//) is funded by **SESAR Joint Undertaking** (WPE Call for Tender) and is managed by Eurocontrol.
-===== Topic =====
-Evaluation and validation methodologies are integral parts of Air Traffic Management (ATM). They are
-well understood for safety, environmental and other business cases – for which operational validation
-guidelines exist which are well defined and widely used. In contrast, the effectiveness of risk
-assessment practices for security, as well as comparative evaluation of such practices, is largely
-uncharted territory. We don't know to what degree the practices and their activities provide security
-and whether or not they give return on investment. Furthermore, we currently don't know how to
-evaluate or compare security practices; there are no accepted metrics to decide that activity X works
-better than activity Y in a given setting. This becomes even more true in an uncertain and rapidly
-changing environment with changing demands by users and changing threats.
-The question is: //How can SESAR stakeholders know that their methods for ensuring security in
-the complex ATM domain really work? Would additional expensive security analysis and
-measures be worth the cost?//
-One cannot simply use proven techniques from safety and just replace "safety" with “security”: safety
-risk analysis assumes a game against Nature (including involuntary human errors), while security
-risks are a game against Man. Nature might not deliberately trigger two faults; while Man can. On the
-opposite side, Nature is never running short of budget or motivation, while Man does.
-The purpose of this project is to provide ways of evaluating and comparing risk assessment methods
-for security in ATM, especially in relation to human factors. The goal is to provide relevant
-stakeholders with the means to select the risk assessment methods are best suited for the task at hand,
-for example security assessment in relation to introduction of a particular new system by taking into
-account the specific aspect of security.
-The only way to know the actual effectiveness of a risk assessment activity is to empirically investigate
-it. In this project we will therefore subject risk assessment methods to scientific empirical methods. It
-is obviously unfeasible to investigate all existing methods, so a selection of methods to investigate will
-be made. While the project will evaluate this selection of existing methods, the overall framework
-(concepts, terminology, study designs and metrics) that must be developed to do this evaluation will
-be of a general nature so as to enable later replications and comparable studies.
-===== Partners =====
-University of Trento (Coordinator, Italy), SINTEF, DeepBlue and University of Southampton.
-===== Project Internal Information =====
-Please check [[https://trinity.disi.unitn.it/emfase/|SVN Repository]] (Restricted Access)
-===== Project presentation =====
-{{:projects:emfase:deliverable:emfase_poster_35x50_cmyk_small.pdf|EMFASE Poster presented at SID 2013}}
-===== Current Activities =====
-Below is the list of experiments and related activities.
-Trento's results are also reported in [[validation_of_risk_and_security_requirements_methodologies|Empirical Validation of Risk and Security Methodologies]].
-==== Criteria identification and validation ====
-  - SESAR Jamboree Nov 2013
-    * Participants: ATM experts
-    * Feedback: questionnaire, focus group interview
-==== Experiments ====
-=== Comparison of Security Risk Assessment methods ===
-  - UNITN Security Engineering course 2013-14:
-    * Participants: 29 MSc students enrolled to Security Engineering course at the University of Trento
-    * Method: CORAS vs Eurocontrol SECRAM (*)
-    * Case Study: SmartGrid
-    * Final result: excel file with threats and controls, presentations, report
-    * Feedback: questionnaire, interview
-  - First International Week with Italian Post on Cyber Security in Complex Information Systems 2014 (Rome, Italy):
-    * Participants: students - around 60 sort of controlled participants
-    * Method: CORAS vs SESAR SECRAM (*)
-    * Case Study: Online Banking
-    * Final result: excel file with threats and controls, report
-    * Feedback: questionnaire
-  - UNITN Security Engineering course 2014-15:
-    * Participants: MSc students - around 30 sort of controlled participants
-    * Method: CORAS vs SESAR SecRAM (*)
-    * Case Study: Remotely Operated Tower (ATM) (*)
-    * Final result: excel file with threats and controls, presentations, report
-    * Feedback: questionnaire, focus groups interview
-  - UNITN Security Engineering course 2015-16:
-    * Participants: MSc students - around 50 sort of controlled participants
-    * Method: CORAS vs SESAR SecRAM (*)
-    * Case Study: Unmanned Aerial System Traffic Management (UTM)
-    * Final result: excel file with threats and controls, presentations, report
-    * Feedback: questionnaire, focus groups interview
-=== Effectiveness of Catalogues of Threats and Security Controls in Security Risk Assessment ===
-  - EIT Winter School 2014:
-    * Participants: students around 20 sort of controlled participants
-    * Method: SESAR SecRAM (*) + [ BSI Catalog vs SECRAM Catalog (*) ]
-    * Case Study: Remotely Operated Tower (*)
-    * Final result: excel file with requirements, hand-drawn poster for result presentation, report
-    * Feedback: questionnaire
-  - EMFASE SecRAM Evaluation Workshop  2014:
-    * Participants: professionals around 15 sort of controlled participants
-    * Method: SESAR SecRAM (*) + [ BSI catalogue vs SECRAM catalogue (*) vs No catalogue (control group)]
-    * Case Study: Remotely Operated Tower (*)
-    * Final result: excel file with requirements, report
-    * Feedback: questionnaire, focus groups interview
-=== An Empirical Comparison of Tabular vs. Graphical Risk Model Representations ===
-  - UNITN Security Engineering course 2014-15:
-    * Participants: 35 MSc students - controlled participants
-    * Representation: Graphical (CORAS) vs Tabular (NIST)
-    * Scenario: Online Banking and Health Care Network
-    * Final result: responses to the online comprehensibility task
-    * Feedback: post-task questionnaire
-  - University of Oslo Model Engineering course 2014-2015:
-    * Participants: 11 MSc students - controlled participants
-    * Representation: Graphical (CORAS) vs Tabular (NIST)
-    * Scenario: Online Banking
-    * Final result: responses to the online comprehensibility task
-    * Feedback: post-task questionnaire
-  - PUCRS Information Systems course 2014-15:
-    * Participants: 27 MSc and 13 BSc students - controlled participants
-    * Representation: Graphical (CORAS) vs Tabular (NIST)
-    * Scenario: Online Banking and Health Care Network
-    * Final result: responses to the online comprehensibility task
-    * Feedback: post-task questionnaire
-  - University of Calabria Cybersecurity professional master course - September 2015:
-    * Participants: 52 MSc students - controlled participants
-    * Representation: Graphical (CORAS) vs Tabular (NIST)
-    * Scenario: Online Banking and Health Care Network
-    * Final result: responses to the online comprehensibility task
-    * Feedback: post-task questionnaire
-  - UNITN Security Engineering course 2015-16:
-    * Participants: 51 MSc students - controlled participants
-    * Representation: Graphical (CORAS) vs Tabular (NIST)
-    * Scenario: Online Banking and Health Care Network
-    * Final result: responses to the online comprehensibility task
-    * Feedback: post-task questionnaire
-  - EMFASE - Security Risk Assessment Tutorial at SESAR Innovation Days 2015 (Bologna, Italy):
-    * Participants: 14 professionals - sort of controlled participants
-    * Representation: Graphical (CORAS) vs Tabular (SESAR SecRAM)
-    * Scenario: Online Banking
-    * Final result: responses to the paper-based comprehensibility task
-    * Feedback: post-task questionnaire
-  - EMFASE Online Study on Comprehensibility of Risk Models:
-    * Participants: 60 professionals
-    * Representation: Graphical (CORAS) vs Tabular (NIST)
-    * Scenario: Online Banking
-    * Final result: responses to the online comprehensibility task
-    * Feedback: post-task questionnaire
-In part (*) means confidential documents are distributed
-===== Deliverables =====
-  - {{:projects:emfase:e.02.32_d1.1_selection_of_risk_assessment_methods_object_of_study_00.01.03.pdf|D1.1 Selection of risk assessment methods object of study}}
-  - {{:projects:emfase:deliverable:d1-2_firstempiricalevaluationframework_v000102.pdf|D1.2 First Empirical Evaluation Framework}}
-  - {{:projects:emfase:deliverable:e.02.32_d1.3_refinedempiricalevaluationframework_v000100.pdf|D1.3 Refined Empirical Evaluation Framework}}
-  - {{:projects:emfase:deliverable:d2_1_scenariodescriptions_v00_01_03.pdf|D2.1 Scenario Descriptions}}
-  - {{:projects:emfase:deliverable:e.02.32_-_emfase_-_d2.2_-_first_evaluation_report_ed.00.01.00.pdf|D2.2 First Evaluation Report}}
-  - {{:projects:emfase:deliverable:e_02_32_-_emfase_-_d3_1_-_draft_causal_explanations-ed.00.01.00.pdf|D3.1 Draft Causal Explanations}}
-===== Publications =====
-    * K. Labunets, Y. Li, F. Massacci, F. Paci, M. Ragosta, B. Solhaug, K. Stølen, A. Tedeschi. **Preliminary Experiments on the Relative Comprehensibility of Tabular and Graphical Risk Models**, In //the Proceedings of 5th SESAR Innovation Days (SIDs'15).// {{:research_activities:experiments:2014-comprehensibility:labunets-etal-sids_2015_paper_32.pdf|PDF}}
-    * K. Labunets, F. Paci, F. Massacci. **Which Security Catalogue Is Better for Novices?** In //Proc. of EmpiRE Workshop at IEEE RE'15.// {{:research_activities:experiments:2014-winter-school:labunets-etal-empire-re15-preprint.pdf|PDF (preprint)}}
-  * M. de Gramatica, K. Labunets, F. Massacci, F. Paci, and A. Tedeschi. **The Role of Catalogues of Threats and Security Controls in Security Risk Assessment: An Empirical Study with ATM Professionals.** In //Proc. of REFSQ'15//. {{:research_activities:experiments:2014-rome-deepblue:gramatica-etal-refsq2015.pdf|PDF}}
-  * K. Labunets, F. Massacci, F. Paci, M. Ragosta, B. Solhaug, K. Stølen, A. Tedeschi. **A First Empirical Evaluation Framework for Security Risk Assessment Methods in the ATM Domain**, In //the Proceedings of 4th SESAR Innovation Days (SIDs'14).// {{:research_activities:experiments:2014-seceng:labunets-etal-sids_2014_paper_40.pdf|PDF}}
-  * M. Giacalone, R. Mammoliti, F. Massacci, F. Paci, R. Perugino, and C. Selli. **Security Triage: A Report of a Lean Security Requirements Methodology for Cost-Effective Security Analysis.** A short summary appears In //Proc. of EmpiRE Workshop at IEEE RE'14//. {{:research_activities:experiments:giacalone-etal-re14-preprint.pdf|3 pages PDF}}. A longer Industry report appears in //Proc. of ESEM'2014//. {{:research_activities:security_requirements_engineering:paper-207-esem-2014.pdf|PDF (preprint)}}
-  * K. Labunets, F. Paci, F. Massacci, and R. Ruprai. **An Experiment on Comparing Textual vs. Visual Industrial Methods for Security Risk Assessment.** In //Proc. of EmpiRE Workshop at IEEE RE'14// {{:research_activities:experiments:labunets-etal-empire-re14-preprint.pdf|PDF}}

DISI Security Research Group Wiki

User Tools

Site Tools

Differences

Page Tools