User Tools

Site Tools


erise_2011

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
erise_2011 [2013/04/08 16:53]
katsiaryna.labunets@unitn.it
erise_2011 [2021/01/29 10:58] (current)
Line 1: Line 1:
 ===== eRISE Challenge 2011 ==== ===== eRISE Challenge 2011 ====
-The eRISE challenge 2011 was conducted for empirical evaluation of security engineering methods. The event was carried out in May 2011. Both parts of experimenttraining and application,​ took place at Dauphine University, Paris, France. +The eRISE challenge 2011 was conducted for empirical evaluation of security engineering methods. The event was carried out in May 2011. Both parts of experimenttraining and application ​phases, took place at Dauphine University, Paris, France.
- +
-eRISE event has the objective of providing the method designer with:   +
-  * Empirical evaluation and Benchmarking of security engineering methods;  +
-  * Knowledge of how and why participants intend to adopt a method; +
-  * Feedback to improve a security method by investigating strengths, weakness and limitations of the method.  +
- +
-eRISE aims to provide the participants with the benefit of:  +
-  * Knowledge about various state-of-the art methods in the research field, on analyzing security risks and requirements of a system; +
-  * Opportunity to participate and work on an international collaborative project remotely.  +
  
 +You can look a video of eRISE 2011 presentation on [[http://​youtu.be/​F7fUbBZzH-U|YouTube]] and download {{:​research_activities:​erise:​erise_2011:​tutorials:​e_rise2011.ppt|slides}}. See the [[validation_of_risk_and_security_requirements_methodologies|main page]] for our work on empirical validation of security risk assessment methods and other experiments.
 ==== Participants ==== ==== Participants ====
-In eRISE 2011 were involved the following participants:​+In eRISE 2011 were involved the following participants: ​{{ :​research_activities:​erise:​erise_2011:​photos:​applciation02.jpeg?​250|}}
   * **Customers** ​   * **Customers** ​
-     * Yudistira Asnar (University of Trento) +     ​* ​//Yudistira Asnar// (University of Trento) 
-     * Federica Paci (University of Trento)+     ​* ​//Federica Paci// (University of Trento)
   * **Method Designers**:​   * **Method Designers**:​
-     * Atle Refsdal - SINTEF (CORAS) +     ​* ​//Atle Refsdal// - SINTEF (CORAS). Interview on [[http://​youtu.be/​xQ8_6EACwnQ|YouTube]]. 
-     * Thein Than Tun - Open University (Security Argumentation)  +     ​* ​//Thein Than Tun// - Open University (Security Argumentation). Interview on [[http://​youtu.be/​YaHKyRJhTw4|YouTube]]. ​ 
-     * Michalis Pavlidis, Shareeful Islam - University of East London (Secure Tropos) +     ​* ​//Michalis Pavlidis, Shareeful Islam// - University of East London (Secure Tropos). Interview on [[http://​youtu.be/​LhYR_kYbJOM|YouTube]]. 
-     * Fabio Massacci - University of Trento (Si*) +     ​* ​//Fabio Massacci// - University of Trento (Si*) 
-  * **Participants**:​ +  * **Participants**: ​{{ :​research_activities:​erise:​erise_2011:​photos:​application01.jpeg?​250|}} 
-     * 13 students were enrolled in the Master in Computer Science at the University of Trento; +     ​* ​//13 students// were enrolled in the Master in Computer Science at the University of Trento; 
-     * 36 professionals were attending a Master Course in Management of Information System Enterprise at Dauphine University. This master has an admission requirement of a minimum of five years of working experience in the field of Auditing in Information Systems +     ​* ​//36 professionals// were attending a Master Course in Management of Information System Enterprise at Dauphine University. This master has an admission requirement of a minimum of five years of working experience in the field of Auditing in Information Systems
 ==== Evaluated Methods ==== ==== Evaluated Methods ====
 The selection of the security requirements methods to be evaluated was driven The selection of the security requirements methods to be evaluated was driven
Line 33: Line 23:
 Four methods have been evaluated and compared during eRISE 2011: Four methods have been evaluated and compared during eRISE 2011:
  
-  * **CORAS** is a model-driven method for risk analysis proposed by SINTEF, Norway. Materials: book chapter, tutorial. +  * **CORAS** is a model-driven method for risk analysis proposed by SINTEF, Norway. Materials: ​{{:​research_activities:​erise:​erise_2012:​tutorials:​coras-intro.pdf|book chapter}}{{:​research_activities:​erise:​erise_2011:​tutorials:​erise2011_coras_pres.pdf|tutorial}}
-  * **SECURITY ARGUMENTATION** is a framework for security requirements elicitation and analysis developed at Open University, Buckinghamshire,​ United Kingdom. Materials: paper, tutorial. +  * **SECURITY ARGUMENTATION** is a framework for security requirements elicitation and analysis developed at Open University, Buckinghamshire,​ United Kingdom. Materials: ​{{:​research_activities:​erise:​erise_2012:​tutorials:​secarg-paper.pdf|paper}}{{:​research_activities:​erise:​erise_2011:​tutorials:​secarg-pres.pptx|tutorial}}
-  * **SECURE TROPOS** is a methodology designed at University of East London, United Kingdom; the methodology supports capturing, analysis and reasoning of security requirements from the early stages of the development process. Materials: paper, tutorial. +  * **SECURE TROPOS** is a methodology designed at University of East London, United Kingdom; the methodology supports capturing, analysis and reasoning of security requirements from the early stages of the development process. Materials: ​{{:​research_activities:​erise:​erise_2011:​tutorials:​secure_tropos-paper.pdf|paper}}{{:​research_activities:​erise:​erise_2011:​tutorials:​secure_tropos-paper2.pdf|paper 2}}, {{:​research_activities:​erise:​erise_2011:​tutorials:​secure_tropos_presentation.pptx|tutorial}}
-  * **SI* ** is a formal framework developed at the University of Trento, Italy for modeling and analyzing security requirements of an organization. Materials: paper, tutorial.+  * **SI* ** is a formal framework developed at the University of Trento, Italy for modeling and analyzing security requirements of an organization. Materials: ​{{:​research_activities:​erise:​erise_2011:​tutorials:​si_star-paper.pdf|paper}}{{:​research_activities:​erise:​erise_2011:​tutorials:​si_star-pres.pptx|tutorial}}. 
 +==== Application scenario ==== 
 +In eRISE 2011 fictional application scenario, Healthcare Collaboration Network(HCN),​ was proposed to the participant for analysis.
  
-==== Application scenarios ==== +Regional HealthCare Authority needs to monitor and alert citizens on occurrence of endemic or pandemic diseases within the region of CityVille. Healthcare Authority decides to create ​Healthcare Collaboration ​network involving data source organizations ​(like hospitals, physicians) and data review organizations ​(like government agencies, health insurers). Participants perform ​the role of consultants in analyzing the main threats; ensuring the information security and privacy protection of Healthcare collaboration network
-In eRISE 2011 Healthcare Collaboration ​Network ​(HCNscenario ​and its extension for monitoring Adverse Drug Event(HCN-ADEwere proposed to the participant for analysis.+
  
-=== Healthcare Collaborative Network ===+The participants,​ during the Training day, received two chapters of [[http://​www.redbooks.ibm.com/​abstracts/​sg246779.html|the HCN book]] (Ch.1 and Ch.6). Moreover the participants received a 1-hour seminar about HCN, which was given by one member of the organizing team.  ​
  
-Regional HealthCare Authority needs to monitor ​and alert citizens ​on occurrence ​of endemic or pandemic diseases within ​the region ​of CityVilleHealthcare Authority decides ​to create Healthcare Collaboration network ​involving ​data source organizations ​(like hospitalsphysicians) and data review organizations (like government agencieshealth insurers). Participants perform ​the role of consultants ​in analyzing ​the main threatsensuring ​the information security ​and privacy protection ​of Healthcare collaboration network.  ​+The materials about this scenario are available online: {{:​research_activities:​erise:​erise_2011:​tutorials:​hcn_chapters.pdf|HCN chapters}}, {{:​research_activities:​erise:​erise_2011:​tutorials:​hcn_pres.pptx|presentation}},​ {{:​research_activities:​erise:​erise_2011:​tutorials:​ceo_note.pdf|customer'​s email}}, {{:​research_activities:​erise:​erise_2011:​tutorials:​ade_faqs.pdf|Adverse Drug Event FAQ}}. 
 +==== Experimental Procedure ==== 
 +eRISE 2011 was conducted in three main phases: {{ :​research_activities:​erise:​erise_2011:​photos:​training.jpeg?​250|}} 
 +  * **Training Phase** on May 13, 2011 (at Dauphine Paris University),​ where participants attended tutorials on the methods under evaluation ​and on the HCN case. 
 +  * **Application Phases** on May 14-27, 2011 (with face-to-face session on May 26-27 at Dauphine Paris University) where participants applied the methods to analyse security issues ​of the HCN case. 
 +  * **Evaluation Phase**, where participants evaluated the methods through focused group interviews while method designers evaluated the final reports. {{ :​research_activities:​erise:​erise_2011:​photos:​application03.jpeg?​250|}} The goal is to assess the correctness ​of the methods application and the quality of the security requirements identified by the participants.  
 +      -  **Two Post-it session** were conducted for each method, each involving six participants,​ apart from one session, which had 7 participantsEach participant was asked to produce a total 20 post-its: 5 each one containing positive aspects about the method, 5 each one containing negative aspects about the method, 5 containing positive aspects about the competition,​ and 5 reporting negative aspects about the competition. All these post-it notes contributed to two Post-it clouds, one about the method and one about the competition,​ of 120 post-its per method (130 for one of the method), for a total of 490 notes. {{ :​research_activities:​erise:​erise_2011:​photos:​post_discussion.jpeg?​250|}} 
 +      - **Two focus group discussions** were conducted for each method, each involving ​six participants, ​(apart from one discussionwhich had 7 participants), the Method Designer ​and one member of the Organizing Team, which also served as moderator. Focus groups had a duration of 90 minutes each and yielded a total of 540 minutes of audio and video recordings. ​  
 +==== Data Collection and Analysis ==== 
 +We have collected different kinds of data
 +  * **Questionnaires** include questions on subjects'​ knowledge of IT securityrisk assessment, and requirements engineering and their evaluation of the methods'​ aspects. Questionnaires contained a combination of open questions and list of adjectives, rated by participants through 7-points Likert scales. The participants were administered **four questionnaires** during the execution of the eRISE 2011:  {{ :​research_activities:​erise:​erise_2011:​photos:​artifact.jpeg?​250|}} 
 +    * **Q1** was administered before the Training phase and aimed at collecting participant’s level of awareness on Information Security.({{:​research_activities:​erise:​erise_2011:​questionnaires:​q1-information_security_awareness.pdf|Q1}}). 
 +    * **Q2** was administered to participants after the Training phase and aimed at collecting participants’ first impression about the method ({{:​research_activities:​erise:​erise_2011:​questionnaires:​q2-erise_method_questionnaire.pdf|Q2}}). 
 +    * **Q3** was administered at the end of remote group collaboration and aimed at collecting participants’ opinion about the method when applied ​in a condition of remote group collaboration. This was also a mid-term overall evaluation of the method ({{:​research_activities:​erise:​erise_2011:​questionnaires:​q3-erise_method_questionnaire.pdf|Q3}}). ​  
 +    * **Q4** was administered at the end of the Application phase, after the sessions of face-to-face group work sessions. This questionnaire aimed at collecting final evaluation by participants about the method ({{:​research_activities:​erise:​erise_2011:​questionnaires:​q4-erise_method_questionnaire.pdf|Q4}}). {{ :​research_activities:​erise:​erise_2011:​photos:​postit_notes.jpeg?​250|}} 
 +  * **Audio/​Video Recordings* ** capture the application of the methods by subjects and the focus groups interviews 
 +  * **Post-it Notes* ** list positive and negative aspects about the methods ​and the study itself; 
 +  * **Focus Group Transcripts* ** report the discussion with method designers a number ​of topics related to the method, its application on the given scenario and the process of evaluation. 
 +  * **Group Presentations* ** by participants summarize the results of method'​s application;​ 
 +  * **Final Reports* ** describe in detail how participants have identified the security requirements following the method.
  
-The materials ​about this scenario ​are available ​online: scenario description and presentation.+* These materials are available ​upon e-mail request
  
-=== HCN: Monitoring Adverse Drug Event ===+=== Data Analysis ​=== 
  
-Healthcare Collaboration Network (HCN) needs to monitor Adverse Drug Event at the CityVille +Questionnaires have been analyzed using //​statistical analysis//. For post-it notes we have used //affinity analysis// in order to group similar feedback on positive and negative aspects of the methods. The transcripts of the focus groups discussions have been analyzed using //coding//, a content analysis technique used in grounded theory. Coding helped us to discover text patterns that are relevant to what makes methods effective in identifying security requirements ​and why.
- +
-The materials about this scenario ​are available online: scenario description ​and presentation. +
- +
- +
-==== Context ==== +
- +
-==== Experimental Procedure ==== +
- +
-==== Data Collection and Analysis ====+
erise_2011.1365440039.txt.gz · Last modified: 2021/01/29 10:58 (external edit)