User Tools
erise_2011
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
erise_2011 [2013/04/09 11:04] katsiaryna.labunets@unitn.it [Data Collection and Analysis] |
erise_2011 [2021/01/29 10:58] (current) |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ===== eRISE Challenge 2011 ==== | ===== eRISE Challenge 2011 ==== | ||
| - | The eRISE challenge 2011 was conducted for empirical evaluation of security engineering methods. The event was carried out in May 2011. Both parts of experiment: training and application, took place at Dauphine University, Paris, France. | + | The eRISE challenge 2011 was conducted for empirical evaluation of security engineering methods. The event was carried out in May 2011. Both parts of experiment, training and application phases, took place at Dauphine University, Paris, France. |
| - | + | ||
| - | eRISE event has the objective of providing the method designer with: | + | |
| - | * Empirical evaluation and Benchmarking of security engineering methods; | + | |
| - | * Knowledge of how and why participants intend to adopt a method; | + | |
| - | * Feedback to improve a security method by investigating strengths, weakness and limitations of the method. | + | |
| - | + | ||
| - | eRISE aims to provide the participants with the benefit of: | + | |
| - | * Knowledge about various state-of-the art methods in the research field, on analyzing security risks and requirements of a system; | + | |
| - | * Opportunity to participate and work on an international collaborative project remotely. | + | |
| + | You can look a video of eRISE 2011 presentation on [[http://youtu.be/F7fUbBZzH-U|YouTube]] and download {{:research_activities:erise:erise_2011:tutorials:e_rise2011.ppt|slides}}. See the [[validation_of_risk_and_security_requirements_methodologies|main page]] for our work on empirical validation of security risk assessment methods and other experiments. | ||
| ==== Participants ==== | ==== Participants ==== | ||
| - | In eRISE 2011 were involved the following participants: | + | In eRISE 2011 were involved the following participants: {{ :research_activities:erise:erise_2011:photos:applciation02.jpeg?250|}} |
| * **Customers** | * **Customers** | ||
| - | * Yudistira Asnar (University of Trento) | + | * //Yudistira Asnar// (University of Trento) |
| - | * Federica Paci (University of Trento) | + | * //Federica Paci// (University of Trento) |
| * **Method Designers**: | * **Method Designers**: | ||
| - | * Atle Refsdal - SINTEF (CORAS) | + | * //Atle Refsdal// - SINTEF (CORAS). Interview on [[http://youtu.be/xQ8_6EACwnQ|YouTube]]. |
| - | * Thein Than Tun - Open University (Security Argumentation) | + | * //Thein Than Tun// - Open University (Security Argumentation). Interview on [[http://youtu.be/YaHKyRJhTw4|YouTube]]. |
| - | * Michalis Pavlidis, Shareeful Islam - University of East London (Secure Tropos) | + | * //Michalis Pavlidis, Shareeful Islam// - University of East London (Secure Tropos). Interview on [[http://youtu.be/LhYR_kYbJOM|YouTube]]. |
| - | * Fabio Massacci - University of Trento (Si*) | + | * //Fabio Massacci// - University of Trento (Si*) |
| - | * **Participants**: | + | * **Participants**: {{ :research_activities:erise:erise_2011:photos:application01.jpeg?250|}} |
| - | * 13 students were enrolled in the Master in Computer Science at the University of Trento; | + | * //13 students// were enrolled in the Master in Computer Science at the University of Trento; |
| - | * 36 professionals were attending a Master Course in Management of Information System Enterprise at Dauphine University. This master has an admission requirement of a minimum of five years of working experience in the field of Auditing in Information Systems | + | * //36 professionals// were attending a Master Course in Management of Information System Enterprise at Dauphine University. This master has an admission requirement of a minimum of five years of working experience in the field of Auditing in Information Systems |
| ==== Evaluated Methods ==== | ==== Evaluated Methods ==== | ||
| The selection of the security requirements methods to be evaluated was driven | The selection of the security requirements methods to be evaluated was driven | ||
| Line 33: | Line 23: | ||
| Four methods have been evaluated and compared during eRISE 2011: | Four methods have been evaluated and compared during eRISE 2011: | ||
| - | * **CORAS** is a model-driven method for risk analysis proposed by SINTEF, Norway. Materials: book chapter, tutorial. | + | * **CORAS** is a model-driven method for risk analysis proposed by SINTEF, Norway. Materials: {{:research_activities:erise:erise_2012:tutorials:coras-intro.pdf|book chapter}}, {{:research_activities:erise:erise_2011:tutorials:erise2011_coras_pres.pdf|tutorial}}. |
| - | * **SECURITY ARGUMENTATION** is a framework for security requirements elicitation and analysis developed at Open University, Buckinghamshire, United Kingdom. Materials: paper, tutorial. | + | * **SECURITY ARGUMENTATION** is a framework for security requirements elicitation and analysis developed at Open University, Buckinghamshire, United Kingdom. Materials: {{:research_activities:erise:erise_2012:tutorials:secarg-paper.pdf|paper}}, {{:research_activities:erise:erise_2011:tutorials:secarg-pres.pptx|tutorial}}. |
| - | * **SECURE TROPOS** is a methodology designed at University of East London, United Kingdom; the methodology supports capturing, analysis and reasoning of security requirements from the early stages of the development process. Materials: paper, tutorial. | + | * **SECURE TROPOS** is a methodology designed at University of East London, United Kingdom; the methodology supports capturing, analysis and reasoning of security requirements from the early stages of the development process. Materials: {{:research_activities:erise:erise_2011:tutorials:secure_tropos-paper.pdf|paper}}, {{:research_activities:erise:erise_2011:tutorials:secure_tropos-paper2.pdf|paper 2}}, {{:research_activities:erise:erise_2011:tutorials:secure_tropos_presentation.pptx|tutorial}}. |
| - | * **SI* ** is a formal framework developed at the University of Trento, Italy for modeling and analyzing security requirements of an organization. Materials: paper, tutorial. | + | * **SI* ** is a formal framework developed at the University of Trento, Italy for modeling and analyzing security requirements of an organization. Materials: {{:research_activities:erise:erise_2011:tutorials:si_star-paper.pdf|paper}}, {{:research_activities:erise:erise_2011:tutorials:si_star-pres.pptx|tutorial}}. |
| ==== Application scenario ==== | ==== Application scenario ==== | ||
| In eRISE 2011 fictional application scenario, Healthcare Collaboration Network(HCN), was proposed to the participant for analysis. | In eRISE 2011 fictional application scenario, Healthcare Collaboration Network(HCN), was proposed to the participant for analysis. | ||
| - | Regional HealthCare Authority needs to monitor and alert citizens on occurrence of endemic or pandemic diseases within the region of CityVille. Healthcare Authority decides to create Healthcare Collaboration network involving data source organizations (like hospitals, physicians) and data review organizations (like government agencies, health insurers). Participants perform the role of consultants in analyzing the main threats; ensuring the information security and privacy protection of Healthcare collaboration network. | + | Regional HealthCare Authority needs to monitor and alert citizens on occurrence of endemic or pandemic diseases within the region of CityVille. Healthcare Authority decides to create Healthcare Collaboration network involving data source organizations (like hospitals, physicians) and data review organizations (like government agencies, health insurers). Participants perform the role of consultants in analyzing the main threats; ensuring the information security and privacy protection of Healthcare collaboration network. |
| - | The materials about this scenario are available online: scenario description and presentation. | + | The participants, during the Training day, received two chapters of [[http://www.redbooks.ibm.com/abstracts/sg246779.html|the HCN book]] (Ch.1 and Ch.6). Moreover the participants received a 1-hour seminar about HCN, which was given by one member of the organizing team. |
| + | The materials about this scenario are available online: {{:research_activities:erise:erise_2011:tutorials:hcn_chapters.pdf|HCN chapters}}, {{:research_activities:erise:erise_2011:tutorials:hcn_pres.pptx|presentation}}, {{:research_activities:erise:erise_2011:tutorials:ceo_note.pdf|customer's email}}, {{:research_activities:erise:erise_2011:tutorials:ade_faqs.pdf|Adverse Drug Event FAQ}}. | ||
| ==== Experimental Procedure ==== | ==== Experimental Procedure ==== | ||
| - | eRISE 2011 was conducted in three main phases: | + | eRISE 2011 was conducted in three main phases: {{ :research_activities:erise:erise_2011:photos:training.jpeg?250|}} |
| * **Training Phase** on May 13, 2011 (at Dauphine Paris University), where participants attended tutorials on the methods under evaluation and on the HCN case. | * **Training Phase** on May 13, 2011 (at Dauphine Paris University), where participants attended tutorials on the methods under evaluation and on the HCN case. | ||
| * **Application Phases** on May 14-27, 2011 (with face-to-face session on May 26-27 at Dauphine Paris University) where participants applied the methods to analyse security issues of the HCN case. | * **Application Phases** on May 14-27, 2011 (with face-to-face session on May 26-27 at Dauphine Paris University) where participants applied the methods to analyse security issues of the HCN case. | ||
| - | * **Evaluation Phase**, where participants evaluated the methods through focused group interviews while method designers evaluated the final reports. The goal is to assess the correctness of the methods application and the quality of the security requirements identified by the participants. | + | * **Evaluation Phase**, where participants evaluated the methods through focused group interviews while method designers evaluated the final reports. {{ :research_activities:erise:erise_2011:photos:application03.jpeg?250|}} The goal is to assess the correctness of the methods application and the quality of the security requirements identified by the participants. |
| + | - **Two Post-it session** were conducted for each method, each involving six participants, apart from one session, which had 7 participants. Each participant was asked to produce a total 20 post-its: 5 each one containing positive aspects about the method, 5 each one containing negative aspects about the method, 5 containing positive aspects about the competition, and 5 reporting negative aspects about the competition. All these post-it notes contributed to two Post-it clouds, one about the method and one about the competition, of 120 post-its per method (130 for one of the method), for a total of 490 notes. {{ :research_activities:erise:erise_2011:photos:post_discussion.jpeg?250|}} | ||
| + | - **Two focus group discussions** were conducted for each method, each involving six participants, (apart from one discussion, which had 7 participants), the Method Designer and one member of the Organizing Team, which also served as moderator. Focus groups had a duration of 90 minutes each and yielded a total of 540 minutes of audio and video recordings. | ||
| ==== Data Collection and Analysis ==== | ==== Data Collection and Analysis ==== | ||
| We have collected different kinds of data: | We have collected different kinds of data: | ||
| - | * **Questionnaires** include questions on subjects' knowledge of IT security, risk assessment, and requirements engineering and their evaluation of the methods' aspects. The participants were administered **four questionnaires** during the execution of the eRISE 2011: | + | * **Questionnaires** include questions on subjects' knowledge of IT security, risk assessment, and requirements engineering and their evaluation of the methods' aspects. Questionnaires contained a combination of open questions and list of adjectives, rated by participants through 7-points Likert scales. The participants were administered **four questionnaires** during the execution of the eRISE 2011: {{ :research_activities:erise:erise_2011:photos:artifact.jpeg?250|}} |
| - | * **Q1** was administered at the beginning of the Training phase to collect participants' background ({{:research_activities:erise:erise_2011:questionnaires:q1-information_security_awareness.pdf|Q1}}); | + | * **Q1** was administered before the Training phase and aimed at collecting participant’s level of awareness on Information Security.({{:research_activities:erise:erise_2011:questionnaires:q1-information_security_awareness.pdf|Q1}}). |
| - | * **Q2** was distributed at the end of Training phase to collect participants' impression about the method that they were working with, regardless of the scenario on which they were asked to apply the method ({{:research_activities:erise:erise_2011:questionnaires:q2-erise_method_questionnaire.pdf|Q2}}); | + | * **Q2** was administered to participants after the Training phase and aimed at collecting participants’ first impression about the method ({{:research_activities:erise:erise_2011:questionnaires:q2-erise_method_questionnaire.pdf|Q2}}). |
| - | * **Q3** was administered at the end of remote part of the Application phase to collect participants' impression about the method that they were assigned to, after using it via remote collaboration tools ({{:research_activities:erise:erise_2011:questionnaires:q3-erise_method_questionnaire.pdf|Q3}}); | + | * **Q3** was administered at the end of remote group collaboration and aimed at collecting participants’ opinion about the method when applied in a condition of remote group collaboration. This was also a mid-term overall evaluation of the method ({{:research_activities:erise:erise_2011:questionnaires:q3-erise_method_questionnaire.pdf|Q3}}). |
| - | * **Q4** was administered after face-to-face collaboration part of the Application phase ({{:research_activities:erise:erise_2011:questionnaires:q4-erise_method_questionnaire.pdf|Q4}}); | + | * **Q4** was administered at the end of the Application phase, after the sessions of face-to-face group work sessions. This questionnaire aimed at collecting final evaluation by participants about the method ({{:research_activities:erise:erise_2011:questionnaires:q4-erise_method_questionnaire.pdf|Q4}}). {{ :research_activities:erise:erise_2011:photos:postit_notes.jpeg?250|}} |
| * **Audio/Video Recordings* ** capture the application of the methods by subjects and the focus groups interviews; | * **Audio/Video Recordings* ** capture the application of the methods by subjects and the focus groups interviews; | ||
| * **Post-it Notes* ** list positive and negative aspects about the methods and the study itself; | * **Post-it Notes* ** list positive and negative aspects about the methods and the study itself; | ||
erise_2011.1365498264.txt.gz · Last modified: 2021/01/29 10:58 (external edit)
Page Tools
