User Tools
erise_2012
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
erise_2012 [2013/04/08 12:56] katsiaryna.labunets@unitn.it [Experimental Procedure] |
erise_2012 [2021/01/29 10:58] (current) |
||
|---|---|---|---|
| Line 2: | Line 2: | ||
| The eRISE 2012 challenge was conducted to empirically evaluate security engineering and risk analysis methods. The event was carried out in May and June 2012. The first part of experiment took place at the University of Trento, Italy, the second at Dauphine University, Paris, France. | The eRISE 2012 challenge was conducted to empirically evaluate security engineering and risk analysis methods. The event was carried out in May and June 2012. The first part of experiment took place at the University of Trento, Italy, the second at Dauphine University, Paris, France. | ||
| - | To have a glimpse of eRISE 2012 you can look a video on YouTube [[http://youtu.be/hyxg56RTQsw|eRISE 2012]]. | + | To have a glimpse of eRISE 2012 you can look a video on YouTube [[http://youtu.be/hyxg56RTQsw|eRISE 2012]]. See the [[validation_of_risk_and_security_requirements_methodologies|main page]] for our work on empirical validation of security risk assessment methods and other experiments. |
| ==== Participants ==== | ==== Participants ==== | ||
| + | {{ :research_activities:erise:erise_2012:photo:training01.jpeg?250|}} | ||
| In eRISE 2012 were involved the following participants: | In eRISE 2012 were involved the following participants: | ||
| * **Customers** | * **Customers** | ||
| - | * Marina Egea (Atos Research); | + | * //Marina Egea// (Atos Research); |
| - | * Jorge Cuellar (Siemens AG); | + | * //Jorge Cuellar// (Siemens AG); |
| * **Method Designers**: | * **Method Designers**: | ||
| - | * Le Minh Sang Tran - SINTEF/University of Trento (CORAS) | + | * //Le Minh Sang Tran// - SINTEF/University of Trento (CORAS) |
| - | * Kim Wuyts, Riccardo Scandariato - Katholieke Universiteit Leuven (LINDDUN) | + | * //Kim Wuyts, Riccardo Scandariato// - Katholieke Universiteit Leuven (LINDDUN) |
| - | * Thein Than Tun - Open University (Security Argumentation) | + | * //Thein Than Tun// - Open University (Security Argumentation) |
| - | * Michalis Pavlidis - University of East London (Secure Tropos) | + | * //Michalis Pavlidis// - University of East London (Secure Tropos) |
| - | * Daniel G. Mellado - University of Castilla La Mancha (SREP) | + | * //Daniel G. Mellado// - University of Castilla La Mancha (SREP) |
| * **Observers**: | * **Observers**: | ||
| - | * Sarila Rana | + | * //Sarila Rana// |
| - | * Martina Degramatica | + | * //Martina Degramatica// |
| - | * Deepa Nagaraj | + | * //Deepa Nagaraj// |
| - | * Elda Paja | + | * //Elda Paja// |
| - | * Jennifer Horkoff | + | * //Jennifer Horkoff// |
| * **Participants**: | * **Participants**: | ||
| - | * 15 students were enrolled in the Master in Computer Science at the University of Trento and had a background in Security Engineering and Information Systems | + | * //15 students// were enrolled in the Master in Computer Science at the University of Trento and had a background in Security Engineering and Information Systems |
| - | * 27 professionals were attending a Master Course in Audit for Information System in Enterprises at Dauphine University. This master has an admission requirement of a minimum of five years of working experience in the field of Auditing in Information Systems | + | * //27 professionals// were attending a Master Course in Audit for Information System in Enterprises at Dauphine University. This master has an admission requirement of a minimum of five years of working experience in the field of Auditing in Information Systems |
| ==== Evaluated Methods ==== | ==== Evaluated Methods ==== | ||
| + | |||
| + | {{ :research_activities:erise:erise_2012:photo:training.jpeg?250|}} | ||
| The selection of the security requirements methods to be evaluated was driven | The selection of the security requirements methods to be evaluated was driven | ||
| by three main factors: the number of citations, the fact that research on the | by three main factors: the number of citations, the fact that research on the | ||
| method is still ongoing, and availability of the methods designers. | method is still ongoing, and availability of the methods designers. | ||
| - | |||
| - | {{ :research_activities:erise:erise_2012:photo:training.jpeg?250|}} | ||
| Five methods have been evaluated and compared during eRISE 2012: | Five methods have been evaluated and compared during eRISE 2012: | ||
| Line 55: | Line 56: | ||
| ==== Experimental Procedure ==== | ==== Experimental Procedure ==== | ||
| eRISE 2012 was conducted in three main phases: | eRISE 2012 was conducted in three main phases: | ||
| - | * **Training Phase** where participants attended tutorials on the methods under evaluation and on the eHealth and Smart Grid industrial cases: {{ :research_activities:erise:erise_2012:photo:artifact.jpeg?175|}} | + | {{ :research_activities:erise:erise_2012:photo:application_01.jpeg?250|}} |
| - | - May 7-9, 2012 at the University of Trento, Trento, Italy | + | * **Training Phase** where participants attended tutorials on the methods under evaluation and on the eHealth and Smart Grid industrial cases: |
| + | - May 7-9, 2012 at the University of Trento, Italy | ||
| * **Application Phases**, where participants applied the methods to analyse security issues of the eHealth and Smart Grid industrial cases: | * **Application Phases**, where participants applied the methods to analyse security issues of the eHealth and Smart Grid industrial cases: | ||
| - | - May 10-11, 2012 at the University of Trento, Trento, Italy | + | - May 10-11, 2012 at the University of Trento, Italy |
| - June 14-15, 2012 at Dauphine University, Paris, France | - June 14-15, 2012 at Dauphine University, Paris, France | ||
| - | |||
| * **Evaluation Phase**, where participants evaluated the methods through focused group interviews and post-it notes sessions while method designers and customers evaluated the final reports. The goal is to assess the correctness of the methods application and the quality of the security requirements identified by the participants. | * **Evaluation Phase**, where participants evaluated the methods through focused group interviews and post-it notes sessions while method designers and customers evaluated the final reports. The goal is to assess the correctness of the methods application and the quality of the security requirements identified by the participants. | ||
| - June 15, 2012 Focus Groups and Post-it notes sessions with participants, at Dauphine University, Paris, France | - June 15, 2012 Focus Groups and Post-it notes sessions with participants, at Dauphine University, Paris, France | ||
| Line 66: | Line 67: | ||
| ==== Data Collection and Analysis ==== | ==== Data Collection and Analysis ==== | ||
| We have collected different kinds of data: | We have collected different kinds of data: | ||
| + | {{ :research_activities:erise:erise_2012:photo:presentation.jpeg?250|}} | ||
| * **Questionnaires** include questions on subjects' knowledge of IT security, risk assessment, and requirements engineering and their evaluation of the methods' aspects. The participants were administered **five questionnaires** during the execution of the eRISE 2012: | * **Questionnaires** include questions on subjects' knowledge of IT security, risk assessment, and requirements engineering and their evaluation of the methods' aspects. The participants were administered **five questionnaires** during the execution of the eRISE 2012: | ||
| * **Q1** was administered at the beginning of the Training phase to collect participants' background ({{:research_activities:erise:erise_2012:tutorials:q1-background.docx|Q1}}); | * **Q1** was administered at the beginning of the Training phase to collect participants' background ({{:research_activities:erise:erise_2012:tutorials:q1-background.docx|Q1}}); | ||
| * **Q2** was distributed at the end of the Training phase ({{:research_activities:erise:erise_2012:tutorials:q2-method_assessment.docx|Q2}}); | * **Q2** was distributed at the end of the Training phase ({{:research_activities:erise:erise_2012:tutorials:q2-method_assessment.docx|Q2}}); | ||
| * **Q3** and **Q4** were administered during the two Application phases ({{:research_activities:erise:erise_2012:tutorials:q3-method_assessment.docx|Q3}} and {{:research_activities:erise:erise_2012:tutorials:q4-method_assessment.docx|Q4}}); | * **Q3** and **Q4** were administered during the two Application phases ({{:research_activities:erise:erise_2012:tutorials:q3-method_assessment.docx|Q3}} and {{:research_activities:erise:erise_2012:tutorials:q4-method_assessment.docx|Q4}}); | ||
| - | * **Q5** was administered at the end of the Application phase to compare the method applied by the participants with other methods they may already knew ({{:research_activities:erise:erise_2012:tutorials:q5-_comparison_.pdf|Q5}}); {{ :research_activities:erise:erise_2012:photo:postit_notes.jpg?300|}} | + | * **Q5** was administered at the end of the Application phase to compare the method applied by the participants with other methods they may already knew ({{:research_activities:erise:erise_2012:tutorials:q5-_comparison_.pdf|Q5}}); {{ :research_activities:erise:erise_2012:photo:postit_notes.jpg?250|}} |
| * **Audio/Video Recordings* ** capture the application of the methods by subjects and the focus groups interviews; | * **Audio/Video Recordings* ** capture the application of the methods by subjects and the focus groups interviews; | ||
| * **Post-it Notes* ** list positive and negative aspects about the methods and the study itself; | * **Post-it Notes* ** list positive and negative aspects about the methods and the study itself; | ||
| Line 79: | Line 81: | ||
| * These materials are available upon e-mail request. | * These materials are available upon e-mail request. | ||
| - | === Data Analysis === | + | === Data Analysis === |
| + | |||
| + | {{:research_activities:erise:erise_2012:photo:artifact.jpeg?175 |}} | ||
| Questionnaires have been analyzed using //statistical analysis//. For post-it notes we have used //affinity analysis// in order to group similar feedback on positive and negative aspects of the methods. The transcripts of the focus groups discussions have been analyzed using //coding//, a content analysis technique used in grounded theory. Coding helped us to discover text patterns that are relevant to what makes methods effective in identifying security requirements and why. We have performed a qualitative analysis of the final reports. | Questionnaires have been analyzed using //statistical analysis//. For post-it notes we have used //affinity analysis// in order to group similar feedback on positive and negative aspects of the methods. The transcripts of the focus groups discussions have been analyzed using //coding//, a content analysis technique used in grounded theory. Coding helped us to discover text patterns that are relevant to what makes methods effective in identifying security requirements and why. We have performed a qualitative analysis of the final reports. | ||
erise_2012.1365418561.txt.gz · Last modified: 2021/01/29 10:58 (external edit)
Page Tools
