This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
erise_2012 [2013/04/08 17:20] federica.paci@unitn.it [Evaluated Methods] |
erise_2012 [2013/04/08 17:58] katsiaryna.labunets@unitn.it [Experimental Procedure] |
||
---|---|---|---|
Line 25: | Line 25: | ||
==== Evaluated Methods ==== | ==== Evaluated Methods ==== | ||
+ | |||
+ | {{ :research_activities:erise:erise_2012:photo:training.jpeg?200|}} | ||
The selection of the security requirements methods to be evaluated was driven | The selection of the security requirements methods to be evaluated was driven | ||
by three main factors: the number of citations, the fact that research on the | by three main factors: the number of citations, the fact that research on the | ||
Line 53: | Line 55: | ||
==== Experimental Procedure ==== | ==== Experimental Procedure ==== | ||
eRISE 2012 was conducted in three main phases: | eRISE 2012 was conducted in three main phases: | ||
- | * **Training Phase** where participants attended tutorials on the methods under evaluation and on the eHealth and Smart Grid industrial cases: {{ :research_activities:erise:erise_2012:photo:artifact.jpeg?175|}} | + | {{ :research_activities:erise:erise_2012:photo:application_01.jpeg?250|}} |
+ | * **Training Phase** where participants attended tutorials on the methods under evaluation and on the eHealth and Smart Grid industrial cases: | ||
- May 7-9, 2012 at the University of Trento, Trento, Italy | - May 7-9, 2012 at the University of Trento, Trento, Italy | ||
* **Application Phases**, where participants applied the methods to analyse security issues of the eHealth and Smart Grid industrial cases: | * **Application Phases**, where participants applied the methods to analyse security issues of the eHealth and Smart Grid industrial cases: | ||
- May 10-11, 2012 at the University of Trento, Trento, Italy | - May 10-11, 2012 at the University of Trento, Trento, Italy | ||
- June 14-15, 2012 at Dauphine University, Paris, France | - June 14-15, 2012 at Dauphine University, Paris, France | ||
- | |||
* **Evaluation Phase**, where participants evaluated the methods through focused group interviews and post-it notes sessions while method designers and customers evaluated the final reports. The goal is to assess the correctness of the methods application and the quality of the security requirements identified by the participants. | * **Evaluation Phase**, where participants evaluated the methods through focused group interviews and post-it notes sessions while method designers and customers evaluated the final reports. The goal is to assess the correctness of the methods application and the quality of the security requirements identified by the participants. | ||
- June 15, 2012 Focus Groups and Post-it notes sessions with participants, at Dauphine University, Paris, France | - June 15, 2012 Focus Groups and Post-it notes sessions with participants, at Dauphine University, Paris, France | ||
Line 64: | Line 66: | ||
==== Data Collection and Analysis ==== | ==== Data Collection and Analysis ==== | ||
We have collected different kinds of data: | We have collected different kinds of data: | ||
+ | {{ :research_activities:erise:erise_2012:photo:presentation.jpeg?250|}} | ||
* **Questionnaires** include questions on subjects' knowledge of IT security, risk assessment, and requirements engineering and their evaluation of the methods' aspects. The participants were administered **five questionnaires** during the execution of the eRISE 2012: | * **Questionnaires** include questions on subjects' knowledge of IT security, risk assessment, and requirements engineering and their evaluation of the methods' aspects. The participants were administered **five questionnaires** during the execution of the eRISE 2012: | ||
* **Q1** was administered at the beginning of the Training phase to collect participants' background ({{:research_activities:erise:erise_2012:tutorials:q1-background.docx|Q1}}); | * **Q1** was administered at the beginning of the Training phase to collect participants' background ({{:research_activities:erise:erise_2012:tutorials:q1-background.docx|Q1}}); | ||
* **Q2** was distributed at the end of the Training phase ({{:research_activities:erise:erise_2012:tutorials:q2-method_assessment.docx|Q2}}); | * **Q2** was distributed at the end of the Training phase ({{:research_activities:erise:erise_2012:tutorials:q2-method_assessment.docx|Q2}}); | ||
* **Q3** and **Q4** were administered during the two Application phases ({{:research_activities:erise:erise_2012:tutorials:q3-method_assessment.docx|Q3}} and {{:research_activities:erise:erise_2012:tutorials:q4-method_assessment.docx|Q4}}); | * **Q3** and **Q4** were administered during the two Application phases ({{:research_activities:erise:erise_2012:tutorials:q3-method_assessment.docx|Q3}} and {{:research_activities:erise:erise_2012:tutorials:q4-method_assessment.docx|Q4}}); | ||
- | * **Q5** was administered at the end of the Application phase to compare the method applied by the participants with other methods they may already knew ({{:research_activities:erise:erise_2012:tutorials:q5-_comparison_.pdf|Q5}}); {{ :research_activities:erise:erise_2012:photo:postit_notes.jpg?300|}} | + | * **Q5** was administered at the end of the Application phase to compare the method applied by the participants with other methods they may already knew ({{:research_activities:erise:erise_2012:tutorials:q5-_comparison_.pdf|Q5}}); {{ :research_activities:erise:erise_2012:photo:postit_notes.jpg?250|}} |
* **Audio/Video Recordings* ** capture the application of the methods by subjects and the focus groups interviews; | * **Audio/Video Recordings* ** capture the application of the methods by subjects and the focus groups interviews; | ||
* **Post-it Notes* ** list positive and negative aspects about the methods and the study itself; | * **Post-it Notes* ** list positive and negative aspects about the methods and the study itself; | ||
Line 77: | Line 80: | ||
* These materials are available upon e-mail request. | * These materials are available upon e-mail request. | ||
- | === Data Analysis === | + | === Data Analysis === |
+ | |||
+ | {{:research_activities:erise:erise_2012:photo:artifact.jpeg?175 |}} | ||
Questionnaires have been analyzed using //statistical analysis//. For post-it notes we have used //affinity analysis// in order to group similar feedback on positive and negative aspects of the methods. The transcripts of the focus groups discussions have been analyzed using //coding//, a content analysis technique used in grounded theory. Coding helped us to discover text patterns that are relevant to what makes methods effective in identifying security requirements and why. We have performed a qualitative analysis of the final reports. | Questionnaires have been analyzed using //statistical analysis//. For post-it notes we have used //affinity analysis// in order to group similar feedback on positive and negative aspects of the methods. The transcripts of the focus groups discussions have been analyzed using //coding//, a content analysis technique used in grounded theory. Coding helped us to discover text patterns that are relevant to what makes methods effective in identifying security requirements and why. We have performed a qualitative analysis of the final reports. |