Differences

This shows you the differences between two versions of the page.

--- erise [2013/04/04 12:35]
federica.paci@unitn.it [Research Question]
+++ erise [2021/01/29 10:58] (current)
@@ Line 1: / Line 1: @@
-===== Engineering of Risk and Security Requirement Challenge ====
+===== eRISE Challenge ====
-The eRISE challenge is a series of empirical studies that aim to compare security engineering methods. Two editions of eRISE challenge has been held in 2011 [[eRISE 2011]] and 2012 [[eRISE 2012]]
+The eRISE (engineering RIsks and SEcurity Requirements) challenge is a series of empirical studies that aim to compare security engineering methods sponsored by [[http://www.nessos-project.eu|NESSoS]] European Project and [[http://www.eitictlabs.eu|EIT ICT Labs]]. Three editions of eRISE challenge has been held [[eRISE 2011]], [[eRISE 2012]], and [[eRISE 2013]]. See the [[validation_of_risk_and_security_requirements_methodologies|main page]] for our work on empirical validation of security risk assessment methods and other experiments.
-The organization of the third edition [[eRISE 2013]] is currently ongoing.
+The idea of eRISE challenge is to bring together researchers, young students and practitioners to understand if security methods are effective and what features determine their effectiveness.
+With eRISE we want to be able to tell whether "//it is not a method to find security recommendations..//", or at least "//it helps us to represent the model but does not help in finding solution//", or hopefully "//it helps to find out specific security requirement//." (quoting some of the participants of our experiments).
 **eRISE provides  method designer with**:
-  * Empirical evaluation and Benchmarking of security engineering methods;
+  * Empirical evaluation and benchmarking of security engineering methods;
-  * Knowledge of how and why participants intend to adopt a method;
+  * Understand if the proposed method works in practice
-  * Feedback to improve a security method by investigating strengths, weakness and   limitations of the   method.
+  * Feedback to improve a security method by investigating its strengths, weakness and limitations.
 **eRISE aims to provide the participants with the benefit of:**
   * Knowledge about various state-of-the art methods in the research field, on analyzing security risks  and requirements of a system;
-   * Opportunity to participate and work on an international collaborative project remotely.
+   * Opportunity to participate and work in international collaborative project.
+==== Research Questions ====
+  * **RQ1** //Are security requirements and risk methods effective when applied by someone different than their own inventor?//
+  * **RQ2** //Why are the methods effective? Why they are not?//
+A method is //effective// when it assists the analyst to produce high quality security requirements with less time and less effort.
+==== Research Approach ====
+Since our research questions are exploratory in nature, we applied a **mix-method** experimental methodology combining both qualitative and quantitative data collection and analysis techniques.  We evaluate methods' //effectiveness// based on the reports delivered by the participants, while we investigate the //why//s methods are effective by means of questionnaires, focus group interviews and post-it notes (RQ2).
+==== Experimental Protocol ====
+One of our goals is to investigate whether the methods under evaluation could be used
+effectively by users who have no prior knowledge of the methods. Therefore we have designed a protocol
+to conduct comparative empirical studies in this setting. The protocol consists of three main phases:
+  * **Training**. First, participants are administered a questionnaire to collect information about their level of expertise in requirement engineering, security and on other methods they may know. Then, they are divided in groups where each group is composed of one master students and two professionals. The groups are assigned to a security requirements or a risk analysis method and to an industrial case to be analyzed using the method. The participants have to attend lectures about the method and on the industrial application scenario. At the end of the Training phase, the  participants are administered a questionnaire to determine their level of understanding of the methods and of the industrial applications scenario.
+  * **Application**. Participants work in groups and apply the method to analyze the application scenario.Group collaboration takes place both face-to-face and remotely by using multiple communication channels (e.g. mail, chat, video conferencing facilities). At the end of this phase, participants are involved in focus groups interviews, and they are requested to fill in post-it notes and a questionnaire about their impressions on the method. To document the application of the methods, the groups are audio-video recorded. In addition, groups have to deliver a final report.
-==== Research Question ====
+  * **Evaluation**.  On one side, participants assess the methods' effectiveness: they are involved in focus groups interviews, and they are requested to fill in post-it notes and a questionnaire about their impressions on the method. On the other side, method designers assess if the participants have a followed the method while customers, instead, evaluate if the groups have identified a set of security requirements or countermeasures that are specific for the application scenario, and if they are able to justify their results based on the method's application.
-===== Research Approach ====
-===== Experimental Protocol ====
+The eRISE Experimental Protocol involves five types of actors:
+  - **Method Designer** is the researcher who has proposed one of the method under evaluation. His main responsibility is to train participants in the method and to answer participants' questions during the Application phase. S/he also contributes to the assessment of the methods'effectiveness by analyzing groups' reports.
+  - **Customer** is an industrial partner who introduces the industrial application scenario to the participants. S/he also has to be available during the Application phase to answers all possible questions that participants may raise during analysis.
+  - **Observer** plays an important role during the Application phase because they supplement audio-video recording with information about the behavior of participants e.g (if the Participants work in group vs work alone) and the difficulties that they face during the application of the method. The observer also interviews the groups and leads the post-it notes sessions.
+  - **Researcher** takes care of the organization, sets the research questions, selects the participants, invites the method designers and the customers, and analyzes the data collected during the study.
+  - **Participant** is the most important role. Participants work in group and apply a method provided by one of the method designers to analyze the risk and security issues of the scenario provided by the customer.

DISI Security Research Group Wiki

User Tools

Site Tools

Differences

Page Tools