User Tools

Site Tools


erise

This is an old revision of the document!


Engineering of Risk and Security Requirement Challenge

The eRISE challenge is a series of empirical studies that aim to compare security engineering methods. Two editions of eRISE challenge has been held eRISE 2011 and eRISE 2012.The organization of the third edition eRISE 2013 is currently ongoing.

The idea of eRISE challenge is to bring together researchers, young students and practitioners who interact to understand if security methods are effective and what features determine their effectiveness.

With eRISE we want to be able to tell whether it is not a method to find security recommendations..it helps us to represent the model but does not help in finding solution or it helps to find out specific security requirement.

eRISE provides method designer with:

  • Empirical evaluation and Benchmarking of security engineering methods;
  • Knowledge of how and why participants intend to adopt a method;
  • Feedback to improve a security method by investigating strengths, weakness and limitations of the method.

eRISE aims to provide the participants with the benefit of:

  • Knowledge about various state-of-the art methods in the research field, on analyzing security risks and requirements of a system;
  • Opportunity to participate and work on an international collaborative project remotely.

Research Question

  • RQ1Are security requirements and risk methods effective when applied by someone different than their own inventor?
  • RQ2 Why are the methods effective? Why they are not?

Effectiveness means that a method assists the analyst to produce high quality security requirements with less time and less effort.

Research Approach

Since our research questions are exploratory in nature, we applied a mix-method experimental methodology combining both qualitative and quantitative data collection and analysis techniques. We evaluate methods' effectiveness based on the reports delivered by the participants, while we investigate the whys methods are effective by means of questionnaires, focus group interviews and post-it notes (RQ2).

Experimental Protocol

One of our goals is to investigate whether the methods under evaluation could be used effectively by users who have no prior knowledge of the methods. Therefore we have designed a protocol to conduct comparative empirical studies in this setting. The protocol consists of three main phases:

  • Training. First, participants are administered a questionnaire to collect information about their level of expertise in requirement engineering, security and on other methods they may know. Then, they are divided in groups where each group is composed of one master students and two professionals. The groups are assigned to a security requirements or a risk analysis method and to an industrial case to be analyzed using the method. The participants have to attend lectures about the method and on the industrial application scenario. At the end of the Training phase, the participants are administered a questionnaire to determine their level of understanding of the methods and of the industrial applications scenario.
  • Application. Participants work in groups and apply the method to analyze the application scenario.Group collaboration takes place both face-to-face and remotely by using multiple communication channels (e.g. mail, chat, video conferencing facilities). At the end of this phase, participants are involved in focus groups interviews, and they are requested to fill in post-it notes and a questionnaire about their impressions on the method. To document the application of the methods, the groups are audio-video recorded. In addition, groups have to deliver a final report.
  • Evaluation. On one side, participants assess the methods' effectiveness: they are involved in focus groups interviews, and they are requested to fill in post-it notes and a questionnaire about their impressions on the method. On the other side, method designers assess if the participants have a followed the method while customers, instead, evaluate if the groups have identified a set of security requirements or countermeasures that are specific for the application scenario, and if they are able to justify their results based on the method's application.

The eRISE Experimental Protocol involves five types of actors:

  1. Method Designer is the researcher who has proposed one of the method under evaluation. His main responsibility is to train participants in the method and to answer participants' questions during the Application phase. S/he also contributes to the assessment of the methods'effectiveness by analyzing groups' reports.
  2. Customer is an industrial partner who introduces the industrial application scenario to the participants. S/he also has to be available during the Application phase to answers all possible questions that participants may raise during analysis.
  3. Observer plays an important role during the Application phase because they supplement audio-video recording with information about the behavior of participants e.g (if the Participants work in group vs work alone) and the difficulties that they face during the application of the method. The observer also interviews the groups and leads the post-it notes sessions.
  4. Researcher takes care of the organization, sets the research questions, selects the participants, invites the method designers and the customers, and analyzes the data collected during the study.
  5. Participant is the most important role. Participants work in group and apply a method provided by one of the method designers to analyze the risk and security issues of the scenario provided by the customer.
erise.1365177865.txt.gz · Last modified: 2021/01/29 10:58 (external edit)