The Sec4AI4Sec project focuses on the security of systems implemented with artificial intelligence (AI) and addresses the challenges introduced by the wide deployment of AI, involving both the presence of intelligent components within implemented systems (Sec4AI) and the use of AI by DevOps to support secure software development (AI4Sec).

Figure 1.a: AI-augmented systems have a larger attack surface because they include many different assets (in red). Our project objectives (in green) have the ambition to cover these assets holistically.

Sec4AI4Sec is aware of the two realities in which artificial intelligence (AI) manifests itself-as shown in Figure 1.a : Sec4AI, where AI is part of deployed systems and has new vulnerabilities and problematic behaviors, and AI4Sec, where AI is used by DevOps but has reliability issues. The project will address these facets of AI to achieve deep scientific, economic and technological impact, while also seeking to address crucial societal issues. The project will focus on three key scenarios of the EU Digital Compass: virtualization of the 5G core, autonomy for safety systems in aviation and security, and quality for third-party software evaluation and certification. In addition, the project addresses the lack of real-world examples for the effective application of data-driven code analysis methods. Repository mining tools will be developed to extract realistic code samples used for learning.

Figure 1.b: Technical challenges of Sec4AI4Sec.

List of partecipants: Sec4AI4Sec has brought together a team with 5 leading universities (Amsterdam, Cagliari, Hamburg, Lugano, Trento), 2 innovative SMEs (FrontEndART, Pluribus One), 3 large companies (Airbus, SAP, Thales), and 1 center for digital innovation (Cefriel). The project will generate a set of innovative techniques and open-source tools, new methodologies for the secure design and certification of AIaugmented systems, and benchmarks that can be used to standardize the evaluation of research results in the secure software research community. Coordinator | 1 University of Trento UNITN IT Partner | 2 SAP S.E. SAP DE Partner | 3 Technische Universität Hamburg TUHH DE Partner | 4 Airbus Operations GmbH AIRBUS DE Partner | 5 University of Cagliari UNICA IT Partner | 6 Thales SIX GTS France THALES FR Partner | 7 FrontEndART Szoftver Kft. FEA HU Partner | 8 Pluribus One PLURIBUS IT Partner | 9 Cefriel Società Consortile a Responsabilità Limitata CEF IT Partner |10 Vrije Universiteit Amsterdam VUA NL Partner | 11 Oppida SAS OPPIDA FR Associated partner |12 Università della Svizzera Italiana USI CH The University of Trento (UNITN) will play the following key roles in the Sec4AI4Sec project: Project Management (WP1): UNITN will handle administrative responsibilities and coordination of the financial and technical aspects of the project, ensuring the effective completion of activities according to Horizon Europe rules. Communication, Dissemination and Exploitation (WP2): UNITN will investigate and pursue opportunities to make WP3 and WP5 results available to the community, helping to communicate them regularly through the website and social media. Code and Package Evaluation (WP3): UNITN will support experimentation and data mining from repository mining, studying how to hybridize existing Machine Learning-based approaches for vulnerability detection, and conducting experiments with human testers to evaluate vulnerabilities reported by SOTA and the ML-based hybridized method. Robustness testing of artificial intelligence (Trustworthy AI) models (WP4): UNITN will support the definition of AI/ML vulnerability taxonomy and provide support for testing AI testing methodologies. Generation of code fixes (WP5): UNITN will contribute to building the vulnerability dataset, comparing state-of-the-art tools with AI-based tools for automatic vulnerability repair, and conducting experiments with students. Pilots (WP7): UNITN will participate in the pilot design, providing WP4 results to support the pilots and WP3 and WP5 results for the pilot. The project aims to achieve significant scientific, economic and technological impact and will provide new methodologies, open-source tools and benchmarks to standardize the evaluation of results in the secure software research community.