Deliverable 5.1 proposes a holistic validation approach to be applied to the testing of the RAYUELA serious game according to the landscape specifications compiled in WP1, WP2 and WP3. The validation plan in this document also defines the iterative validation and refinement approach for the initial prototypes of the serious game. It aims to serve as a practical guide of action for the whole period of validation on the field. It defines the timeline of each piloting activities along with the mechanisms that will be applied to monitor and guide users to obtain the specific objectives of each one. 

RAYUELA validation approach builds on a set of concrete methodologies, presenting a complete assessment approach of the project’s serious game and attempting to offer a set of useful insights into user behaviour and knowledge in relation to aspects of cybercrime as experienced through the RAYUELA game: 

• A systematic usability survey, consisting of an initial usability study, will be followed by implementing a Technology Acceptance approach. 
• A pre- and post-self-Reflection survey on aspects of knowledge on cybercrimes such as Cyberbullying, Online Grooming, Human Trafficking, Misinformation -Deception and Technological Threats. 
• An ethnographic validation aiming to observe children during playing the game, and 
• An evaluation of aspects of in-game player decisions and behaviours, based on the study of the anonymous log files created by the system. These data will be analysed as part of Task 6.4. Interpretation of collected data on the impact of human factors on young cybercrime victims and offenders. 

This document offers the detailed piloting plan, followed by a short presentation of the cybercrimes to be addressed and 5 (out of almost 50) possible scenarios. Furthermore, a presentation of the validation methods and an explanation of the questionnaires designed is followed by entries on Ethics, with an emphasis on anonymisation and GDPR, and the preliminary Key Performance Indicators (KPIs). All the proposed instruments are presented as annexes at the end of this document. To be more precise, the deliverable is organized as follows. 

Chapter 2 describes the landscape of the pilot implementation. Data to populate and feed RAYUELA’s serious game will be gathered using the pilot actions envisioned, which will maximize psychological and sociological impact compared to traditional approaches, by reaching a significant number of users in a friendly and non-invasive manner. 

Chapter 3 describes the context of implementation that is defined by the key areas of cybercrime and the related scenarios that will be tested in pilot sites. A number of indicative scenarios are presented for the reader to have a holistic overview of the proposed intervention. 

Chapter 4 describes the Validation Framework of the project. Validation process in RAYUELA will include three main phases of activities in the piloting countries (Spain, Greece, Belgium, Estonia and UK’s Northern Ireland), focusing progressively on a) the technical usability and acceptance, b) the potential of RAYUELA’s serious game as a powerful educational tool and c) raising awareness and offer opportunities for users to enhance awareness and knowledge on cybersafety and motivating them to 

learn more, in order to acquire cybersafety routines in their everyday lives. The framework for the provision of training and support to teachers is also discussed. 

Chapter 5presents the usability survey that has been designed for the RAYUELA pilots, consisting of an initial usability study, to be followed by the implementation of a Technology Acceptance approach. The instruments that will be used are described and presented in detail. 

Chapter 6focuses on the ethnographic evaluation in the form of participant observation. The aim here is to explore various behavioural perspectives by studying the target population, namely minors, during their engagement with the RAYUELA gaming solution. 

Chapter 7 describes the methodology that will be used to assess the educational value of the RAYUELA game. The proposed study will focus on cognitive and affective outcomes. The study will focus on highlighting the impact of the RAYUELA game to raise students’ awareness on cybercrime, to increase their intrinsic motivation to get involved in related tasks and to assess the game’s learning effectiveness. 

Chapter 8 highlights that all piloting activities must be ethically compliant and be carried out in compliance with the rules of the GDPR. The Ethical considerations are very important for the success of the pilots and the delivery of quality outcomes. 

Chapter 9 presents the KPIs related with the pilots and the data acquisition in the framework of the project. These indicators will govern the overall validation plan and the related activities. 

Finally, Chapter 10 concludes the deliverable highlighting that the core evaluation endeavour of the RAYUELA project is to collect meaningful feedback from our target groups on the contribution of the RAYUELA serious game in the fight against cybercrime in minors.