Exploring the use of a technology scouting methodology to integrate innovative solutions from startups into an aerospace industry

. Technological innovation is extremely important in the industrial world, as it allows companies to remain competitive and improve the efficiency, productivity, and sustainability of their activities. Companies that invest in technological innovation can obtain numerous advantages, including improved product quality, cost reduction, greater flexibility, and the ability to quickly respond to customer needs. An approach to open innovation that has become widespread in recent years is to conduct technology scouting through the vast array of solutions provided by innovative startups. With a large number of startups proposing new technologies in aerospace sector, could be challenging for companies to identify the most promising solutions. Therefore, a structured methodology for evaluating technology proposed by startups is essential to ensure the identification and implementation of the best solutions and the effective allocation of resources. This paper presents a case study that describes the process of technology scouting proposed by startups within an aerospace industry company, based on a company-defined roadmap.


Introduction
The aerospace industry is constantly pushing the boundaries of technology, and startups have a significant role to play in this process.That is why it is important for aerospace companies to conduct scouting of technology proposed by startups, in order to stay at the forefront of innovation.During each "innovation call," regardless of the proposed technology, there are many candidate startups, and the need to evaluate and choose the one that best aligns with the company's trend is very important.The purpose of this paper is to present a case study in which a technological scouting methodology has been applied.
A structured methodology for evaluating technology proposed by startups allows for the continuous improvement of the evaluation process and builds trust and confidence among all stakeholders.
The entire scouting process is based on a company-defined roadmap.The steps begin when the company defines the problem in which it is interested in scouting, and continue until the selection of the best alternative among those presented.

2.
Scouting Process: Roadmap The industry scouting process for the selection of a technology proposed by a startup is a multistep process that involves several phases.It involves a combination of factors, such as the alignment of the technology with the company's business needs, the potential impact of the technology, and the startup's ability to execute on their plans.
The entire scouting process can be imagined as a set of activities to be followed in succession.[1] The roadmap on which it develops is shown in Fig 1 .It is characterized by two nested processes: the scouting process and the technology evaluation methodology process.The two processes are complementary: -The first process (colored in red) concerns all activities carried out internally by the company or potentially by startup acceleration companies and relates to the selection, screening, and finally management of collaboration contracts with the chosen alternatives.-The second process (colored in light blue) is a more complex and articulated process.It includes step-by-step evaluation activities of the technology proposed by the startup.-The following paragraphs, describe detail the different steps that make up the roadmap.
Each of these phases has been implemented according to the pattern shown in the Fig 1.

2.1
Problem Definition Problem definition is the process of clearly identifying and understanding the problem or need that technology is intended to address, Fig 2.

Understanding and defining business needs & Priority processing
Structuring a collection of business needs is important for innovation scouting because it helps to identify the specific areas where new technology is needed and where it can have the most impact.By having a clear understanding of business needs, companies can better focus their innovation scouting efforts on areas that are greatest relevant to their operations and where they can add the most value.This allows for a more efficient use of resources and a higher likelihood of successfully identifying and implementing new technology that can help the company achieve its goals.
At the corporate level, the relevant business unit has prepared a "needs sheet" which is a collection of all the requirements.It highlights the reference problem, the possible directions of solutions, and the main elements that should characterize the technological solution proposed by the startup.
Following the receipt of this document, the open innovation pilot team takes care of elaborating the data, defining reference keywords of the chosen topic, organizing the scouting mode, and structuring various support tools for the entire process.

2.2
Problem Structuring Problem structuring is the second phase of the process, in which the problem is organized and categorized in a way that allows a comprehensive analysis (see Fig 3).
During this phase, with the support of the Business Unit related to the topic selected during the Problem Structuring phase, a panel of experts has been structured, to participate in the evaluation of the startup candidates.The selection of the panel was done carefully, as the final selections of the scouting process will depend on it.In this case, the expert panel for the evaluation of startup candidates has been purposely formed in a multidisciplinary way in order to obtain a broad evaluation of the candidate companies.
In this project the panel is made up of an expert in the topic from the relevant corporate Business Unit, a full professor from the Politecnico di Torino, part of the department of management and production engineering, a Ph.D student with sector expertise, and two professionals from the Open Innovation Team who manage the scouting project.

2.3
Parameters & Objects Definition This phase of the roadmap is the process of defining key parameters and objectives that will be used to evaluate the proposed technology, (see Fig 4).
Before going into the details of this phase, it is important to specify its structure.At the company level, regardless of the scouting process, which we will refer to as Dealflow from now on, it was decided to evaluate the candidate startups based on four macro areas which we will define as "parameters".Specifically, the evaluation parameters chosen by the management to assess the suitability objective are Team, Finance, Community and Technology.
Regarding the first three parameters, the company decided to keep the defining criteria for each group unchanged for every Dealflow.The only parameter subject to change for each Dealflow is that related to technology.This parameter has different needs based on the type, field of application, difficulty of implementation, and structure.
In this phase, the panel of experts is tasked with deciding the weight to give to each parameter and its associated criteria (Team, Finance, Community, and Technology).The panel of experts will also define the criteria and objectives that will be used to evaluate the proposed technology.This step is critical in determining the technology's value proposition and potential for success.

Definition of Criteria and Scores
This stage of the process is structured into the following subphases: • Definition of parameters weight s and consistency verification The first step is to assign a weight to each parameter in the evaluation areas.Through a brainstorming technique, the panel of experts expressed the importance ratios of each parameter relative to the others.
Subsequently, the consistency of the weights was verified through pairwise comparison using the Analytic Hierarchy Process (AHP) technique, followed by the eigenvector method [2,3,4].The result, shown in Figure 5, indicates that the evaluators expressed a strong preference for the parameter related to Technology, which was assigned a weight of 62%, while the parameters of Team, Finance, and Community were assigned final levels of importance of 18%, 10%, and 10%, respectively, Fig 5.
• Criteria associated with the parameters Team, Finance, and Community As previously mentioned, the criteria associated with the three parameters related to the areas of Team, Finance, and Community are unchanged for each Dealflow, Fig 6.
• Definition of Technology Parameter Criteria: The crucial part of this phase of the process is to define the criteria for evaluating the different technologies proposed by the startups.This phase of the process has been structured in a mixed Delphi-NTG-Brainstorming approach.All these techniques were leveraged to achieve the methodology with the best outcome [5].
• Delphi: Sharing the topic with the panel of experts The Delphi technique is a method used to gather and process information from a group of experts in order to reach a consensus on a specific issue [6].An invitation to a meeting has been sent to the panel of experts, attaching the topic of the meeting and highlighting the main objective of the scouting process in question.At this point, the experts become more aware of the topic and begin to have an idea of what the truly important aspects are for the company and what the main objectives are to achieve.
• Nominal technique group NTG -Presentation of solutions The nominal group technique is a method used to collect and prioritize information and ideas from a group of people.In the case study under consideration, the respondents are the 5 people who make up the previously defined panel of experts.This technique can be used in the selection of criteria for evaluating technology proposed by startups.The previously defined panel of experts generates a list of suitable criteria for evaluating the performance of startups.The group worked together to prioritize the criteria in order to assess the technology proposed by startups effectively.• Brainstorming -Final Selection of Suitable Criteria In this phase of the process, once N criteria have been defined, the panel of experts is called to come together to refine the choices made in order to obtain a smaller number of criteria, which will make it easier to start the evaluation process in a more streamlined way.The experts of the scouting process chose to use a brainstorming technique.
Once all the ideas have been generated, the group then goes through each one, grouping similar ideas together and eliminating duplicates.From there, the group may use a voting process or other criteria to prioritize the ideas and select the final set for the decision-making process.In this case, the criteria deemed most suitable for the scouting case in question for the evaluation of technology in the digital twin field were the following in Table 1.Each evaluator was asked to rank the criteria they deemed most appropriate by assigning a score from 1 to 5 (with 1 being less important and 5 being more important).The sum of votes for each criterion was then calculated horizontally.As shown in Table 1, three macro-groups of criteria can be identified.The first is characterized by all those criteria that did not receive any scores.This was a result of the ideation process that led many evaluators to reconsider their ideas and prefer others.The second macro-group consists of criteria that received low scores such as cost efficiency, go-to-market strategy, customer traction, and intellectual property.Finally, there are criteria characterized by a higher score that have emerged as the object of greater interest by unanimity.The latter, highlighted in gray in the table, are Accuracy, Technological Advancements, Real-Time Capability, Data Management, and Risk Evaluation.The final choice therefore fell on 5 out of the total 17 criteria previously proposed.
• AHP process After the selection, the weight was calculated again through pairwise comparison using the AHP technique [2,3,4].With the help of the panel of experts, the levels of importance necessary to complete the pairwise comparison matrix were unanimously decided by the evaluators through brainstorming.In order to verify the consistency of the weights assigned to these criteria, the Consistency Index was calculated using the eigenvector technique.This procedure resulted in an acceptable consistency among the parameters, with the CI value of 0.022 being lower than 10%, Fig 7. Through various search engines, Startups with technologies deemed of interest for the examined business problem were selected.

Table 1 Final Selection of Criteria
In the case at hand, Dealflow, led to the identification of 20 startups with technology that has good potential for the company.

2.5
Acquisition of additional Data In this phase of the process, the previously selected companies must be evaluated based on their complementarity with the company's needs, Fig 9 .After storing the list of startups related to the scouting flow and their preliminary information in a database using a management tool, a report containing the profile sheets of each startup is generated.This reference report is then sent to a panel of experts for a preliminary evaluation.The aim of this initial assessment is to exclude those companies that are definitely not suitable for the scouting call and that, according to the experts, may not have the required skills.For the startups that are deemed valid at first glance, further information will be acquired to deepen the scouting process.This type of evaluation is mainly carried out by the direct stakeholders of the relevant business unit who evaluate the technological aspect of the startup and its adaptability to the company's core business.The first screening by the expert panel led to a screening of approximately 70% of the candidates.The scouting process continued with 7 candidates.

2.6
Experience Verification The "Experience Verification" phase of the roadmap is the process of verifying the startup's experience and track record in the field, Fig. 10.
Company's open innovation team prepared several questionnaires to be submitted to selected startups with questions aimed at deepening the topics of interest.If the preliminary evaluation, supported by the initial information, is positive, after having further examined the characteristics of the companies, they can proceed with the first individual meeting with the panel of experts.Otherwise, they will be excluded from the final selection but will still remain within the company database for future reconsideration.The experience verification phase aims to understand the actual technology of the company, attend presentations by the candidates, delve into the topic of technology, the startup's mission, and the organizational and work method.In the case study, a Pitch Day was set up, during which each startup presented their activities, objectives, and proposed technology.The set of meetings was held entirely within a single day in which the company attended, in succession, the 7 individual startup pitches, each lasting 30 minutes, followed by 5 minutes in which the panel of experts expressed their evaluation.To do this, the individuals managing the scouting process prepared a questionnaire to send to the panel of experts so that at the end of each pitch, they could give their evaluation of the startup in a simple and immediate way.This questionnaire was produced using an Online Module to facilitate the creation, sending, and collection of results and is divided into four modules.It is divided into five sections and aims to collect scores on a Likert scale ranging from 1 to 7 for each previously defined criterion [7].The Likert scale range captures the intensity of individuals' perception for the different proposed startups.In the present case, and following a careful literature analysis, it was decided to use a 7-point Likert scale of preference [8].The Online Module organization has provided for the presence of 5 sections related to the selection of evaluation parameters: Startup header, Technology, Team, Community, Financial.

Startup Ranking
The "Startup Ranking" phase of the roadmap is the process of ranking the startups based on the criteria/parameters established in the previous steps.This step is important as it is complex at the same time.Many approaches are possible, one of the possible ones has been chosen here to determine which technologies and startups warrant further investment and support (see Fig 11).Automatically, the previous Form, presented a final result for each criteria, given by the average of the scores obtained by the 7 components of the expert panel.The results obtained from the Likert scale were analyzed as a whole and weighted again according to the relative importance of the parameters in question.The final calculation shown in the table resulted in a ranking of suitability for each startup, taking into account all previously analyzed

Conclusions
The use of a technology scouting methodology to integrate innovative solutions from startups into the aerospace industry is crucial to remain competitive and improve the efficiency, productivity, and sustainability of the company's activities.Through the case study presented, we discussed a structured methodology for evaluating technology proposed by startups.By continuously improving the evaluation processes, companies can build trust and confidence among all stakeholders, and ultimately achieve success in their innovation efforts.
To date, while the proposed technology scouting methodology has shown promising results, it is important to acknowledge its limitations.These limitations are mainly due to the assumptions made during the development of the methodology, such as the use of specific aggregation methods and the Analytic Hierarchy Process (AHP) for evaluation.Future work should focus on deepening the understanding of these limitations and exploring alternatives approaches.

Figure 3
Figure 3 Problem Structuring

Figure 5 Figure 6
Figure 5 Definition of parameters weights

Figure 7
Figure 7 AHP Process

Figure 8
Figure 8 Interest Exploration

Figure
Figure 10 Experience Verification

Figure 11
Figure 11 Startup Ranking