The first blog post of 2021 was about the topic of “DiGA” and data collection. In this article we would like to go into the preparation task in more detail and that is why this time it is about the evaluation concept. This must also be submitted together with the test plan for the DiGA study with the application for inclusion in the reimbursement list. This blog post explains what it is all about, how it is best created and everything that needs to be taken into account.

Abbreviations

BOB (higher federal authority)

BtB (Business to Business)

BtC (Business to Customer)

DiGA (digital health application)

MDR (medical device regulation; EU regulation 2017/745)

Underlying regulations

Digital Care Act (DVG)
Digital Health Applications Ordinance (DiGAV)
DiGA Guide
EU Regulation 2017/745 (MDR)
ISO 14155

1 Introduction

In order to be included as a DiGA in the reimbursement directory (DiGA directory), various requirements must be met and the review process at the BfArM must be successfully completed. This includes, among other things, if a study that meets the DiGA criteria has not yet been carried out, an evaluation concept and a clinical study based on it. January blog post important information about the DiGA study

This article deals with the evaluation concept for the positive care effect of the DiGA. The DiGA guidelines go into this in more detail in Chapter 4.5.2:

„With the application, the manufacturer also submits an evaluation concept drawn up in accordance with generally recognized scientific standards that appropriately takes into account the results of the systematic data evaluation. The study protocol of the intended study should be part of the evaluation concept. The choice of outcomes and study design of the selected comparison and the reality of care must be justified. It must be explained why and how the evidence of the desired pVE emerges from the selected evaluation concept. This must have been created by a manufacturer-independent scientific institute .“

“The scientific evaluation concept to be presented should appropriately take into account the results of the systematic data evaluation in accordance with Section 15 DiGAV.”

Excerpt from: Brönneke, Jan B. “DiGA VADEMECUM: What you need to know about digital health applications (German Edition).

The manufacturer does not have to create this himself because the law requires it to be created in the guidelines but also in the DVG by an independent scientific institute. Nevertheless, it contributes a significant part to the creation, because the study concept, including the endpoints to be proven for the positive care effect, require in-depth study of this topic. This article would like to explain in more detail what this means. At the same time, it is shown how the evaluation concept can be used for a DiGA (software as a medical product) that has been on the market for a long time or for a e.g. B. a DiGA that is currently in the development process or one that has just been approved under the MDD can be created.

2. DiGA evaluation concept

In the DiGA regulations, the evaluation concept is defined as follows:

„If an application for testing is to be submitted, it must be accompanied by a scientific evaluation concept. This must be prepared by a manufacturer-independent institution to demonstrate the positive care effect according to generally recognized scientific standards.”

(Source: DiGA Guide )

This includes in particular the following information on the planned study project in order to demonstrate the positive care effect of the DiGA:

• an indication of the testing period (maximum 12 months)
• systematic data evaluation with the DiGA itself
• Description using the PICO scheme in the short version of the positive supply effect
• Specifying the patient group by specifying the corresponding ICD codes
• Type of positive care effects of DiGA: medical benefits and/or patient-relevant procedural and structural improvements
• Information on research design and results
• Information on the quality-assured application of the DiGA and exclusion criteria
• Information about the scientific and manufacturer-independent institute involved

For example, we structure our evaluation concept as follows:

Fig. 1: Content and structure of the evaluation concept

An essential part of the scientific evaluation concept is systematic data evaluation as part of the application of the DiGA. Therefore, this data evaluation must be carried out with the approved medical product. This is not a problem if DiGA is already on the market and data has already been collected through its application. However, for products that are still in development or have just been approved, there may not yet be any data available for a corresponding evaluation. Such a survey phase must therefore be planned following approval before an application for provisional inclusion in the register can be submitted.

Regardless of this, the manufacturer must think about the planned supply path to be followed with the DiGA, for which the positive supply effect is to be proven. As part of the data collection for the evaluation concept, various endpoints should already be established that can be checked for validity in this context.

The aim and purpose of the data evaluation should be to define the endpoints of the DiGA study, which can be used to demonstrate the positive care effect on the intended care path. It therefore makes sense to have a selection option from the areas

• medical benefits
• patient-relevant process and structural improvements

But how should this data be collected so that it can then be evaluated for the evaluation concept?

2.1 Already approved DiGA

Many of the DiGAs already listed are approved medical devices that were already on the market. This means that it is possible to use a study that has already been carried out and that meets the DiGA criteria in order to be immediately and definitively included in the directory. In this case, we strongly recommend that you seek advice from the BfArM. Especially when there is uncertainty as to whether the data collected is sufficient and it is therefore unclear whether it is going in the right direction and whether an application should be submitted for provisional or final inclusion.

If no study has yet been carried out, in this case it is advisable to retrospectively evaluate data that was collected as part of the application with the DiGA and is available from the manufacturer via the app.

Note: This is possible without any problems if there is a BtC relationship between the manufacturer and the user. Doesn't this exist because the DiGA, for example, is not directly from the manufacturer, but e.g. B. is provided by therapists (with whom there is a BtB relationship with the manufacturer), the data is not held by the manufacturer.

If the manufacturer has access to the data that is automatically collected by DiGA as part of the application, it can be evaluated anonymously. In this case, this is done via a so-called observation plan; the collection refers to “real world data” and since it is completely anonymous, it can be collected accordingly without involving an ethics committee or authority. The observation plan defines the parameters to be collected, which relate to the above-mentioned aspects

• medical benefits
• patient-relevant process and structural improvements

should refer. Our March article “Medical products and real world data as well as real world evidence” is also recommended for real world data (link: https://www.medxteam.de/index.php/medxteam-blog/15-medizinprodukte-und-real-world -data and real-world evidence).

2.2 DiGAs that have not yet been approved or have just been approved

No personal data may have been collected using these products. In this case, the clinical assessment is usually also carried out using performance data (see also Article 61 Section 1 of the MDR), since clinical data can be dispensed with for these Class I or IIa products in order to meet the basic safety and performance requirements .

This means that no approval studies are usually carried out. Guidelines for state-of-the-art chapters in clinical assessment are usually only available for the underlying indications and alternative applications, as DiGAs are more innovative in nature and are not yet comprehensively anchored in guidelines.

However, for the clinical evaluation, claims for clinical performance, safety and clinical benefit must already be defined in the clinical evaluation plan and then supported with data in the clinical evaluation report.

Note: This is exactly where it is recommended to use the interface on the DiGA topic of “medical benefits” or patient-relevant structural and procedural improvements, because this data can then be used to update the clinical assessment after the DiGA study.

In addition, the DiGA process does not end with the listing in the DiGA directory. Negotiations with the health insurance companies then begin.

to include keywords from medical benefits or patient-relevant structural and process improvements, if possible, when defining the intended purpose of the medical product and in the claims This makes later negotiations easier. It’s also best not to talk about “software” but rather about digital health applications.

Therefore, once the medical device has been approved, data collection for the evaluation concept must be planned.

Here too, it is advisable to collect the data collected with the DiGA as part of its use by the manufacturer. This survey is not retrospective, but prospective into the future.

However, the data can also be evaluated retrospectively after a defined period of application of the DiGA.

Here too, it is important that the data

• with the DiGA itself and
• anonymous

be collected. This also works best in the BtC case. But even in the BtB case, if the manufacturer does not have direct access to the app data, an observation study and collection of real world data can be carried out. Essentially, all that needs to be ensured here is anonymized provision in the BtB ratio.

2.3 Summary

A key aspect of the evaluation concept is data on the care path taken and the necessary evidence of this

• medical benefit
• or patient-relevant procedural and structural improvements.

Parameters are therefore defined in advance that should be evaluated accordingly after data collection. These should come from the following areas:

Medical Benefits:

• improving health status,
• shortening the duration of illness,
• Prolonging survival or
• Improving the quality of life

Patient-relevant structural and procedural improvements:

• in the context of the detection, monitoring, treatment or mitigation of diseases or the detection, treatment, mitigation or compensation of injuries or disabilities and
• aimed at supporting the health care activities of patients or integrating the processes between patients and service providers and
• include in particular the areas of

1. coordination of treatment processes,
2. Alignment of treatment with guidelines and recognized standards,
3. adherence,
4. Facilitating access to care,
5. patient safety,
6. health literacy,
7. patient sovereignty,
8. Coping with illness-related difficulties in everyday life or
9. Reduction of therapy-related expenses and burdens on patients and their relatives.

3. What we can do for you

We act as a scientific, manufacturer-independent institution (CRO). As such, we create your evaluation concept and advise you in an interface-compliant manner in your early development phase with regard to your claims for the medical product or determination of the intended purpose. All of this is done with DiGA requirements in mind, so you can kill two birds with one stone.

If your technical documentation is already available, we look at possible DiGA endpoints either based on your documentation or based on the care path subsequently taken with the DiGA and make a sensible preliminary selection so that the data analysis for your evaluation concept is targeted and does not get out of hand. After all, we want clarity and not to fish in clouds.

4. How we can help you

At medXteam we clarify whether and if so which clinical trial needs to be carried out under what conditions and according to what requirements during the pre-study phase: In 3 steps we determine the correct and cost-effective strategy in relation to the clinical trial required in your case Data collection. This also applies to your evaluation concept and your DiGA study!

Do you already have some initial questions?

You can get a free initial consultation here: free initial consultation