July 14, 2017 Minutes
Attendees: Wendy, Heather, Bhaskar, Paul
Apologies: Mike, Mitch, Jasmine, Eric
- For reference, here are the topics that we came up with during our discussion at CSS:
- Use of Data Visualizations for Data Anomaly Detection
- Data Quality
- Anomaly Detection
- Data Cleaning
- Metrics & Operational Performance
- Central Monitoring
- Signal Detection
- Registries & Rare Diseases
Improvements in the Tools for Reviewers:
Paul provided an update on the above. Most of the visualisations locally are fairly static and are run as a service by one group then passed to another. We are trying to find clinical trial sites for further inspection which is based on looking at a large (13) number of attributes. These are then ranked and scored. There are initial attempts to adapt tools like Jump using JumpClinical capabilities to diagnose some problems.
Still trying to understand what the analytic approach is.
There have been some issues vs CROs in the generic field. Generic trials aren't normally double blinded trials that we are used too in the new drug application world. FDA is trying to find a better system to detect inappropriate practices.
The best starting point for this group is to define what measures do contribute to data quality.
- At a site level
- Subject level
- Study level
Start to make a grid on each of these levels to look at the data interactively. What parameters are the key ones? What's the best way to visualise those?
KRI/KRA's (Key Risk Attributes/Indicators) Various tools are set up to look at these such as:
- Number of AE's
- Number of Serious AE's
- Non Serious AE's
Essentially data types at the study level. Various programs currently exist to work with this. JumpClinical as an e.g., has such a system that can help with the threshold values.
Look at the subject level data itself and do some analysis based on that. Then look at the interactive feature that could be displayed.
Is there a way that we can detect data anomalies without pre defining any measures that we are looking for? Is there a way of aggregating the data and then visualising that some method could be identified?
Benign data issues vs fraud. Areas such as date or decimal points can cause issues.
The user base most interested in this is during the study conduct aid.
- Data cleaning.
- Significant trend for a particular site or group of sites
Could we introduce some of these elements into one tool, is this possible?
Even the data that is supposed to be cleaned isn't always the case. EG, A drug based on a pre-existing one with 2 doses available. One of the medical reviewers noticed that the AE rate on the low dose was actually higher than the treatment emergent adverse event rate than the higher dose and queried what was going on. The CRO charged with performing the data analyses had reversed the codes for high/low dose. There was a quality control issue here. Its a type of data anomaly.
If there is any death during the study then it's often recorded in multiple data files. There have been some situations that different deaths are recorded in different files. There could be a death files, adverse event file etc. This involves doing consistent checks.
How often do we have protocol deviation? If the patient is supposed to have a follow up visit, how often do we see them coming at that interval + - 1 week.
Must have a base line to compare. Do we have the baseline done at the right time frame? Sometimes baseline done 6 months before so how can we identify those scenarios?
We can look to make recommendations in protocol deviations. In terms of implementing, do we want to say we are trying to create generic enough to run across studies and how do we fit in the missing pieces at protocol level. Would this be different protocol to protocol?
What extent can these can be generalised across Theraputic Areas? What would be most interesting? Identify the ones most generalisable.?
In dealing with this issue. When the form has been employed it's to count the number of protocol deviation. If you have a large site, that site will contribute disproportionally. So you can normalise by dividing by the number of subjects enrolled at that site. Which gives you a rough capita protocol deviation rate. If you use this as a value, if you have a high/low one then this may be a cause for further investigation.
ACTION: ALL Team should review the Transcelerate White Paper. Look at what the key things are to make progress.
ACTION: Mitch will do some research within his own company and see what he can share during next meeting.
Future Meetings: Confirmed bi-weekly meetings on Friday until October and then review again there.
ACTION: Link to Wiki Page that needs to be populated
Wendy to talk through the use of Teamwork during next call.