Evaluation Of Alternative Transport Formats

From PhUSE Wiki
Jump to: navigation, search

Project Title

Evaluate and Prototype Alternative Transport Formats for Submission to support both Structure and Content Requirements.

Status

White paper is complete and reviewed. This will be made available via the usual PhUSE publication channels.

Project Team Members

Name Role Contact
Geoff Low Co-lead geoff.low (a) phuse.eu
Wendy Dobson PhUSE wendy@phuse.eu

Affected Stakeholders

We have identified the following Stakeholders:

  • Industry users preparing data for submission to regulatory agencies
  • Users within regulatory agencies receiving and reviewing data
  • Technology vendors
  • Anyone with a need to transport structured clinical data

Project Meeting Frequency

  • Not Applicable

Links

Definition(s) of Problem/Issue/Challenge

For decades, the submission format for clinical data has been the SAS v5 transport format and just recently those formats have contained CDISC SDTM and ADaM datasets. In 2012, the FDA held a public meeting to discuss the challenges and gaps in the SAS v5 transport format that limited their ability to review the data. The challenges identified included a combination of technical issues (e.g. limitations in variables and label lengths) and content issues (e.g. inability to define relationships without external metadata).

The FDA initiated a Pilot project in 2013 to evaluate CDISC Dataset-XML as an alternative transport format. They conducted this pilot over a year and summarized their findings. The FDA concluded that the DS-XML format helped to address basic technical challenges; however, the size of the data was still an issue. While content issues were not addressed in the scope of the pilot, follow up discussions have indicated that content gaps are also part of the challenge.

Potential Technological Solution(s)

In order to evaluate the proposed alternatives we are going to define a set of criteria against which the existing and proposed transport formats can be evaluated against. While this is not the use cases we would have desired we think it should function as a useful proxy. These are being developed in a series of meetings and will be released for public comment at the completion of the internal proposal.

We will release the set of criteria, along with clear definitions thereof as part of a survey allowing the industry to rank the criteria. Using ranked criteria, we can subjectively evaluate any suggested replacements; but with the subjective view reflecting the priorities of the wider industry.

Other Initiatives that can be leveraged/merged

Recommended Solution(s)

In evaluating data transfers of the future we must consider two aspects:

  • how data is transferred
  • what data is being transferred

In the past the thrust of the evaluation has been to focus on the former, but we must also begin to consider the latter. One of the chief (not related to field widths and formats) concerns is the ability to draw relationships between elements (such as the link between an adverse event and related concomitant medications) - changing the file format is not necessarily enough to suit the use cases of today and the future.

Methodology for Approach

In the absence of formal use cases for replacement of the XPT format we have chosen to focus on building a set of criteria that can be used to objectively (as much as possible) assess the new candidates. The finalised set of criteria will be sent out to people on the PhUSE and CDISC mailing lists, to allow participants to rank them. In doing so we will have a ranked set of criteria that we can use to assess according to the wants/needs of the industry.

Supporting Materials

The criteria for assessment for the transport formats are reported on the following page.

Evaluation Criteria for Transport Formats

Specific Actions

Phase I: Evaluate Alternative Transport Formats to support structure and content

Phase I will explore alternative transport formats/models to address structural and content gaps within the existing transport format and associated content. The following steps are proposed:

  • Identify a set of common criteria that a future proof transport format should satisfy
  • Put the set of criteria out for review by the industry
  • Collate the criteria into a ranked set of criteria.

Phase 2: Prototype Model Mapping SDTM to transport format

If Phase I identifies a clear path forward, Phase 2 will leverage existing SDTM pilot data and the prototype model developed in phase I to map pilot SDTM data to the model thus providing an alternative transfer format. The following steps are proposed:

  • Represent pilot SDTM data using the prototype model.
  • Test whether new transport format can support use cases
  • Evaluate the performance of the new model against the existing models along a pre-determined set of quality metrics (such as ease of use, data quality)

Deliverables and Dates:

  • Phase I
    • Finalised Evaluation Criteria - Start of July
    • Questionnaire out for Industry Evaluation of Criteria - 01 Jul 2016
    • Responses to Questionnaire - 22 Jul 2016
    • Whitepaper first Draft - 15 Sep 2016
    • Final White Paper - 22 Oct 2016
  • Phase II
    • Project Planning - 01 Nov 2016
    • Identification of 6-8 candidates for Alt. Trans - 14 Nov 2016
    • Start Evaluation of candidates - 27 Nov 2016

PhUSE EU CSS 2016

  • Finalise set of criteria for evaluation of alternate submission formats
  • Finalise Questionnaire for evaluation
  • Work on White Paper

PhUSE CSS 2017

  • Work on White Paper
  • Participate and facilitate linkathon

Last revision by Laurenwhite,10/5/2017