WG5 Project 02

From PhUSE Wiki
Jump to: navigation, search


Project Team

Project Leads:

  • Gustav Bernard (Gustav.Bernard (at) Quintiles.com)
  • Andrew T Miskel


Project Members, 2017:

  • Jared Slain
  • Sam Tomioka
  • Nancy Brucken
  • Dante Di Tommaso
  • Rebeka Revis
  • Nina Worden
  • Vineet Sharma
  • Andrew T Miskell
  • Frank Senk
  • Joy Li
  • Pei Sern
  • Rohit Kamath
  • Subha Jamburajan
  • Valerie Williams
  • Alex Ford
  • Mat Soukup
  • Jeno Pizarro
  • Hal Li


Project Members, 2015:

  • Anders Vidstrup
  • Austin Taylor
  • Dirk Spruck
  • Frank Senk
  • Gustav Bernard
  • Jared Slain
  • Jeno Pizzaro
  • Lina Jørgensen
  • Mercy Navarro
  • Peter Glass
  • Rebeka Tabbey
  • Shubha Manjunath
  • Suzie Perigaud
  • Terry Walsh
  • Yingshan You


If you want to participate, please simply contact Gustav or Andy, above.

Project Description

Name: Repository Content and Delivery

Description:
One of the projects under Standard Analysis and Code Sharing.

This working group relies on distributed volunteers to review, develop, test and qualify Standard Scripts for general use. Volunteers should be able to easily find ways to contribute, either by improving our processes or working directly on the development and qualification of Standard Scripts. The process we are following were defined in 2014 and associated roles for qualifying repository code, which we have further refined and exercised in 2015 and 2016. See below.

Core Resources and References

Moving forward in 2017
We will:

  • folder WPCT contains scripts for the White Paper on Central Tendency analyses
  • sub-folders separate outputs by script language, so we can store same-name outputs generated by both R and SAS scripts
  • output_r contain R outputs (same file names as SAS outputs)
  • output_sas contain SAS outputs (same file names as R outputs)
  • folder utilities contains SAS macros that the template programs require
  • folder specification contains consolidated specs for each Central Tendency target, extracted from the white paper and stored in yml-format
  • folder qualification contains test scripts that document the qualification of each component in WPCT and utilities


Ways to Contribute & Why this matters

How can I contribute?

There are many ways to contribute further, such as:

  • Stay informed by joining 3-weekly team TCs (contact Gustav or Andy, above)
  • Review existing scripts, and there are a few types of code so far:
  1. Read scripts in SAS and R for Central Tendency targets - Assess suitability for contributor and end-users. Suggest improvements. (See the WG5 Box plot challenge for R and SAS starting points.)
  2. Test scripts - Same as with SAS and R scripts, but rather than focusing on reading, assessing code, try running and feeding back your experience.
  • Test & Qualify scripts (either in the style proposed, or a style you propose and demonstrate)
  • Review & Improve SDTM and ADaM test data, esp. for conformance with CDISC specification
  • Decisions in our CDISC SDTM/ADaM test data affect scripts (e.g., different variable names for low/high reference limits in ADVS vs. ADLB).
  • Can you explain and provide references for such decisions?
  • We have also included these CDISC test data in our phuse/css github project.
  • Write scripts for remaining targets (and, yes, basically all targets are still "remaining" at this time)
  • Other ideas (infrastructure, development, test automation, etc.)


Why does this matter? We all have to juggle multiple responsibilities each day. Why is it important for volunteers within our industry to make time, and gain support within their organization, for contributions to this project? Here we offer you a few key points for promoting contributions from your group:

  1. Building standards for data visualization upon ADaM standards simply makes sense
  2. The White Paper subteam are making good progress to release a standard view (via standard analyses) of clinical trials databases
  3. Qualified, trusted scripts to deliver an accepted "standard view" of clinical trials data will unlock the potential efficiencies of the standard analyses in the white papers
  4. Heath Authorities are interested in gaining these efficiencies, to receive the core analyses as familiar and consistent data displays to accelerate review
  5. Participating, contributing and using these common analyses and industry scripts provides learning experiences for all -- both for industry as well as technical SAS and R expertise


Project Progress

Approach for 2015

  • Specify improvements to Repository metadata and interface
  • Specify improvements to Specifications for white paper targets, including how and where we document and publish
  • Revise the Qualification process


Notes from CSS annual meeting: 16-17 MAR2015

  1. Proceed with qualification process as proposed
  2. Exercise process on specific deliverable for 2015, the Central Tendencies package
  3. Use contributed code for ideas rather than trying to qualify contributed code
  4. Develop and deliver "Template programs with extras", as proposed in WG5 P02 Qualification Process for Standard Analyses


Qualification Process Version 1.1

Qualification of Scripts: The Steps, States and Tasks Version 1.1
The following table describes the steps of qualifying a script, artifacts, and how the progress is tracked by a state indicating how the script is progressing through the steps.

Step Resulting State Actors Artifacts Description
Requirements Analysis & Initial Script Development Created
  • Developer
  • SME
Establish
  • Script including header with To Do list (SAS)
  • Script YML
  • Create initial documentation for the script
  • Maintain history of changes in Git
Developer
  • Extracts the specification from the white paper.
  • Consults with SME as needed
  • Creates the initial version of the script

Subject Matter Expert (SME)

  • Knows the intent of the data display
  • Advises and clarifies the requirements as needed

Developer & SME

  • Identify required data input, structure of input data, and any restrictions that might apply to the script or the data
Script Completion Completed
  • SME
  • Developer
Finalize initial artifacts.

Establish script outputs (SAS).

  • Maintain history of changes in Git
  • Code complete status in software development, the scripts are considered to be
  • This requires testing by the developer, and review by the SME. This typically requires some iterations to establish intended behavior within an agreed scope.
Review and Qualification Tested
  • QA Expert
  • SME

Objectives of this step:

  • Ensure Code and Specification is Robust.
  • Confirm results in the formal qualification of the script.
  • Ensure Code runs within multiple SAS Environments
Script Release Released
  • QA Expert
  • Technical Writer
QA Expert updates as needed

Tech. Writer updates source specification as needed (e.g., white paper)

  • Review and accept tests.
  • Document the completeness of this development cycle.
  • This step includes a final check that all files are in place and accessible before updating the script state to be "Qualified".

Qualification Process

Qualification of Scripts: The Steps, States and Tasks
The following table describes the steps of qualifying a script, artifacts, and how the progress is tracked by a state indicating how the script is progressing through the steps.

Step Resulting State Actors Artifacts Description
Requirements Analysis & Initial Script Development Created
  • Developer
  • SME
Establish
  • Specification YML
  • Script including header with To Do list (SAS)
  • Script YML
  • Create initial documentation for the script
  • Maintain history of changes in Git
Developer
  • Extracts the specification from the white paper.
  • Consults with SME as needed
  • Creates the initial version of the script

Subject Matter Expert (SME)

  • Knows the intent of the data display
  • Advises and clarifies the requirements as needed

Developer & SME

  • Identify required data input, structure of input data, and any restrictions that might apply to the script or the data
Script Completion Completed
  • SME
  • Developer
Finalize initial artifacts.

Establish script outputs (SAS or R).

  • Maintain history of changes in Git
  • Code complete status in software development, the scripts are considered to be
  • This requires testing by the developer, and review by the SME. This typically requires some iterations to establish intended behavior within an agreed scope.
Review and Qualification Tested
  • QA Expert
  • SME
  • Developer

Updated as needed

Objectives of this step:

  • Define test strategy and create and execute tests.
  • Confirm results in the formal qualification of the script.

Who does what:

  • QA Expert creates & SME reviews
    • Specify tests that establish intended behavior of the script and the test environment to be used (for example, OS, SAS/R version, etc.)
  • Developer and/or QA Expert create
    • Manual or automated test steps for the script]
  • QA Expert and/or SME review
    • Manual or automated test steps for the script
    • Expected test outputs (aka reference output)
  • QA Expert executes
    • Manual or automated test steps for the script
  • QA Expert and/or Developer updates
    • all artifacts as needed
Script Release Released
  • QA Expert
  • Technical Writer
QA Expert updates as needed

Tech. Writer updates source specification as needed (e.g., white paper)

  • Review and accept tests.
  • Document the completeness of this development cycle.
  • This step includes a final check that all files are in place and accessible before updating the script state to be "Qualified".

Comments from the team:

  • <add your comments, suggestions here>






Project Documents

Key Documents


Posters and Presentations


FAQ

Feel free to post questions and answer questions from colleagues here:

  • Q1: I cannot access Trello from work, to create an account or work on our boards (network is blocked, browser is out-of-date, etc.). What can I do?
    • A1: If you have a mobile device, check for the Trello app. It provides basic functionality.
    • A2: Explore options within your organization to gain access. Other contributors have used this project as a business case for access or to install a modern browser (see Why This Matters, above.)
  • Q2: I cannot access test data directly from SAS, as described here and implemented in the scripts. What can I do?
    • A1: The SAS scripts by default access our test data sets via the SAS macro "util_access_test_data.sas". We have now added an options to override default behavior by passing in a local folder. You must first download the CSS/PhUSE data sets, store them locally, and then specify in a script to %util_access_test_data(..., local=your-local-path).


Project Meetings

2018 Agendas & Minutes

2017 Agendas & Minutes

2016 Agendas & Minutes

2015 Agendas & Minutes


2013 CSS - Notes from the annual meeting



Last revised by GustavBernard,01/29/2018