WG5 Project 02
Project Team
Project Leads:
- Dante Di Tommaso (dante.di_tommaso (at) roche.com)
- Peter Schaefer (pschaefernet (at) yahoo.com)
Project Members, 2015:
- Anders Vidstrup
- Austin Taylor
- Dirk Spruck
- Frank Senk
- Gustav Bernard
- Jared Slain
- Jeno Pizzaro
- Lina Jørgensen
- Mercy Navarro
- Peter Glass
- Rebeka Tabbey
- Shubha Manjunath
- Suzie Perigaud
- Terry Walsh
- Yingshan You
If you want to participate, please simply contact Dante or Peter, above.
Project Description
Name: Repository Content and Delivery
Description:
One of the projects under WG5 Standard Scripts.
This working group relies on distributed volunteers to review, develop, test and qualify Standard Scripts for general use. Volunteers should be able to easily find ways to contribute, either by improving our processes or working directly on the qualification of Standard Scripts. In 2014, we defined a process and associated roles for qualifying repository code, which we have further refined and exercised in 2015. See below.
Core Resources and References
- Project 08 within our Working Group 5 - the team writing the white papers that define the standard analyses which we then deliver in SAS and R
- Our qualification process for these scripts
- Our script repository in Github
- Our draft User Guide for Central Tendency Scripts
- CDISC ADaM test data in GitHub, originally published as a test project by CDISC.
Moving forward in 2015
We will:
- Exercise and refine our qualification process while delivering a "Central Tendencies" package based on the published CT white paper from P08 - Analysis and Display White papers.
- Develop scripts according to our Project Programming Guidelines
- Work on the phuse-scripts repository in Github.
- ( here's a link to a git e-book, a helpful reference )
- folder WPCT contains template programs for White Paper Central Tendencies
- sub-folders separate outputs by script language, so we can store same-name outputs generated by both R and SAS scripts
- output_r contain R outputs (same file names as SAS outputs)
- output_sas contain SAS outputs (same file names as R outputs)
- folder utilities contains SAS macros that the template programs require
- folder specification contains consolidated specs for each Central Tendency target, extracted from the white paper and stored in yml-format
- folder qualification contains test scripts that document the qualification of each component in WPCT and utilities
- Notes about qualification:
- 2014 proposal was to create a File:Qualification of CSS scripts.xlsx for each component to explain the qualification process and document tests and checks, as implemented for certification and confirmation stages
- This now seems burdensome without real gain in accessibility or transparency
- So we are now considering a simpler approach that remains similar in spirit: simply documenting tests and checks in scripts in the qualification folder, above.
Ways to Contribute & Why this matters
How can I contribute?
Take the WG5 Box plot challenge and contribute your best method for creating an industry standard!
There are many ways to contribute further, such as:
- Stay informed by joining 3-weekly team TCs (contact Dante or Peter, above)
- Review existing scripts, and there are a few types of code so far:
- Read scripts in SAS and R for Central Tendency targets - Assess suitability for contributor and end-users. Suggest improvements. (See the WG5 Box plot challenge for R and SAS starting points.)
- Test scripts - Same as with SAS and R scripts, but rather than focusing on reading, assessing code, try running and feeding back your experience.
- scriptathon 2014 code, both SAS and R - Browse this repository of contributed code. Feed back any that you like and consider suitable for our "Central Tendancy" package in SAS and R.
- Test & Qualify scripts (either in the style proposed, or a style you propose and demonstrate)
- Review & Improve SDTM and ADaM test data, esp. for conformance with CDISC specification
- Decisions in our CDISC SDTM/ADaM test data affect scripts (e.g., different variable names for low/high reference limits in ADVS vs. ADLB).
- Can you explain and provide references for such decisions?
- We have also included these CDISC test data in our phuse/css github project.
- Write scripts for remaining targets (and, yes, basically all targets are still "remaining" at this time)
- Review YML metadata files. We propose 2 types:
- YML for scripts (SAS and R) that describe the script, its purpose and its state
- YML for targets, language-independent specifications for the target output created by the SAS and R scripts
- Other ideas (infrastructure, development, test automation, etc.)
Why does this matter? We all have to juggle multiple responsibilities each day. Why is it important for volunteers within our industry to make time, and gain support within their organization, for contributions to this project? Here we offer you a few key points for promoting contributions from your group:
- Building standards for data visualization upon ADaM standards simply makes sense
- The White Paper subteam are making good progress to release a standard view (via standard analyses) of clinical trials databases
- Qualified, trusted scripts to deliver an accepted "standard view" of clinical trials data will unlock the potential efficiencies of the standard analyses in the white papers
- Heath Authorities are interested in gaining these efficiencies, to receive the core analyses as familiar and consistent data displays to accelerate review
- Participating, contributing and using these common analyses and industry scripts provides learning experiences for all -- both for industry as well as technical SAS and R expertise
Project Progress
Approach for 2015
- Deliver a Central Tendencies package, based on WG5 P08 Central Tendencies white paper
- Exercise the process for File:Qualification of CSS scripts.xlsx
- To facilitate both contribution and adoption of standard scripts
- Specify improvements to Repository metadata and interface
- Specify improvements to Specifications for white paper targets, including how and where we document and publish
- Revise the Qualification process
Notes from CSS annual meeting: 16-17 MAR2015
- Proceed with qualification process as proposed
- Exercise process on specific deliverable for 2015, the Central Tendencies package
- Use contributed code for ideas rather than trying to qualify contributed code
- Develop and deliver "Template programs with extras", as proposed in WG5 P02 Qualification Process for Standard Analyses
Qualification Process
Qualification of Scripts: The Steps, States and Tasks
The following table describes the steps of qualifying a script, artifacts, and how the progress is tracked by a state indicating how the script is progressing through the steps.
Step | Resulting State | Actors | Artifacts | Description |
---|---|---|---|---|
Requirements Analysis & Initial Script Development | Created |
|
Establish
|
Developer
Subject Matter Expert (SME)
Developer & SME
|
Script Completion | Completed |
|
Finalize initial artifacts.
Establish script outputs (SAS or R).
|
|
Review and Qualification | Tested |
|
Updated as needed
|
Objectives of this step:
Who does what:
|
Script Release | Released |
|
QA Expert updates as needed
Tech. Writer updates source specification as needed (e.g., white paper) |
|
Comments from the team:
- <add your comments, suggestions here>
Project Documents
Key Documents
- (latest)File:Qualification of CSS scripts.xlsx for Qualification of scripts, template for test definition
- File:WG5 P02 Proposal - 2014.pptx for Qualification of scripts
- File:FDA Scrips.ppt from 2013
Posters and Presentations
- File:ScriptRepo+Qual-Abstract PhUSECSS 2015.pptx, Poster for Script Repository and Qualification Process, PhUSE CSS 2015 annual conference
- WG5 P02 Qualification Process for Standard Analyses, Presentation to the Working Group during the 2015 CSS Annual Meeting.
FAQ
Feel free to post questions and answer questions from colleagues here:
- Q1: I cannot access Trello from work, to create an account or work on our boards (network is blocked, browser is out-of-date, etc.). What can I do?
- A1: If you have a mobile device, check for the Trello app. It provides basic functionality.
- A2: Explore options within your organization to gain access. Other contributors have used this project as a business case for access or to install a modern browser (see Why This Matters, above.)
- Q2: I cannot access test data directly from SAS, as described here and implemented in the scripts. What can I do?
- A1: The SAS scripts by default access our test data sets via the SAS macro "util_access_test_data.sas". We have now added an options to override default behavior by passing in a local folder. You must first download the CSS/PhUSE data sets, store them locally, and then specify in a script to %util_access_test_data(..., local=your-local-path).
Project Meetings
2015 Agendas & Minutes
- WG5 P02 Agenda 2015-05-05
- WG5 P02 Agenda 2015-05-26
- WG5 P02 Agenda 2015-06-16 - NB: Please prepare as mentioned in the agenda
- WG5 P02 Agenda 2015-07-07
- WG5 P02 Agenda 2015-07-29
- WG5 P02 Agenda 2015-08-18
- WG5 P02 Agenda 2015-09-09
- WG5 P02 Agenda 2015-09-29
- WG5 P02 Agenda 2015-11-10
- WG5 P02 Agenda 2015-12-01
- WG5 P02 Agenda 2016-02-01
2013 CSS - Notes from the annual meeting
Last revised by DanteDT,03/3/2016