WG5 Project 02

From PHUSE Wiki
Revision as of 03:16, 3 March 2016 by DanteDT (talk | contribs) (→‎Project Description)
Jump to: navigation, search


Project Team

Project Leads:

  • Dante Di Tommaso (dante.di_tommaso (at) roche.com)
  • Peter Schaefer (pschaefernet (at) yahoo.com)


Project Members, 2015:

  • Anders Vidstrup
  • Austin Taylor
  • Dirk Spruck
  • Frank Senk
  • Gustav Bernard
  • Jared Slain
  • Jeno Pizzaro
  • Lina Jørgensen
  • Mercy Navarro
  • Peter Glass
  • Rebeka Tabbey
  • Shubha Manjunath
  • Suzie Perigaud
  • Terry Walsh
  • Yingshan You


If you want to participate, please simply contact Dante or Peter, above.

Project Description

Name: Repository Content and Delivery

Description:
One of the projects under WG5 Standard Scripts.

This working group relies on distributed volunteers to review, develop, test and qualify Standard Scripts for general use. Volunteers should be able to easily find ways to contribute, either by improving our processes or working directly on the qualification of Standard Scripts. In 2014, we defined a process and associated roles for qualifying repository code, which we have further refined and exercised in 2015. See below.

Core Resources and References

Moving forward in 2015
We will:

  • folder WPCT contains template programs for White Paper Central Tendencies
  • sub-folders separate outputs by script language, so we can store same-name outputs generated by both R and SAS scripts
  • output_r contain R outputs (same file names as SAS outputs)
  • output_sas contain SAS outputs (same file names as R outputs)
  • folder utilities contains SAS macros that the template programs require
  • folder specification contains consolidated specs for each Central Tendency target, extracted from the white paper and stored in yml-format
  • folder qualification contains test scripts that document the qualification of each component in WPCT and utilities
  • Notes about qualification:
  • 2014 proposal was to create a File:Qualification of CSS scripts.xlsx for each component to explain the qualification process and document tests and checks, as implemented for certification and confirmation stages
  • This now seems burdensome without real gain in accessibility or transparency
  • So we are now considering a simpler approach that remains similar in spirit: simply documenting tests and checks in scripts in the qualification folder, above.


Ways to Contribute & Why this matters

How can I contribute?

Take the WG5 Box plot challenge and contribute your best method for creating an industry standard!

There are many ways to contribute further, such as:

  • Stay informed by joining 3-weekly team TCs (contact Dante or Peter, above)
  • Review existing scripts, and there are a few types of code so far:
  1. Read scripts in SAS and R for Central Tendency targets - Assess suitability for contributor and end-users. Suggest improvements. (See the WG5 Box plot challenge for R and SAS starting points.)
  2. Test scripts - Same as with SAS and R scripts, but rather than focusing on reading, assessing code, try running and feeding back your experience.
  3. scriptathon 2014 code, both SAS and R - Browse this repository of contributed code. Feed back any that you like and consider suitable for our "Central Tendancy" package in SAS and R.
  • Test & Qualify scripts (either in the style proposed, or a style you propose and demonstrate)
  • Review & Improve SDTM and ADaM test data, esp. for conformance with CDISC specification
  • Decisions in our CDISC SDTM/ADaM test data affect scripts (e.g., different variable names for low/high reference limits in ADVS vs. ADLB).
  • Can you explain and provide references for such decisions?
  • We have also included these CDISC test data in our phuse/css github project.
  • Write scripts for remaining targets (and, yes, basically all targets are still "remaining" at this time)
  • Review YML metadata files. We propose 2 types:
  1. YML for scripts (SAS and R) that describe the script, its purpose and its state
  2. YML for targets, language-independent specifications for the target output created by the SAS and R scripts
  • Other ideas (infrastructure, development, test automation, etc.)


Why does this matter? We all have to juggle multiple responsibilities each day. Why is it important for volunteers within our industry to make time, and gain support within their organization, for contributions to this project? Here we offer you a few key points for promoting contributions from your group:

  1. Building standards for data visualization upon ADaM standards simply makes sense
  2. The White Paper subteam are making good progress to release a standard view (via standard analyses) of clinical trials databases
  3. Qualified, trusted scripts to deliver an accepted "standard view" of clinical trials data will unlock the potential efficiencies of the standard analyses in the white papers
  4. Heath Authorities are interested in gaining these efficiencies, to receive the core analyses as familiar and consistent data displays to accelerate review
  5. Participating, contributing and using these common analyses and industry scripts provides learning experiences for all -- both for industry as well as technical SAS and R expertise


Project Progress

Approach for 2015

  • Specify improvements to Repository metadata and interface
  • Specify improvements to Specifications for white paper targets, including how and where we document and publish
  • Revise the Qualification process


Notes from CSS annual meeting: 16-17 MAR2015

  1. Proceed with qualification process as proposed
  2. Exercise process on specific deliverable for 2015, the Central Tendencies package
  3. Use contributed code for ideas rather than trying to qualify contributed code
  4. Develop and deliver "Template programs with extras", as proposed in WG5 P02 Qualification Process for Standard Analyses


Qualification Process

Qualification of Scripts: The Steps, States and Tasks
The following table describes the steps of qualifying a script, artifacts, and how the progress is tracked by a state indicating how the script is progressing through the steps.

Step Resulting State Actors Artifacts Description
Requirements Analysis & Initial Script Development Created
  • Developer
  • SME
 Establish
  • Specification YML
  • Script including header with To Do list (SAS or R)
  • Script YML
  • Create initial documentation for the script
  • Task card(s) in Trello
  • Maintain history of changes in Git
 Developer
  • Extracts the specification from the white paper.
  • Consults with SME as needed
  • Creates the initial version of the script

Subject Matter Expert (SME)

  • Knows the intent of the data display
  • Advises and clarifies the requirements as needed

Developer & SME

  • Identify required data input, structure of input data, and any restrictions that might apply to the script or the data
Script Completion Completed
  • SME
  • Developer
 Finalize initial artifacts.

Establish script outputs (SAS or R).

  • Task card(s) in Trello
  • Maintain history of changes in Git
  • Code complete status in software development, the scripts are considered to be
  • This requires testing by the developer, and review by the SME. This typically requires some iterations to establish intended behavior within an agreed scope.
Review and Qualification Tested
  • QA Expert
  • SME
  • Developer

Updated as needed

Objectives of this step:

  • Define test strategy and create and execute tests.
  • Confirm results in the formal qualification of the script.

Who does what:

  • QA Expert creates & SME reviews
    • Specify tests that establish intended behavior of the script and the test environment to be used (for example, OS, SAS/R version, etc.)
  • Developer and/or QA Expert create
    • Manual or automated test steps for the script]
  • QA Expert and/or SME review
    • Manual or automated test steps for the script
    • Expected test outputs (aka reference output)
  • QA Expert executes
    • Manual or automated test steps for the script
  • QA Expert and/or Developer updates
    • all artifacts as needed
Script Release Released
  • QA Expert
  • Technical Writer
QA Expert updates as needed

Tech. Writer updates source specification as needed (e.g., white paper)

  • Review and accept tests.
  • Document the completeness of this development cycle.
  • This step includes a final check that all files are in place and accessible before updating the script state to be "Qualified".

Comments from the team:

  • <add your comments, suggestions here>


Project Documents

Key Documents


Posters and Presentations


FAQ

Feel free to post questions and answer questions from colleagues here:

  • Q1: I cannot access Trello from work, to create an account or work on our boards (network is blocked, browser is out-of-date, etc.). What can I do?
    • A1: If you have a mobile device, check for the Trello app. It provides basic functionality.
    • A2: Explore options within your organization to gain access. Other contributors have used this project as a business case for access or to install a modern browser (see Why This Matters, above.)
  • Q2: I cannot access test data directly from SAS, as described here and implemented in the scripts. What can I do?
    • A1: The SAS scripts by default access our test data sets via the SAS macro "util_access_test_data.sas". We have now added an options to override default behavior by passing in a local folder. You must first download the CSS/PhUSE data sets, store them locally, and then specify in a script to %util_access_test_data(..., local=your-local-path).


Project Meetings

2015 Agendas & Minutes


2013 CSS - Notes from the annual meeting


Last revised by DanteDT,03/3/2016

Retrieved from "https://www.phusewiki.org/wiki/index.php?title=WG5_Project_02&oldid=14961"