Difference between revisions of "WG5 Project 02"

From PHUSE Wiki
Jump to: navigation, search
m (Qualification Process)
m (Qualification Process)
Line 100: Line 100:
 
|Initial Script Implementation || Created || Programmer, SME  
 
|Initial Script Implementation || Created || Programmer, SME  
 
|| Establish
 
|| Establish
* Specifications
+
* [[WG5 P02 Specification Template|Specification YML]]
* Program header
+
* Script header ([[WG5 P02 Script header - SAS|SAS]] or R)
 +
* [[WG5 P02 Script YML Template|Script YML]]
 
* Task card(s) in Trello
 
* Task card(s) in Trello
 
|| The script is implemented based on the specification in the white paper. Sometimes a subject matter expert will help and answer questions or confirm the implementation to be correct.
 
|| The script is implemented based on the specification in the white paper. Sometimes a subject matter expert will help and answer questions or confirm the implementation to be correct.

Revision as of 04:28, 1 May 2015


Project Team

Project Leads:
Dante Di Tommaso (dante.di_tommaso (at) roche.com)
Peter Schaefer (pschaefernet (at) yahoo.com)

Project Members:
New participants from the 2015 annual conference:

  • Austin Taylor
  • Dirk Spruck
  • Frank Senk
  • Gustav Bernard
  • Jeno Pizzaro
  • Mercy Navarro
  • Peter Glass
  • Rebeka Tabbey
  • Terry Walsh

Continuing participants:

  • Yingshan You
  • Anders Vidstrup
  • Lina Jørgensen


If you want to participate, please simply contact Dante or Peter, above.

Project Description

Name: Repository Content and Delivery

Description:
One of the projects under WG5 Standard Scripts.

This working group relies on distributed volunteers to review, develop, test and qualify Standard Scripts for general use. Volunteers should be able to easily find ways to contribute, either by improving our processes or working directly on the qualification of Standard Scripts. In 2014, we defined a process and associated roles for qualifying repository code. See File:Qualification of CSS scripts.xlsx in Project Documents, below.

Moving forward in 2015
We will:

  • folder WPCT contains template programs for White Paper Central Tendencies
  • folder utilities contains SAS macros that the template programs require
  • folder qualification contains test scripts that document the qualification of each component in WPCT and utilities
  • Notes about qualification:
  • 2014 proposal was to create a File:Qualification of CSS scripts.xlsx for each component to explain the qualification process and document tests and checks, as implemented for certification and confirmation stages
  • This now seems burdensome without real gain in accessibility or transparency
  • So we are now considering a simpler approach that remains similar in spirit: simply documenting tests and checks in scripts in the qualification folder, above.


Project Progress

Approach for 2015

  • Specify improvements to Repository metadata and interface
  • Specify improvements to Specifications for white paper targets, including how and where we document and publish
  • Revise the Qualification process

Notes from CSS annual meeting: 16-17 MAR2015

  1. Proceed with qualification process as proposed
  2. Exercise process on specific deliverable for 2015, the Central Tendencies package
  3. Use contributed code for ideas rather than trying to qualify contributed code
  4. Develop and deliver "Template programs with extras", as proposed in WG5 P02 Qualification Process for Standard Analyses

Notes from FDA meeting: 18-19 MAR2013

  1. Define the minimum standards by which a script/tool/program can be considered validated.
  2. Minimum standards by which a script/tool/program can be considered validated:
  • Robust without red errors in companies production environment
  • -or-
  • Robust and used in FDA scripts repository, and some kind of ranking, e.g. with stars : ******.
  • Open CDISC validator or Data Fit used to check input/output.
  • SDTM/ADaM used in input/output.
  • GPP in source.
  • Docmented or perhaps only documented in header.
  • User Guide.
  • Requirement specification.
  • Run according to Requirement specification.
  • Tested by validation plan, test program all Peer reviewed
  • Tested by End users
  1. Moving a script to "production" (fully qualified):
  • Use folder ./src for non validated scripts
  • Use folder ./valid for validated scrips.
  • Discuss by whom and how

Project Documents


Qualification Process

Qualification Process Steps, Tasks, and Status of Scripts
The following table describes the steps during the qualification of a script and how the progress is tracked by a status that is assigned to the script. More details on the tasks in each step are describe below the table.

Process Step Resulting Status of Script Actors Artifacts Description
Initial Script Implementation Created Programmer, SME Establish The script is implemented based on the specification in the white paper. Sometimes a subject matter expert will help and answer questions or confirm the implementation to be correct.
Script Completion Completed SME, Programmer artifacts As the equivalent to the 'code complete' status in software development, the scripts are considered to be correct, complete, and the programming guidelines were observed. This will require some initial testing by a subject matter expert and will typically require some iterations to fix issues.
Review and Qualification Tested QA Expert, SME, Programmer artifacts This would be the development and execution of tests which results in the formal qualification of the script.
Script Release Released QA Expert, Tech. Writer artifacts The testing will be reviewed and accepted and the completeness of the process will be documented. This step should include a final check whether all files are in place and accessible before the scripts are declared to be 'released'.

Tasks of each Qualification Step

  • Initial Script Implementation
  • Script Completion
  • Review and Qualification
  • Script Release

Project Presentations & Posters


Project Meetings

2015 Agendas & Minutes

Conclusion after the 2013 FDA/PhUSE meeting

  • Companies will not allow programmers/statisticians to use a "validated", external script
  • Without some validation according to the companies SOPs
  • The check list will instead be a part of the User Guide: How to add a script to Google Code.
  • Validation will be called "qualification".
  • The qualifications will be based on a rating of each script (Metadata)
  • Qualification will be a part of the Metadata subgroup, which Lina J. will join.
  • Only scripts developed directly to a White Paper should be Validated in the common sense of the word
  • A small paper about this change will be produced together with the Members that joined this group at the FDA meeting.

Last revised by DanteDT,05/1/2015