Intervention Effectiveness Research: Quality Improvement and Program Evaluation
von: Karen A. Monsen
Springer-Verlag, 2017
ISBN: 9783319612461
Sprache: Englisch
214 Seiten, Download: 4205 KB
Format: PDF, auch als Online-Lesen
Foreword | 6 | ||
Preface | 8 | ||
Abbreviations | 17 | ||
Contents | 10 | ||
Part I: Introduction to Intervention Effectiveness Research, Quality Improvement, and Program Evaluation | 18 | ||
1: Key Concepts, Definitions, and Frameworks | 19 | ||
1.1 Introduction | 19 | ||
1.2 Definitions and Descriptions of Intervention Effectiveness Research, Quality Improvement, and Program Evaluation: What They Have in Common and How They Differ | 20 | ||
1.2.1 What Is Quality Improvement? | 20 | ||
1.2.2 What Is Program Evaluation? | 21 | ||
1.3 How Intervention Effectiveness Research, Quality Improvement Activities, and Program Evaluation Are Similar | 21 | ||
1.4 How Intervention Effectiveness Research, Quality Improvement, and Outcome Evaluation Are Different | 22 | ||
1.4.1 Translational Research | 22 | ||
1.4.2 Quality Improvement (QI) | 22 | ||
1.4.3 Six Sigma Quality Improvement | 23 | ||
1.4.4 Health Services Research | 23 | ||
1.4.5 Big Data in Health Care Research | 24 | ||
1.4.6 Program Evaluation | 24 | ||
1.4.7 Implementation Research | 25 | ||
1.5 Definitions of Similar Sounding Terms and What This Book Does Not Attempt | 25 | ||
1.5.1 Comparative Effectiveness Research | 25 | ||
1.5.2 Implementation Science Research | 25 | ||
1.5.3 Dissemination Science | 26 | ||
1.6 Frameworks to Support Intervention Effectiveness Research, Quality Improvement Activities, and Program Evaluation | 26 | ||
1.6.1 Theory | 26 | ||
1.6.2 Logic Models | 28 | ||
1.6.3 Theoretical Framework | 28 | ||
1.6.4 Conceptual Framework | 28 | ||
References | 29 | ||
2: Problem-Intervention-Outcome Meta-Model (PIO MM): A Conceptual Meta Model for Intervention Effectiveness Research, Quality Improvement Activities, and Program Evaluation | 32 | ||
2.1 Introduction to the Problem-Intervention-Outcome Meta-Model (PIO MM) | 32 | ||
2.2 PIO MM and the CDC Logic Model | 34 | ||
2.3 PIO MM and the IHI Quality Improvement Model | 35 | ||
2.4 Using the PIO MM | 37 | ||
2.5 Operationalizing the PIO MM | 40 | ||
2.6 PIO MM Relationship to Change Theory | 41 | ||
2.7 PIO MM Relationship to PICOT | 41 | ||
References | 42 | ||
3: Problem-Intervention-Outcome Meta-Model Project Design | 44 | ||
3.1 Design for Intervention Effectiveness Research, Quality Improvement Activities, and Program Evaluation | 44 | ||
3.1.1 Observational Design | 44 | ||
3.1.2 Retrospective Design | 45 | ||
3.1.3 Prospective Design | 46 | ||
3.2 Intervention and Measurement Timing | 47 | ||
3.3 PIO MM and Research Design | 47 | ||
3.4 Benefits and Challenges of the Single Group Before and After Design | 47 | ||
3.4.1 Threats to Internal Validity | 48 | ||
3.4.2 Enhancing Before and After Design Using Comparisons | 48 | ||
3.4.3 Considerations for Prospective Data Collection | 49 | ||
3.5 Comparisons Using PIO MM Variables | 49 | ||
3.5.1 Problem | 49 | ||
3.5.2 Intervention | 49 | ||
3.5.3 Interventionist | 50 | ||
3.5.4 Outcome | 51 | ||
3.5.5 Population (Individual Characteristics) | 51 | ||
3.5.6 Setting | 51 | ||
3.5.7 Time | 52 | ||
3.6 Mixed Methods: Qualitative Evaluation | 52 | ||
References | 53 | ||
4: Tools for Intervention Effectiveness Research, Quality Improvement Activities, and Program Evaluation | 55 | ||
4.1 Data Sources | 55 | ||
4.2 Checklists for Obtaining New or Existing Data for Operationalizing the PIO MM | 56 | ||
4.3 Electronic Health Record Data | 58 | ||
4.4 Nursing-Specific Data | 58 | ||
4.5 Omaha System | 59 | ||
4.5.1 Problem Classification Scheme | 60 | ||
4.5.2 Intervention Scheme | 60 | ||
4.5.3 Problem Rating Scale for Outcomes | 60 | ||
4.6 Analysis Software and Techniques | 62 | ||
4.7 Power Analysis | 62 | ||
4.8 Software for Descriptive and Inferential Statistical Methods and for Creating Graphs/Charts | 62 | ||
4.8.1 Microsoft Excel | 62 | ||
4.8.2 R | 63 | ||
4.8.3 SAS | 63 | ||
4.9 Big Data (Pattern Detection) Methods | 63 | ||
4.9.1 Clustering | 64 | ||
4.9.2 Visualization | 64 | ||
4.10 Team Approach | 65 | ||
References | 65 | ||
5: Descriptive Analysis and Interpretation | 67 | ||
5.1 Introduction | 67 | ||
5.2 Data Cleaning | 67 | ||
5.2.1 Screening Phase | 68 | ||
5.2.2 Diagnostic Phase | 68 | ||
5.2.3 Treatment Phase | 69 | ||
5.2.4 Missing Data | 69 | ||
5.3 Pre-Processing | 69 | ||
5.3.1 Transforming and Recoding | 69 | ||
5.3.2 Identification and Labeling of Clusters Within a Sample | 70 | ||
5.4 Descriptive Statistics | 71 | ||
5.4.1 Frequency | 72 | ||
5.4.2 Cross Tabulation (Cross Tab) Matrix | 73 | ||
5.4.3 Rank | 73 | ||
5.4.4 Measures of Central Tendency | 73 | ||
5.4.5 Measures of Distribution | 75 | ||
References | 76 | ||
6: Inferential Analysis and Interpretation | 77 | ||
6.1 About Inferential Statistics | 77 | ||
6.2 Comparisons and Statistical Significance | 80 | ||
6.2.1 Comparisons of Sample Characteristics | 80 | ||
6.2.2 Outcomes as Measured by Before and After Comparison | 80 | ||
6.2.3 Benchmarking | 82 | ||
6.2.4 The P-Value in Large Dataset Research | 83 | ||
6.3 Clinical or Practical Significance | 83 | ||
6.3.1 Effect Size (Clinical or Practical Significance of Pchange = PTime2 ? PTime1) | 83 | ||
6.3.2 Interpretation of Effect Size (Clinical or Practical Significance) | 84 | ||
6.4 Associations | 84 | ||
6.4.1 Correlation | 84 | ||
6.4.2 Regression | 86 | ||
6.4.3 Interpretation of Correlations | 86 | ||
6.4.4 Survival Analysis (PTime1, PTime2, … PTimeX) | 86 | ||
6.4.5 Cross Tabs and Chi-Square (?2) | 87 | ||
6.5 Generalizability | 87 | ||
References | 89 | ||
7: Exploratory Data Analysis | 90 | ||
7.1 The Development of Exploratory Data Analysis | 90 | ||
7.2 Interpretation of Exploratory Data Analysis | 91 | ||
7.3 Visualization Techniques | 91 | ||
7.3.1 Heat Map | 91 | ||
7.3.2 Line Graph | 93 | ||
Line Graph with Trend Line | 95 | ||
Parallel Coordinates | 95 | ||
References | 97 | ||
8: Ethical Considerations | 99 | ||
8.1 Minimal Risk | 99 | ||
8.2 Institutional Review | 100 | ||
8.2.1 Where and How to Access an IRB | 100 | ||
8.2.2 When a Project May Be Exempt from IRB Review | 100 | ||
8.2.3 The Special Case of Quality Improvement | 101 | ||
8.2.4 Minimal Risk and IRB Review | 102 | ||
8.2.5 The Special Case of Program Evaluation | 103 | ||
8.3 Informed Consent | 104 | ||
8.3.1 What Is Informed Consent? | 104 | ||
8.3.2 Informed Consent Processes in the Context of Existing Data | 105 | ||
8.4 Data Privacy and Security | 105 | ||
References | 107 | ||
Part II: Practical Guide for Using the Problem-Intervention-Outcome Meta-Model | 108 | ||
9: Use the Worksheets and PIO MM Figure | 109 | ||
9.1 Review of Part I | 109 | ||
9.2 Overview of Part II | 110 | ||
9.2.1 Examples of Projects | 110 | ||
9.3 Starting the Process | 111 | ||
9.3.1 Worksheet Review | 111 | ||
9.3.2 Complete the PIO MM Diagram | 115 | ||
References | 115 | ||
10: Know the Literature (Worksheet A) | 116 | ||
10.1 Preparing to Complete Worksheet A | 116 | ||
10.2 Step-by Step Instructions for Completing Worksheet A | 117 | ||
10.2.1 Population of Interest | 118 | ||
10.2.2 Problem Addressed | 119 | ||
10.2.3 Measure(s) of Outcome | 119 | ||
10.2.4 Intervention(s) Used | 120 | ||
10.2.5 Measures of Intervention | 121 | ||
10.2.6 Measure of Intervention Fidelity | 121 | ||
10.2.7 Demographic Characteristics of a Sample | 122 | ||
10.2.8 Contextual Factors | 122 | ||
10.2.8.1 Contextual Factors – Interventionist | 123 | ||
10.2.9 Analysis Methods | 123 | ||
10.2.10 Comments | 124 | ||
10.2.11 Complete Reference | 124 | ||
10.3 Sources of Information for the PIO MM Matrix | 124 | ||
References | 127 | ||
11: Define the Problem (Worksheet B) | 128 | ||
11.1 Preparing to Complete Worksheet B | 128 | ||
11.2 Step-by Step Instructions for Completing Worksheet B | 129 | ||
11.2.1 Problem | 129 | ||
11.2.2 Definition of the Problem | 130 | ||
11.2.3 Population of Interest | 130 | ||
11.2.4 Background | 130 | ||
11.2.5 Problem Measurement Instrument/Scale | 131 | ||
11.2.6 Anticipated Outcome and Rationale | 132 | ||
11.2.7 What is Not Known/Gap in Knowledge | 132 | ||
References | 137 | ||
12: Describe the Intervention (Worksheet C) | 139 | ||
12.1 Preparing to Complete Worksheet C | 139 | ||
12.2 Step-by Step Instructions for Completing Worksheet C | 140 | ||
12.2.1 Describe the Intervention | 140 | ||
12.2.2 Expected Effectiveness | 140 | ||
12.2.3 Theoretical Framework | 141 | ||
12.2.4 Intervention Content and Essential Core Components | 141 | ||
12.2.5 Describe Intervention Measurement: Amount, Type, Fidelity, Quality | 142 | ||
12.2.6 Describe Interventionist Characteristics: Qualifications, Training, Demographics | 143 | ||
References | 149 | ||
13: Define the Outcome (Worksheet D) | 151 | ||
13.1 Preparing to Complete Worksheet D | 151 | ||
13.2 Step-by-Step Instructions for Completing Worksheet D | 152 | ||
References | 160 | ||
14: Plan the Analysis (Worksheet E) | 162 | ||
14.1 Preparing to Complete Worksheet E | 162 | ||
14.1.1 Step 1. Review Project Statements | 163 | ||
14.1.2 Step 2. Select Statements That Are Most Applicable to the Project and Discipline | 165 | ||
14.1.3 Step 3. Review Design Options | 165 | ||
14.1.4 Step 4. State the Design | 165 | ||
14.1.5 Step 5. Review Variables | 166 | ||
14.1.6 Step 6. Plan for Creating New Variables | 166 | ||
14.2 Step-by-Step Instructions for Completing Worksheet E | 167 | ||
14.2.1 Exploratory Data Analysis | 167 | ||
14.2.2 Sample | 167 | ||
14.2.3 Intervention | 168 | ||
14.2.4 Outcome | 169 | ||
14.2.5 Relationships Among Variables | 170 | ||
References | 171 | ||
15: Interpret the Results (Worksheet F) | 173 | ||
15.1 Preparing to Complete Worksheet F | 173 | ||
15.2 Results Statements and Presentation | 174 | ||
15.2.1 Presenting the Results | 174 | ||
15.2.2 Description of Sample Characteristics | 174 | ||
15.2.3 Description of Interventions | 176 | ||
15.2.4 Description of Outcomes | 177 | ||
15.2.5 Description of Benchmark Attainment | 179 | ||
15.2.6 Correlations Between Interventions and Outcomes | 180 | ||
15.3 Results Interpretation | 181 | ||
15.3.1 Theoretical Framework-Related Interpretation | 182 | ||
15.3.2 Temporality-Related Interpretation | 182 | ||
15.3.3 Give Alternative Explanations for the Findings | 183 | ||
References | 184 | ||
16: Disseminate the Findings | 185 | ||
16.1 Why Dissemination Matters | 185 | ||
16.2 Getting the Most Benefit from This Chapter | 186 | ||
16.3 Iterative Interpretation and Explication of the Overall Story | 186 | ||
16.4 Drafting the Abstract: Summarize the Story in Brief | 190 | ||
16.5 Develop and Display Results | 191 | ||
16.6 Adding Meaningful Interpretation to the Results | 193 | ||
16.7 Limitations | 199 | ||
16.8 The Methods Section | 200 | ||
16.9 The Purpose Statement | 201 | ||
16.10 Background to Set the Stage for the Purpose | 201 | ||
16.11 The Gap in Knowledge | 203 | ||
16.12 Title, Abstract, and Conclusion | 204 | ||
16.13 Rewrite the Abstract | 205 | ||
16.14 Write the Conclusions Section | 205 | ||
16.15 Polishing Tips | 207 | ||
16.16 Styles and Author Guidelines | 207 | ||
References | 208 | ||
17: Synthesis, Next Steps, and Epilogue | 210 | ||
17.1 Planning Next Steps | 210 | ||
17.2 Questions to Inspire Next Steps | 212 | ||
17.3 Building Evidence on Evidence | 213 | ||
References | 214 |