Cover: Developing Army Leaders Developing Army Leaders

Lessons for Teaching Critical Thinking in Distributed, Resident, and Mixed-Delivery Venues

by Susan G. Straus, Michael G. Shanley, James C. Crowley, Douglas Yeung, Sarah H. Bana, Kristin J. Leuschner

Download

Purchase

Research Questions 

How effective is the Common Core — the first phase of the U.S. Army's system for developing critical thinking skills in its officer corps and to what extent are there differences among delivery venues? 

Based on current measures, how can course delivery be improved? 

How well do current methods of evaluation gauge course success and point to needed improvements? 

Abstract

The U.S. Army uses the Command and General Staff Officer Course (CGSOC) as a key component of its system for developing critical thinking skills and abilities in its officer corps. The Common Core is the first phase of CGSOC. The Common Core is taught in three venues: a resident course taught at Fort Leavenworth and at satellite campuses; Advanced Distributed Learning (ADL), a web-based, self-paced course that uses interactive multimedia instruction; and The Army School System (TASS), primarily for Reserve Component officers, which combines resident and interactive multimedia instruction and is taught by the U.S. Army Reserve Command's 97th Brigade and its three subordinate battalions. CGSOC consists of nine blocks of instruction taught as stand-alone modules in the resident course (14–16 weeks long) and organized into three phases in TASS and ADL (designed to be taken over a period of up to 18 months). In response to the interests of Army leadership, this study sought to answer the following questions about the Common Core, focusing on the 2009–2010 academic year: Based on current methods of evaluation, how effective is the Common Core, and to what extent are there differences among distributed, resident, and mixed-delivery venues? Based on current measures, how can course delivery be improved? How well do current methods of evaluation gauge course success and point to needed improvements? To answer these questions, the authors analyzed available data from Command and General Staff School, including responses to student surveys, grades on assignments, and student characteristics. In addition, the authors conducted a quasi-experimental study to assess consistency in grading among faculty members.

Key Findings

• Students in All Venues Were Generally Satisfied with the Common Core — The First Phase of the U.S. Army's System for Developing Critical Thinking Skills in Its Officer Corps 
• Students were generally satisfied with most aspects of the course in all venues. 
• However, students engaged in online instruction were less satisfied with opportunities for performance feedback and wanted more peer interaction. 
• Students reported technical and administrative problems with some aspects of online instructional delivery. 
• Student Grades Were High Across All Venues; Grading Needs Improvement 
• There were no meaningful differences in grades across venues. 
• The average scores on assignments suggest that there was leniency in grading. 
• The authors conducted an exploratory, quasi-experimental study to assess the consistency of grading among faculty on four assignments. Although the number of faculty who participated in the study was small, responses of those who did participate showed that reliability across these graders was generally quite low. 
• Future Evaluation Should Focus on Whether the Best Possible Outcomes Within Venues, Rather Than Equivalent Outcomes Across Venues, Are Achieved 
• The authors found few meaningful differences in students' self-assessed learning and course grades among the three venues. 
• These results should not be interpreted to mean that the venues are equally effective or that differences do not exist. 
• Although the learning goals are the same, the venues are designed to support different student environments and needs. 

Recommendations

• The Command and General Staff School (CGSS) should continue to examine grading practices and, if needed, provide professional development to ensure that faculty grade to a common standard.
• CGSS should investigate the use of blended learning strategies, such as those used in their Advanced Operations Course, to provide Advanced Distributed Learning students with opportunities for instructor and peer interaction.
• CGSS should address technical issues with access to and functions of online courseware. The demands of Phase II on The Army School System students also warrant further investigation, along with consideration of mitigation strategies such as changes in course structure.
• Surveys used in the Common Core should be modified to obtain more diagnostic information: 
• Assess self-efficacy questions before and after training in combination with individual-level explanatory variables to assess training effectiveness. 
• Move from five- to six-point response options for close-ended questions to increase variability in responses. 
• Analyze responses to open-ended questions to better understand students' experiences in the course and to improve future surveys. 
• Use web analytics, such as time spent on online lessons, to assess student behavior in the course and its association with performance in training.
• CGSS should survey course graduates and their supervisors to assess the impact of training on subsequent job performance.

Table of Contents 

Chapter One 

Introduction 

Chapter Two 

Overview of Delivery Venues for the Command and General Staff officer Course Common Core 

Chapter Three 

Students' Post-Training Reactions and Attitudes 

Chapter Four 

Grades on Student Assessments and Faculty Grading Practices 

Chapter Five 

Summary, Conclusions, and the Way Ahead 

Appendix A 

CGSOC Common Core Terminal Learning Objectives 

Appendix B 

Examples of Survey Questions and Coefficient Alpha Statistics for Scales Measuring Perceived Learning and Quality of Assessments 

Appendix C 

Survey Questions and Coefficient Alpha Statistics for Scales Measuring Instructional Delivery and Overall Satisfaction 

Appendix D 

Survey Responses in Resident, ADL, and TASS Venues 

Appendix E 

Association of Student Demographic Characteristics and Survey Responses 

Appendix F 

Grader Characteristics 

Appendix G 

Measures of Learning Goal Orientation and Motivation to Learn 

Research conducted by 

RANDARMY RESEARCH DIVISION

Manpower and Training Program

The research described in this report was sponsored by the United States Army and cond