Proposed Study Could Assess Training Effectiveness
Through the years, the expectations of law enforcement officers has expanded considerably to include responsibilities beyond the traditional role of policing. People often associate the role of peace officers as protecting and serving the public, maintaining public, trust and upholding individual liberties and constitutional rights. However, modern law enforcement roles have expanded to include enhancing the community’s quality of life and social service issues.
Today, society perceives law enforcement to be many things. The community expects a young officer to be a leader, regardless of their title or rank. Officers influence the actions of others and have an obvious leadership role in the community. A proposed study at the Department of Criminal Justice Training seeks to measure the training effectiveness of the basic police academy with the implementation of questionnaires gauging student reaction to training. The study would then compare the surveys to written exam scores taken during academy training.
There has never been a study of this type to measure the effectiveness of police training in Kentucky. Doing so will allow the academy to make any necessary changes to the curriculum content and culture of the academy to ensure the academy maximizes the value of training and operates in a fiscally responsive manner.
Evaluation using Kirkpatrick’s Model
Academic testing as the sole method of evaluation is not an effective measurement of training, given the complexities of the law enforcement profession. Academic testing may measure the recruit’s knowledge, but the behavior and performance of that knowledge are equally important. The expectations of an officer go beyond the performance of specific tasks. If it were just a task-oriented profession, knowledge and skill alone would define an officer’s professionalism. Attitude plays a significant role in how the individual is perceived by his or her fellow officers, command staff and citizens in the community. Officers are visible in the community, whether in uniform or not, and their leadership influences others.
The academy would be using Kirkpatrick’s Model of Evaluation. Kirkpatrick’s model has been applied when measuring police training effectiveness before, but not in Kentucky. In studies, the Kirkpatricks found a student’s perception of training had a positive relationship with learning and behavior. Other studies indicate that recruits having positive reactions to training would have higher exam scores than those with negative reactions. Using Kirkpatrick’s model, it was found that a level 1 (reaction) is positively related to the second level (learning). This indicates that if a student has a positive reaction to training, the student would also have higher academic exam scores and thus, a higher level of learning results from the training.
Confirming this relationship in the academy at DOCJT could have an impact beyond the curriculum content. Information revealing the recruit’s self-efficacy related to demonstrated knowledge and skill proficiency could result in a culture shift for instructional staff. This information is ultimately important for the future of law enforcement training and the safety of our families and community.
Each recruit would be subject to the same surveys and academic exams. Based on Kirkpatrick’s studies, it is believed DOCJT recruits with a positive reaction to training will experience higher exam scores than those with lower reactions. It is also believed recruits with higher than average grades would indicate positively on the final survey that they are more likely to use information (behavior level) from the academy daily in their jobs.
How the Test Would be Conducted
This study uses a mixed-method design. Participants would be law enforcement officers attending a law enforcement basic training class required for certification as peace officers in the commonwealth. Officers would have diverse educational backgrounds and a wide range of prior work experience, including military service. Each student would be asked to take a survey with demographic questions before their arrival for the course. This survey would be distributed by DOCJT’s learning management system.
During the first day of training, participants would be administered a pre-academy written exam to establish a baseline of law enforcement knowledge. The exam consists of selected multiple-choice questions that would also appear on formative written exams during the academy and a summative, post-academy written exam given in the final week of academy training. A comparison of the selected questions on the pre-academy written exam and the post-academy written exam would determine a measurement of learning, to indicate any increase or decrease of the recruits’ knowledge level from the beginning of academy training to the end.
Each question would have a direct link to one or more enabling learning objectives listed on the approved Kentucky Law Enforcement Council curriculum. Subsequently, each enabling learning objective would have a direct link to a training requirement, such as law or task identified on the Job Task Analysis report.
At five-week intervals throughout the academy, recruits would be required to complete questionnaires using the academy’s learning management system. The questionnaires will consist of Likert scale questions (quantitative) covering instructor knowledge, instructor enthusiasm, teaching effectiveness, the learning environment (classrooms and ranges) and facilities (dormitory, cafeteria and training equipment). The questionnaires also include open-ended questions (qualitative) to gain knowledge of what parts of training participants found most useful and what changes they would recommend for future classes. While beyond the scope of this study, a final questionnaire would be sent to students one year after graduation via email, again using the learning management system, to evaluate if knowledge gained during the academy is being applied on the job.
Student demographic data would be collected to allow the classification of participants according to age, educational level and prior work experience, including military service. This data would allow participants to be classified as living in one of 14 regions of the state and by other data such as race and gender. Comparing the pre-test exam at the beginning of training and the summative exam at the end of training would provide a statistical measurement of knowledge gained during the course. This numerical data can be analyzed using an independent t-test to determine the relationship between age groups, educational levels and military experience. The mean and standard deviation of the two groups will also be recorded. The correlation between the scores indicates any increase of knowledge gained during the course of training.
Questionnaires given during the academy would provide information on the participants’ reaction to training and what components of the course they believe would be most useful on the job. The questionnaires would contain Likert Scale questions regarding the information presented, instructor knowledge and skill, and the classroom facilities. Open-ended responses would require information that is more descriptive regarding what the participant found most useful and recommended improvements to the course.
This proposed study could help DOCJT better understand recruit learning and help the academy continue to make improvements that serve the needs of its clients.
