Written by: Jessica Noble for Terry Mitchell.
The Centre for Student Success at Wilfrid Laurier University launched the JUMPStart to Higher Education Program in 2013. The objective of the program is to educate grade 7 and 8 students and their families about the multiple pathways to postsecondary education available to them. JUMPStart has four specific goals, which are to:
In 2015, JUMPStart reached over 2500 grade 7 and 8 students in the Waterloo Region District School Board, although the grade 7 program did not conclude in time for its data to be included in this report. Through interactive, engaging workshop sessions students were informed about the different pathways to postsecondary education (e.g. apprenticeships, college, and university). The workshops sessions provided information on sources of financial support, high school course selection and career/postsecondary planning. During the sessions a video and PowerPoint slideshow, created by the JUMPStart team, were viewed by students that illustrated various career options and their related educational pathways. Additionally, students were provided with workbooks to take home with signature sheets to return to school with the aim of facilitating conversations regarding postsecondary education between students and their parents or guardians. High degrees of family interaction were noted this year with five classes returning 100% of their signature sheets. Based on this high level of participation a total of 80 JUMP campus bursaries were dispersed to students. Technical challenges with data collection and analysis were overcome this year increasing the validity of the intervention outcome findings.
Statistically significant intervention effects were found for all questions of the survey; participant scores increased from the pre-survey to the post-survey. The new knowledge-testing questions also produced a higher number of students selecting the correct response from pre to post-survey. This means that after participating in the session, students exhibited significant increases in perceived knowledge, awareness, and understanding. The JUMPStart program significantly contributes to middle school students’ knowledge, awareness, and understanding of postsecondary education enhancing youth knowledge readiness for the challenges of educational and career planning. Ongoing program evaluation reveals that the Laurier JUMPStart program is meeting its goals of:
Laurier’s JUMPStart program in strongly aligned with the Ministry of Education’s Creating Pathways to Success document* in providing students with information to advance their self-knowledge, a guided exploration of education and career opportunities, information to assist students in making informed curriculum decisions for employment and career goal planning, with increased knowledge and financial planning tools to achieve their personal life goals.
This program evaluation report was made possible because of the support from various stakeholder groups. The research team would like to give a special thanks to the Astley Family Foundation for funding the program. Also, we are grateful for the research support provided by the Centre for Student Services at Laurier and the school support staff at each data collection site. Lastly, we would like to thank all the teachers and students that participated in the JUMPStart program.
The Poverty Reduction Research Group (PRRG) housed within Laurier’s Centre for Community, Research, Learning and Action (CCRLA) was hired under the direction of Dr. Terry Mitchell to conduct a three-year pilot program evaluation for the JUMPStart to Higher Education initiative. The evaluation findings of the third year of the JUMPStart program that was conducted with six select elementary schools within the Waterloo region can be found in this report. The program design, method, evaluation objectives, challenges, and overall findings of the intervention in relation to the program’s objectives can be found in this document. Additionally, results pertaining to the differences in school and gender are also presently. The evaluation report concludes with recommendations for the JUMPStart programs as well as for the evaluation process going forward.
Created by the Centre for Student Success at Laurier, the JUMPStart program began in 2013. The centre is composed of six units:
These units offer resources through a wide variety of program and services set to enhance and support the academic experience of students at Laurier. The programs and services assist with mathematical and writing skills, as well as workshops on time management, exam preparation and stress management.
The Centre for Student Success designed the JUMPStart program to help facilitate the discussion about various pathways to postsecondary education among youth and their families at an optimal age. Few opportunities exist for elementary school children in grades 7 and 8 to access information and resources regarding postsecondary education and thus, they are the target population of the program.
Generally, exposure to information pertaining to higher education occurs when students are enrolled in a mandatory career course. The dearth of exposure to this vital information prior to high school can be challenging as students are required to complete high school course selection during their grade 8 academic year. Students are then required to make decisions about entering the applied or academic stream in high school, without prior knowledge of high school course requirements for postsecondary admission. As a result of these findings, JUMPStart was designed to address this barrier to postsecondary participation by providing interactive workshops about the different pathways into postsecondary education (e.g. apprenticeships, college, and university) available to students.
The JUMPStart program complements the “Creating Pathways to Success” initiative put forth by the Ontario Ministry of Education, which is an education and career/life-planning program designed for students in kindergarten to grade 12. The program goals are to create opportunities inside and outside the classroom for students, parents and the broader community to obtain information that will allow them to make informed decisions regarding their education, career and life choices (Ontario Ministry of Education, 2014).
The JUMPStart program is offered to interested schools within the Waterloo Region District school board. Special focus was given to schools that are situated within neighbourhoods with significantly higher proportions of families who are at risk socially or economically. The main goal of the program is to “jumpstart” the conversation around postsecondary education, so students can make more informed decisions about their futures in grade 8 and throughout high school.
The JUMPStart Program consists of four main objectives, which are to:
The three main program outputs of the JUMPStart program centre are:
To demystify postsecondary education and provide knowledge regarding the multiple pathways to higher education.
Sessions consist of fun and interactive workshops delivered by Faculty of Education students. The sessions are tailored for specific grade levels (e.g. grade 7 and 8) and incorporate information about:
Sessions will be delivered by university administrators at locations and times convenient to students’ families. The sessions will include:
Sessions will cover similar topics to those addressed in the parent and student sessions and will be designed to assist their roles as mentors to the students.
Supplementary marketing and educational materials will be used to provide further information and resources to stakeholders to refer to when making future decisions around education.
Bursary support for students with financial need to attend Laurier’s Junior University Multidisciplinary Program (JUMP), which is a three-day residential program that takes place at Laurier during the months of May and June. The program is delivered to students in grades 5-8, and aims to encourage higher education by having students experience university life through attending courses and various campus activities and events.
The target population of the JUMPStart program has three distinct characteristics in relation to location, age and socioeconomic status.
In the pilot year of JUMPStart, the creators envisioned reaching over 1,000 students, anticipating a larger ripple effect for the program in the future. The target goal was met with over 1,000 students participating in year one and over 1,100 reached in year two, which included an additional 700 grade 7 students. In the current year, year three, approximately 2,500 grade 7 and 8 students participated.
Program staff hope in the long-term the program becomes self-sustaining while maintaining high levels of participation. In other words, it is hoped that JUMPStart students who pursue postsecondary education become ambassadors of the programs, sharing their knowledge of postsecondary pathways and demystifying postsecondary education for their younger siblings and the larger community. The implementation section outlines what aspects of the program actually occurred and what aspects had to be modified based on available resources.
In the first year of the pilot program, sessions were only delivered to students. Due to limited time and resources, sessions were not delivered to families and education providers. In the second year of the program, sessions were delivered to students as well as to the School Council group at Doon P.S. and School Success consultants at the Waterloo Catholic District School Board.
This year the sessions were delivered to students and staff in eight schools within the Waterloo Region District School Board by two Laurier Faculty of Education graduates. Due to time constraints and extensive workloads in regards to the School Council agenda, the sessions were not administered to families and education providers.
The Centre for Student Success at Laurier commissioned the Poverty Reduction Research Group housed within the Centre for Community Research, Learning, and Action (CCRLA) to conduct a summative evaluation (i.e. an evaluation of the components and outcomes of the program). The goal of the evaluation was to determine the extent to which the program is meeting its stated objectives and how the process of meeting those objectives can be improved.
The JUMPStart program was successfully administered to students from eight different schools. Due to unforeseen circumstances regarding a school threat, post-workshop data could not be collected from the first session at Laurentian Public School. Thus, data from this workshop was removed from the analysis process. Additionally, although JUMPStart was delivered to grade 7 and 8 students, the programming with the Grade 7 students did not end in time to include their data in this report. The results of the grade 7 students will be included in the 2016/17 report. Thus, the findings of this report pertain to the programming administered to the grade 8 students. Data was available for 973 students from eight schools (see Table 1). All participants were enrolled in a grade 8 class within the Waterloo Region District School Board.
The average age of participants was 13, with a range of 12 to 14 years. 758 out of the 973 students surveyed provided their age (22.1% missing). 396 (40.7%) of these students identified as male, 390 (40.1%) identified as females, 16 (1.6%) selected response numbers that did not have an identified gender and 171 (17.6%) did not answer this question.
|School Name||Grade||Location||# of Students|
|A.R. Kaufman P.S.||8||Kitchener||40|
|Lincoln Heights P.S.||8||Kitchener||57|
|William G. Davis P.S.||8||Cambridge||194|
Again, the best effort was made to include schools situated in neighbourhoods with families with lower socioeconomic status. However, this is difficult because each school supports a large geographic area composed of families with low, middle and high socioeconomic status. Schools were selected based on the knowledge of the program coordinator acquired through her 36 years of experience as principal and teacher in the Waterloo Region District School board and the screening questions that were discussed with each school representative.
1 Post-survey data (n=100) could not be collected during the first session at Laurentian P.S., therefore their data was removed during data analysis.
Surveys were designed to assess the impact of the intervention session on the students’ knowledge and program satisfaction. The 12-item surveys were administered at the beginning of the session (pre-survey) and the end of the session (post-survey) (see Appendix A). The survey was similar to the version used in the previous two years, however two additional questions were included assessing workshop satisfaction post-survey (i.e. length satisfaction, how informative the session was) and two items were changed to assess students’ knowledge rather than perceived understanding. Table 2 outlines how the survey was designed to assess whether the intervention session was successful in achieving the program’s objectives. The JUMPStart program is aligned with the Ministry of Education's new Creating Pathways to Success document (see Appendix B) with five survey items specific to this document. We assessed participants’ experiences concerning the length of the sessions and how informative the session was. Demographic information (i.e. age and gender) was also collected. Unfortunately, due to error when creating the surveys, the option of “other” was not provided for students when responding to the demographic item pertaining to their identified gender. This will be amended in future surveys.
|Program Objectives||Survey Question|
|Help students and their families to better understand how a postsecondary education can be achievable.||2 and 3|
|Provide information to assist students and their families in their decisions in grade 8 around streams they choose for high school.||5 and 6|
|Provide information about funding available for postsecondary education.||4|
|Demystify postsecondary education for students and their families.||1, 2, and 3|
Creating Pathways to Success Four Areas of Learning:
|8, 9, 10, 11 and 12|
|Program experience.||13 and 14|
Technical challenges with data collection and analysis were overcome this year by utilizing a more compatible format for the session response data. Program facilitators inputted the data manually after each session and students used the same iClicker for both the pre- and post-surveys, ensuring the data were connected to the same students. This method facilitated the work of the evaluation team during the data analysis and enhanced the validity of the intervention outcome findings.
The data was collected electronically using iClicker technology. The 12 pre-survey questions were embedded in the PowerPoint presentation and presented to the students by the facilitators before the workshop. The 14-item post-survey, which included the same items as the pre-survey with two additional items regarding feedback about the presentation, was administered after the workshop using the same iClicker method. All students were given an iClicker remote to respond to the survey questions (using a five-point Likert scale). Unlike the previous year, the students used the same iClicker for the pre- and post-surveys. The responses were then sent to the instructor hub and were processed by the iClicker software. The response data was then entered manually into SPSS by a facilitator of the JUMPStart program.
Due to an unforeseen school threat that interrupted data collection, post-surveys were not administered during the first session at Laurentian Public School. As a result, the data from this school could not be analyzed thus reported sample sizes are lower. Additionally, the data from the grade 7 students could not be included in this report due to the JUMPStart program extending past the expected deadline.
Statistical data analysis of the seven analyzed JUMPStart sites revealed statistically significant intervention effects found on all nine questions (i.e. Q1, Q3, Q5, Q6, Q7, Q8, Q9, Q10, Q12) (ps <.001). See Figure 1 below and Appendix C for means. In other words, students’ post-measure scores were higher than their pre-measure scores. Findings also revealed that more students were choosing the correct response post-survey on the three new knowledge-based questions. The evaluation data indicates that the JUMPStart program increased students’ knowledge, awareness and understanding of the covered topics.
|Survey Question||Pre-Survey||Post-Survey||Mean Change||N|
|Q1: I know the difference between an apprenticeship, college and university.||2.8||4.3||-1.53||973|
|Q2: You can become a licensed tradesperson after you have…||2.9||3.1||-0.22||973|
|Q3: I am aware of how long it takes to complete the different levels of education: Apprenticeship, college, and university.||3.1||4.3||-1.1||973|
|Q4: Which of these sources of financial aid do you have to pay back?||2.1||2.1||-0.04||973|
|Q5: I know the difference between academic and applied high school courses.||4.2||4.5||-0.29||973|
|Q6: I understand that I need to take different high school courses depending on whether I want to attend college, university or a an apprenticeship.||3.9||4.4||-0.58||973|
|Q7: How likely is it that you will apply to college/ university/ apprenticeship?||4.2||4.4||-0.19||973|
Q8: I am aware of how certain opportunities (e.g., recreational, social, volunteer and part-time employment) are related to certain careers.
Q9: I have a good sense of my personal and educational/ career goals.
Q10: I understand the steps required to achieve my educational/career goals.
Q11: A common reason why certain jobs no longer exist is because:
Q12: I understand how global trends will influence careers in the future.
Program objectives 1 and 4 concerned demystifying postsecondary education and helping students and their families better understand how a postsecondary education can be achievable. Survey questions 1, and 3 assessed knowledge, understanding and awareness of these topics. Statistically significant intervention effects were found on all questions (ps < .001). Question 2 produced a small mean change (0.22) as the majority of students were selecting the correct answer during the pre-survey. Post-survey scores produced a slightly higher group score, revealing an increase in knowledge and number of students selecting the correct response, post-workshop.
Question 1 produced one of the lowest mean scores (2.8) on the pre-survey. This means that students did not initially perceive themselves as knowledgeable of the differences between an apprenticeship, college and university. However, the mean change (i.e. the ‘jump’ in score from the pre- to the post-survey) is the first highest in the survey, compared to the second highest in previous years. In other words, although students’ pre-survey scores were very low, they exhibited the greatest increases in perceptions of knowledge on this item. Students exhibited similar increases in perceptions of understanding and awareness on questions 2 and 3. Additionally, the mean change scores for these items were among the highest in the sample. Thus, it can be said that the intervention was especially successful in meeting program objectives 1 and 4 for students.
Likelihood of applying to pursue postsecondary education: An additional question was added that assessed the likelihood that students would apply to college, university or an apprenticeship. Interestingly, a statistically significant intervention effect was found for this question (p < .001).
Overall, students indicated that were already very likely to apply to postsecondary educational programs on the pre-measure. Despite initially high scores on the pre-survey, students still exhibited statistically significant increases in likelihood from the pre-survey to the post-survey. This means that the intervention had the effect of increasing the likelihood that students would pursue postsecondary education. Again, this could be evidence of a demystifying effect of the intervention as well as increased access to information on post-secondary funding (see objective 3).
The second program objective was to provide information to assist students and their families in their decisions in grade 8 around choosing applied or academic streams in high school. This was assessed through survey questions 5 and 6. Statistically significant intervention effects were found on both questions (ps < .001). The significant results provide evidence that objective 2 of the JUMPStart program was met.
The third program objective was to provide information about funding available for postsecondary education. This was assessed by survey question 4. Question 4 assessed students' knowledge of financial aid sources. Findings reveal the smallest mean change in this question. In other words, the majority of students were selecting the correct answer during the pre-survey and post-survey. Out of the three knowledge-based questions, students appeared to have the most knowledge pertaining to specific financial aid that needs to be paid back, pre-workshop. Thus, it can be said that objective 3 of the JUMPStart program was already met prior to the workshop intervention.
The fifth program objective was to align the JUMPStart program with the four areas of learning outlined in the Ministry of Education's “Creating Pathways to Success” document. JUMPStart specifically targeted the following areas of learning:
Statistically significant intervention effects were found for all questions (p < .001).
Question 11 was changed from assessing perceived knowledge to a knowledge-based question during this third year of operation. Findings revealed the same mean score from pre to post-workshop. In other words, the majority of students were selecting the correct response during both the pre and post-survey. It can be assumed students held prior knowledge regarding global trends of employment. Question 12 and 8 revealed a relatively large mean (i.e. the ‘jump’ in score from the pre- to the post-survey) exhibiting an increase in perceptions of knowledge on this question relating to career opportunities and global trends Therefore, although students already possessed high levels of perceived awareness of how certain opportunities are related to certain careers, the intervention increased this perceived awareness further. Thus, it can be said that the intervention was successful in meeting program objective 5: exploring opportunities for students.
Question 9 produced a somewhat high pre-survey score and an even higher score on the post-measure. So, although students already perceived themselves as having a good sense of their personal and educational/career goals, the intervention increased this sense further. Thus, it can be said that the intervention was successful in meeting program objective 5: making decisions and setting goals for students.
Similar to effects found for question 9 and 10 produced a somewhat high pre-survey score and an even higher score on the post-measure. So, although students already perceived themselves as having a good understanding of the steps required to achieve their education/ career goals, the intervention increased this understanding further. Thus, it can be said that the intervention was successful in meeting program objective 5: achieving goals and making transitions for students.
The sixth program objective concerns participants’ experiences of the program. This objective was assessed through post-measure questions 13 and 14 that asked students how informative they found the session and how they felt about the length of the session.
The mean response for the group was 4.17 meaning that on average, students found the session to be informative. When asked about the length of the presentation the mean response was 3.73, meaning that on average, students found the length of the session to be just right. These responses were similar to previous years.
It is suggested, similar to previous years, that any unforeseen circumstances be noted which may have led to missing data. It was noted that for this report there were no such circumstances that may have led to missing data. However, multiple other reasons may have resulted in the missing data. This included:
During the third year of operation, the JUMPStart program was able to input recommended changes from the research team into the PowerPoint presentation. The facilitators were able to send the first school data collections sessions to the research team as requested. It is recommended this practice continues and if possible a meeting between the JUMPStart facilitators and research team is conducted prior to the first school presentation to conduct a tutorial on data collection and entry methods, as well as discuss any changes to data collection/instruments.
The use of iClicker technology remained beneficial to students, who found them interactive and fun, as well as program staff who no longer had to deliver and pick-up paper materials, or pay for printing and other costs (e.g. travel). Collecting data in-session also has its benefits as intervention effects are being measured in real-time rather than the next day. However, although more costly and time-consuming, data sets were more complete than when paper surveys were administered.
Many students did not complete all electronic survey questions, so there are unequal numbers of students within the same session. This could be due to in-session distractions resulting in students missing questions; the dynamic of completing a questionnaire in a session surrounded by peers is much different than completing a paper survey in a quiet classroom. For example, there may be less privacy when responding, and more pressure to conform to peer responses. Furthermore, although designed to automate data collection (i.e. no longer require members of the research team to manually enter survey data), iClickers only provide non-numeric (i.e. A, B, C, D, E) multiple choice responses with no numeric option; in order to perform statistical analyses, the data need to be numeric. Thus, a member of the JUMPStart team had to manually convert the non-numeric response data into numeric form (i.e. A=1, B=2, C=3, D=4, E=5). In comparison to last year, data entry was completed after each school session rather than at the end of the JUMPStart program.
This practice saved time for the research team conducting data analysis and report writing. Early data entry also allowed for quicker detection and applied solutions of any errors. Going forward, it is recommended that data is input after each sessions and the data file is sent to the research team for review periodically throughout the JUMPStart program. Lastly, it is important that when new questions are added or revised that their response options are compatible with iClicker’s response set. The demographic question pertaining to gender was changed from open-ended to a response option of male or female. As a result, students were given selective responses and were unable to input an “other” option. It is recommended that future JUMPStart surveys ensure there is an “other” response option for students.
Each class should collectively fill out a feedback form and these forms should be a mandatory component of the program as they provided very useful comments on how to improve the overall effectiveness and implementation of the session.
Prior to the launch of the intervention, detailed usability testing on intervention materials should be conducted with all major stakeholders. The testing should focus on appropriateness of language, method of delivery and content levels for each specific grade.
As was the case in the first year of the program, parental engagement was primarily achieved through the workbook that was sent home with students. The return rate for workbook slips was higher than last year at 100%. Early engagement and support from parents is crucial in order to create large scale program buy-in and commitment. Parents could potentially be engaged though general school information nights that take place at the start of the semester, announcements at school events, parent committee meetings and/or parent-teacher nights. Educational flyers could be sent home with students with a space for evaluation check boxes on the workbook slips.
The Research Team and JUMPStart facilitators should work closely together to design and execute the evaluation process of the program. This will allow for stronger evaluation instruments, identify any errors early, and apply a solution quickly. Continued collaboration will increase the quality of the data going forward.
In previous years, some questions attended to perceptions of knowledge, awareness and understanding and not actual levels. In response to this concern, three questions were revised to assess students’ actual knowledge (Q2, Q4, Q11). The response for these questions, however, revealed that students held previous knowledge prior to the workshop intervention and thus the evaluation tool was unable to detect the extent of knowledge increase.
It is recommended that future knowledge-based questions are tested previous to data collection to ensure there is a bell curve. Preferably, knowledge-testing questions will produce an even distribution across response range, prior to the workshop intervention in order to reveal program effects. It is recommended that the research team and JUMPStart facilitators work closely together to ensure grade level of survey questions are adequate. Additionally, it is possible that students may have been sharing responses with one another. It is recommended that students are reminded not to talk to each other during the pre- and post-survey in order to increase privacy as well as strengthen the quality of data.
Lastly, it is possible students held higher levels of knowledge pertaining to the new questions (i.e. trades, financial aid, and global trends of employment).
The JUMPStart to Higher Education program has reached over 3000 students in the Waterloo Region in the past three years. Results show that the intervention has had a positive impact on student knowledge, awareness, and understanding of postsecondary education pathways and funding opportunities. Significant intervention effects were found on all nine questions of the survey confirming that the JUMPStart program was successful in meeting its objectives. Participant evaluation data provides evidence that the students’ understanding of how post-secondary education can be funded and achieved was increased. The program helped demystify postsecondary education and increased knowledge discerning the differences between academic and applied streams in high school.
In the previous evaluations of JUMPStart, knowledge regarding the different streams of postsecondary education and funding opportunities was assessed through perceived knowledge questions. It was recommended by the evaluation team to change these perceived knowledge questions into actual knowledge-based questions. Although students already possessed high levels of knowledge regarding postsecondary education and a high likelihood of applying to postsecondary programs, the intervention increased knowledge about post secondary school pathways and funding opportunities with increased reported likelihood of attending post secondary education institutions. Overall, the students found the presentation to be informative and were satisfied with the length of the presentation.
The JUMPStart program significantly contributes to middle school students’ knowledge, awareness, and understanding of postsecondary education enhancing youth knowledge readiness for the challenges of educational and career planning. Ongoing program evaluation reveals that the Laurier JUMPStart program is meeting its goals of:
In conclusion, Laurier’s JUMPStart program in strongly aligned with the Ministry of Education's Creating Pathways to Success document in providing students with information to advance their self-knowledge, a guided exploration of education and career opportunities, with information to assist students in making informed curriculum decisions for employment and career goal planning with increased knowledge and financial planning tools to achieve their personal life goals.
We see you are accessing our website on IE8. We recommend you view in Chrome, Safari, Firefox or IE9+ instead.×