How to use the GSE to validate and improve course performance

9737

One of the challenges when working in a school is making sure that students are achieving the learning outcomes designed by the institution. When I was working as a professor teaching conversation classes for freshman and sophomore students, I felt quite good about my ability to lead a class, but often wondered about the objectives outlined by the institution. What my students accomplished in my classrooms was very different from the student learning outcomes, or SLO, that were published by the school. What was the disconnect? Of course, my students performed well, and usually well above the standards, so everyone was happy. I wasn’t always satisfied, though, but aside from my teacher’s intuition, I had no real way to challenge that the published outcomes were, in fact, below what students should be able to do.

This was way back in 2008, long before I learned about the Common European Framework of Reference (CEFR). Looking back now, I realise that my intuition was correct because of the clarity provided in the CEFR. Having an externally validated framework helps to take the guesswork and intuition out of the process of aligning learning outcomes.

Learning Outcomes vs. Learning Objectives

In the field, teachers and administrators may use the terms learning outcomes and learning objectives interchangeably. However, these two things are not the same and it is useful to understand the difference between one and the other. The simplest way to describe the difference is as follows:

Learning objectives is a term that describes the specific learning teachers and professors aim to achieve at the classroom level. Each distinct class will have a specific learning objective aligned to the skills, content and context of the learning presented in the classroom.

Learning outcomes is a term that describes the collective learning achievements presented in a course and that a student is expected to be able to demonstrate mastery of after completing a course of learning.

Student learning outcomes

Why is this difference important?

If you think of the English classroom as a recipe, the learning outcomes are the dish I will prepare and the learning objectives are the ingredients that make up that dish. Learning outcomes tell me, the teacher, what I need to achieve by the end of the course, but don’t prescribe the manner in which I will achieve the outcomes. I have the freedom to choose my pedagogical strategy, create my curriculum, organize my content and plan my class activities as necessary to meet the learning outcomes described by my institution.[i] It is important to make the distinction, as a good course is not so prescriptive that teachers can’t respond to the needs of individual learners in the classroom in order to achieve learning.

Many teachers in the field know that from year to year, semester to semester, course to course, even day to day, what our students need from us in the classroom will change. It’s important that teachers have control over the learning objectives used in the classroom in order to achieve the institution outcomes provided by administrators, directors and deans. At the same time, without clear learning outcomes from the top, there would be little consistency in the quality of courses and learning delivered in a school. Both learning outcomes and learning objectives are important for understanding course needs and requirements and for helping students prepare for the expectations of the learning environment.

Validating Learning Outcomes: A School in Profile

As the learning outcomes are the goal of classes, it is important that these outcomes are designed to demonstrate mastery and progress of learning. Outcomes should allow teachers freedom while also clearly describing performance expectations and measurable achievements by students. Because of this, it is extremely important that learning outcomes be at an appropriate level and show progress when viewed across courses.

When the outcomes set learning goals that are too challenging, students will not meet expectations and will most likely fail, fall behind or be forced to repeat learning. If the outcomes are low hanging, students may feel a false sense of accomplishment and may be sorely challenged and surprised when they discover that despite their success in their language courses, they are not yet fully prepared to tackle complex English challenges. This is where using tools like the CEFR and the Global Scale of English can help to assess and validate the outcomes of the course, helping administrators, teachers and students find balance between teaching and demonstrable learning.

To help understand how this works, we will use as an example a made-up school called Learning Academy. Here is the profile of our pretend school:

School Name Learning Academy
Programs English for Business (1–3)

General English (1–10)

English Pathways for University (1–3)

Length of a Level Six weeks

Let’s look at the imaginary General English program at Learning Academy. The course is described as follows:

Learning Academy Course
General English (1–10) This General English course will take you from beginner to mastery in under a year. Improve your English to more fully experience the world.
Course Overview: Each course meets five days a week with five hours a day of class time. Weekly homework includes an additional three to four hours of study. Weekend field trips to local cultural sites are optional. (Expected course hours per level = 180 hours.)
Level 1 CEFR A1
Level 2 CEFR A1–A2
Level 3 CEFR A2
Level 4 CEFR A2–B1
Level 5 CEFR B1
Level 6 CEFR B1–B2
Level 7 CEFR B2
Level 8 CEFR B2–C1
Level 9 CEFR C1
Level 10 CEFR C1–C2

On paper, it looks like Learning Academy has a pretty well-organized program with reasonable expectations of what learners will be able to achieve by the end of a course of study. Let’s imagine, though, that the school has noted a problem with Level 6. In our school, the expectation is for students to perform at a B2 level by the end of the six weeks of study. Teachers have noted that students consistently fail to meet the learning outcomes for the course and must repeat the section. Students are complaining that the course is not delivering on expectations and feel that their money was not well spent. Administrators are frustrated and are not sure if the problem is poor teaching or poor placement of students based on test scores.

Problem: Student completion of Level 6 is consistently poor
Students Frustrated with repeating section. Feel teachers are to blame. Angry and feel it is a waste of money spent.
Teachers Frustrated with poor performance. Spend more time making materials. Angry and feel the course is taking too much personal time, the course material is inadequate and the program is poorly aligned to actual performance.
Administrators Frustrated with poor performance. Worried about decreased recruitment because of bad reputation from previous students. Angry with placement exams, frustrated with curricular materials and worried about underperforming teachers.

What Is the Solution?

When we step back and look at the problem from an outside perspective, one thing quickly becomes clear: The expectation of what students will achieve is unrealistic based on the number of hours of potential input. Based on what we know about this course, learners will have about 180 hours of guided learning in the program. What we know about CEFR levels from research is that once learners have moved into the B levels, it takes longer to move upwards in performance.[ii] With this knowledge, there are a few potential solutions for Level 6 of the program: break Level 6 into two courses, increase the number of hours for teaching and learning, and improve the alignment of learning outcomes to achieve satisfactory performance.

As a curriculum consultant, I would strongly encourage Learning Academy to add an additional two or three levels to the General English course to begin with. The next thing I would want to do is look at the learning outcomes published for the Learning Academy Level 6 course. The administrative team at Learning Academy wants to solve the problem, so we look at the learning outcomes. Here are some of the learning outcomes for the course. As an example:

Learning Academy Level 6 Student Learning Outcomes (Sample)
Speaking Narrate stories in multiple tenses

Ask for and give personal information

Organize and give short presentations and talks on a variety of topics

By looking closely at the learning outcomes, we can map them using the Global Scale of English Teacher Toolkit. Knowing that the expected level of the course is from B1 to B2, it is important that the range examined be the B1 to B2 range. Using the range, we look for descriptors that are a good match, moving outside the range only when no descriptor can be found. By doing this, it is possible to use the Global Scale of English to validate learning outcomes against the GSE/CEFR.

Learning Academy Level 6 Student Learning Outcomes GSE Descriptor and Level
Speaking Narrate stories in multiple tenses

Ask for and give personal information

Organize and give short presentations and talks on a variety of topics

Can narrate a story (45, B1).

Can convey simple relevant information emphasizing the most important point (45, B1).

Can give a short rehearsed talk or presentation on a familiar topic (53, B1+).

For the Level 6 program, Learning Academy had a total of 35 student learning outcomes. Fifteen were related to speaking, 10 were related to listening, eight were related to reading and two were related to writing. Using mapping to validate descriptors, we can get real insight into the challenges of the program, which will help to identify whether there are specific gaps in the program. After mapping the 35 descriptors with the Global Scale of English, it was easy for administrators to see a clear problem with the program. Of the 35 learning outcomes mapped, 20 were at a C1 level, 5 were at B1+, 5 were at B1 and 5 were at A2 or lower!

Distribution of Learning Academy Descriptors at Level 6: Mapped to GSE/CEFR
58% of descriptors are C1
14% of descriptors are B1+
14% of descriptors are B1
14% of descriptors are A2 or lower

Before mapping and validating the learning outcomes, administrators and teachers were looking for answers to understand why students were not achieving course expectations. After mapping, a clear problem emerges: The student learning outcomes are not appropriate and it will be difficult or impossible for teachers and learners to meet expectations because of these considerable gaps. More than half of the program’s learning outcomes are far too high for students to achieve. Additionally, the outcomes are not very well aligned to more realistic expectations of what students should achieve during the course. Now administrators and teachers can work together to create an updated plan of action for Level 6 of the course. Recommended changes are to reduce the number of C1 level descriptors and include more scaffolded progression through the skills. A revised program might look as follows:

Learning Academy Level 6 Revision Plan
Descriptor Alignment Current Revision
15 Speaking 10 C1

1 A2

3 B1

1 B1+

3 B1

8 B1+

4 B2

10 Listening 8 C1

1 A2

1 B1

2 B1+

3 B1

5 B1+

2 B2

8 Reading 2 C1

1 B1

2 B1+

2 B1

2 B1+

1 B2

2 Writing 1 A2

1 A1

2 B1+

From here, Learning Academy can begin to implement a new course of action: creating new student learning outcomes to meet the revision plan. Additionally, the school may decide to change the published expectations of course levels or add more courses. Using the Global Scale of English as a validation tool does not mean the school will need to adopt GSE learning objectives, only that the creation of learning outcomes for the course can represent a better distribution of expectations and more achievable and measurable progress.

As changes to the course are implemented, administrators can monitor student performance, the course pass rate, and student and teacher evaluations of the changes, using this information to make additional changes and adjustments to the learning outcomes of the course as the program continues. The use of the Global Scale of English as a validation tool provides clear direction for the future.

Validation to Improve Performance

Think about the example of the Learning Academy and your own institution. Do you have a course that is underperforming? Perhaps you have a course that is performing adequately but you think there is room for improvement. By using the Global Scale of English Teacher Toolkit, you can go beyond intuition and feelings about level and appropriacy and examine with some objectivity the course and its alignment to expectations. The value of external validation helps everyone improve their performance and achieve their personal performance and learning objectives.

I hope that you’re as excited as I am about the endless possibilities afforded us by using the GSE Teacher Toolkit. I’ve already blogged about several of my favourite uses of the teacher toolkit, including using it to create performance-based rubrics, using it as an assessment tool and, finally, using it as an inspiration for interesting and unique content.

I’d love to hear how you might use the GSE Teacher Toolkit—please share your ideas in the comments section below!

[i] Diamond, R. (1998). “Clarifying Instructional Goals and Objectives.” In Designing and Assessing Courses and Curricula: A Practical Guide (Revised ed.). San Francisco: Jossey-Bass.*

[ii] Davila, S., Mayor, M. (2016 March). “Granular Insight into Learner Assessment, Progress and Performance,” conference session presented at the International Teachers of English to Speakers of Other Languages 50th annual conference and language expo, Baltimore, MD.

In this article