Washback of English proficiency test in classroom activities at national university of arts education

VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15 1 Original Article Washback of English Proficiency Test in Classroom Activities at National University of Arts Education Dinh Thi Phuong Hoa* Dean of Foreign Languages, Hanoi Law University, 87 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam Received 05 November 2019 Revised 10 February 2020; Accepted 17 February 2020 Abstract: A study in Vietnam concerning the effects of The Vietnam Six-levels of Foreign Language Proficiency Framework, specially English Proficiency Tests for graduates, on classroom teaching and learning activities are reported. The study explores the phenomenon of washback or backwash, the influences of testing on 9 teachers and 679 non-English major students. It is cited as the only known research investigating washback in language education through classroom observation. The study was conducted at National University of Art Education, and combined classroom observations with data from interview, questionaire responses and document analysis to determine whether washback exist, to what degree it operates, and whether it is a positive or negative force in this educational context. The insights from the findings indicate that washback of English Proficiency Tests for graduates occurred in both positive and negative forms, to some degree, in teaching and learing content, methods and styles. Evidence of washback, both positive and negative, on the way teachers design tests was also found. This should help Vietnamese educators to prepare favourable conditions for enhancing the benificial washback of EPT. The findings have contributed to the knowledge of a nature of washback and consequently opened a new understanding to recognize the dissimilar levels of washback. further research is recommended. Keywords: Washback, English Proficiency Tests, classrooms activities. 1. Introduction * Today, English has become a global language that offers the chances to integrate into all the professions. Khamkhien (2010, p. _______ * Corresponding author. E-mail address: dinhphuonghoa.ecas@gmail.com https://doi.org/10.25073/2588-1159/vnuer.4335 757) stated that, “the importance of English has flashed an increasing concentration in the development of English language teaching in numerous countries” [1]. In Vietnam, English has been instructed nationwide as a compulsory subject at both lower, upper secondary level and tertiary level; and as an elective subject at primary level from 1980s to present (Nguyen, 1997, p.5) [2]. Notwithstanding its impact, D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15 2 English language teaching and learning for non- major learners in Vietnam are contradictory to all expectations as the language skills of listening, speaking, reading and writing have not been appreciated. Furthermore, most of the teachers have still taught English with traditional techniques, as teacher-centered or the grammar-translation method for many years. Conversely, for fulfilling the needs of a modern society in the globalization epoch, Vietnamese Prime Minister issued Decision No 1400/QD-Ttg of September 30, 2008, approving the scheme “Foreign Language Teaching and Learning in the national education system during 2008 - 2020” and now this scheme is extended to 2025 (National Foreign Languages Project for short) [3]. The scheme aims at implementing an educational innovation and evaluation of foreign language teaching and learning at all levels in the national education system. Accordingly, Minister of Education and Training issued the Circular N0 01/2014/TT-BGDĐT of January 24, 2014, approving The Vietnam Six-levels of Foreign Language Proficiency Framework (henceforth VNFLPF). This framework consists of six levels that are compatible with the Common European Framework of Reference for Languages [4] (CEFR for short) and other common international language proficiency levels and used as reference when writing curriculums and teaching plans. According to National Foreign Languages Project, with undergraduate institutions that are not specialized in foreign languages, the new language-training program must require a language proficiency of VNFLPF level 3 upon graduation. Based on this framework, English Proficiency Test from level 2 to level 5 (henceforth EPT.2 and EPT.3-5) is conducted and issued. Among these, EPT.2 is compatible with A2 of CEFR and EPT.3-5 is compatible with B1, C1, and C2 of CEFR. It thus became a very high-stakes test with serious consequences for non-English major students. National University of Art Education (henceforth NUAE) is Undergraduate institution that is not specialized in foreign languages; the new language-training program must require a language proficiency of VNFLPF level 3 upon graduation. However, because of limitation of training time and English in mixed-big sized classes (from 55 to over 65 students) in a large room where is 105 square meters in area, no microphone, and thus, some students could not listen to all lessons clearly. 679 students were from 18 to 22 years of age. They were from different Northern areas of Vietnam. Although, they had 3 years of learning English at high schools, their English proficiency was at beginner level (A0), therefore, the Rector of NUAE decided to apply for English proficiency of VNFLPF level 2 (A2) upon graduation. As a result, EPT.2 (A2) of VNFLPF is a compulsoty requirement for NUAE graduation from 2016 and EPT.3 (B1) will start being used from 2021. On the basic of the background of the Vietnam educational innovation context, particularly the context at NUAE, the study attemped to address these issues: 1) Whether English Proficiency Test will positively influence the English language teaching process at National University of Art Education, Vietnam. 2) Whether the changes in the teaching process will beneficially affect teaching strategies, which will lead to changes in learning style at National University of Art Education, Vietnam. 2. Literature review 2.1. The definition of washback in this study The term “washback” is predominant in language teaching and testing literature as well as general education. However, the term “washback” has been defined and interchangeably by many researchers and organizations worldwide. D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15 3 In applied linguistics, the term “washback” or backwash is defined as the influence or impact of tests on curriculum/syllabus design, language teaching and language testing [5]. Accordingly, tests can influence teachers and learners, and thus influence teaching and learning activities. The influences may be either positive or negative, depending on various facets not yet defined. Nevertherless, whether a seperate and distinguishable phenomenon of washback exists is still open to debate; and there appear to be very few emprical studies directly investigating this phenomenon [6]. In the educational evaluation literature, washback is considered the influences of testing on teaching and learning practices. Therefore, tests can drive teaching and learning that is also mentioned as measurement-driven instruction [7]. Fitz-Gibbon (1996) defined impact as any effect of the service [or of an event or initiative] on an individual or group [8]. This definition accepts that the impact can be positive or negative and may be intended or accidental. When holding this definition, measuring impact is about identifying and evaluating change [9]. Messick (1989) expanded the concept of consequential validity, changing the previous notions about score interpretation and test use. The concept of washback in test validity research is primarily associated with Messick’s concept of consequential validity. Therefore, washback is defined as an “instance of the consequential aspect of construct validity and a focal point of validity research” [10], which covers components of test use, the impact of testing on test-takers and educators, the interpretation of results by decision-makers, and any possible misuses, abuses, and unintentional effects of tests. The influences of tests on teachers, students, institutions, and society are accordingly considered one type of validity evidence. Many other researchers have also emphasized the meaning of justifying test use and exploring its consequences ([11, 12]). Therefore, washback also plays a key role in the process of educational innovation and assessment in language teaching and learning [13]. In short, for the purpose of this paper, the term “washback/backwash” is understood to be the influences that tests have on teachers and students in terms of the methods/activities they use in their classrooms to teach/study English as Foreign Language. 2.2. The Vietnam Six-levels of foreign language proficiency framework The CEFR provides a detailed description of learner level by skills, in a language-neutral format. Therefore, the CEFR is used for many dissimilar practical purposes because its influence goes beyond merely describing language proficiency of learners, they are: teacher training programs, developing syllabuses, creating tests/exams, marking exams, evaluating language learning needs, designing courses, developing learning materials and describing language policies continuous/self-assessment. Accordingly, VNFLPF is designed based on CEFR in the Vietnam educational context. This framework consists of six levels and its Can-do descriptors that are compatible with CEFR and other common international language proficiency levels. Therefore, VNFLPF is used as reference when writing curriculums teaching plans, assessement and designing test. VNFLPF describes foreign language proficiency at three broad bands with six main levels: level 1 and level 2, level 3 and level 4, level 5 and level 6. The scale starts at level 1 and finishes at level 6 that is compatial with CEFR from A1 to C2 as the following: For the purpose of this paper, the usage of VNFLPF helps to define clearly certain requirements for competency, capacity in listening, speaking, reading and writing, and thus English level 2 (A2) of VNFLPF learner is actived in the performance of the four main language activities, including listening, speaking (spoken interaction), reading, writing (written production) in the public, the personal, the educational and the occupational domains with some types of text and questions. D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15 4 d Table 1. The 6 levels of the VNFLPF VNFLPF (Level) General Descriptions A - Basic user Level 1 (A1 - Breakthrough) Can communicate in basic English with help from the listener Level 2 (A2- Way-stage) Can communicate in English within a limited range of contexts B - Independent user Level 3 (B1 - Threshold) Can communicate essential points Level 4 (B2 - Vantage) Can use English effectively, with some fluency, in a range of contexts C - Proficient user Level 5 (C1 - Effective Operational Proficiency) Can use English fluently and flexibly in a wide range of contexts. Level 6 (C2 -Mastery or Highly proficient) Can use English, very fluently, precisely and sensitively, in most contexts D 2.3. Some washback studies Studies on washback reveal varied and sometimes different findings. The following section discusses the washback influences on teaching or learning activities in classroom. The field of washback has been investigated by many researchers around the world. Among these, the washback model of Alderson and Wall (1993) is considered a classic and landmark study. Alderson and Wall (1993) used obsevation method to carry out their Srilanka study on investigating the washback existing of English teaching and learning activities in classroom. Alderson and Wall (1993, p. 120-121) developed the fifteen hypotheses (WHs for short) that combined different possible aspects of washback, including the effect on what to teach/learn, how to teach/learn, the rate and sequence of teaching/learning, the degree and depth of teaching/learning and the attitudes to content, method, etc. of teaching/learning [14]. Alderson and Hamp-Lyons’s model (1996, p. 296) used interviews and one-week-classroom observations of teachers to review and correct WHs of Alderson and Wall (1993) that “tests will have different amounts and types of washback on some teachers and some learners than other teachers and learners” [15]. The studies of Cheng (1999 and 2004) focused on old and new HongKong Certificate Examination in English (HKCEE) ([16, 17]). Cheng (1999) used classroom observation that combined her data of baseline study and Part A of Communicative Orientation of Language Teaching [18] and interview methods to compare “teachers’ perceptions toward both old and new HKCEE”. Cheng (2004) based on a combined research framework that employed multiple approaches to explore both the macro level (including the main parties within the HongKong educational context) and the micro level in schools (concerning different aspects of English teaching and learning) to recognize the washback phenomena by using English questionaires that consisted of three parts. Part 1 discovered the general information of teacher. Part 2 with 5-point Likert scale of agreement discovered teacher’s perceptions and 5-point Likert scale of frequency of Part 3 discovered teacher’s reactions to the new HKCEE through their classroom teaching and learning activities. Regarding the washback of CEFR, Pan and Newfields (2012) worked on discovering how English proficiency graduation requirements have impacted 17 tertiary educational institutions in Taiwan by using extensive questionnaire and interview data [19]. Among them, the survey contained two types of questions: multiple-choice questions with categorical responses and 5-point Likert scale questions with pseudo-ordinal responses. Since 2003, Taiwan’s Ministry of Education (TME) established a list of recommended tests to set English thresholds for graduates to generate a level of English proficiency, which were modified according to the CEFR B1 or A2 levels. They included two local tests: the D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15 5 General English Proficiency Test (GEPT) and the College Student English Proficiency Test (CSEPT). The GEPT was a 5-level, four-skill general English proficiency examination commissioned by TME in 1999. The CSEPT was 2-level, listening-reading-grammar test for university-level students in Taiwan. Accordingly, Pan and Newfields (2012) conducted their study after the inception of English certification graduation requirements in Taiwan; so a comparison of the baseline and a follow-up study to determine the consequences brought about by the tests was not viable. Therefore, a comparison of the differences between the schools with graduation requirements and those without graduation requirements will be used to reveal test effects. In short, this part focuses on some washback studies published between 1993 and 2012. The first part also reviews how these studies have investigated washback. All studies cited here explore different aspects of washback and use various instruments. Alderson and Wall investigated evident of both beneficial and harmful washback on the content of teaching and on ways of assessing, but not on teaching methodology. Alderson and Hamp-Lyons ascertained the influence of the TOEFL on class teaching and TOELF affected both what and how teachers teach, but the effects differed from teacher to teacher. However, the study of Alderson and Hamp-Lyons had three significant limitations. Firstly, they did not include questionaires. Secondly, they choiced participants and lastly, they dealt with washback primarily from perspectives of teachers, hardly addressing students’ opinions. Cheng contributed to the few washback studies by using both quantitative and qualitative methods. Cheng’s study was useful because her study attemped to evaluate the effects of the new examination, however, a longitudinal research with a longer timeframe than the one used by Cheng might shed better light on the influences of the new HKCEE. Pan and Newfields aimed to discover the test effects brought about by graduation requirements in the Taiwanese tertiary educational framework from the perspectives of students. Comparatively little research of Pan and Newfields was conducted regarding the effects of tests on the learning processes, in contrast to the significant number of studies on the effects of tests on teaching. Their study confirmed the argument of other washback studies that standardized tests were not a panacea that always succeeded in changing students’ study habits. Therefore, their study also made it clear that the test requirements did not lead to a notable amount of “studying for the test” a phenomenon often reported in examination- oriented societies. However, the study of Pan and Newfields had three noteworthy limitations. Firstly, the study of Pan and Newfields was conducted at a period when many institutions in Taiwan were eager to adopt the government’s EFL graduation exam policy and thus, washback appeared to be ineffective. Secondly, their study has relied on self-reported student data and thus, such information was easily prone to expectancy bias. Subsequent investigation should include more classroom observational data and seek to corroborate student data with other data sources from teachers and school administrators. This should allow their study to get a more accurate and dynamic picture of how washback patterns are perceived by different test stakeholders. Lastly, one goal of introducing graduation requirements was to improve the ability of graduates to communicate effectively in English in the office that would be very difficult for the researchers to measure, further research should pay more attention to this aspect of washback. All of the reviewed studies have been conducted in primary and secondary schools or tertiary educational institutions in Srilanka, HongKong, Taiwan. The methods were used involved either written questionaires or interview/observations. They found evidence of washback influences on teachers’ behaviours or learning. Accordingly, there has no previous research into washback effects arising from D.T.P. Hoa / VNU Journal of Science: Education Research, Vol. 36, No. 1 (2020) 1-15 6 EPT.2 or standardised tests in Vietnam National University of Art Education. Thus, further research into this area is still needed. Drawing on some washback models and some empirical studies on language teaching or learning activities in classroom of Alderson and Hamp-Lyon (1996), Cheng (1999 and 2004) and Pan and Newfields (2012), this study will be designed to investigate “Washback of English Proficiency Test in Classroom Activities at NUAE”. The study concentrated on an exploration of the effects of washback on teachers/teaching process may offer insights about how VNFLPF and EPT.2 influence language teaching or learning activities in classroom at the educatio