|
PACIFIC ISLANDS FORUM SECRETARIAT PIFS(04)FEDMA.05 EDUCATION MINISTERS MEETING Apia, Samoa 28-29 January 2004 AGENDA ITEM 5 THE ROLE OF ASSESSMENT IN IMPROVING QUALITY IN EDUCATION; THE SHIFT FROM ASSESSMENT OF LEARNING TOWARDS ASSESSMENT FOR LEARNING This paper was prepared for the Forum Secretariat by Dr Visesio Pongi, Director of the South Pacific Board for Educational Assessment. 1 PACIFIC ISLANDS FORUM SECRETARIAT PIFS(04)FEDMA.05 EDUCATION MINISTERS MEETING Apia, Samoa 28-29 January 2004 Summary brief THE ROLE OF ASSESSMENT IN IMPROVING QUALITY IN EDUCATION; THE SHIFT FROM ASSESSMENT OF LEARNING TOWARDS ASSESSMENT FOR LEARNING Purpose The paper addresses the issue of quality of education, which is often addressed at the international or regional level, but tends to be insufficiently addressed at national level. It focuses on how assessment, or the information that assessment produces, can be used to improve student achievement in the education process. Issues 2. There are many aspects to the quality of education. The results of student achievement need to be considered together with information on the context and environment in which the student is learning, in order to provide a more complete picture of factors that influence the quality of education. 3. Student achievement has been considered as a key indicator of educational quality, mainly by using the results of high-stakes examination as a measurement. This is, however, a crude indication of quality. 4. Assessment is used in many parts of the world to identify problem areas in education systems. This approach ultimately leads to improvements in the quality of learning and teaching. Assessment is not an end in itself but a means for improving student learning. This is assessment for learning, rather than assessment of learning. 5. Where selection is the predominant factor in assessment, as in many countries of the Pacific region, it results in a narrow approach to teaching and learning as well as to policy setting. 2 6. While some Pacific countries have attempted to make improvements in their assessment processes, they are often hampered by the over-arching influence of high- stake examinations that act as selection devices to further stages of education. 7. The pressure for selection means that students focus their learning only on those outcomes of the course that get examined, often at the expense of those that do not easily lend themselves to external examinations. 8. Teachers often use past years examination papers to define the curriculum and what should be taught. Thus stereotyped teaching and learning still dominates both primary and secondary schools in the region despite efforts to introduce other forms of assessment such as outcomes-oriented learning. This is partly due to a lack of sufficient training in assessment during teachers’ training. The training of teachers is critical in ensuring that any innovative practices succeed. 9. Since the 1990s, countries in the Pacific region have been seeking alternatives to screening-device type examinations. The alternative methods are those that focus on quality of student performances, such as standards-based assessment, for the ultimate purpose of improving student learning and teachers’ teaching, and hence the quality of education. 10. School-based assessment has increased in many countries in order to broaden the base of assessment, but this is usually part of the high-stakes examination process. 11. Assessment, like curricula, needs to be responsive to student needs, as the purpose is to determine what students can do, as well as identifying their strengths and weaknesses. Besides providing information for sound decision-making about student achievements, a valid and reliable assessment also highlights strengths and weaknesses in the curriculum. 12. School examinations tend to imitate external examinations and research shows that such tests have a negative impact on student motivation. Little has been done to assess the whole range of skills and outcomes anticipated in the curriculum. 13. There is a lack of information on the quality of student learning in the first five years of schooling in most Pacific countries, despite the existence of many primary level examinations. This is because these examinations test student relative performance rather than learning outcomes. There are also few remedial programmes in the region to cater for children who are not achieving expected outcomes. 14. Most of the education systems in the region have not yet put in place strategies for monitoring changes in the standards of education, either at the school or national level, or procedures for identifying areas in the curriculum that are problematic. 15. The challenge facing teachers in schools is making sure that any new assessment strategy takes on board the diverse needs of the range of children attending schools. 3 16. Standards based assessment that is based on defined learning outcomes needs to be introduced at the school level as well as nationally, so that individual teachers and schools as well as national systems can identify and address student weaknesses. 17. It is proposed that each country should develop an assessment framework, in line with its curriculum framework. This strategy emphasises the role of assessment as one of promoting teaching and learning. 18. Curriculum outcomes for each level should be transformed into clearly defined and measurable learning outcomes, which form the basis for assessment. In order to ascertain national standards, specially designed standards-based standardised tests can be developed. 19. This strategy is not proposing an entirely new strategy, rather an attempt to refocus assessment and the way in which it is conducted. The overall objective is to improve student learning by getting teachers more involved and adopting teaching strategies that would enhance effective learning. Recommendation 20. It is recommended that Ministers: (a) Note the contents of this paper; and (b) Consider developing an assessment framework, in line with the national curriculum framework. 4 PACIFIC ISLANDS FORUM SECRETARIAT PIFS(04)FEDMA.05 EDUCATION MINISTERS MEETING Apia, Samoa 28-29 January 2004 THE ROLE OF ASSESSMENT IN IMPROVING QUALITY IN EDUCATION; THE SHIFT FROM ASSESSMENT OF LEARNING TOWARDS ASSESSMENT FOR LEARNING Introduction The 1948 United Nation’s Universal Declaration of Human Rights stipulates that education is a fundamental right and more recently the 1990 Convention on the Right of the Child recognised education as a condition for social development (ADB, 2003). It is only recently, within the last two decades, that education has been recognised as a key sector in economic development through its influence on human resource development. The World Summit on Education for All (Jomtien Framework for Action, 2000), the World Education Forum (Dakar Framework for Action, 2000) and the UN Millennium Summit (Millennium Development Goals, 2000) all recognised the importance of education and the need to improve its quality especially in developing nations. The Forum Islands Education Minister’s Basic Education Action Plan (2001) further emphasised, amongst other things, the need to improve the quality of basic education in all Forum Island Countries. 2. The increasing international commitment to education, as indicated above, is a reflection of the importance of education as a determinant in the welfare of every nation especially in the global effort to reduce poverty. Because of its importance, education takes up a significant proportion of total public expenditure in many of our Forum Island Countries each year (Figure 1) and is often at the centre of policy discussions relating to human resource development. 5 Total Public Expenditure on Education in Some FICs 20 18 16 14 12 10 8 6 4 2 0 % of Total Public Expenditure Cook Is Fiji Nauru PNG Samoa Solomon Tonga Tuvalu Vanuatu Is Country Figure 1: Total Public Expenditure in Education in some FICs (Source: UNESCO Institute of Statistics; for 1999 to 2000 school year) 3. The increasing international commitment to education in recent decades has seen a gradual change in the delivery of education from one that focuses on access to basic education to one where focus is more on improving quality. However, the differing social and economic circumstances among our countries in the region have resulted in the disparity in the delivery of education. While some countries are still grappling with the issue of access to basic education, some have achieved universal basic education and are now focusing on quality improvement. 4. While education authorities in some of our countries are now starting to focus on those quality-related goals of both the Jomtien and Dakar Conventions of 2000, many are not really sure what quality education is let alone how to improve the quality of education currently being delivered. The quality of education is perhaps one of the most talked about issues in education but trying to define precisely “what constitutes quality” is not easy (UNESCO, 2002). Consequently countries tend to adopt their own ideas, and hence their own indicators, for measuring and hence monitoring the quality or changes in the quality of their education programmes. 5. This is perhaps one of the biggest challenges facing education in our part of the world today; what do we consider as quality education? What do we need to do or put in place in order to improve education quality? and how can we monitor changes in the quality of education in our countries over time? While there may not be any straight 6 forward answers to some of the questions raised, they need to be seriously considered and education authorities in each country need to identify and put in place strategies that would help improve the quality of education in their respective country. 6. The intention of this paper is to look at how we can use assessment or the information that our assessments produce as a means of improving one aspect of the education process, and that is student achievements. The notion of quality articulated in this paper is based on the assumption that a system where a higher proportion of students achieve a certain standard (proportion of the learning outcomes in the prescribed curriculum), is deemed to be of a better quality than one where only a few of the students achieve the anticipated outcomes. Because of the complexities associated with the issue of quality, and the economies of scale in our various countries, we should be looking favourably at working in close partnership with each other as well as with the regional and international education communities in our efforts to improve education quality in our respective countries. 7. With the help of international agencies such as UNESCO, ADB, World Bank, etc. we need to identify and agree upon criteria for assessing and/or measuring quality. Indicators for measuring these criteria also needs to be developed if we are to monitor the quality or changes in the quality of the education programmes we deliver to our children. Each country needs to be well aware of the situation in their country and for this, baseline data on the situation for each country in each of the quality criteria needs to be established if any sensible effort is to be in place for monitoring any changes in the quality of education. A word of caution however, that efforts to improve quality are not only costly but also time consuming as it may take years before any indication of improvement in the quality of education can be observed. The Quality of Education in Pacific Island Countries 8. Despite the global concerns about education quality in developing countries such as our region, and the effort by the international and regional educational communities to improve education quality, our region remains as one of the vulnerable regions in as far as the quality of education is concerned. Some of the authors who have been developing the PRIDE initiative (2002) refer to our region as “an economic, social and political time bomb” because a large proportion of our youths are either unemployed or underemployed, thus contributing to incipient poverty including overcrowding, health issues such as too early and too close pregnancies, HIV/AIDS and high risk behaviours, high rates of crime and potential for social instability generally. 9. In a number of the countries in our region, access to primary education is no longer a priority concern as they have achieved universal primary education. However, “lack of appropriate education and training that realistically reflects and links into further training and or employment/self-employment opportunities in the formal or informal sectors” is now the main issue of concern leading to the problem with our youths today. While most of our countries have made great strides in their effort to improve the quality of their education, there is growing concern amongst the regional and international 7 education community that all is not well in our education sectors. Available data show very compellingly that large numbers of male and female school-leavers who have gone through our education system lack basic life skills or possess insufficient skills, knowledge and attitudes to enable them to either progress to further training or to secure a job, or to make a living for themselves. 10. The international community has taken steps to address the problems associated with the quality of education in under-developed and developing nations. During the Conference on Education for All (EFA) in Jomtien, Thailand in 1990, Education was affirmed as key to development. The 155 nations that participated in the conference, including all of the Forum Island Countries, were urged to intensify their efforts to address the basic learning needs of all their citizens. A Framework for Action to meet these commitments was also approved and with support from the Pacific UNESCO Regional Office, all countries in our region made a commitment to work towards achieving the goals of the Education For All (EFA) initiative. Each country has now developed its own EFA Action Plan. Aspects of the EFA goals that were highlighted as relevant to the region include those relating to; quality of formal basic education provision including measurable learning outcomes in literacy, numeracy and life-skill areas; equity and delivery of appropriate basic education programs, with a focus on marginalized/vulnerable and ‘at- risk’ groups; and community/parent education programs. 11. Countries that participated at the World Education Forum held in Dakar, Senegal in 2000 committed themselves to attaining certain goals that would ensure significant improvement in the quality of education. One of these goals relates to the improvement of “all aspects of the quality of education and ensuring excellence of all so that recognised and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills” (ADB, 2003). 12. Of the 8 Millennium Development Goals endorsed by all Forum Island countries in 2000, two relate directly to education and training; § To ensure, that by 2015, children everywhere, boys and girls alike, will be able to complete a full course of primary schooling. § To eliminate gender disparity in primary and secondary education preferably by 2005 and in all levels of education no later than 2015 13. Both these goals reinforce the commitments made previously in the Jomtien EFA Conference in 1990 and the Dakar World Education Forum in 2000. 14. During the first meeting of the Pacific Forum Islands Ministers of Education held in Auckland in May 2001, ministers reaffirmed their commitment to the Dakar 2000 EFA Framework for Action goals and noted the actions taken so far by Forum Island Countries at the national level. A Vision Statement for the Pacific Basic Education Action Plan (BEAP) was adopted and the meeting concluded that basic education is the fundamental building block for economic and social development, and that a mechanism should be in 8 place to ensure that the Pacific Vision and Strategies for Basic Education were actively pursued. 15. With the goal for the BEAP focusing on achieving “universal and equitable educational participation and achievement, and to ensure access and equity and improve quality and outcomes”, ministers identified specific areas that the BEAP needs to consider. These include Education Policies and Planning; Improving Quality in Basic Education; Financing Education; Non-Formal Education; Gender and Equity Issues in Education; Teaching of Governance and Civics; Technical and Vocational Education and Training (TVET); as well as Developing Partnerships. 16. Although countries in the region vary in the patterns of participation and achievement in education vary, many of the issues of concern are common, with differences more a question of scale, priority in a particular context, or the cultural specifics of the context. In spite of the differences, most countries in the region share the common objective of provision of quality relevant basic education, and many of the constraining factors are common; including economic constraints, geographic spread and numbers of school age populations, difficulties in resourcing isolated remote communities especially, teacher shortages and quality of the teacher training experience, mismatch between education outcomes and skill requirements for a diverse range of post school options, etc (UNDP, 1999). 17. In spite of the efforts and commitments by the international and regional communities towards education, there is no doubt that the quality of education in our region is a concern. It is important for the education authority in each country to take the initiative and put in place mechanisms for identifying key problem areas in the education system and then define realistic strategies to address such problems. Participation in regional as well as in international efforts aimed at improving the quality of education would help in identifying the relative extent of the problem. 18. At the regional level, efforts aimed at identifying the literacy and numeracy levels at the end of Years 4 and 6 (PILLS) as well as the various tests constructed by South Pacific Board for Educational Assessment (SPBEA) for the same purpose with some of its member states (Standardised Test of Achievements in Kiribati, Tonga Tuvalu, Solomon Islands and Vanuatu) should be encouraged and extended to other countries. The data collected from such efforts over the years should be thoroughly analysed and results seriously considered with appropriate follow-up actions immediately put in place. 19. Published figures of literacy for Forum Island Countries, such as those shown in Figure 2, are relatively high and often misleading as this may not reflect the true situation in each country. However, the true level of literacy depends on what we mean by literacy. There is a general understanding that despite the high level of literacy published, there is a high level of hidden illiteracy resulting in a significant proportion of children completing school but still lacking the basic literacy skills. 9 Pacific Developing Literacy Rates (%) for 2000 Member Countries (15 years and over) Female Male Overall Rate Cook Islands 94 93 (1998) 94 Fiji Islands 91 95 93 Kiribati 91 94(1998) 93 Marshall Islands 97 97 (1999) 97 FSM 66 77 72 Nauru 95 95 (1998) 95 Papua New Guinea 57 71 64 Samoa 98 99 99 Solomon Is 20 40 30 Tonga 99 99 (1998) 99 Tuvalu 95 95 (1998) 95 Vanuatu 30 37 (1998) 34 (Sources: ADB, 2003, UNDP Human Development Report 1999)) Figure 2: Literacy Rates for 15 years and over for PICs 20. The Pacific Islands Literacy Levels test (PILLs) introduced as part of the Basic Education Life Skills (BELS) project in 1994 showed that the literacy and numeracy situation in many of the countries in our region was not as bright as originally reported. However, such results bear no resemblance on the figures shown in Figure 2. One must be cautious however in drawing conclusions from such results as the two sets of data were based on different definitions of literacy (and numeracy in the case of PILLs). Figure 3 shows the literacy and numeracy situation in some of the countries in the region at the end of Year 4 in primary based on the definition of literacy and numeracy given below. Literacy “is the confident, appropriate and accurate use of spoken and written language for the wide variety of personal, public and creative uses demanded by the society in which the user lives. Numeracy “ is the confident, appropriate and accurate use of number and the language of mathematics for the wide variety of personal and public uses demanded by the society in which the user lives. (PILLs, 1992) 21. A comparison of the literacy and numeracy levels at the end of Year 4 based on the PILL results at the beginning and at the end of the PILLs Project (1994 and 1998/2000) revealed no improvement over the 5/6 year period despite the effort and the resources put in under the BELS programme. Figure 3 gives a picture of the overall situation at the beginning and at the end of the BELS programme but only for those countries that took part in the PILLs project. 22. Figure 3 shows that about a third of the Year 4 students in countries who took part in the test at the end of the year (between 1998 and 2000) hardly achieved any of the basic literacy and numeracy skills expected at the end of Year 4 and were therefore considered 10 to be “at risk” (AR) with less than 10% having acquired most or all of the basic literacy and numeracy expected at the end of Year 4 (L5). More importantly, no improvement was detected over the 4 to 5 year period despite the efforts and resources put in towards improving the situation. If this is any indication of the situation in countries in the region now, then we should be alarmed as it shows that the majority of the primary school students that proceed from one level to another are not yet ready for the challenges at the next level. Comparison of PILL Results at Beginning (1994) and at end of PILL Project (1998/2000) 35 30 25 20 % 15 10 5 0 L5 L4 L3 L2 AR PILL 1994 Level Achieved PILL 1998/2000 Figure 3: Comparison of the overall literacy and numeracy situation for SPBEA member countries at the beginning (1994) and at the end (1998/2000) of the PILLs Project. 23. The experiences with the PILLs test tend to support the general beliefs that many of the students in schools throughout our region go through schools but fail to acquire the basic life skills such as those of literacy and numeracy. Unfortunately such information is not readily available to policy and decision makers, and in situations where such information is available, they are not taken seriously and hence appropriate follow up actions are often not forthcoming. If the results shown by the PILLs test as well as the general beliefs about students in our schools are true, then the issue of student’s readiness for studies at the next level is one that needs urgent consideration. Each country must implement strategies that would address this problem if the quality of learning and hence education in our countries are to improve. 11 Linking Assessment to Education Quality 24. While most of our countries have made huge strides in their effort to provide basic education to the majority of their youths, provision of quality education remains a concern. It is not surprising that the issue of quality in education featured prominently in both the Jomtien Framework for Action (1990), the Dakar Framework for Action (2000) and the Forum Basic Education Action Plan (2001). 25. Although the quality of education in Pacific Island Countries has been on the agenda of international as well as regional educational agencies for sometimes, it is one that has not been given sufficient attention at the national level. This is partly due to the preoccupation of most countries with the issue of access and partly due to a general lack of understanding by national education authorities of what constitutes quality education. While few topics within the education policy circles receive, as much attention as that of quality, defining precisely what constitutes quality is not an easy task. Attempts to use assessment to monitor the quality of education can only be successful if we are sure of what quality is, and its characteristics clearly identified. 26. In the last decade the information needs of the educational community both within the region and internationally have grown. More students with more diverse needs are being educated in our schools for an increasingly complex, demanding world. Public interest and involvement in education has intensified, and with this trend comes the demand for greater accountability in education. 27. The need to know how well our students are doing, how well are our schools and systems doing in achieving the anticipated educational outcomes should be given priority. Results of student achievements need to be considered in tandem with information on the environment in which the student is learning in order to provide a more complete picture of the various factors that influence the quality of education in each of our countries in the region. Educational Indicators need to be identified and agreed upon before information relating to education quality is gathered and sensibly analysed. However this is easier said than done because countries have had less experience with the issue of quality in our education system. Unlike other social and economic indicators such as the consumer price index, the unemployment rate in economics, and mortality rates, etc. which have played a key role in public policy decisions for a long time, their use is widely known and accepted when compared to education quality indicators. 28. While both the international as well as the regional educational communities have shown interests in efforts to measure and monitor education quality in countries in the region, they are aware of the difficulties associated with such initiative. In its effort to define the “characteristics of quality” in primary education, the ADB (2002) included the following components; § teaching methodologies (designed to encourage independent thinking), § teachers (capable, motivated and well trained) § curriculum (appropriate and well designed) § learning materials (effective including textbooks) 12 § learning environment (safe and well maintained) § examination system (valid and reliable) § school leadership including supervision (effective) § direct instructional time ( ample) § financing (adequate) § organizational structure and support (effective) 29. Several other efforts have identified other but more specific indicators such as enrolment, class sizes, resources, student achievements, parent involvement, student socio and economic background, etc. However measuring, let alone monitoring, these various dimensions are highly problematic resulting in the lack of appropriate data being collected. While educators may have kept accounts of indicators relating to system inputs, with elaborate reporting on such factors as enrolments and class sizes, with focus on the resources invested in education, reports based on system inputs are inadequate for well- informed decision-making. This is due primarily to the difficulty associated with collecting the necessary data. 30. The growing concern about the quality of education in our region has led to the attempt to include other measures to describe a detailed profile of the students’ learning context and what society receives for its investment in education. This approach requires a new educational audit initiative that is not only comprehensive but includes inputs, context, processes and results so that educators have up-to-date and accurate data both to identify areas for improvement and to credibly demonstrate system quality and productivity. 31. Student achievement has been considered as a key indicator of educational quality although student performances on large-scale high stake assessments have been the subject of much public scrutiny in many of the countries in the region. Assessment therefore provides the mechanism and platform for measuring and monitoring student achievement over time. However, performances on such assessments can only be interpreted meaningfully in the context of the system that produced them and the purposes accorded to such assessments. To understand and evaluate the overall quality of education in our countries we not only need quantitative measures such as test scores in the high stake assessments, but a more vivid picture of the unique and complex character of the educational system in each country. 32. Many factors influence student learning, some of them outside the control of the school. Contextual features, such as parental involvement in the school, students’ background and community socio-economic status also influence student achievement to varying degrees. Significant factors must not only be identified and measured, but also addressed through educational programs designed to maximize or mitigate their influence. 33. Analysis of the multiple components is therefore necessary to assess the education system and determine the policies and programs needed to improve student learning and educate students for the 21st century. While education specialists often cite improvement of school quality as a necessary component of development strategies for the future, they 13 fail to provide specifics on how to improve or achieve quality. This is perhaps the biggest challenge for education systems in our region, and for the international and regional education communities, to propose initiatives that would enable the overarching issue of education quality to be measured and monitored solely for the purpose of improvement. 34. Many of our countries however experience difficulties in bringing about such major changes on their own given the cost and personnel constraints that such changes require. Any major shift beyond the limited interventions currently undertaken to recast curriculum and teaching at this level, in order to bring about the attainment of Basic Education outcomes for the majority of students, requiring corresponding shifts in curriculum orientation, pedagogy, ways of assessing students, and teacher training, would necessitate external support (PRIDE, 2002). 35. A regional approach such as that anticipated in the PRIDE project would be appropriate and welcomed. What is certain however is that unless such a shift in the focus in education and assessment in our countries takes place, the majority of our young people will continue to leave our education system ill-prepared for what lies ahead. The PRIDE initiative is not only timely but one that would hopefully address the concerns about the quality of basic education in our region. One must be cautious however in trying to introduce strategies to bring about change to ensure that the intentions of the change are achieved. Hargreaves (1997) cautioned that in spite of the knowledge developed on strategies for implementing educational change, many efforts to bring about such change do not often meet expectations because of the multi-dimensional nature of the process. Understanding assessment 36. Changing the way we assess the educational achievement of our students is not an easy undertaking. While there may be good grounds for wanting to change our assessment philosophies and hence assessment approaches, it is often difficult for such changes to be accepted unless the alternative we intend to put in place provides more useful information about student achievements than the current system does. 37. In recent years there have been significant changes not only in the emphasis and focus in the assessment process but also in the way in which assessment information is being interpreted and used. The global shift where assessment is increasingly being perceived as part and parcel of the teaching/learning process has brought about significant changes not only in the way in which the assessment process is conducted but also in how the results of the assessment are being used. 38. The notion of “assessment for learning” (AfL), where emphasis is on using the assessment information to improve student learning, is becoming more prominent and accepted as the way forward in efforts to improve the quality of education. As an information gathering process, any assessment is only as good as the instruments used and how the information gathered is used. An understanding of the process is crucial if the assessment is to have a significant impact in any effort to improve the quality of education. 14 39. Assessment has been used widely in other parts of the world to identify problem areas in education systems. It has helped not only in identifying problems with literacy and numeracy but also in establishing levels of student performances in literacy and numeracy compared to curriculum expectations. Such use of assessment contributes more towards the improvement of the quality of education compared to the current situation where emphasis is on “assessment for ranking” (AFR) based on the relative achievement of students. 40. Decision and policy-makers need to have a clear insight of the assessment process, what it is and what is involved, before the changes and the direction such changes are likely to take are expected to be accepted. More than often, changes recommended are based on economic, social, demographic or political considerations rather than educational ones. Generally speaking a broad approach to assessment is preferred in situations where pressure of selection is not an issue. Where selection becomes the predominant factor in the assessment, one would expect a narrow approach to be adopted (Boyer & Ewell, 1988; McGaw, 1987). 41. With selection still being the dominant factor in the assessment in most, if not all, of the education systems in our region, it is not surprising that most of our countries adopt a very narrow approach and perceptions of assessment. So much so that even in countries and situations where selection is no longer the key factor in the assessment, a narrow approach to assessment is still being adopted. The unfortunate part is that such narrow perceptions of assessment are being reflected in how teachers teach, how students learn and more importantly how policy and decision makers set policies. 42. A challenge for the education systems in our region is to work towards putting in place an assessment system that not only assess the full range of outcomes important to a particular area of students’ learning, but also caters for the diverse and often competing demands of the various stakeholders and users of the information (Masters & Hill, 1988) while maintaining a coherent programme. In emphasizing this challenge, Jones and Bray (1986) assert that; “…the greatest challenge to policy-makers in schools is to achieve coherent whole-school policies while maintaining flexibility; to cater for differing needs of different subjects, differing classes and differing age groups and ability levels, while avoiding fragmentation”. (p.8) 43. While the views expressed above may seem far-fetched in as far as education in our region is concerned, it provides an indication of what the future holds and what any development in the area should try to achieve if the quality of the education is to improve. 44. But before countries start focusing on the possible impact assessment has on the quality of education, they need to be clear of what assessment is so as to fully appreciate the changes that are currently taking place or likely to take place. Stobart (2003) points out that assessments take different forms ranging from large-scale international 15 monitoring assessments to assessments carried out by teachers in the classroom, each having its own purpose and consequences. While data from high stakes assessments, such as public examinations, may be welcomed by policy and decision makers, the responses from students as well as those directly involved in the process may be very different. 45. For far too long, assessment in many of our countries has been narrowly restricted to high stake examinations with screening and ranking as their ultimate purpose. Success rates in such examinations are often measured by the proportion of students selected to the next level of education although acceptance to the next level is often a matter of availability of space and not of achieving the required standard. More than often, the proportion of students accepted to the next level is misleadingly associated with education quality. Such misconception however reflects the narrow emphasis and focus of assessment in many of our countries. 46. Assessment is an important stage in the education cycle and is increasingly being considered as an integral part of the teaching and learning process. It is not an end in itself but is a vehicle or a means for improving student learning and hence the quality of education in the system as a whole (AAHE, 1998). Assessment on the one hand focuses on seeking information (or evidence) about the status of student learning, what they have achieved as well as what they have yet to achieve, and how effective the teaching has been. On the other hand, it is seen as a regulatory process aimed at assuring stakeholders of how well students learn or teachers teach. Figure 4 gives an indication of the changing direction that assessment is currently undergoing, from one that focuses on the assessment of what students have learned towards one that focuses on how the assessment can improve learning. 47. Masters & Hill (1988) agree and assert that it is ongoing and focuses on improving student learning and involves; setting clear expectations (objectives) with appropriate criteria and standards that are closely linked to the expectations, systematically gathering evidence of how well actual student performances (outcomes) match expectations, and then using such evidence to match student performances against expectations. 48. Assessment involves two main processes; that of gathering appropriate information about student learning as well as teachers teaching and how such information can be used to improve both teaching and learning. The quality of the information (or evidence) collected is therefore crucial as that may be the deciding factor in any decision made relating to the educational future of students or in deciding on what strategy needs to be put in place in order to improve on any inherent weakness in the overall system. This means that the evidence must not only be exhaustive but also cover most if not all of the key areas of the curriculum. However, the quality of the evidence one collects is only as good as the instrument put in place for the purpose. It is for this reason that we must ensure that the instrument(s) put in place must provide most, if not all, of the evidence that is needed. But as no single instrument or battery of instruments will totally assess what needs to be assessed, one can only hope to narrow, but probably never completely close the gap between what we would like to assess and the techniques that are available to assess what we want to assess (Jones & Bray, 1986). 16 Teaching Aims Objectives Teaching Assessmen Learning Assessment Objectives t Outcomes Trend Instruction Instruction Assessment of Learning Assessment for Learning Figure 4: Current changes in the assessment process 49. While significant changes have taken place in the area of assessment worldwide, it is only recently that our region has started to come to grips with the implications of these changes. In recent times assessment is increasingly being directed towards improving student learning through improvement to how and what students learn as well as towards improvement to how teachers teach. Such efforts however are being hampered by the over-arching influence of the high-stake examinations that for so long have cast a spell on any initiative that may have led to improvements in quality of learning. 50. Being a goal-oriented process that entails comparing student performances to educational purposes and expectations, attention is given to both outcomes as well as the experiences that lead to those outcomes. Although it is important to try and find out where students eventually end up, it is equally important to try and find out about student’s experiences along the way, the teaching as well as student effort that leads to where they end up. Assessment provides the opportunity for us to understand which students learn best under what conditions, what problems do they have and in what area. Fredricksen & Collins (1989) argue that, “…the goal of assessment has to be, above all, to support the improvement of learning and teaching” (p.32). However such a vision about what assessment can do to improve quality will continue to remain a mirage as long as the spell cast by the high-stake examinations over the education system is allowed to prevail. Looking Ahead in Assessment 51. Some of the assessment practices currently used in our region today have been in place for a long time. It is only recently, over the last two decades, that serious efforts were made to review the various processes used in assessing student performance. Since the early 1990’s countries in the region have become increasingly aware of the desirability to explore other methods of assessing student achievements. The tendency has been to look for alternatives to the high-stakes examinations, which focus almost entirely on screening students for selection. The alternative methods are those that focus on quality of 17 student performances, such as standards-based assessment, for the ultimate purpose of improving student learning and teachers’ teaching, and hence the quality of education. 52. The changes in assessment practices brought about by changes in assessment beliefs and philosophies, as well as the increasing call to improve education quality, has resulted in a broadening of the process. Efforts were made to include areas that would not normally be assessed due to the difficulty associated with including them in the examinations. This change in focus has led to a gradual recession of the one-off high stake public examination and a resurgent of the school-based assessment directed more toward the improvement of the teaching and learning. But realistically these efforts are continuously being undermined by the high-stakes examinations that have been allowed to dominate the assessment. 53. Although the changes currently taking place in assessment are numerous, their intentions are the same, that of improving the process and hence the quality of education provided. One of the key changes has been the change in focus from the teacher and what is taught to the students and what they have achieved (student outcomes) and what have yet to be achieved. This has resulted in the assessment becoming concerned with the individual student strengths and weaknesses in the various tasks in comparison to relative performances. While there is still a need for information about students’ relative performances, there is increasing focus on student’s individual level of achievement. Glaser (1986) predicted that assessments would become more concerned with student competence in the various tasks rather than the tests we are familiar with. 54. Recently both teachers as well as educational professionals support the move towards a broad-based assessment that provides opportunities for the range of interest of students. With the increasing opportunities and pathways now available as well as the diverse interest of students, it is no longer desirable to assume that holding all students within common studies suffice, or to assume that a single standard may become a common goal for all students. Like curriculum, assessments need to be responsive to student needs, after all its ultimate purpose is to determine what students can do and what their strengths and weaknesses are. 55. It is important for the instrument or instruments used to incorporate most, if not all, of the outcomes imbedded in the various areas of study. While the assessment procedures and instruments used may be of no educational value in themselves; they are important in that they provide the means for collecting the evidence about student performance. In this regard, choosing the most appropriate form of assessment to use is therefore a key consideration. 56. This move to broaden the base of the assessment is in line with the need to ensure that the assessment incorporates all of the outcomes considered to be important for the development of students. While such move has been prompted by the failure of the traditional examinations to provide the necessary information, the difficulty associated with collecting alternative information as well as the cost implications has resulted in the continued dominance of the high-stake examinations. 18 Assessments and Education Quality 57. As earlier mentioned, assessment is a data gathering process and is not an end in itself but a means to the end. The process does not directly result in improvement in the quality of education but provides the basis and means for developing strategies that would eventually result in improving the quality of education. It also provides the information upon which decisions relating to any efforts to improve quality can be made. As a process that gathers information directly relating to the performance of students and effectiveness of teaching, assessment has an important role to play in improving the quality of teaching and learning, and hence education as a whole. 58. Besides providing information for sound decision making about student achievements, a valid and reliable assessment also provides information that highlights strengths and/or weaknesses in any areas of the teachers’ teaching or students’ learning as well as any apparent weakness in the curriculum. Such information is crucial in any strategy that the education authority may put in place in its effort to improve the quality of its education programme. As reported in its report on Key Indicators for 2003, the ADB highlights the low quality of schools in its developing member countries as one of the most pressing problems. 59. Student achievements as well as curriculum are key areas of education and assessment has an important role to play in efforts aimed at improving education quality. Through its role in assessing not only the standard and level of student achievements but also the effectiveness of the teaching as well as the curriculum in place, assessment provides the information necessary for monitoring how changes to any of the inputs in the education system affect student performances and achievements. The current efforts to measure and monitor literacy and numeracy standards at certain levels in primary education in some of our countries are indicative of the crucial role assessment has in the monitoring of standards and hence the changes in the quality of education. Key Issues for Consideration 60. Some of the key issues in the assessment process that have significant impact on the quality of teaching and learning, and hence the quality of education in both primary and secondary throughout the region are outlined below for discussion. While the main focus on these issues is confined to the nature of the assessment practices in place, the influences of such practices in other areas of education need to be clearly understood Emphasis on assessment for screening 61. Assessing the achievement of students in primary and secondary schools in most, if not all, of the countries in our region is still very much dominated by one-off high-stake external examinations. While these examinations have played an important role in the development of our various education systems over the years, their primary role has always been, and still remain one of screening to determine those students to proceed to 19 the next level. But with limited places available at the next level, the examinations and hence the screening process became so competitive that success in these examinations became the primary focus in the teaching and learning throughout primary and secondary. 62. Consequently teachers in both primary and secondary adopt end of term or end of year examinations that closely mirror the external examinations at the end. In such an environment, emphasis in both the teaching and learning is on improving students’ rank position, thus improving their chances of being selected. What the students can or cannot do or what skills have they acquired are compromised with the push for better ranking. 63. Studies carried out by experts throughout the world on the impact of assessment on student learning are well documented. A review by Harlen & Crick (2003) on the impact of testing as well as other forms of summative assessments on students’ motivation for learning found that tests have negative impacts on student motivation. In their review on the impact of high stake examinations on students, Madaus and Clarke (1999) concluded that; § high-stakes high-standards tests do not have a markedly positive effect on teaching and learning. § high stakes tests do not motivate unmotivated students § authentic forms of high–stakes assessment are not a more equitable way to assess progress of students who differ in race, culture, native language or culture. § high-stakes tests increase dropout rates, particularly among minority student population. (Madaus & Clarke, pp 172-173) 64. Although efforts to improve the quality of the assessment have been in place for many years, they have so far been restricted to improvement of the assessment instrument. Invariably this has resulted in the assessment focusing only on those skills that can easily be assessed by the instrument used while little has been done to assess the whole range of skills and outcomes anticipated in the curriculum. This raises questions about the validity of the assessment currently in place in most countries. 65. The worldwide trend to transfer some of the responsibilities for the assessment of student achievements to teachers has yet to achieve its purpose despite the efforts by regional organisations such as USP and SPBEA. Not only do countries have a long way to go, but they also have a lot to learn from the experiences of countries that have faced the same concerns about the quality of their education system in recent years. Lack of Information on Quality of Learning 66. While there is a lot of similarity in the assessment framework of many of our countries in the region, a “one that fits all” solution may not necessarily be the best approach. As indicated in Figure 5, most countries have national examinations conducted at the end of either Year 6 for most countries or Year 8 in some countries such as Kiribati, Tuvalu and Samoa. These end of primary examinations serve a predominantly selection 20 function although the extent to which the selection is carried out varies from country to country. Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Fiji FIE FEYE Nauru Y6E Y7E Kiribati STAKI 1 SEE F2E Tonga STAT 1 STAT 2 STAT 3 Tokelau Tuvalu Y8E Samoa SPELL Y8E SEE Solomon Is Y6E Vanuatu In place On trial Under Development To be Developed Figure 5: Primary Assessment Framework in some PICs 67. An obvious feature of the assessment framework in many of our countries in our region is the limited information available on the quality of student learning throughout the first five years of primary schooling. This is a reflection of the current perception of many of the countries towards assessment as being a process of ranking students to facilitate selection. Such an environment has deprived the education authority in most countries of the crucial information relating to the quality and standard of teaching and learning, especially in the crucial years of schooling, pre-school and early primary years. 21 68. Studies carried out in other developing countries such as the Latin American countries in the late 20th century show that “it was misleading to assume that the success of the best university graduates…” (Schiefelbein & Schiefelbein, 2003) studying in universities in developed countries indicates good quality. Further studies have also shown that availability of quality assessment information helps in identifying appropriate strategies for improving quality. Using information from standardised tests designed to identify problem on key areas of the curriculum such as reading, writing and numeracy has helped identify teaching strategies and initiatives for improvement. The link between assessment and better learning needs to be widely and better understood so that the assessment information can be effectively used to make informed decisions. 69. The lack of information regarding the standard of education throughout the first years of education (early childhood and primary), together with the social promotions policy adopted by many of the countries throughout the region, has resulted in the prevailing situation where many of the teachers are unaware of the specific weaknesses and strengths of students particularly in relation to the expectations of the curriculum. This has also resulted in the absence of any established strategy to monitor student achievements throughout primary education. This is due partly to the heavy emphasis on students’ relative performance. Consequently many of the students completing primary education in schools throughout the region fail to achieve many of the outcomes specified in the curriculum. 70. Experiences from the BELS programme suggest that the system that has been in place over the years in most of our countries in the region need to change if any significant improvement in the quality of education, in both primary and secondary, is to improve. One such change is to refocus both the teaching and learning on achieving the outcomes stipulated in the curriculum rather than on students’ relative performance and ranking on the high-stake examinations. 71. Remedial programmes need to be in place for those that the assessment identifies as yet to achieve the curriculum outcomes (slow learners). At the same time enrichment programmes need to be put in place for those that the assessment identifies as having achieved the outcomes (fast learners). This would undoubtedly result in an overall improvement in the quality of learning and hence education. However, this can only be put in place if information relating to the level of achievement of each student on each of the outcomes is made available to teachers. Teaching for the Examination 72. The adoption of high stake examinations at the end of either Year 6, or Year 8 in primary and end of Form 6 or 7 in secondary has over the years resulted in the current environment where ranking in such examinations dominates the teaching and learning. The pressure for selection has over the years directed the focus in the teaching, not only throughout primary but also in secondary education. Every effort is made so students achieve good marks in the selection examination rather than achieving the curriculum outcomes. Consequently, students focus their learning only on those outcomes of the 22 course that get examined often at the expense of those that do not easily lend themselves to the external examinations. 73. In an environment where teachers are accountable for students’ test scores and their ranking instead of the effectiveness of teaching, teachers spend a great deal of their time and effort in preparing the students for the examinations (Pollard et al, 2000). It is becoming a common practice in many classrooms throughout the region for teachers to administer practice tests that not only take away valuable teaching time simply to confirm to the high achievers that they are good and to the low achievers that they are failures. Moreover, teachers actively coach students to pass the tests rather than helping them to understand what is being tested (Leonard & Davey, 2001). According to Gordon and Reeves (1997) students can be coached and pass the test eventhough they may not have learned and understood the concepts on which they are being tested. 74. Johnston and McClune (2000) found that where external examinations are predominant, they have constricting effects on the curriculum and on teaching methods. Even when teachers are not directly teaching to the tests, they often change their approach and adjust their teaching in ways they perceived as necessary because of the tests. They even spend more time in their teaching on direct instruction and less time for students to learn through enquiry, research and problem solving. 75. Tests often lead to a “measurement-driven instruction” where teachers use past years examination papers to define the curriculum and what should be taught, paying particular attention not just to the content but also to the form of the test. Such an environment encourages teachers to become more performance-centered thus forcing students to adjust and learn only those they believe are valued and will gain teacher’s approval. 76. There is evidence to suggest that teachers influence students to focus more on learning processes (Perry, 1998). However, students are not likely to adjust their learning as long as teacher assessments and teaching methods still implicitly, and explicitly in some cases reflect performance goals. A study by Roderick and Engel (2001) found that more of the slow learners would not give up on themselves if teachers focus more on task and learning-centered goals and using the assessment to help them succeed rather than using the assessment to confirm their failure. 77. Examinations cannot improve learning, nor can they necessarily improve the quality of education. However they provide information that could be used to identify possible problem areas as well as strategies for addressing such problems. Tests such as standardized achievement tests have helped in identifying problems with literacy and numeracy in some of the countries in the region. 78. While the high-stake examinations have been successful in producing the required national ranking of students, which facilitates the selection process, it has at the same time shifted the focus in both the teaching and learning from the real issue of achieving the curriculum outcomes towards achieving better ranking in the selection examination. Even 23 the teaching at levels where external examinations do not exist the teaching and learning are still directed more towards improving students’ skills in answering examination-type questions rather than acquiring the skills specified in the various curriculum outcomes at the end of each level. 79. The stereotyped teaching, and learning to a lesser extent, still dominates both the teaching and learning throughout primary and secondary education despite efforts to introduce other forms of assessment. The heavy emphasis of the assessment on relative performance continues to encourage rote learning of examinable areas and non-coverage of non-examinable outcomes. Teachers tend to depend to a large extent on the information from teacher-designed tests and examinations without due regard to the fact that a large number of the learning outcomes do not easily lend themselves to such assessments. To provide valid and reliable assessment of student learning requires teachers to put in place an assessment plan that not only covers the range of student learning outcomes but also uses the most appropriate styles of assessment. Such a plan should focus on collecting evidence of the extent to which students have attained the specific learning outcomes within the various learning areas or strands in the curriculum. 80. In spite of what has been said about the effect of high-stake assessment on learning, teachers are still expected to continue to prepare their students for such assessment, whether or not students are learning much in the process or not. For as long as there are limited places available at the upper level, examinations will continue to be used as selection devices (Kellaghan & Greaney, 1992). Student Readiness and Social Promotions 81. One of the common policies adopted in the education systems throughout the region is one where, because of limited spaces available as well as other factors, students are allowed to move on to the next level with no regard to their readiness to the challenges of the next level. This is commonly referred to as the policy of “social promotion”. This means that students are allowed to proceed to the next level not because they are ready but because of the need to make space to those coming up. With little attempt to monitor or to determine whether they have achieved the learning outcomes set out in the curriculum for each level, it would be possible for students to progress up the ladder and complete either primary or secondary education without having acquired any of the basic life skills such as reading or writing. 82. The lack of monitoring means that teachers at each level may not have information on which of the curriculum outcomes for each level has each student achieved. Likewise, teachers at the next level have little or no information on what each student can or cannot do. More than often teachers assume that all students moving up to a new level have achieved the same standard in each of the outcomes. Teachers therefore target the ideal student in their teaching rather than the specific weaknesses of students. 83. While social promotions is a common policy in both primary and secondary schools in our region, its influence in the quality of education is far greater than what one 24 would normally expect. For those students who have achieved most if not all of the anticipated curriculum outcomes for a level, social promotion is necessary. For those who have yet to acquire the minimum competency, especially in literacy and numeracy, social promotion becomes problematic as such students will be expected to overcome their shortcomings from the previous level as well as having to face the challenges of the new level. Thus for such students their problem is compounded. The key concern therefore becomes one of matching what students have achieved and what the curriculum expects them to achieve at the end of each level. 84. In most situations teachers do not have access to specific information relating to what students can or cannot do. This is the main problem with social promotions where students who are not yet ready for the challenges of the next level are allowed to move on to the next level. The problem is compounded for such students especially in stair-casing situations where the outcomes at each level build on those of the previous level. 85. Most primary and secondary school teachers in our region either do not have the expertise to be able to collect the kinds of information that would enable them to determine which students have achieved which outcomes or they do not have the time to gather such information because of their preoccupation with preparing students to the high-stake assessments. Consequently the majority of teachers adopt end of topic or end of term tests to collect relative data about student performance rather than standards-based data that they need. Teachers of schools in the 21st century need to be well aware of what information they need and how to obtain such specific information so they are in a position to decide on how best to assist the students. 86. The adoption of the social promotions policy and its consequences should be seriously considered given its impact on the overall quality of education. However, the influence of other factors such as teacher commitment and teacher expertise also need to be considered as they all contribute to the current situation. A compromise needs to be reached where students’ readiness to proceed to the next level as well as the availability of resources are taken into consideration. Promoting only those students who have been assessed to be ready or who have achieved the minimum competency standard required to proceed to the next level, with no consideration for the resources available, may create further problems. Effective record keeping on the part of teachers is crucial to enable them to get a clear picture of student’s achievements and how best to help address their weaknesses. Lack of monitoring of standards 87. Most of the education systems in our region have not yet put in place any strategy for monitoring changes in the standards of education, both at the school and at the national level let alone strategies for addressing areas of learning, teaching or curriculum that have been identified as being problematic. This is partly due to each country being preoccupied with the pressing issue of access in primary and lower secondary. Lack of personnel with the appropriate expertise in assessment also contributes to the failure in the education systems to redirect focus in education towards improving the quality of education. 25 88. While some of the countries in the region are starting to seriously consider the quality of their education programmes, particularly those in primary, focus is still very much on access. The level of access to primary and secondary education varies considerably from country to country, with access figures in some countries such as Fiji, Nauru, Tonga and Samoa over 90% while figures for other countries such as Kiribati, Solomon Islands and Vanuatu are considerably lower. Figure 6 shows the proportion of children of primary school age who are actually in school. Member Country Gross Primary Gross Secondary Enrolment Enrolment Fiji 90 36 Kiribati 77 44 Nauru 96 34 Tonga 90 67 Tuvalu 88 74 Tokelau na na Samoa 94 70 Solomon Is 39 24 Vanuatu 72 44 Average 80 49 Sources: Pacific Human Development Report 1999; - EFA 2000 Report; UNDP HDR Report 2002 Figure 6: Gross enrolment for primary and secondary in Board member countries 89. Over the years, emphasis on access in primary education, compared to outcomes, has seen little effort by countries to monitor the quality and standard of education both in primary and in secondary. Results of students’ performance in the high-stakes examinations are often used to indicate the standard of the education system and quality of schools. Consequently most of the children who leave school at the end of primary, Year 6 in some countries and Year 8 in others, have been proven to possess insufficient skills to be able to cope with the challenges ahead. 90. The lack of monitoring of individual student achievements at the school level and at the national level suggest that teachers often do not have full knowledge of the capabilities of each of their students. What can each student do or cannot do in terms of the various learning outcomes? But with increasing commitment by the regional and international education communities, countries are becoming aware of the shortcomings and are starting to make arrangements to address some of these issues. 91. Lack of focus of the teaching and learning on learning outcomes due to the lack of appropriate information, coupled with the social promotions policy, means that even students who have yet to achieve the learning outcomes are given little extra assistance. Instead they are expected to be able to keep up with those who have been assessed to be ready, thus compounding their problems. 26 92. If the quality of education in our region is to improve, the education authority in each country needs to redirect their focus in assessment towards using assessment to improve the teaching and learning and put in place strategies that would ensure the achievement of the various learning outcomes enshrined in the various key learning areas of the curriculum. At the primary school level the achievement of the literacy and numeracy outcomes are crucial because of their impact on other areas of learning. Studies in other parts of the world have established the close link between literacy and numeracy and student performances in other disciplines. Elley (1992) agrees that poor reading and writing skills become effective constraints for learning in other subjects. Lack of corrective (remedial) programmes 93. Considering the high proportion of students who fail to achieve the learning outcomes prescribed by the curriculum for each level, but are allowed to proceed to the next level because of the social promotions policy, one would expect recovery or remedial programmes to be an important feature of the programme at each level in both primary and secondary. Surprisingly it is left up to the individual school to decide whether such programmes or strategies are necessary. But with the rush to cover the crowded curriculum and the pressure imposed by the high-stake assessments at the end, little effort has been made to put in place strategies for implementing corrective measures for those having problems with some of the outcomes. 94. While the majority of students who start primary do complete six years of education in most of our countries, many struggle to achieve the learning outcomes prescribed in the curriculum. It would be interesting to find out the gap that exists at each level between what students can do and what the curriculum expects. 95. Experiences from the PILLs tests (1994 – 1998) have found that it is not uncommon for students to be at Year 6 for example, but still struggling to achieve some of the learning outcomes prescribed for Years 4 and 5. With no formal arrangement for any corrective measures in place, the problem is compounded each year as students move up to a point where they find that they can no longer cope with the demands of the curriculum. The situation can only be resolved if teachers are in a position to identify what each student can or cannot do and put in place strategies for addressing the weaknesses. Having access to such information, teachers would then be in a position to design remedial programmes for non-achievers as well as enrichment programmes for first-time achievers. 96. Figure 7 gives a possible strategy that could be considered for adoption as it provides a corrective pathway for those that fail to achieve the learning outcome in the first instance as well as a pathway for those that achieve the learning outcome in the first instance. The question that needs to be considered is how many times would a student be allowed to pursue the corrective pathway before they are allowed to proceed to the next one. 27 Learning Outcome to be assessed Identify Specific Outcome to be assessed Design appropriate assessment tasks Proceed to next outcome Carry out corrective instructions Administer assessment tasks No Yes Specific Outcome achieved? All specific outcomes achieved Determine next learning outcome Figure 7: A proposed strategy for assessing achievement of students in the classroom situation 28 Teachers’ assessment expertise 97. One of the key issues that has contributed to the persistence of stereotype teaching and learning in both primary and secondary schools in the region has been teachers’ lack of knowledge and expertise especially in the use of assessment to gather information that would help improve their teaching. A perusal of the syllabus of some of the key teacher training institutions in the region reveal the need for more in depth coverage of assessment especially when one considers the important role assessment plays in the teaching and learning process. Yet when student teachers complete their training, they are expected to be able to carry out valid and reliable assessment of their students’ achievements. It is not surprising therefore for teachers in both primary and secondary not to have the appropriate skills and expertise to be able to carry out a proper assessment of their students’ achievements. 98. Teachers’ lack of assessment expertise has continued resulted in the continued misconception of assessment being synonymous with examinations/tests. Consequently, they consider teacher-designed tests and examinations particularly end of term/year examinations, as well as external examinations, as the only form of assessment that can provide valid and reliable information about their students’ achievements. Without the knowledge and skills in good assessment practices teachers will continue to use pen and paper tests and examinations as the only form of assessment without realising that they are only assessing a minor proportion of students’ achievements. 99. Before teachers can be expected to change the way they conduct their assessment, and hence improve the way they teach, they need specific training to enable them to use the multiplicity of assessment methods and techniques available. The question of choosing the assessment method that is “fit for the purpose” only becomes an issue if teachers are able to choose and use the method that is most appropriate. Despite the ongoing debate on the use of assessment information either to improve learning or for selection, the fact remains that teachers need to be trained so they acquire assessment skills even if only to allow them to choose the most appropriate method for the task. 100. Gathering data about students’ relative performances in a particular learning area requires a different method, a norm-referenced instrument, than gathering data about students’ level of achievement in the various outcomes within the same learning area, standards-based instruments. Without the proper knowledge and training most teachers would continue to use norm-referenced instruments to serve both the purposes mentioned above. After all, an assessment is only as good as the instrument one uses, and how the data gathered are being used and for what purpose. 101. The main issue however is not one of knowing the various types of assessment but one of having the assessment expertise to be able to use the assessment information appropriately. Gipps (1996) argues that with proper training teachers can use assessment for selection and accountability to improve learning provided that teachers are well trained to use the data from one assessment to serve the purpose of another. One must not be misled however into believing that introducing a new assessment innovation would 29 automatically improve learning and hence education quality. It is the struggle to make such innovation work and achieve its purpose (Sebatane, 1998). Sheppard agrees with the concern about the expertise of teachers and argues that all effort must be made to support and train teachers. After all, the success of any innovation aimed at improving the quality of learning would depend to a large extent on the ability of teachers to implement such innovation. “If teachers are being asked to make fundamental changes in what they teach and how they teach it, then they need sustained support to try out new practices, learn new theories, and make it their own” (Sheppard, 1995). Rigid Structure of Education System 102. The structure of the education system in most countries in the region has been in place for quite some time, 6 to 8 years of primary followed by 4 to 6 years of secondary. The curriculum for both primary and secondary is then organised around years with a 6- year curriculum for primary and 6-year curriculum for secondary. Few countries have other arrangements. Invariably the curriculum lays out clear objectives, or learning outcomes in some cases, that students are expected to achieve at each level or year. This means that it is possible for both teachers and students to be made aware early in the process of the learning outcomes students are expected to achieve at each level. 103. Because of the limited places available, as well as the nature of the assessment adopted throughout the school system, with its heavy emphasis on assessment for ranking and selection, teachers are not given the opportunity to focus the assessment on identifying the level achieved by each student in each outcome. Instead, students are given a mark or grade for each subjects and at times labelled as either FAIL or PASS although such labels are often based on the arbitrary magic mark of 50. Ranking is often based on an aggregate of the marks or grades awarded for the various subjects taken and acceptance to the next level of education is often considered as the PASS standard. 104. Unfortunately the rigid structure of the education system in many countries does not allow students who have partially achieved all of the learning outcomes prescribed for each subject at any given level to proceed to the next level. Where selection is necessary, students are required to achieve a specific aggregate arbitrary score, often based on the number of places available at the next level, before they are allowed to proceed further. Those who do not make the arbitrary score either drop out, for some of those at the upper end of secondary, or repeat all the subjects again at the same level, for many of those taking high-stake examinations such as those at the end of primary or end of secondary. 105. The rigidity of the structure of the education system in many countries, compounded by the highly selective assessment system in place, has resulted in an increasing proportion of students, especially at the exit levels, opting to repeat the whole years programme in the hope of improving their performance and hence their chances of being selected. While students, parents, and even the education authorities see repetition as giving students a second chance, education experts see repetition as a “priority problem” that is linked to weaknesses in the education system (Schiefelbein & Schiefelbein, 2003). Experts suggest that the high level of repetition is linked to the low 30 quality of education and all efforts should be made to address the situation as it indicates the failure of the system to cater for the needs of those opting to repeat. Figure 8 shows the total number of repeaters at PSSC award (end of Form 6) from 1999 to 2003. Number of Repeaters at PSSC from 1999 to 2003, by gender 500 450 400 350 300 250 200 Number 150 100 50 Boys 0 Girls 1999 2000 2001 2002 2003 Total Year Figure 8: Number of repeaters for PSSC from 1999 to 2003 by gender. 106. The inflexibility of the curriculum and the rigid structure of the education system have to some extent contributed to the current situation where the majority of the students at any level fail to achieve the level of performances the curriculum expect. With the social promotions policy in place the performance of students on key basic skills such as literacy and numeracy could only be assessed if test instruments are put in place for such purpose. The system also does not allow for students to be assessed at the level of their achievements although they could be at another level (multi-level study). 107. Introducing multi-level studying to schools poses a far greater challenge than one would expect. There is a need to carefully consider the implications of such an initiative as it has both resource and personnel implications. Not only would it require teachers to be skilled in multi-level teaching as well as assessing students on outcomes that span more than one level, it would also require careful timetabling so that it would be possible for students attending classes at one level to be assessed on outcomes from courses offered at other levels. On the bright side, introducing multi-level study especially at those levels with high stake assessments would allow those who would repeat all their subjects because they fail to be selected to the next level, to improve their performance on those subjects they fail to perform well in while pursuing those subjects they perform well in at the next level. For such an initiative to be possible the rigid structure of the education system, especially the level structure of the school system, needs to be relaxed. As well, the curriculum needs to be restructured around key learning areas or strands thus making it possible for students to be assessed on those outcomes prescribed for each strand. Why would a Year 4 student have to wait a year before he/she can be assessed in the Year 5 31 outcomes of a given strand if he/she has shown that he has achieved all of those outcomes prescribed for Year 4? Curriculum and teaching mismatch 108. While variations are evident in the curriculum of Pacific Island countries, there are similarities not only in their content but also in their structure. In most countries, the curriculum invariably has statements that indicate what students are expected to be able to achieve at the end of each level. These statements or outcomes portray the skills and expected level of mastery or achievement in each area. 109. The outcome statements provide guidelines that teachers could use as the basis for their teaching while the over arching influence of high stake examinations in later years often distort the teaching from focusing on the curriculum outcomes to improving skills in answering examination-type questions. This has resulted in the mismatch that currently exists in what teachers emphasise in their teaching and what the curriculum expects. 110. Because teachers are not targeting the curriculum outcomes in their teaching and also in the assessment they carry out, it means that students are not focusing their learning on achieving the outcomes to the extent anticipated in the curriculum. Information represented by the graphs in Figure 4 supports this claim and indicates that while teachers may be working hard in teaching their students, only a small proportion do actually achieve at the level expected in the curriculum. This mismatch between the teaching and what the curriculum expects is a major issue that needs to be carefully considered. What is needed is for teachers to focus their teaching on getting students to achieve as many of the prescribed outcomes as possible. 111. The idea of focusing the teaching as well as the assessment on the outcomes prescribed in the curriculum is one that teachers need to be in-serviced in, as it requires the teacher to refocus their teaching. At the same time, teachers are expected to be knowledgeable of the various standard of performance and capable of developing assessment instruments that would determine the level of performance of each student in any of the prescribed outcome. Challenges for the future 112. In recent years there has been considerable discussions worldwide on the issue of good assessment practices and what system should be put in place to provide the information that education authorities need. Outcome of these discussions has seen a redirection in the focus in assessment from students’ relative performance towards a broad-based assessment with emphasis on the specific outcomes each student has achieved. This redirection in assessment has enabled teachers to find out more about their students’ learning as well as their specific areas of strengths and weaknesses. These place teachers in a better position to provide specific assistance to students. 32 113. Here are some of the challenges that the education authority in each country may wish to consider in its endeavour to improve the quality of education it provides. Changing the Emphasis in Assessment 114. One of the key challenges facing education in countries throughout the region relates to the current focus of the assessment process. In many of the countries in the region, the main focus of the assessment, especially the high-stake assessments, is one of ranking. Persistence of the assessment with ranking will continue to deprive stakeholders of the crucial information relating to the standard achieved by individual students, the school, the province and the country as a whole. 115. The education authority in each country needs to put in place strategies that would promote “assessment for learning” instead of “assessment for ranking” if the quality of education is to improve. For as long as the focus of the assessment is on ranking in high stake assessments, teachers will continue to focus their teaching on maximising student ranking in the assessment, students will continue to focus their learning on what is likely to be included in the examination and parents as well as stakeholders will focus their interests on the relative ranking of students. It is only then that teachers will have access to the information they need in order to provide the badly needed assistance students need. At the same time the education authority will also have access to the information they need in order to put in place strategies that would help address the common problems facing the system. 116. It is time that countries should try to look beyond the performance of students in the high-stake assessments and their relative ranking in such assessments and focus more on the level of achievement of students relative to the learning outcomes stipulated in the curriculum. While there would still be a role for high-stake examinations, because of the limited available places at the next level and hence the need for selection, their influences on the curriculum, the teaching and student learning should be seriously considered. All effort should be made to minimise the influence of these high-stake examinations so that teachers could focus more on the important issue of improving teaching and students concentrate on achieving the ranging of learning outcomes in the curriculum. Improving Teachers’ Assessment Expertise 117. With the re-focusing of the assessment on the improvement of teaching and learning, it is anticipated that with intensive in-service training on good classroom assessment techniques, teachers would be able to design simple assessment tasks that would provide the diagnostic information they need in order to improve their teaching. Specially developed instruments that focus on the outcomes of the key areas of the curriculum in any chosen level or levels can also be designed to provide the baseline information needed by the education authority, or the school authority, to monitor the overall standard of performance of the school on any given domain(s) over time. Such information would assist the school or the national education authority in deployment of resources and the review of the curriculum, etc. 33 118. A perusal of the curriculum of teacher training institutions in the region, as well as experiences with teachers in schools support the observation that teachers, both primary and secondary, throughout the region are not well prepared to face the challenges facing our schools in the 21st century. With training institutions focusing more on broadening the knowledge of student teachers rather than their teaching expertise, it is not surprising that teachers focus their teaching on what has been in place over the years. 119. To ensure that teachers focus their teaching on improving student learning, the education authority needs to ensure that serving teachers are in-serviced not only on good teaching techniques but also on good classroom assessment techniques so they would be able diagnose students’ learning and hence provide the appropriate assistance students need. Teacher training institutions should also review its programme to ensure that good classroom assessment practices are part and parcel of their curriculum. With the important role assessment plays in the teaching and learning processes, and the focus on assessment for learning, teachers in our schools today are not only expected to be competent teachers but also competent assessors. 120. The expectation of teachers today, both in terms of the pedagogy and assessment strategy, far exceed their level of expertise and training resulting in most teachers reverting to the method that they are most comfortable with. In terms of assessment, teachers tend to over use classroom tests as the only means of gathering information about the level of achievement of students often at the expense of the plethora of more appropriate methods available. To ensure that the assessment achieves its purpose, it is important that all teachers are able to identify the overarching outcomes prescribed in the curriculum and also to unpack such outcomes into measurable learning outcomes. 121. Teachers need to acquire the appropriate assessment expertise before they are able to carry out the assessment responsibilities expected of them. Although teachers are expected to focus their teaching on the various learning outcomes, it would only be possible if they were able to identify such outcomes. This task would be made much easier if the learning outcomes are clearly and explicitly stated, thus making the task of matching students’ actual performance with expected outcomes far easier. Broadening the scope of the assessment 122. Most curriculum documents either explicitly or implicitly state the range of outcomes students are expected to achieve. With the current emphasis of the assessment on high-stake examinations that facilitate ranking and hence selection, only those outcomes that can be examined get assessed. This means that under the current focus on “assessment for ranking” many of the important learning outcomes do not get assessed. Consequently no information is available on the achievement of students on the non- academic but often more important skill areas such as practical skills. 123. If this situation prevails, the assessment process will continue to focus only on a small portion of what students have achieved, that is, only on achievements in the 34 cognitive area. Achievements in other areas such as the psychomotor and affective domains remain outside the boundary of the domain areas being assessed. This means that student achievements in areas that are important to life in the community but do not lend themselves to the high-stake examinations often do not get assessed. 124. To be able to provide a true indication of the level of achievement of students in the range of learning outcomes outlined in the curriculum, education authorities should consider reviewing their assessment framework to allow student achievements in the range of learning outcomes to be included in the assessment. This would mean putting in place an assessment framework that would give greater recognition of teachers’ classroom assessments as well as adopting other more appropriate assessment methods such as performance-based assessment. 125. Failure to put in place a broad-based assessment framework that takes into consideration student achievements in all of the outcomes stipulated in the curriculum would continue to promote the current system that not only focuses on a small part of student achievements but also encourages a narrow focus in the teaching and learning process. Such a situation would not only continue to undermine any effort to improve the quality of teaching and learning but also continue to encourage a narrow focus in what teachers teach as well as what students learn Reviewing the curriculum and Assessment Frameworks 126. The rigid structure of the curriculum, organised into levels or years, coupled with an assessment system that focuses on ranking, has over the years resulted in the current system where students are allowed to progress to the next level with little consideration given to their level of achievement. In a level where a high-stake assessment is in place, performance in the examinations provides the basis for progressing to the next level. Students not achieving at the required standard to enable them to proceed to the school of their choice often find themselves having to repeat the whole programme with the hope of improving their chances of being selected. 127. The education authority in each country needs to seriously consider reviewing both the curriculum and the assessment framework to allow for students who have achieved the required level in some of the learning areas to proceed to the next level while working towards achieving the required level in other areas of learning at another year, that is, multi-levelling. Multi-levelling not only removes the need for students to repeat the same programme but it would also encourage those students who would not normally get selected to stay on in school. 128. The assessment framework in each country needs to be in a position to be able to measure the standards in key areas of the education system such as literacy and numeracy to be measured and for the quality of education to be monitored. This may require putting in place instruments specially designed to gather information relating to the standard of achievement of students and hence the quality of education. 35 Reporting on the achievement of the whole child Sample reporting of standards monitoring Overall Multiplication Achievement Levels for Boys and Girls at end of Year 6 45 40 35 30 25 % 20 15 10 5 0 L5 L4 L3 L2 L1 Girls Achievement Levels Boys Achievement Levels Level 1 Cannot carry out multiplication of 3 or 4-igit numbers with a 2-digit number for both whole and decimal numbers. Level 2 Can multiply only up to 3-digit numbers with 2-digit numbers for whole numbers correctly most times, but cannot multiply decimal numbers. Level 3 Can multiply up to 4-digit numbers with 2-digit numbers for whole numbers correctly most times but have problems with decimal numbers. Level 4 Consistently multiply up to 4-digit numbers with 2-digit numbers correctly for whole numbers correctly but may have problems with decimal numbers especially the placing of decimal points. Level 5 Consistently multiply up to 4-digit numbers with 2-digit numbers correctly for both whole numbers and decimal numbers correctly Figure 8: Sample reporting strategy of standard monitoring of a learning outcome (multiplication) in a Year 6 mathematics curriculum 129. While the policy of allowing students to proceed to the next level, with little consideration given to their readiness to proceed, education authorities need to put in place an assessment strategy that allows for the achievement of students in the curriculum outcomes specified for each level to be measured before they are allowed to proceed. This is important especially in literacy and numeracy because of their influence in the other areas of study. Teachers need to refocus their teaching on the anticipated learning 36 outcomes for each level instead of couching students for the examinations at the end. At the same time students achievement in the various outcomes need to be clearly reported so that stakeholders are fully aware of the weaknesses and strengths of students. 130. Reporting of the level of achievement of students in any area of study needs to indicate the standard achieved by students. An example of a possible reporting strategy for a given learning area or strand in Year 6 mathematics course is given in Figure 8. 131. Reporting of the level of achievement of students in any area of study needs to indicate the standard achieved by students. An example of a possible reporting strategy for a given learning area or strand in Year 6 mathematics course is given in Figure 8. Conclusion 132. While concerns about the quality of education in countries throughout our region are not new, perhaps the time has come for us to step back and take a serious look at the direction our education systems are taking. We need to be able to identify the range of skills and values that our societies today expect of our children so that we can redirect the focus and emphasis in our education systems to ensure that the majority of our children leave school ready for what lies ahead. The continued focus of our education systems on assessment for ranking will only benefit the few who find themselves at the top of the rank. But for the majority, they consider themselves as failures and while at the same time lack the basic skills life in the society require. 133. To improve the quality of the education that countries in the region offer their children, education authorities need to look beyond the marks, or grades, and students’ relative performance and focus more on the achievement of the whole child. Education authorities need to be confident that what is on offer would provide the range of basic life skills that children need in order to become successful members in the society they live in. Only then that we can focus on determining the achievements of students in the both the academic and non-academic skills that make up the curriculum. It would then become meaningful to talk about the quality of education and consider ways and means of monitoring quality of our education over time. Recommendation 134. It is recommended that Ministers: (a) Note the contents of this paper; and (b) Consider developing an assessment framework, in line with the national curriculum framework. 37 Bibliography ADB (2002), The Quality of Education, Dimensions and Strategies. In Education in Developing Asia, Vol.5, Manila; ADP ADB (2003), Asian Development Outlook 2003. Special Chapter, “Competitiveness in Developing Asia”, Manila; ADP Boyer, C.M & Ewell, P.T (1988), Acting out state-mandated assessment; Evidence from five states. Change, 20, 41-47. Fredricksen, J.R. & Collins, A. (1989), A system approach to educational testing, Educational Researcher, 18(9), 27-32. Gipps, C. (1996), Assessment for Learning, in A. Little & A. Wolf (Eds) Assessment in Transition:Learning, Monitoring and Selection in International Perspective (Oxford, Pergamon Press) Glaser, R. (1986), The integration of instruction and testing. In ETS (1986) The redesign of testing for the 21st century. Princeton,: Educational Testing Service. Harlen, W. & Crick, R.D. (2003), Testing and motivation for learning, Assessment in Education, 10(2), 169-207. Hargreaves, A. (1997). Introduction; Pushing the Boundaries of Educational Change. In A. Hargreaves et al (eds). International Handbook of Educational Change. I.(pp. 281-294). Dordrecht:Kluwer. Madaus, G & Clarke, M (1999), The adverse impact of high stakes testing on minority students; Evidence from 100 years of test data. High stakes K-12 Testing Conference, Harvard University. Masters, G.N & Hill, P.W.(1988), Reforming the assessment of student achievement in senior secondary school. Australian Journal of Education. 32(3), 274-286. McGaw, B. (1987), Overview of issues in school assessment. In NZ/ACER, The best of SET assessment. Victoria; NZ/ACER. Jones, R.L & Bray, E. (1986), Assessment; Form principles to action, Guides to assessment in assessment. London MacMillan. PINZ (2002), PRIDE, Pacific Regional Initiatives For the Delivery of Basic Education, Project Design Document, December 2002 38 Schiefelbein, E & Schiefelbein, P (2003), From Screening to Improving Quality; the case of Latin America, Assessment in Education, 10 (2), 141-159. Sebatane, E.M. (1998), Assessment and Classroom Learning: A Response to Black & William, Assessment in Education, 5(1), 1998. Sheppard, L.A. (1995), Using Assessment to improve learning: Educational Leadership, 52(5), pp.38-43. Stobart, G. (2003) Editorial. The Impact of assessment: Intended and Unintended Consequences; Assessment in Education, Vol. 10, No. 2, July 2003. UNDP (1999), Pacific Development Report Withers, G. (1992), Pacific Islands Literacy Levels; A criterion-referenced study. UNESCO Office for the Pacific States, 1992. 39
© 2006, USP Library. Copyright & Disclaimer
Contact Us
|