Students

APPL924 – Language Testing and Evaluation

2014 – S2 Day

General Information

Download as PDF
Unit convenor and teaching staff Unit convenor and teaching staff Convener and Lecturer
Mehdi Riazi
Contact via email
C5A 575
by email
Margaret Wood
Credit points Credit points
4
Prerequisites Prerequisites
Admission to MAppLing in (TESOL or LSP) or PGDipAppLing in TESOL or MTransInterMAppLing or MCommDis or MTransIntPed
Corequisites Corequisites
Co-badged status Co-badged status
2014 S2 External
Unit description Unit description
This unit explains key concepts and issues in language testing and provides students with principles and techniques for designing and evaluating language tests and assessment tasks. Functions of language tests, models of language proficiency, and assessment of language skills and components are discussed in this unit. The unit also focuses on other issues such as the reliability and validity of language tests in social contexts.

Important Academic Dates

Information about important academic dates including deadlines for withdrawing from units are available at https://www.mq.edu.au/study/calendar-of-dates

Learning Outcomes

On successful completion of this unit, you will be able to:

  • • Define and describe key terms and concepts in language testing and assessment
  • • Identify a range of purposes for assessment in language learning programs
  • Apply knowledge of language assessment to the construction of tests and assessment tasks
  • • Appraise issues and challenges in assessing language skills and components
  • • Demonstrate familiarity with current issues and debates concerning the nature of language proficiency and how it is assessed, and
  • • Identify key features of quality assessment and apply them to language tests and assessment tasks in use
  • • Evaluate test usefulness using available frameworks

General Assessment Information

Assignment Submission

Please Note: The assignment submission process has changed.

 

All assignments must be uploaded in iLearn on due dates. You need to login into iLearn and upload your assignment file there.

 

Please do NOT submit the assignment in hardcopy.

 

For each assignment, you must:

 

  • submit the assignment in Word or RTF format
  • fill in and include the cover sheet provided on the Linguistics web site. Follow the links from:  http://www.ling.mq.edu.au/support/coversheet.htm
  • add a footer to each page of the assignment, with page numbering, your name and student number, and “APPL924” clearly marked
  • type double-spaced
  • use the appropriate file name for the attachment (see below).

 

Assignments will not be accepted unless they have both the cover sheet and the footer. Please paste the coversheet in as the first page of your assignment – do not send it as a separate file. You do not need to print and scan the coversheet, just fill in your name and details on the electronic copy.

 

Assignments submitted electronically will be marked and returned electronically.

 

Please submit assignments one time only unless specifically requested.

 

Subject headers

When uploading assignments, please use the subject headings:

  • APPL924 ass1
  • APPL924 ass2
  • APPL924 ass3

 

a) Unless students have negotiated an extension based on documented evidence of significant disruption to their studies, a penalty of 5% per day will apply to late submission of assignments. 

b) Again, unless otherwise negotiated, assignments will not be accepted at all AFTER the date on which the marked assignments are returned to all students in the unit. 

Note that you must keep a copy of each assignment as proof that it was completed and submitted, just in case the submitted assignment is misplaced or damaged.

4d. Return of Marked Assignments

Students’ marked assignments will, in general, be returned to them within 2 to 4 weeks of submission. Please note that assignments cannot be accepted after the return of marked materials. Marked assignments may be sent to you by email to your students@mq address or, they may be returned through iLearn system.

Assessment Tasks

Name Weighting Due
Task 1: Annotated bibliography 20% week 4
Task 2: Review Paper 30% Week 8
Task 3: Test Review 40% Week 13
Active Participation 10% Semester wide

Task 1: Annotated bibliography

Due: week 4
Weighting: 20%

Choose one of the four language skills (listening, reading, speaking, writing), and prepare an Annotated Bibliography on the challenges of testing the skill. The challenge may be related to different aspects of assessing the skill from the construct definition (how the skill is defined), using appropriate test materials, setting criteria for assessment (e.g, rating scales, e.g.) to the scoring and interpretation of performance. You may choose to focus on one of these themes or a combination of them, but please be advised that the main focus of the assignment is on “challenges”. 


On successful completion you will be able to:
  • • Appraise issues and challenges in assessing language skills and components

Task 2: Review Paper

Due: Week 8
Weighting: 30%

This assignment requires you to change your AB (Assignment 1) into a review paper. Use your AB (the introduction, the annotations and the reflections, and the conclusion) and write an essay to discuss and highlight the major problems and challenges of assessing the communication skill you have chosen. The AB should provide you with enough content to write the review paper, however, you may want to do further research and add more content if necessary to enhance your review paper. Attend to the comments you received on your AB to improve your second assignment. The structure of the review paper will follow an introduction, body and conclusion. For further elaboration on each section of the paper refer to your unit outline. 


On successful completion you will be able to:
  • • Define and describe key terms and concepts in language testing and assessment
  • • Appraise issues and challenges in assessing language skills and components
  • • Demonstrate familiarity with current issues and debates concerning the nature of language proficiency and how it is assessed, and
  • • Identify key features of quality assessment and apply them to language tests and assessment tasks in use

Task 3: Test Review

Due: Week 13
Weighting: 40%

This is an extension of Assignment 2 and involves an evaluative commentary on a language test of your choice. You may wish to look at a well-known standardised language proficiency test (such as the IELTS or TOEFL) or you may want to examine a test that is used in the context in which you work.

You may choose to evaluate either the whole test or one module of it, such as the reading or the speaking module. If you choose to evaluate a large-scale test you will probably find that the latter option is more manageable. Use the six categories in Bachman and Palmer (1996) test usefulness framework (authenticity, reliability, validity, practicality, interactiveness, and impact) as a way of organizing your evaluation and review of the test or the test module. You may choose to include all the six features or you may choose some of them depending on the scope and focus of your evaluation.

You will need to search for the related literature (evaluation of the tests or their components), read them carefully and include relevant resources in your evaluative commentary.  


On successful completion you will be able to:
  • • Define and describe key terms and concepts in language testing and assessment
  • • Demonstrate familiarity with current issues and debates concerning the nature of language proficiency and how it is assessed, and
  • • Identify key features of quality assessment and apply them to language tests and assessment tasks in use
  • • Evaluate test usefulness using available frameworks

Active Participation

Due: Semester wide
Weighting: 10%

Both internal (on-campus) and external (distance) students should take part in teaching and learning activities throughout the semester. These will include attending lectures (for internal students), and participation in face-to-face in-class, and online discussion. Use “Forum” facility in iLearn and post your comments and discuss different topics related to the unit. You are encouraged to post a brief report of the papers you find and read on each of the unit topics. Groups may be formed of those who share similar areas of interest to further discuss their topics and exchange resources. Also, use the “Glossary” part to define (briefly) and describe research concepts and terms. This will help you to consolidate your understanding of key terms and will give others a chance to read and understand those concepts. There is also a Wiki where you can post more lengthy definitions as well as useful sources you come across.


On successful completion you will be able to:
  • • Identify a range of purposes for assessment in language learning programs
  • Apply knowledge of language assessment to the construction of tests and assessment tasks
  • • Identify key features of quality assessment and apply them to language tests and assessment tasks in use

Delivery and Resources

Delivery and Resources

3a. Teaching and Learning Strategies

This unit is offered both face-to-face and online. The face-to-face mode will include weekly lectures and on-campus students must attend classes. Online mode covers the same topics as listed in the unit schedule and distance students must access and read unit materials and perform learning activities from folders in iLearn. Both internal, on-campus, and external students are encouraged to participate in online discussion of the topics and post their reflections in general “Forum”. They are also urged to use “Glossary” to post their concise definition of key terms and concepts related to language testing and assessment.

3c. Required and Recommended Texts and/or Materials

The required textbook for this unit is:

Hughes, A. (2008) (7th printing). Testing for language teachers. Cambridge: Cambridge University Press.

This book provides a general introduction to the field of language testing and is aimed at language teaching practitioners.

In addition to the required textbook, some journal articles and book chapters related to the topics covered in the unit are recommended.

As a Macquarie University student you are entitled to access a variety of sources including journal articles through the library’s page: http://www.mq.edu.au/on_campus/library/

Recommended other texts:

References with asterisks will be available on e-reserve and seven day loan.

Alderson, J. C. (2000). Assessing reading. Cambridge: CUP. (Chapter 3)

*Bachman, L. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press. (Chapter 3)

Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press. (Chapter 2)

Brown, H.D. (2004). Language assessment: Principles and classroom practices. London: Longman.

Brown, J.D. (2005). Testing in language programs. Upper Saddle River, NJ: Prentice Hall Regents.

Fulcher, G., & Davidson, F. (2007). Language testing and assessment: An advanced resource book. New York: Routledge.

Gottlieb, M., & Nguyen, D. (2007). Assessment and accountability in language education programs. Philadelphia, PA: Caslon Publishing.

Henning, G. (1987). A guide to language testing. Cambridge, MA: Newbury House.

Luoma, S. (2004). Assessing speaking. Cambridge: CUP. (Chapter 7)

*McNamara, T. (2000). Language testing. Oxford: Oxford University Press. (Chapter 7)

*McNamar, T., & Roever, C. (2006). Language testing: The social dimension. Malden, MA: Blackwell. (Chapter 2)

*Weigle, S. C. (2002). Assessing writing. Cambridge: CUP. (Chapter 5)

Recommended articles:

*Brindley, G. (1998). Assessing listening abilities. Annual Review of Applied Linguistics, 18: 171-191.

*Brindley, G. (2013). Task-based language testing. In C.A. Chapelle (ed.), The encyclopaedia of applied linguistics. Blackwell Publishing Ltd. Accessible online from MQ online library.

Brown, J.D. (2004). Performance assessment. Second Language Studies, 22(2), 91-139.

Celce-Murcia, M., Dornyei, Z., & Thurrell, S. (1995). Communicative competence: A pedagogically motivated model with content specifications. Issues in Applied Linguistics, 6(2), 5-35.

*Davidson, F. (2013). Test specification. In C.A. Chapelle (ed.), The encyclopaedia of applied linguistics. Blackwell Publishing Ltd. Accessible online from MQ online library.

*Ginther, A. (2013). Assessment of speaking. In C.A. Chapelle (ed.), The encyclopaedia of applied linguistics. Blackwell Publishing Ltd. Accessible online from MQ online library.

*Ham-Lyons, L. (2000). Social, professional and individual responsibility in language testing. System, 28, 579-591.

*Koda, K. (2013). Assessment of reading. In C.A. Chapelle (ed.), The encyclopaedia of applied linguistics. Blackwell Publishing Ltd. Accessible online from MQ online library.

Language Testing (2002) (Special Issue). Task-based language assessment.

*Ockey, G.J. (2013). Assessment of listening. In C.A. Chapelle (ed.), The encyclopaedia of applied linguistics. Blackwell Publishing Ltd. Accessible online from MQ online library.

*Weigle, S. (2013). Assessment of writing. In C.A. Chapelle (ed.), The encyclopaedia of applied linguistics. Blackwell Publishing Ltd. Accessible online from MQ online library.

Unit Schedule

Week

Date

Topic

Lecturer  

Readings

Required

Additional

1

 

07/08/14

 

Introduction to the unit (Review of the unit outline & assessment tasks) + Participants’ backgrounds & interests

Mehdi Riazi

Hughes (2008) Chapter 1

 

--

 

2

 

14/08/14

 

 

Why testing? Purposes of language tests

Mehdi Riazi

Hughes (2008) Chapters 3 (pp. 11-19)

 

 

Bachman (1990) Chapter 3 (pp. 54-72)

 

3

 

21/08/14

Principles of good practice in test development

Mehdi Riazi

EALTA Principles of good practice+ ETS Guidelines for best test development

 

 

--

4

 

28/08/14

What to test & how to test? Designing a language test: Stages in test construction + Test specifications

Mehdi Riazi

 

Hughes (2008) Chapter 7

 

 

*Davidson (2013)

5

 

04/09/14

 

Language testing and assessment in social contexts

Stephen Moore

McNamara (2000) Chapter 7

Hamp-Lyons (2000)

 

 

McNamara & Roever (2006) Chapter 2

6

 

11/09/14

 

Language testing and assessment in social contexts

Stephen Moore

McNamara (2000) Chapter 7

Hamp-Lyons (2000)

McNamara & Roever (2006) Chapter 2

7

 

18/09/14

Task-based Language Assessment (TBLA)

Stephen Moore

Language Testing (2002). The special issue on TBLA

*Brindley (2013)

Mid-semester break Sept. 20th – Oct. 6th

8

 

09/10/14

 

Interpreting test scores: Norm-referenced vs. Criterion-referenced tests

 

Mehdi Riazi

Hughes (2008) Chapters 3 (pp. 19-25)

 

 

Bachman (1990) Chapter 3 (pp. 72-80)

 

 

9

 

16/10/14

Assessing Listening

 

Mehdi Riazi

 

Hughes (2008) Chapter 12

*Brindley (1998)

*Ockey (2013)

 

10

 

23/10/14

 

Assessing Reading

 

Mehdi Riazi

 

Hughes (2008) Chapter 11

 

 

*Koda (2013)

11

 

30/10/14

Assessing Speaking

 

Mehdi Riazi

 

Hughes (2008) Chapter 10

 

 

*Ginther (2013)

12

 

06/11/14

Assessing Writing

Mehdi Riazi

 

Hughes (2008) Chapter 9

 

*Weigle (2002) Chapter 5

*Weigle (2013)

13

 

13/11/14

 

Quality features of language tests: Test usefulness framework

 

Mehdi Riazi

 

Hughes (2008) Chapters 4 & 5

Bachman & Palmer (1996) Chapter 2

Policies and Procedures

Macquarie University policies and procedures are accessible from Policy Central. Students should be aware of the following policies in particular with regard to Learning and Teaching:

Academic Honesty Policy http://mq.edu.au/policy/docs/academic_honesty/policy.html

Assessment Policy  http://mq.edu.au/policy/docs/assessment/policy.html

Grading Policy http://mq.edu.au/policy/docs/grading/policy.html

Grade Appeal Policy http://mq.edu.au/policy/docs/gradeappeal/policy.html

Grievance Management Policy http://mq.edu.au/policy/docs/grievance_management/policy.html

Disruption to Studies Policy http://www.mq.edu.au/policy/docs/disruption_studies/policy.html The Disruption to Studies Policy is effective from March 3 2014 and replaces the Special Consideration Policy.

In addition, a number of other policies can be found in the Learning and Teaching Category of Policy Central.

Student Code of Conduct

Macquarie University students have a responsibility to be familiar with the Student Code of Conduct: https://students.mq.edu.au/support/student_conduct/

Student Support

Macquarie University provides a range of support services for students. For details, visit http://students.mq.edu.au/support/

Learning Skills

Learning Skills (mq.edu.au/learningskills) provides academic writing resources and study strategies to improve your marks and take control of your study.

Student Services and Support

Students with a disability are encouraged to contact the Disability Service who can provide appropriate help with any issues that arise during their studies.

Student Enquiries

For all student enquiries, visit Student Connect at ask.mq.edu.au

IT Help

For help with University computer systems and technology, visit http://informatics.mq.edu.au/help/

When using the University's IT, you must adhere to the Acceptable Use Policy. The policy applies to all who connect to the MQ network including students.

Graduate Capabilities

PG - Discipline Knowledge and Skills

Our postgraduates will be able to demonstrate a significantly enhanced depth and breadth of knowledge, scholarly understanding, and specific subject content knowledge in their chosen fields.

This graduate capability is supported by:

Learning outcomes

  • • Define and describe key terms and concepts in language testing and assessment
  • • Identify a range of purposes for assessment in language learning programs
  • Apply knowledge of language assessment to the construction of tests and assessment tasks
  • • Appraise issues and challenges in assessing language skills and components
  • • Demonstrate familiarity with current issues and debates concerning the nature of language proficiency and how it is assessed, and
  • • Identify key features of quality assessment and apply them to language tests and assessment tasks in use
  • • Evaluate test usefulness using available frameworks

Assessment tasks

  • Task 1: Annotated bibliography
  • Task 2: Review Paper
  • Task 3: Test Review
  • Active Participation

PG - Critical, Analytical and Integrative Thinking

Our postgraduates will be capable of utilising and reflecting on prior knowledge and experience, of applying higher level critical thinking skills, and of integrating and synthesising learning and knowledge from a range of sources and environments. A characteristic of this form of thinking is the generation of new, professionally oriented knowledge through personal or group-based critique of practice and theory.

This graduate capability is supported by:

Learning outcomes

  • Apply knowledge of language assessment to the construction of tests and assessment tasks
  • • Appraise issues and challenges in assessing language skills and components
  • • Identify key features of quality assessment and apply them to language tests and assessment tasks in use
  • • Evaluate test usefulness using available frameworks

Assessment tasks

  • Task 1: Annotated bibliography
  • Task 2: Review Paper
  • Task 3: Test Review

PG - Research and Problem Solving Capability

Our postgraduates will be capable of systematic enquiry; able to use research skills to create new knowledge that can be applied to real world issues, or contribute to a field of study or practice to enhance society. They will be capable of creative questioning, problem finding and problem solving.

This graduate capability is supported by:

Learning outcomes

  • • Identify a range of purposes for assessment in language learning programs
  • • Evaluate test usefulness using available frameworks

Assessment tasks

  • Task 1: Annotated bibliography
  • Task 2: Review Paper
  • Task 3: Test Review

PG - Effective Communication

Our postgraduates will be able to communicate effectively and convey their views to different social, cultural, and professional audiences. They will be able to use a variety of technologically supported media to communicate with empathy using a range of written, spoken or visual formats.

This graduate capability is supported by:

Learning outcomes

  • Apply knowledge of language assessment to the construction of tests and assessment tasks
  • • Appraise issues and challenges in assessing language skills and components
  • • Demonstrate familiarity with current issues and debates concerning the nature of language proficiency and how it is assessed, and
  • • Evaluate test usefulness using available frameworks

Assessment tasks

  • Task 1: Annotated bibliography
  • Task 2: Review Paper
  • Task 3: Test Review
  • Active Participation

PG - Engaged and Responsible, Active and Ethical Citizens

Our postgraduates will be ethically aware and capable of confident transformative action in relation to their professional responsibilities and the wider community. They will have a sense of connectedness with others and country and have a sense of mutual obligation. They will be able to appreciate the impact of their professional roles for social justice and inclusion related to national and global issues

This graduate capability is supported by:

Learning outcomes

  • Apply knowledge of language assessment to the construction of tests and assessment tasks
  • • Appraise issues and challenges in assessing language skills and components
  • • Identify key features of quality assessment and apply them to language tests and assessment tasks in use
  • • Evaluate test usefulness using available frameworks

Assessment tasks

  • Task 1: Annotated bibliography
  • Task 2: Review Paper
  • Task 3: Test Review
  • Active Participation

PG - Capable of Professional and Personal Judgment and Initiative

Our postgraduates will demonstrate a high standard of discernment and common sense in their professional and personal judgment. They will have the ability to make informed choices and decisions that reflect both the nature of their professional work and their personal perspectives.

This graduate capability is supported by:

Learning outcomes

  • Apply knowledge of language assessment to the construction of tests and assessment tasks
  • • Appraise issues and challenges in assessing language skills and components
  • • Demonstrate familiarity with current issues and debates concerning the nature of language proficiency and how it is assessed, and
  • • Identify key features of quality assessment and apply them to language tests and assessment tasks in use
  • • Evaluate test usefulness using available frameworks

Assessment tasks

  • Task 1: Annotated bibliography
  • Task 2: Review Paper
  • Task 3: Test Review

Scoring Policy

5. POLICIES

5a. University Policy on Grading

Academic Senate has a set of guidelines on the distribution of grades across the range from fail to high distinction.  Your final result will include one of these grades plus a standardised numerical grade (SNG). 

The following descriptions apply to assessment grades:

HD  High Distinction  85-100: denotes performance which meets all unit objectives in such an exceptional way and with such marked excellence that it deserves the highest level of recognition.  

D  Distinction  75-84: denotes performance which clearly deserves a very high level of recognition as an excellent achievement in the unit.  

Cr  Credit  65-74: denotes performance which is substantially better than would normally be expected of competent students in the unit.

 

P  Pass  50-64: denotes performance which satisfies unit objectives.  

F  Fail  0-49: denotes that a candidate has failed to complete a unit satisfactorily.

Please note that

  1. The above grades and descriptions apply to undergraduate degrees, postgraduate coursework degrees, diploma and graduate certificate programs
  2. Marks assigned in assignments are raw marks. These are subsequently converted into Standard Numerical Grades (SNG) in order to determine letter grades for the unit.

On occasion the SNG which you receive may not be the same as the total of your marks for each assessment item. Under the Senate guidelines, results may be scaled to ensure that there is a degree of comparability across the university, so that units with the same past performances of their students should achieve similar results.

The process of scaling does not change the order of marks among students.

For an explanation of the policy see -

http://www.mq.edu.au/policy/docs/grading/policy.html

http://senate.mq.edu.au/rules/detailedguidelines.doc

5b. Plagiarism

Plagiarism is considered to be extremely serious by the Department and the University. Students who submit plagiarised material will most likely receive zero. All assignments are scanned on a database and compared with other assignments (past and present) and with content on the internet.

Please refer to the Linguistics Department’s policy on plagiarism at http://www.ling.mq.edu.au/support/policies_plagiarism.htm  and to Macquarie University’s policy at http://www.mq.edu.au/policy/docs/academic_honesty/policy.html

It is expected that you will be familiar with the department’s policy before you begin work on the assignments.

As a good student, you are responsible for ensuring academic integrity practices are followed at all times. Your first step is to read the University's Academic Honesty Policy, and make sure you know what constitutes good practice. Then make sure you know how to reference and cite correctly. There are other practices we need to consider, and one of these is the potential for collusion.

Informal study groups are encouraged as a good way to assist your learning, but please remember that all your independently assessed assignments must be totally independently completed. Unless you are doing a group project where each member contributes to producing one piece of work, for which you get the one mark, using part or all of someone else's work constitutes collusion and breaches the University's Academic Honesty policy.

Do not collude with any other student by selling, giving, lending, explaining or showing all or parts of your independently assessed work/answers/past or current assignments, and do not ask to buy, borrow, see and use all or parts of the work of another student.