Students

SOC 818 – Evaluation and the Policy Process

2015 – S2 Evening

General Information

Download as PDF
Unit convenor and teaching staff Unit convenor and teaching staff Unit Convenor
Markus Hadler
Contact via markus.hadler@mq.edu.au
Instructor
Charlotte Overgaard
Credit points Credit points
4
Prerequisites Prerequisites
Admission to MPASR or PGDipPASR or PGCertPASR or GradCertPASR or GradDipPASR or MAppAnth or PGDipAppAnth or PGCertAppAnth or MPP or PGDipPP or PGCertPP or MIntPubDip
Corequisites Corequisites
Co-badged status Co-badged status
Unit description Unit description
This unit introduces students to policy evaluation and accounts for its place in the policy process and social sciences respectively. Students will acquire a knowledge of different approaches to evaluation as well as the ethical and political issues surrounding this process. The unit also focuses on how trends informing public sector management have an influence on the evaluation process and the actual implementation of evaluations.

Important Academic Dates

Information about important academic dates including deadlines for withdrawing from units are available at https://www.mq.edu.au/study/calendar-of-dates

Learning Outcomes

On successful completion of this unit, you will be able to:

  • A practical introduction to evaluation methods & policy
  • Insight into theoretical accounts about evaluation and policy
  • Professionally useful skills in writing evaluation reports
  • Professionally useful data analysis skills

Assessment Tasks

Name Weighting Due
Participation 30% Ongoing
Key Issues Summary 30% Week 6
Evaluation Proposal 40% Week 13

Participation

Due: Ongoing
Weighting: 30%

This assignment consists of three separate tasks which are meant to encourage active participation and engagement with the course material.

10%: Attendance 

Your grade is directly related to your presence in class. If you are unable to attend, you need to contact the convenor prior to class and submit documentation. You, however, can miss two units without providing documentation.

10%: Being prepared + active participation in class

This part of the assignment is designed to award those students who come prepared and who actively engage in class the course material and with each other.

10%: Leading the discussion

This part of the assignment requires students to engage in more detail with one of the readings during the course. Students will nominate themselves one week in advance of presentation. The task requires each student to prepare maximum 3 slides, and maximum 3 questions for the group to engage with (or alternatively a group exercise). The presentation and discussion should take maximum 20 minutes. This assignment will be discussed in more detail in week 1.


On successful completion you will be able to:
  • Insight into theoretical accounts about evaluation and policy
  • Professionally useful data analysis skills

Key Issues Summary

Due: Week 6
Weighting: 30%

Word limit: 1,500 words                 

Due date: 31 August 2015 (Monday of week 6)

Assignment value: 30%

Write a concise review of the literature presented in the required readings for weeks 1-6. Your review should identify key themes in the literature and relate hose themes to your own learning objectives and fields of interest or practice.

The purpose of the assessment task is to ensure that you understand the key principles of and discussions relating to evaluations. It is essential to identify you own learning objectives for this course in this assignment. You are required to demonstrate a critical understanding of the readings and the issues they raise. It is intended to be an exercise that provides you with a clear overview of the field and the issues and discussions that take place. It is also intended to make you read all the course material.

The reference list is not included in the word limit.

Grade

%

 

Fail

≤49

Failed to complete all aspects of the assigned task. Fail papers do not succeed in demonstrating an understanding of key issues or will not relate those issues to the student’s learning objectives or field of interest or practice. If you fail this assignment, you should consult with unit convenor (Charlotte)

Pass

50-64

This is the standard required to pass this assignment. To achieve a pass you must write clearly and coherently. You must demonstrate an understanding of the issues discussed in the literature and their relevance to your learning objectives. You must provide accurate referencing.

Credit

65-74

Credit range assignments must fulfil the requirement for a pass grade. Additionally, it must provide a convincing integration of the literature rather than a series of loosely related review of individual papers.

Distinction

75-84

In addition to the above, assignments must demonstrate in-depth thought and reflection on the way that the key issues in the literature relate to evaluation practices. Distinction grade papers must demonstrate very strong communication skills, including an ability to deal with complexity in engaging and effective ways.

High Distinction

≥85

In addition to above, assignments will demonstrate exceptional understanding and engagement with the material. It will demonstrate capacity for deep personal reflection and clarity about personal learning objectives in the unit. High Distinction grade assignments must show exceptional communication skills, analytical understanding self-awareness.

 


On successful completion you will be able to:
  • Insight into theoretical accounts about evaluation and policy

Evaluation Proposal

Due: Week 13
Weighting: 40%

Word limit: 2,000 words     

Due date: 9 November 2015 (Monday of week 14)     

Assignment value: 40%

This research assignment requires you to develop an evaluation proposal for a program.

The purpose of this assignment is to make you able to enter the field (relatively) confident that you are able to undertake at least small scale evaluations. This assignment is designed to give you those skills.

In week 8, we will meet with Andrew Ellery, who is a Project Officer for the Department of Family and Community Services. Andrew presents some current projects/programs. This assignment is to write an evaluation proposal for one of those (so don’t miss this week). The assignment will be explained in more detail following Andrew’s visit.

There are two components to this assignment:

10%: A preliminary presentation/consultation with Andrew and MQ staff:

In week 13, Andrew will again visit us. This is your chance to engage and consult with Andrew, Charlotte and Markus. You will be able to present up to 6 A4 pages with your preliminary outline/considerations.

30%: The proposal:

You should organise the proposal so that it includes the elements outlined in the table below.

Note: The sections shaded grey on Table 1 (summary, research timetable, budget, personnel, appendices) normally feature in research proposals and grant applications, but due to the word limit for this task they do not have to be included here (and will not count toward your task). References (which are compulsory) do not have to count toward your word limit. Also, please make sure that you make linkages between the different sections of this report where appropriate.

A marking rubric will be provided later in the course.

 

Table 1: Evaluation Proposal

Cover letter

You should include a cover letter and address the evaluation proposal to Andrew Ellery

Title

The title should indicate the topic of the study in 1- 15 words

 

Summary

100 to 200 word summary of the entire project

Background/

Significance

 

This section should outline the program that you will be evaluating in the proposed research and indicate why evaluation is needed. For instance, are there controversies about the program or the social issue it responds to? Why is there a need for further information about this program? When outlining the program, you should present the program logic in diagrammatic form and explain how the key elements of the program relate to each other. When indicating why the evaluation is needed, you should discuss the program, the social issue that it responds to and what the evaluation will contribute to our knowledge.

Literature Review

A concise summary of the main references relevant to evaluating the selected program. Make sure that you review the findings of other studies and discuss their methodology. At the end of this review, draw out the important messages for the evaluation and the methodology in particular. It is imperative that multiple sources with a range of authors are consulted to allow for a diversity of views to be explored and identification of controversies about the program (or similar types of programs). Please do not take any sources at face value – be critical but reasonable.

Aims and

Objectives

Make it clear who the evaluation is being commissioned by. Identify the study’s main objective and the key research questions that will guide the evaluation. After identifying the main objective, it might help to consider whether you need information on the effectiveness, appropriateness, and efficiency and so on of the program.

Approach

Discuss the suggested approach and justify why you have chosen that approach. It is good practice to discuss (briefly) the merits of the main alternative approaches. Use references if appropriate. What methods will you use? What will be the outcome of using these methods? Which stakeholders will be involved? It is imperative that you explain the links between the evaluation question(s), the evaluation approach, the methods and how these will be operationalised in this particular case.

Ethical and

Political Issues

Identify any specific issues arising from the research. Pay careful attention also to generic

ethical issues in research with human subjects – Respect for Persons: Informed Consent; confidentiality; possible risks for subjects and any possible benefits. State whether Ethics approval has, or will be sought, and if so, from whom. Also, consider the political dimensions of this report – are there controversies about the program or social issue? Are there any conflicts of interest (i.e. as internal or external evaluators)? And, consider the ethical issues that are likely to arise because this is evaluation research.

Research

Timetable

Set out the best information available on the time needed for the various stages of the research: preparation, data collection, analysis, write up, publication/dissemination.

Personnel

Where appropriate, identify the main people (including yourself as primary researcher)

are likely to be involved and summarise the experience and expertise they would need to bring in order to produce high quality data.

Budget

This is required for many grant applications, consultancies, etc. It is often useful to complete in other cases as well.

References

List any books, articles, reports, web sites etc. you have cited in the proposal.

Appendices

Attach any important documents (e.g. questionnaires, ethics approvals, CVs, budget, sampling framework) referred to in the body of the proposal.

 


On successful completion you will be able to:
  • A practical introduction to evaluation methods & policy
  • Professionally useful skills in writing evaluation reports
  • Professionally useful data analysis skills

Delivery and Resources

The course is offered in a two-hour seminar format, comprising a lecture, group discussion and practice based workshop each week.

The readings for each week can be accessed from e-reserve and/or iLearn. As a general background, we would recommend the following textbooks, that you may find useful for this course and as a resource for evaluation research more generally.

  • Fitzpatrick, Jody, et al. (2012) Program Evaluation. Alternative Approaches and Practical Guidelines (Fourth Edition) Pearson.
  • Owen, John (2006) Program Evaluation. Forms and Approaches (Third Edition) Allen & Unwin.

Unit Schedule

1

27 July

Introduction

What is evaluation?

Introductions.

Learning outcomes.

Assignments.

 

2

3 August

Evaluation Approaches

Wicked problems.

Evaluation approaches.

Truth and knowledge

 

3

10 August

Evaluation approaches, continued

Old and new approaches.

Performance and outcome evaluations

 

4

17 August

Evaluation and policy processes

The Politics of Research Utilisation

 

 

5

24 August

Evaluation and policy processes, continued.

 

The policy process

Ethics

Whose values?

Stakeholders/participation

 

 

6

31 August

Evidence based policy

Evidence-based policy

Catching up

 

7

7  Sep

Managing small projects

Doing evaluations

Program Logic.

Small scale evaluations.

 

Break

8

28 Sep

Programs to evaluate for final assignment

Real wicked problems (for final assignment)

 

9

5 October

Managing large evaluation projects

Large projects, writing proposal for and winning a tender

+ getting a job.

+ what it is like working in consultancy

 

 

10

12 October

Planning and collecting evidence

Planning + design (what are we looking for, how do we find it)

Everyday ethics

 

11

19 October

Collecting evidence, continued

Collecting evidence

Evaluation tools

 

12

26 October

Utilisation/

Impact

Writing up

Making a difference.

Utilization and action research

 

13

2 November

Workshop for final assignment

Preliminary consultation, final assignment

 

Policies and Procedures

Macquarie University policies and procedures are accessible from Policy Central. Students should be aware of the following policies in particular with regard to Learning and Teaching:

Academic Honesty Policy http://mq.edu.au/policy/docs/academic_honesty/policy.html

Assessment Policy  http://mq.edu.au/policy/docs/assessment/policy.html

Grading Policy http://mq.edu.au/policy/docs/grading/policy.html

Grade Appeal Policy http://mq.edu.au/policy/docs/gradeappeal/policy.html

Grievance Management Policy http://mq.edu.au/policy/docs/grievance_management/policy.html

Disruption to Studies Policy http://www.mq.edu.au/policy/docs/disruption_studies/policy.html The Disruption to Studies Policy is effective from March 3 2014 and replaces the Special Consideration Policy.

In addition, a number of other policies can be found in the Learning and Teaching Category of Policy Central.

Student Code of Conduct

Macquarie University students have a responsibility to be familiar with the Student Code of Conduct: https://students.mq.edu.au/support/student_conduct/

Results

Results shown in iLearn, or released directly by your Unit Convenor, are not confirmed as they are subject to final approval by the University. Once approved, final results will be sent to your student email address and will be made available in eStudent. For more information visit ask.mq.edu.au.

Late Penalties

For all written assessment tasks, the Department applies the following penalties for late work that does not have an extension:

5 per cent for the first day (or weekend if the assessment is due on Friday and submitted the following Monday); 1 per cent for each subsequent day.

Student Support

Macquarie University provides a range of support services for students. For details, visit http://students.mq.edu.au/support/

Learning Skills

Learning Skills (mq.edu.au/learningskills) provides academic writing resources and study strategies to improve your marks and take control of your study.

Student Services and Support

Students with a disability are encouraged to contact the Disability Service who can provide appropriate help with any issues that arise during their studies.

Student Enquiries

For all student enquiries, visit Student Connect at ask.mq.edu.au

IT Help

For help with University computer systems and technology, visit http://informatics.mq.edu.au/help/

When using the University's IT, you must adhere to the Acceptable Use Policy. The policy applies to all who connect to the MQ network including students.

Graduate Capabilities

PG - Capable of Professional and Personal Judgment and Initiative

Our postgraduates will demonstrate a high standard of discernment and common sense in their professional and personal judgment. They will have the ability to make informed choices and decisions that reflect both the nature of their professional work and their personal perspectives.

This graduate capability is supported by:

Learning outcomes

  • A practical introduction to evaluation methods & policy
  • Insight into theoretical accounts about evaluation and policy
  • Professionally useful data analysis skills

Assessment task

  • Key Issues Summary

PG - Discipline Knowledge and Skills

Our postgraduates will be able to demonstrate a significantly enhanced depth and breadth of knowledge, scholarly understanding, and specific subject content knowledge in their chosen fields.

This graduate capability is supported by:

Learning outcomes

  • A practical introduction to evaluation methods & policy
  • Insight into theoretical accounts about evaluation and policy
  • Professionally useful data analysis skills

Assessment tasks

  • Key Issues Summary
  • Evaluation Proposal

PG - Critical, Analytical and Integrative Thinking

Our postgraduates will be capable of utilising and reflecting on prior knowledge and experience, of applying higher level critical thinking skills, and of integrating and synthesising learning and knowledge from a range of sources and environments. A characteristic of this form of thinking is the generation of new, professionally oriented knowledge through personal or group-based critique of practice and theory.

This graduate capability is supported by:

Learning outcomes

  • A practical introduction to evaluation methods & policy
  • Insight into theoretical accounts about evaluation and policy
  • Professionally useful skills in writing evaluation reports
  • Professionally useful data analysis skills

Assessment tasks

  • Participation
  • Key Issues Summary
  • Evaluation Proposal

PG - Research and Problem Solving Capability

Our postgraduates will be capable of systematic enquiry; able to use research skills to create new knowledge that can be applied to real world issues, or contribute to a field of study or practice to enhance society. They will be capable of creative questioning, problem finding and problem solving.

This graduate capability is supported by:

Learning outcomes

  • A practical introduction to evaluation methods & policy
  • Professionally useful skills in writing evaluation reports
  • Professionally useful data analysis skills

Assessment tasks

  • Participation
  • Key Issues Summary
  • Evaluation Proposal

PG - Effective Communication

Our postgraduates will be able to communicate effectively and convey their views to different social, cultural, and professional audiences. They will be able to use a variety of technologically supported media to communicate with empathy using a range of written, spoken or visual formats.

This graduate capability is supported by:

Learning outcome

  • Professionally useful skills in writing evaluation reports

Assessment tasks

  • Participation
  • Evaluation Proposal

PG - Engaged and Responsible, Active and Ethical Citizens

Our postgraduates will be ethically aware and capable of confident transformative action in relation to their professional responsibilities and the wider community. They will have a sense of connectedness with others and country and have a sense of mutual obligation. They will be able to appreciate the impact of their professional roles for social justice and inclusion related to national and global issues

This graduate capability is supported by:

Learning outcomes

  • A practical introduction to evaluation methods & policy
  • Professionally useful skills in writing evaluation reports

Assessment task

  • Evaluation Proposal