Students

SOC 818 – Evaluation and the Policy Process

2017 – S1 Evening

General Information

Download as PDF
Unit convenor and teaching staff Unit convenor and teaching staff
Charlotte Overgaard
Credit points Credit points
4
Prerequisites Prerequisites
Admission to MPASR or GradCertPASR or GradDipPASR or MAppAnth or MIntPubDip or MPPP or GradCertPP or GradDipPP or MPlan
Corequisites Corequisites
Co-badged status Co-badged status
Unit description Unit description
This unit introduces students to policy evaluation and accounts for its place in the policy process and social sciences respectively. Students will acquire a knowledge of different approaches to evaluation as well as the ethical and political issues surrounding this process. The unit also focuses on how trends informing public sector management have an influence on the evaluation process and the actual implemenation of evaluations.

Important Academic Dates

Information about important academic dates including deadlines for withdrawing from units are available at https://www.mq.edu.au/study/calendar-of-dates

Learning Outcomes

On successful completion of this unit, you will be able to:

  • To give students a practical introduction to evaluation.
  • To give students the ability to critically think about claims about what research can and should do.
  • To give students the ability to (somehow confident) carry out small scale evaluations.
  • To give students the practical skills to develop a research proposal.
  • To give students the practical skills to develop a program logic model and use it.
  • To give students an understanding the key principles of and discussions relating to evaluations for selected topics.
  • To give students an opportunity to apply knowledge to practical examples.

General Assessment Information

Assignment Submission

All written assignments are to be submitted via the turnitin links set up for each assignment on the unit iLearn page.

Quizzes must be completed online within the set time frames.

Assessment Tasks

Name Weighting Hurdle Due
Online quiz 1 5% No 17th to 22nd March
Research task 40% No 23 April
Online quiz 2 10% No 19th May to 24th May
Evaluation Proposal 35% No 8 June
Workshop participation 10% No Ongoing

Online quiz 1

Due: 17th to 22nd March
Weighting: 5%

This online quiz will consist of 10 multiple choice questions. It is worth 5 per cent of your total mark. You will be asked to read each question and select the BEST response from the available options. It is designed to provide you with an early insight into how you are going in the course and will draw on both the course readings and lecture materials from weeks 1 to 3.  

The online quiz will become available on 17th of March and you can undertake the quiz until 22nd of March (at 11:59 pm EST). It will become available via the iLearn page. The quiz can only be taken once. 

You will have up to 15 minutes to complete it.

Marks will become available online after the quiz has closed.

If you have questions about this information, we advise you to ask them before taking the quiz. 

N.B. In the event of technical difficulties, it is your responsibility to contact course staff before the due date. We advise you to ensure that you can access the link to the quiz by 17th March (without starting the quiz unless you want to). 


On successful completion you will be able to:
  • To give students the ability to critically think about claims about what research can and should do.
  • To give students the ability to (somehow confident) carry out small scale evaluations.
  • To give students an understanding the key principles of and discussions relating to evaluations for selected topics.

Research task

Due: 23 April
Weighting: 40%

This assignment includes two separate tasks: Key Issues Review (1a) AND Program Logic (1b).

Your overall mark reflects on a separate assessment of the two tasks in which the Key Issues review counts 25% and the Program Logic counts 15% of your overall mark in the unit.

2a: Key Issues Review

Word limit: 1500 words.

For the Key Issues Review task you have to write a concise review of eight (8) readings pertaining to one or two of the topics listed below. The purpose of the assessment task is to ensure that you understand the key principles of and discussions relating to evaluations for your chosen topic. You are required to demonstrate a critical understanding of the readings and the issues they raise. It is intended to be an exercise that provides you with a clear overview of the field and the issues and discussions that take place.

The aim is to allow you to choose a topic that you have an interest in. It is therefore essential to identify you own learning objectives for this assignment and frame your review around those learning objectives.

For the review you can choose one or two of the following topics:

1. The nature of social science and what it means for evaluations

2. The role of evidence in the policy cycle/process

3. The nature of social problems and what it means for evaluators

4. The role of stakeholders in evaluations

5. Utilisation of evaluation research

6. Values and evaluations

If you have another wish for a topic, please consult teaching staff prior to writing your assignment.

The readings can be a mix of readings provided in this unit and readings you find through a literature search in the library, or similar. You can choose (say) 3 readings from the list of mandatory readings as well as (say) 3 readings from the list of elective readings in addition to (say) 2 readings found through a literature search.

The reference list is not included in the word limit.

2b: Program Logic Task

Word Limit: 750 words

The Program Logic task requires you to develop a program logic for the “Art Heart” project and to explain the logic of change that underpins it.

Program logics are one of the most widely used tools in evaluation research. This assignment is designed to allow you to develop skills associated with using this tool.

You should present the program logic in diagrammatic form or a table and draw on suitable resources, such as “Does your project make a difference”? or a similar. You should also describe the logic in the text of your paper and how to make sense of your diagram or table. And, you should briefly discuss the assumptions and limitations of the program logic model presented.

The word count does not include diagrams or tables.


On successful completion you will be able to:
  • To give students the practical skills to develop a program logic model and use it.

Online quiz 2

Due: 19th May to 24th May
Weighting: 10%

This online quiz will consist of 20 multiple choice and/or short answer questions. It is designed to provide you with ongoing insight into how you are going in the unit. It will draw on course readings, workshop materials and the lectures from week 1 to 10. It will also require you to analyse data tables and use Microsoft Excel, drawing on detailed course notes provided by the lecturer. 

The online quiz will become available on the 19th May and you can undertake the quiz until 24th May (at 11:59 pm EST). It will become available via the iLearn page. The quiz can only be taken once. You will be given 25 minutes to complete it. 

N.B. In the event of technical difficulties, it is your responsibility to contact course staff before the due date. We advise you to ensure that you can access the link to the quiz on 19 May (without starting the quiz unless you want to).


On successful completion you will be able to:
  • To give students the ability to critically think about claims about what research can and should do.
  • To give students the ability to (somehow confident) carry out small scale evaluations.
  • To give students an understanding the key principles of and discussions relating to evaluations for selected topics.

Evaluation Proposal

Due: 8 June
Weighting: 35%

Evaluation Proposal

This research assignment requires you to develop an evaluation proposal for a program or policy.

The purpose of this assignment is to make you able to enter the field (relatively) confident that you are able to undertake at least small scale evaluations. This assignment is designed to give you the skills that enables you to design an evaluation.

To allow you to develop skills that you can use in your future career, you are encouraged to choose a program or policy that you have an interest in. Please consult teaching staff to discuss suitability. Alternatively, choose one of these topics:

* Work for the Dole (reforms for individuals aged under 30 years)

* Sydney Lock Out Rules (a regulation for licensed premises operating in some parts of Sydney)

457 Immigration Visa Program (a temporary visa program for skilled migrants who are sponsored by an approved business)

Hornsby Kur-ing-gai Women's Shelter (a community based program that provides crisis support for women)

Sydney Medically Supervised Injecting Centre (a program that medically supervises drug injecting)

You should organise the proposal so that it includes the elements outlined in the table below.  

Note: The sections indicated with an asterix * (cover letter, research timetable, budget, personnel, appendices) normally feature in research proposals and grant applications, but due to the word limit for this task they do not have to be included here (and will not count toward your task). References (which are compulsory) do not count toward your word limit. Also, please make sure that you make linkages between the different sections of this report where appropriate.

 

Table 1: Evaluation Proposal

Cover letter*

You should include a cover letter and address the evaluation proposal

Title

The title should indicate the topic of the study in 1- 15 words

 

Summary

100 to 200 word summary of the entire project

Background/

Significance

 

This section should outline the program that you will be evaluating in the proposed research and indicate why evaluation is needed. For instance, are there controversies about the program or the social issue it responds to? Why is there a need for further information about this program? When indicating why the evaluation is needed, you should discuss the program, the social issue that it responds to and what the evaluation will contribute to our knowledge.

Literature Review

A concise summary of the main references relevant to evaluating the selected program. Make sure that you review the findings of other studies and discuss their methodology. At the end of this review, draw out the important messages for the evaluation and the methodology. It is imperative that multiple sources with a range of authors are consulted to allow for a diversity of views to be explored and identification of controversies about the program (or similar types of programs).

Aims and

Objectives

Identify the study’s main objective and the key research questions that will guide the evaluation. After identifying the main objective, it might help to consider whether you need information on the effectiveness, appropriateness, and efficiency and so on of the program.

Approach and methods

Discuss the suggested approach and justify why you have chosen that approach. What methods will you use? What will be the outcome of using these methods? Which stakeholders will be involved? It is imperative that you explain the links between the evaluation question(s), the evaluation approach, the methods and how these will be operationalised in this particular case. Use references if appropriate.

Ethical and

Political Issues

Identify any specific issues arising from the research. Pay careful attention also to generic ethical issues in research with human subjects – Respect for Persons: Informed Consent; confidentiality; possible risks for subjects and any possible benefits. State whether Ethics approval has, or will be sought, and if so, from whom. Also, consider the political dimensions of this report – are there controversies about the program or social issue? Are there any conflicts of interest (i.e. as internal or external evaluators)? And, consider the ethical issues that are likely to arise because this is evaluation research. Use references (including those covered in the unit) if appropriate.

Research

Timetable*

Set out the best information available on the time needed for the various stages of the research: preparation, data collection, analysis, write up, publication/dissemination.

Personnel*

Where appropriate, identify the main people (including yourself as primary researcher)

are likely to be involved and summarise the experience and expertise they would need to bring in order to produce high quality data.

Budget*

This is required for many grant applications, consultancies, etc. It is often useful to complete in other cases as well.

References

List any books, articles, reports, web sites etc. you have cited in the proposal.

Appendices*

Attach any important documents (e.g. questionnaires, ethics approvals, CVs, budget, sampling framework) referred to in the body of the proposal.

 


On successful completion you will be able to:
  • To give students a practical introduction to evaluation.
  • To give students the ability to (somehow confident) carry out small scale evaluations.
  • To give students the practical skills to develop a research proposal.
  • To give students an opportunity to apply knowledge to practical examples.

Workshop participation

Due: Ongoing
Weighting: 10%

Workshop participation

This assessment task is intended to encourage and reward you for actively engaging in the workshops. If you want to do well on this assignment, you will need to come prepared to the workshops (having read the set readings) and be prepared to engage with the lecturer and other students.


On successful completion you will be able to:
  • To give students a practical introduction to evaluation.
  • To give students the ability to critically think about claims about what research can and should do.
  • To give students the practical skills to develop a program logic model and use it.

Delivery and Resources

Course Delivery

The teaching program consists of:

  • Short weekly lectures,
  • Workshop exercises,
  • Related readings; and 
  • Independent research tasks over the course of the unit.

As this is an advanced course in the use of applied social science, it is vital for students to participate in all components of the unit. The lecture material and readings provide the conceptual basis of the course, examining the theory for each topic. The lectures and the workshop exercises are designed to complement this by providing the opportunity to exchange ideas, discuss research, and develop a critical and collaborative approach.

Required Readings and Recommended Texts

Readings for each week’s topics are listed on Ilearn and you are expected to have read them before class each week. Active participation is a requirement of the course.

There is no textbook for this course. All readings will be made available on Ilearn.

Unit Schedule

Week

Dates

Topics

1

2 March

Introduction to evaluation research

2

9 March

The nature of social problems, wicked problems

3

16 March

Research Paradigms

4

23 March

Engaging with the policy process

5

30 March

Evidence in policy

6

6 April

Values, Stakeholders

7

13 April

Developing program logics

Break

8

4 May

Evaluation approaches

9

11 May

Evaluation approaches, continued

10

18 May

Methods with a focus on quantitative data

11

25 May

Methods, continued

12

1 June

Ethics and politics

13

8 June

Study week, no lecture or readings

Policies and Procedures

Macquarie University policies and procedures are accessible from Policy Central. Students should be aware of the following policies in particular with regard to Learning and Teaching:

Academic Honesty Policy http://mq.edu.au/policy/docs/academic_honesty/policy.html

Assessment Policy http://mq.edu.au/policy/docs/assessment/policy_2016.html

Grade Appeal Policy http://mq.edu.au/policy/docs/gradeappeal/policy.html

Complaint Management Procedure for Students and Members of the Public http://www.mq.edu.au/policy/docs/complaint_management/procedure.html​

Disruption to Studies Policy (in effect until Dec 4th, 2017): http://www.mq.edu.au/policy/docs/disruption_studies/policy.html

Special Consideration Policy (in effect from Dec 4th, 2017): https://staff.mq.edu.au/work/strategy-planning-and-governance/university-policies-and-procedures/policies/special-consideration

In addition, a number of other policies can be found in the Learning and Teaching Category of Policy Central.

Student Code of Conduct

Macquarie University students have a responsibility to be familiar with the Student Code of Conduct: https://students.mq.edu.au/support/student_conduct/

Results

Results shown in iLearn, or released directly by your Unit Convenor, are not confirmed as they are subject to final approval by the University. Once approved, final results will be sent to your student email address and will be made available in eStudent. For more information visit ask.mq.edu.au.

Student Support

Macquarie University provides a range of support services for students. For details, visit http://students.mq.edu.au/support/

Learning Skills

Learning Skills (mq.edu.au/learningskills) provides academic writing resources and study strategies to improve your marks and take control of your study.

Student Services and Support

Students with a disability are encouraged to contact the Disability Service who can provide appropriate help with any issues that arise during their studies.

Student Enquiries

For all student enquiries, visit Student Connect at ask.mq.edu.au

IT Help

For help with University computer systems and technology, visit http://www.mq.edu.au/about_us/offices_and_units/information_technology/help/

When using the University's IT, you must adhere to the Acceptable Use of IT Resources Policy. The policy applies to all who connect to the MQ network including students.

Graduate Capabilities

PG - Capable of Professional and Personal Judgment and Initiative

Our postgraduates will demonstrate a high standard of discernment and common sense in their professional and personal judgment. They will have the ability to make informed choices and decisions that reflect both the nature of their professional work and their personal perspectives.

This graduate capability is supported by:

Learning outcomes

  • To give students a practical introduction to evaluation.
  • To give students the ability to critically think about claims about what research can and should do.
  • To give students the ability to (somehow confident) carry out small scale evaluations.
  • To give students the practical skills to develop a research proposal.
  • To give students an understanding the key principles of and discussions relating to evaluations for selected topics.

Assessment tasks

  • Online quiz 1
  • Research task
  • Online quiz 2
  • Evaluation Proposal
  • Workshop participation

PG - Discipline Knowledge and Skills

Our postgraduates will be able to demonstrate a significantly enhanced depth and breadth of knowledge, scholarly understanding, and specific subject content knowledge in their chosen fields.

This graduate capability is supported by:

Learning outcomes

  • To give students a practical introduction to evaluation.
  • To give students the ability to critically think about claims about what research can and should do.
  • To give students the ability to (somehow confident) carry out small scale evaluations.
  • To give students the practical skills to develop a research proposal.
  • To give students the practical skills to develop a program logic model and use it.
  • To give students an opportunity to apply knowledge to practical examples.

Assessment tasks

  • Online quiz 1
  • Research task
  • Online quiz 2
  • Evaluation Proposal
  • Workshop participation

PG - Critical, Analytical and Integrative Thinking

Our postgraduates will be capable of utilising and reflecting on prior knowledge and experience, of applying higher level critical thinking skills, and of integrating and synthesising learning and knowledge from a range of sources and environments. A characteristic of this form of thinking is the generation of new, professionally oriented knowledge through personal or group-based critique of practice and theory.

This graduate capability is supported by:

Learning outcomes

  • To give students the ability to critically think about claims about what research can and should do.
  • To give students the practical skills to develop a research proposal.
  • To give students the practical skills to develop a program logic model and use it.
  • To give students an understanding the key principles of and discussions relating to evaluations for selected topics.

Assessment tasks

  • Online quiz 1
  • Research task
  • Online quiz 2
  • Evaluation Proposal
  • Workshop participation

PG - Research and Problem Solving Capability

Our postgraduates will be capable of systematic enquiry; able to use research skills to create new knowledge that can be applied to real world issues, or contribute to a field of study or practice to enhance society. They will be capable of creative questioning, problem finding and problem solving.

This graduate capability is supported by:

Learning outcomes

  • To give students a practical introduction to evaluation.
  • To give students the ability to (somehow confident) carry out small scale evaluations.
  • To give students the practical skills to develop a research proposal.
  • To give students the practical skills to develop a program logic model and use it.
  • To give students an opportunity to apply knowledge to practical examples.

Assessment tasks

  • Online quiz 1
  • Research task
  • Online quiz 2
  • Evaluation Proposal
  • Workshop participation

PG - Effective Communication

Our postgraduates will be able to communicate effectively and convey their views to different social, cultural, and professional audiences. They will be able to use a variety of technologically supported media to communicate with empathy using a range of written, spoken or visual formats.

This graduate capability is supported by:

Learning outcomes

  • To give students the ability to (somehow confident) carry out small scale evaluations.
  • To give students the practical skills to develop a research proposal.
  • To give students an understanding the key principles of and discussions relating to evaluations for selected topics.
  • To give students an opportunity to apply knowledge to practical examples.

Assessment tasks

  • Online quiz 1
  • Online quiz 2
  • Evaluation Proposal

PG - Engaged and Responsible, Active and Ethical Citizens

Our postgraduates will be ethically aware and capable of confident transformative action in relation to their professional responsibilities and the wider community. They will have a sense of connectedness with others and country and have a sense of mutual obligation. They will be able to appreciate the impact of their professional roles for social justice and inclusion related to national and global issues

This graduate capability is supported by:

Learning outcomes

  • To give students the ability to critically think about claims about what research can and should do.
  • To give students an understanding the key principles of and discussions relating to evaluations for selected topics.

Assessment tasks

  • Online quiz 1
  • Online quiz 2
  • Workshop participation