Unit convenor and teaching staff |
Unit convenor and teaching staff
Unit convenor and lecturer
Yan Wang
Contact via +61-2-9850 9539
Room 354, BD Building
By Appointment
Lecturer
Guanfeng Liu
Tutor
Pengfei Ding
Tutor
Shreyas Kumar Singh
|
---|---|
Credit points |
Credit points
10
|
Prerequisites |
Prerequisites
COMP6200 and Admission to MDataSc or MScInnovationIT or GradCertInfoTech or MBusAnalytics
|
Corequisites |
Corequisites
|
Co-badged status |
Co-badged status
|
Unit description |
Unit description
Even simple tasks like counting elements can seem impossible when the amount of data to process is huge. This unit explores some of the key aspects related to processing and mining information from large volumes of data. We present technology commonly used in industry such as map-reduce, and show how a range of data processing methods can be realised using map-reduce. Especial emphasis will be placed in the adaptation of data mining techniques for large volumes of data and for data streaming. |
Information about important academic dates including deadlines for withdrawing from units are available at https://www.mq.edu.au/study/calendar-of-dates
On successful completion of this unit, you will be able to:
From 1 July 2022, Students enrolled in Session based units with written assessments will have the following university standard late penalty applied. Please see https://students.mq.edu.au/study/assessment-exams/assessments for more information.
Unless a Special Consideration request has been submitted and approved, a 5% penalty (of the total possible mark) will be applied each day a written assessment is not submitted, up until the 7th day (including weekends). After the 7th day, a grade of '0' will be awarded even if the assessment is submitted. Submission time for all written assessments is set at 11:55 pm. A 1-hour grace period is provided to students who experience a technical concern.
For any late submission of time-sensitive tasks, such as scheduled tests/exams, performance assessments/presentations, and/or scheduled practical assessments/labs, students need to submit an application for Special Consideration.
Assessments where Late Submissions will be accepted In this unit, late submissions will accepted as follows:
• Assignmanet 1 - YES, Standard Late Penalty applies
• Assignmanet 2 - YES, Standard Late Penalty applies
Name | Weighting | Hurdle | Due |
---|---|---|---|
Assignment 1 | 20% | No | Week 7-8 |
Assignment 2 | 20% | No | Week 12-13 |
Final examination | 60% | No | TBA |
Assessment Type 1: Practice-based task
Indicative Time on Task 2: 30 hours
Due: Week 7-8
Weighting: 20%
In this assignment you will implement MapReduce techniques for the processing of Big Data. You will build your assignment on top of Hadoop.
Assessment Type 1: Practice-based task
Indicative Time on Task 2: 30 hours
Due: Week 12-13
Weighting: 20%
In this assignment you will implement a non-trivial problem that processes Big Data.
Assessment Type 1: Examination
Indicative Time on Task 2: 15 hours
Due: TBA
Weighting: 60%
The final exam will focus on the theoretical aspects of the unit, including algorithms and implementation issues.
1 If you need help with your assignment, please contact:
2 Indicative time-on-task is an estimate of the time required for completion of the assessment task and is subject to individual variation
For details of days, times and rooms consult the timetables webpage.
Some of the contents of the unit will be based on the following books:
Additional material including lecture notes will be made available during the semester. See the unit schedule for a listing of the most relevant reading for each week.
The following software is used in COMP6210:
This software is installed in the labs; you should also ensure that you have working copies of all the above on your own machine. Note that some of this software requires internet access.
Many packages come in various versions; to avoid potential incompatibilities, you should install versions as close as possible to those used in the labs.
The unit web page will be hosted in iLearn, where you will need to login using your Student One ID and password. The unit will make extensive use of discussion boards also hosted in iLearn. Please post questions there, they will be monitored by the staff on the unit.
Note: Lectures will be online.
Week 1: Data and Big Data
Week 2: Organizing Big Data
Week 3: Curating Big Data
Week 4: Processing Big Data (Cloud Computing)
Week 5: Processing Big Data (MapReduce)
Week 6: Big Data Platforms (Guest Lecture)
Week 7: Big Data with High Dimensions
Week 8: Indexing Big Data
Week 9: Searching Big Data
Week 10: Multidimensional Divide and Conquer
Week 11: Grid Decomposition in Big Data
Week 12: Advanced Topic in Big Data (Guest Lecture)
Week 13: Unit Review
Macquarie University policies and procedures are accessible from Policy Central (https://policies.mq.edu.au). Students should be aware of the following policies in particular with regard to Learning and Teaching:
Students seeking more policy resources can visit Student Policies (https://students.mq.edu.au/support/study/policies). It is your one-stop-shop for the key policies you need to know about throughout your undergraduate student journey.
To find other policies relating to Teaching and Learning, visit Policy Central (https://policies.mq.edu.au) and use the search tool.
Macquarie University students have a responsibility to be familiar with the Student Code of Conduct: https://students.mq.edu.au/admin/other-resources/student-conduct
Results published on platform other than eStudent, (eg. iLearn, Coursera etc.) or released directly by your Unit Convenor, are not confirmed as they are subject to final approval by the University. Once approved, final results will be sent to your student email address and will be made available in eStudent. For more information visit ask.mq.edu.au or if you are a Global MBA student contact globalmba.support@mq.edu.au
At Macquarie, we believe academic integrity – honesty, respect, trust, responsibility, fairness and courage – is at the core of learning, teaching and research. We recognise that meeting the expectations required to complete your assessments can be challenging. So, we offer you a range of resources and services to help you reach your potential, including free online writing and maths support, academic skills development and wellbeing consultations.
Macquarie University provides a range of support services for students. For details, visit http://students.mq.edu.au/support/
The Writing Centre provides resources to develop your English language proficiency, academic writing, and communication skills.
The Library provides online and face to face support to help you find and use relevant information resources.
Macquarie University offers a range of Student Support Services including:
Got a question? Ask us via AskMQ, or contact Service Connect.
For help with University computer systems and technology, visit http://www.mq.edu.au/about_us/offices_and_units/information_technology/help/.
When using the University's IT, you must adhere to the Acceptable Use of IT Resources Policy. The policy applies to all who connect to the MQ network including students.
No change.
Date | Description |
---|---|
25/07/2022 | no change |
Unit information based on version 2022.04 of the Handbook