Macquarie University students’ exam scores up by nearly 10 per cent thanks to new AI-powered chatbot

Aerial view of Macquarie University Campus

Established in 1964 to challenge traditional learning models and strengthen connections between academia and industry, the university has become known for its innovative and solutions-led approach, from co-inventing Wi-Fi with CSIRO to designing personalised, industry-aligned degrees that enhance graduate employability. Today, it ranks among the top 1 per cent of universities globally, as recognised in the Times Higher Education World University Rankings 2025 and the QS World University Rankings 2025.

Building on its longstanding embrace of innovation, Macquarie University has also laid the foundation for a digitally enhanced future with scalable and secure technology infrastructure. The university recently undertook a significant cloud transformation and established a Microsoft Azure Landing Zone, which has enabled it to embark on ambitious AI initiatives to support its more than 44,000 students and 3,000 staff members.

“The way we approach AI is not simply technology-driven, meaning it’s not about adopting AI just because we can,” says Phil Laufenberg, Head of AI at Macquarie University. “Instead, we take a challenge-driven approach, looking at the difficulties our students and staff face and identifying where AI is the right solution to improve existing processes and systems.”

Profile shot of Phil Laufenberg in white collared shirt and blue suit jacket.
Phil Laufenberg, Head of AI at Macquarie University

One of the most significant challenges in higher education is ensuring all students receive timely, personalised academic support. In response, Macquarie University developed Virtual Peer, an AI-powered chatbot designed to provide students with real-time, 24/7 support for academic and administrative queries.

A smarter way to learn

Virtual Peer was developed to serve as an AI companion for students, helping them navigate their academic studies and university processes in a structured and responsible manner. The chatbot allows students to learn course concepts at their own pace, anytime and anywhere, by asking questions about class materials as needed.

It also alleviates the administrative load on teaching staff charged with responding to routine student queries – often about deadlines and the necessary details to include in a report, for example – that would otherwise require constant email correspondence or one-on-one consultations relaying the same information each time. By reducing the volume of these emails, it enables teaching staff to respond more quickly to pressing or unique student queries.

“Convenors can miss important queries if a student feels too shy to ask, fearing their question might be seen as silly, or they face language difficulties,” says Patrick Nalepka, Lecturer at Macquarie University’s School of Psychological Sciences. “Virtual Peer is a safe, non-judgemental tool that students can use to seek help.”

Students sitting around a table in library with laptops

The chatbot’s intelligence is built on retrieval-augmented generation, meaning it does not generate answers independently but instead references a curated knowledge base. This knowledge base is stored in a Microsoft SharePoint folder and consists of lecture slides, transcripts, university policies and Q&A materials provided by unit conveners. This ensures Virtual Peer sources its responses directly from verified university materials, making it more reliable than generic AI chatbots.

“From a user perspective, it functions like a ChatGPT-style interface,” explains Nalepka. “The key difference is that, as a unit convener or course director, I can upload a list of documents into its knowledge base, which it then draws upon.

“Another key feature for educators is the ability to review anonymous chat logs of students’ interactions with the chatbot. This helps me identify which topics students find confusing so that I can tailor revision activities for the next lecture.”

To ensure accuracy, Virtual Peer has been programmed with strict constraints. AI-generated misinformation – often referred to as ‘hallucination’ – is a key concern in educational settings, but Macquarie University has mitigated this risk by limiting the chatbot’s response scope.

“We have instructed the bot to only answer based on verified, grounded data,” says Laufenberg. “Every response includes a direct source reference, allowing students to click and open the exact document where the answer was found.”

The science behind Virtual Peer

A fundamental challenge in education inspired the creation of Virtual Peer: the difficulty of providing scalable, high-quality tutoring for all students. This challenge is best understood through Bloom’s 2 Sigma Problem, an observation that one-on-one tutoring can improve student performance by two standard deviations compared to traditional classroom teaching. While this level of personalised learning is ideal, it is not feasible at scale, particularly at large institutions like Macquarie University.

“We strive to create the best possible learning environment at Macquarie University, so we asked ourselves: how can we move towards a one-on-one tutoring experience?” says Laufenberg. “One approach is peer-led learning, where students support each other. Another approach is using AI agents as virtual peers, which is where the name Virtual Peer originates.”

Macquarie University logo on a building.

Research has shown that AI can enhance or hinder learning, depending on its implementation. A study involving 1,000 high school students found that the performance of those using a standard AI chatbot for academic support improved by 48 per cent, while students using an enhanced AI tutor – one that included Socratic questioning and guided feedback – improved by 127 per cent. However, students using the standard AI chatbot, who became overly reliant on AI-generated answers, performed worse once access to the tool was removed.

These insights shaped Macquarie’s design philosophy for Virtual Peer. Rather than simply providing answers, the chatbot is structured to encourage active learning. Students can request explanations in simpler terms, ask for alternative perspectives on complex topics, and receive references to the relevant lecture materials – all without replacing the learning process itself.

Under the AI hood

Macquarie University used Microsoft Azure OpenAI Service to develop Virtual Peer and leveraged provisioned throughput units (PTUs) to ensure high performance, reliability and cost predictability.

“We pay a fixed amount each month for PTUs, which allows us to plan ahead more effectively compared to a pay-as-you-go model,” says Laufenberg. “PTUs also provide dedicated bandwidth to ensure fast response times and a more seamless user experience. If Virtual Peer takes 20 seconds or more to respond, the interaction no longer feels like a real conversation and users are likely to disengage.”

Macquarie University also used Microsoft’s Azure AI Search, Azure API Management, Azure App Service, Azure Cosmos DB and Copilot Studio technologies to power Virtual Peer. These services enable the bot to efficiently retrieve information from its knowledge base to provide students with accurate, context-aware responses.

Virtual Peer’s creation was a highly collaborative effort between Macquarie University and Microsoft. The development process began with an opportunity discovery workshop and a three-day rapid prototyping session at the Microsoft Innovation Hub in Sydney. This intensive design sprint involved refining the chatbot’s core functionality and user experience to ensure it aligned with student needs.

Following the prototyping phase, Macquarie University engaged Celebal Technologies, a Microsoft partner, to help it develop a minimum viable product. The project team worked in phases, first securing funding approval in August 2024 and then moving to a structured development phase in October 2024. This approach ensured that Macquarie University retained full ownership of the AI chatbot’s design while benefiting from Microsoft’s and Celebal’s expertise.

Laufenberg reflects on the critical role that the two companies played in accelerating the project. “Historically, the university has had limited in-house AI software development capability,” he explains. “I’ve recently had the opportunity to hire and build a team to enhance our internal capabilities, and having Microsoft and Celebal as trusted development partners was crucial in meeting our required timeframes.”

By late 2024, the Virtual Peer prototype was operational. Macquarie University was ready to move into a real-world pilot phase to evaluate the chatbot’s effectiveness in a live teaching environment.

AI-powered learning yields significant improvements

The university conducted a pilot study involving 1,400 students enrolled in a second-year psychology unit in October 2024 to assess Virtual Peer’s effectiveness. The pilot was designed to test the chatbot’s ability to assist students in the lead-up to their final exams.

During the initial two-week test phase, students engaged with Virtual Peer extensively, exchanging over 20,800 messages – 80 per cent of which were sent outside of standard university operating hours. While engagement was steady throughout the first week, there was a dramatic spike in usage during the weekend before the final exam.

This revealed a critical insight: students experience peak academic stress and uncertainty outside standard university operating hours. Traditionally, academic support is only available during normal business hours, meaning students studying late at night or on weekends often lack access to assistance.

Laufenberg notes that Virtual Peer effectively addressed this gap. “Virtual Peer allows Macquarie University to provide a premium student experience by offering guidance and support when students need it, rather than being restricted by regular office hours” he says.

The results were significant from an academic performance perspective. Controlling for past academic achievement, an analysis of exam scores found that students who used Virtual Peer in the two weeks before the exam scored nearly five marks higher on average – a 9.45 per cent improvement – compared to non-users.

Beyond academic outcomes, student feedback was overwhelmingly positive. The chatbot achieved a Net Promoter Score of 92, an exceptionally high rating for a new technology solution in education. Additionally, 72 per cent of survey respondents stated they would be very disappointed if they lost access to Virtual Peer.

Two students walking through the University

One of the chatbot’s standout features is its ability to provide unlimited practice questions. Nalepka explains how this feature became particularly useful for students dealing with exam anxiety.

“There are always limitations on my end when structuring practice exams or practice questions,” he says. “While there are other methods – such as question banks or student-generated questions – Virtual Peer can generate an unlimited number of scenario-based, definition-based, short-answer and multiple-choice questions.”

Macquarie University also assessed Virtual Peer’s ability to answer administrative questions accurately. A retrospective evaluation involved tutors revisiting past student enquiries and comparing their responses to those generated by Virtual Peer. The analysis found that tutors rated 84 per cent of Virtual Peer’s answers as equal to or better than human responses, even preferring 38 per cent of the AI-generated answers due to their clarity and completeness.

Laufenberg acknowledges that the chatbot’s effectiveness depends on the quality of the data it was trained on and that minor gaps in documentation were identified during the pilot.

“The remaining 16 per cent of responses that were not optimal were largely due to gaps in our documentation,” he says. “We have since updated our onboarding process to ensure more standardised Q&A templates, helping academics provide clear, structured information for Virtual Peer.”

Scaling up to support thousands of students

Following the pilot’s success, Macquarie University is rolling out an extended trial of Virtual Peer across multiple faculties in 2025, involving more than 3,000 students. The long-term vision is to integrate Virtual Peer across hundreds of course units, potentially benefiting tens of thousands of students by 2026.

“We’re taking a controlled approach to ensure we can replicate the positive results across different faculties before scaling further,” says Laufenberg.

“Macquarie University has always been bold in challenging conventions, and Virtual Peer is another example of how we are pioneering AI-enhanced education to improve student outcomes.”

Aerial view of Macquarie University campus