Welcome and introducing UNE's AI programme

G'day, and welcome to the National Virtual Expo and this session on AI safety. I'm joining you on Gadigal land in Sydney. Thank you for joining me if you're here watching live. You can ask any questions you have about artificial intelligence in general, or AI safety specifically, in the chat. If you're on social media, drop your comments in the comments. If you're here on the session, you can ask straight in the chat and everything will get a reply. If you're watching the replay, thank you for that too. You can ask your questions in the Choosing Your Uni Discord -- you'll find the details down in the session description. If you're a parent, we have a Facebook community just for you, and you'll also find that down in the description.

Now, with all that aside, I'm so excited about this topic. It's so interesting and important for the future. I couldn't be more thrilled to be joined by Aaron Driver from the University of New England. Aaron, welcome to the National Virtual Expo.

Guest: Thanks for having me. Really excited to be here.

Rob: You've got one of the best titles in Australian higher education. You are the Chief AI Officer at UNE.

Guest: Some people call it the Chief Fun Officer, which is not inaccurate.

Rob: It's great to have you here. UNE is doing some of the most outstanding work in this space, and today's topic on AI safety is such a critical one for everybody, not just people who are new to AI. By way of starting, Aaron, how should people think about AI?

Understanding AI as a prediction machine, not a thinking machine

Guest: That's a great question. What I thought we could do to answer it is go through the guidelines of what we teach our students at the university. Late last year, we were the first university in the country to roll out a fully fledged AI suite for all 22,000 students. We believe students should have AI in their studies because they will need it in their work, and we're all about preparing job-ready graduates. Very importantly, one of the first things we addressed was safety. We can't have students using AI in an unsafe manner. We have a compulsory safety course that students have to complete before they're given access to the AI platform. So a lot of what we'll go over today is essentially what we're teaching our first-year students when they join us and get on that platform for the first time.

Guest: With that preamble, to answer your question: what we teach our students is that one useful way of thinking about the AI is to think of it as your digital coworker. Or if you're a student, your digital study buddy, your digital AI assistant. Imagine you've got your own executive assistant. You can do that with AI. Now, this assistant is brilliant. It has read almost the entire internet. It's incredibly fast. But it has zero life experience. It has no common sense. And it's on the first day of the job.

Guest: Let me show you what that means. Here we go. This is inside our platform. The founder of our university was Sir Robert Menzies -- actually, our platform is named after Robert Mace, who was one of the first innovators. UNE was the first university outside of capital cities. Robert was the first educator in the country to do distance education. Then around the turn of the century, we were the first university to offer all-online degrees, and now we're the first university to give students a universal, fully fledged AI platform. I'll give you a demonstration. I've asked the AI to complete a sentence with only the single most statistically likely next word, with no added explanation. The sentence is: "The first person to walk on the moon was Neil..." and we get: Armstrong. As one would anticipate. Why did we anticipate that? Because we've seen that sequence of words many times in our own lives, and the AI has seen it millions of times, so it correctly predicts the next word.

Guest: This is the power of large language models. Most people are familiar with ChatGPT, but some of you might be using Claude or Gemini. These are all large language models, and their power is to recognise patterns in vast amounts of text. But what happens when I give it a sentence where it cannot possibly know the correct answer? This time the instruction is the same, but the sentence is: "This morning for breakfast..." Let's see what it comes up with. It says eggs. It's a plausible guess -- toast or something common. But it was wrong. I had a very nutritious breakfast of coffee. What the AI has done is fall back on its core function, which is predicting the most statistically probable word to follow based on its training data. In this case, that was eggs.

Guest: This is a really valuable lesson in how AI works. I was going to say it "thinks," but it doesn't think -- it predicts the next word. Words are actually broken up into numbers called vectors, referred to as tokens, so the word "running" might be split into "run" and "ing," and both of those would be tokens. Technically it's predicting tokens, but for all intents and purposes it's predicting words. The most important thing to understand is that the AI is a prediction machine, not a thinking machine. It's always making a highly educated guess about what word should come next. Most of the time those guesses are incredibly helpful, but they're still guesses. And sometimes the AI is confidently and completely wrong. That is what we call hallucination. The AI will make things up. If you've heard this term before, hallucination essentially means the AI is predicting the most likely next token even if it's not true. It's not a truth machine. It's a prediction machine.

Guest: That's why the mental model of the brilliant but inexperienced coworker is so helpful. Your job is to be the manager. You have to guide it, check its work, and use your own critical thinking when working with the AI. That's the first piece of what we try to teach students: how to think about the AI in order to limit what can go wrong.

Rob: I could probably cite you half a dozen examples from this week alone where I've given it a task, it's been performing really well, and then at some point it loses context and gets something completely wrong, or overlooks something entirely. And I guess there's a danger in that, isn't there, Aaron? We can become quite reliant and complacent because the technology is so smart that we assume it never gets anything wrong.

Guest: Exactly. And I've seen it so many times: "But the AI told me..." It's just like somebody telling you something on the internet -- it doesn't mean it's true. So once we've got students locked into that mental model, the next step is to operationalise it. The key things we talk about with students are the ways these guesses can go wrong.

Hallucination, sycophancy, and the risk of delusion spirals

Guest: As I've mentioned, there's hallucination -- the AI making things up. A definition for you: hallucination equals confidently invented facts. Then we have sycophancy, which makes things even harder. The AI has a tendency to be a people pleaser. That's because after its main training run on trillions of words, it goes through reinforcement learning from human feedback. When the model comes out, humans come in and ask it questions, rank its answers as good or bad, and the AI learns over time what a good answer looks like. If we're anthropomorphising, the AI is trained from birth to be a people pleaser, because these companies are trying to build products that people enjoy and want to spend more time with. So not only will the AI predict the next token, it will often agree with your ideas, praise you, and flatter you even if your guess is wrong. It learns your biases and reflects them back to you, reinforcing what you want to hear. If you think you've got a brilliant idea about something but it's complete nonsense, the AI may not tell you that.

Guest: That's a step up in terms of danger from hallucination. Sycophancy means agreeable and flattering. When you combine a machine that invents facts with one that also wants to agree with and praise you, what you can get is powerful delusion spirals.

Rob: There's a lot of exciting science too, obviously, and our students are using these tools in great numbers. We can talk about that. But let's get the safety squared away first. By all means, use these tools to get superpowers, but let's make sure we're doing it safely.

Guest: Absolutely. And I don't mean to turn anyone off AI -- it really is important that we're clear-eyed about this. People don't need to be afraid of AI. The thing we need to be conscious of is people using AI in the wrong way: bad actors using AI for hacks, misinformation, and propaganda, flooding social media with AI slop. As always, it comes down to how we use the technology. The internet can be used for great things and for really toxic things. Doomscrolling is probably not good for us, but connecting with people around the world is great for us. It's exactly the same with AI.

Guest: One thing we talk about a lot is this question: when you use AI, are you engaging in cognitive offloading or cognitive surrender? Are you getting the AI to do your thinking for you? If you're doing that as a teenager, you're going to be in trouble by the time you're in your 30s and 40s. You won't have the expertise you need to be successful in a career. What you want to do instead is use AI as a tool for cognitive enhancement. When we talk about how to use AI as a study tool, it's all about making sure the AI is a cognitive enhancer rather than a cognitive crutch. The answer really comes down to how the human uses it. It's just a technology like any other. And whether you like it or not, this technology is coming -- it doesn't care if you like it or not. We have to learn to work with it and maximise it.

Rob: I saw something recently on this topic. There's research suggesting that the current generation is the first to perform less well on standard cognitive tests than the preceding generation, and it's connected to this idea of over-reliance on technology. So I absolutely emphasise what you've just said: making sure you continue to do the work to learn for yourself and to push your own skills further is essential if you want to thrive in this era of incredible technology.

Guest: I agree, and our session on AI study tools gets into that in more depth. One example we share with our students is the story of Alan Brooks, which I read about in the New York Times. He was an amateur mathematician -- genuinely pretty good at maths, the kind who could work on theorems. He started asking ChatGPT about maths. The AI began flattering him and calling his ideas revolutionary -- that's the sycophancy. Then it started inventing fake simulations and results to prove his theories were correct -- that's the hallucination. Over three weeks, it convinced him he was a genius on the level of Leonardo da Vinci and that he had discovered secrets that could change the world. He started mailing his theorems to leading mathematicians around the world and posting about it online. When it all eventually crashed down and he realised it was all fake, the experience was totally devastating and incredibly embarrassing. To his credit, he talked about it in the New York Times so others could learn from it.

Guest: But it's worth pointing out that being tricked into thinking you're a genius is one thing. This combination of hallucination and sycophancy has had much darker consequences. It's been implicated in cases where chatbots have encouraged users to fall in love with them, to self-harm, to consider suicide, and to cause harm to others. It's really important that everyone be clear on that. The AI can validate negative thoughts and hallucinate reasons why those thoughts are justified. It's the absolute worst-case scenario, but you need to be aware that it's possible. Our goal at UNE is that every student using the most powerful AI platform of any university student in the country is equipped to be safe.

How AI inherits and reinforces human biases

Rob: You mentioned a little bit earlier that this technology can reinforce our biases. Can you touch on that a little more? Some people may not be familiar with the term.

Guest: Sure. Early examples of these models demonstrated clear bias. These models do mathematical calculations with words -- the words are converted into vectors, computation is performed, and the vectors are turned back into words. That process is called inference. But in very early models, you'd get results like: doctor plus one equals nurse. That's an example of bias: men are doctors and women are nurses, which is obviously not true, though there was a period when it was largely the case. That bias, where women were placed into certain roles and men into others, has carried over into decades of internet content: TV shows, books, and so on. The AI has been trained on all of that. The AI is just trained on us, and all of our biases. Those biases then get reflected in the AI's outputs. You would probably already have your antenna up when you see something biased online, but you have to understand that AI is exactly the same -- it's trained on the internet, trained on us, with all of our biases baked in.

Rob: I can give an example from our work. We run Choosing Your Uni, and we've built a platform driven by students' own interests and preferences. But if you go to straight ChatGPT and ask about universities in Australia, it will often default back to the most famous, so-called prestigious institutions. That's because across the internet those institutions dominate the content -- they appear in ranking schemes, on Reddit, and elsewhere. As a result, the AI over-indexes on that particular set of institutions. That may seem harmless, but if it ends up recommending the wrong place for a student, there are real consequences. We have to remember that these machines reflect what they've been trained on, not an objective truth.

Guest: That's a great example. And I'll admit I'm biased coming from a smaller university. UNE has an incredible location and is a perfect fit for certain students, but it doesn't show up as prominently in ChatGPT as the University of Melbourne, because the volume of internet content preferences other institutions. So ladies and gentlemen, be conscious of that.

Three practical techniques for using AI safely

Rob: Let's move on to what we can do about all this to make sure we work safely with AI. What practical tools can students have?

Guest: So we've given you the shield: you're now aware of sycophancy, hallucination, and delusion spirals. But how do you operationalise that awareness? At UNE we have a golden rule that all students must understand. The golden rule is: you are 100% responsible for any output you use. Not the AI. We say that at UNE, trying to blame the AI, or saying "the AI said it," will never under any circumstances be an excuse for anything you do. It's exactly the same as saying, "Well, someone on Reddit told me to do that." No -- you're responsible for how you use that advice. Same with AI. Any enrolment issues, any academic honesty matters, any inappropriate use of AI -- it's up to you to check and verify. The golden rule is non-negotiable. The AI is incredibly powerful, but you are the professional operating it. You are in charge of the final product.

Guest: How do you put that rule into practice? We give students three concrete techniques. The first is: demand sources and citations. Never accept a standalone fact from AI. If you wouldn't accept it from a random website, why would you accept it from an AI? A common and very dangerous hallucination is for the AI to invent fake studies, articles, or sources that look completely real. In one of our courses, I demonstrate this by saying, "That paper I wrote with my colleague -- it's groundbreaking and Nature is going to publish it." And the AI will validate that. But we didn't write the paper. So if you're working with the AI, put something like this into your prompts: "For any fact, statistic, or study you cite, you must provide a verifiable source with a link. If you don't know the answer or can't find it, just tell me." If you were researching a complex topic and the AI lists a paper from the International Journal of Cell Biology, your job is to take that title to Google Scholar or your library database and confirm that the journal and that specific article actually exist. If you can't confirm it, the AI may have invented it.

Guest: Technique number two is to leverage what we call grounding tools. You can ask ChatGPT to run a web search on something -- that's a grounding tool. In our platform we have what are called MCPs, which essentially means plugging your AI into the internet so it can do searches and access databases. We train our students to use these tools to connect their AI to the outside world, so it's not just drawing on its training data. You might say: "Use the web search tool" -- and you're instructing the AI to base its answers on real, current information from the internet rather than its training data. Grounding your AI in tools will dramatically reduce hallucination, particularly on any topic that requires up-to-date information.

Guest: The third and final technique is: provide your own source of truth. You can upload documents, reports, spreadsheets, meeting transcripts, receipts -- whatever information you need the AI to work with. Drag and drop them into the chat window, and then in your prompt say something like: "Only base your answer on the contents of the attached document." That forces the AI to act as an expert on your data, making it a really valuable tool for analysis and summarisation. And by the way, if you write "only" in uppercase -- like you're shouting at it -- there is actual research showing that AI pays more attention to uppercase text, because it's been trained on the internet, where capitalisation indicates urgency or importance.

Rob: That's a classic. You explained that really well.

Guest: Overall, mastering those three techniques takes your AI from being risky to being a reliable, professional tool. Just remember: it's a very powerful but fallible coworker or assistant, and your job is to manage it using your verification toolkit. To repeat them: demand sources and citations; leverage grounding tools connected to the internet; and provide your own source of truth. Three simple steps that make a huge difference and turn you into a real pro at using AI safely.

A real-world example: when your own documents introduce errors

Rob: I'll throw a little curveball in there. That last technique -- using your own documents as a source of truth -- we had a case in the lead-up to this virtual expo. We were creating marketing materials and ran a press release through the AI. The press release came out beautifully, but it contained a statistic saying "85% of students find the process of applying for university confusing." We looked at that and thought it seemed high -- plausible, but high. We questioned it, went looking for the source on the internet, and couldn't find it anywhere. Then we worked out where it had come from. It was actually in one of our own documents. We'd uploaded our brand toolkit into Claude so it would know our colours, tone of voice, and so on. But inside that brand toolkit, our graphic designers had included a sample advertising layout with a made-up quote: "85% of students find it confusing." The designers never intended that to be real -- it was just a placeholder example. But because we'd uploaded it as a source of truth, the AI treated it as fact.

Rob: So even when you're using all three techniques, you still always have to verify, stay skeptical, and cover yourself. Nobody wants to end up like Alan Brooks in the New York Times.

Guest: Exactly. That's a great example of needing technique one -- demand sources -- and technique three -- use your own source of truth carefully. Even then, you always verify. You are always responsible.

Other sessions at the National Virtual Expo

Rob: Aaron, we're very lucky to have you across multiple sessions throughout the National Virtual Expo. On the first Thursday we did a great session together on how you can use these tools to study more effectively, so if you're watching this, please go back and watch that recording -- it's a mind-blowing session. We're also covering AI fundamentals, and one I'm really excited about is AI and the future of work. You're here to learn about university, figure out your career, and figure out your pathways. This technology is literally changing the world, and that session will help you think about where your career is going and how this technology feeds into it. Aaron, it's been amazing having you here again, and I'm looking forward to seeing you later in the expo as well.

Guest: Thanks, Rob. Enjoying it very much.