Researchers: Australia has its first framework for AI use in schools, but it needs to proceed with caution

Federal and state governments have just released a national framework for generative AI in schools. This paves the way for generative AI—algorithms that can create new content—to be used routinely in classrooms around the country.

This provides much-needed guidance, a full year after the launch of ChatGPT. Over the past 12 months, schools have had a range of responses to the technology from outright banning to trying to incorporate it into learning.

What is in the framework and what is missing?

What is the framework?

The framework was agreed by state and federal education ministers in October and released publicly last week.

It is designed to help schools use generative AI "in a safe and effective way." It notes it has "great potential to assist teaching and learning and reduce administrative workload in Australian schools." But at the same time it warns of risk and consequences, including "the potential for errors and algorithmic bias in generative AI content; the misuse of personal or confidential information; and the use of generative AI for inappropriate purposes, such as to discriminate against individuals or groups, or to undermine the integrity of student assessments."

Federal Education Minister Jason Clare also stressed "schools should not use generative AI products that sell student data."

What is in the framework?

The framework itself is just two pages long, and includes six overarching principles and 25 "guiding statements." The six principles are:

• teaching and learning, including schools teaching students about how these tools work, including their potential limitations and biases
• human and social well-being, including using tools in a way that avoids reinforcing biases
• transparency, including disclosing when tools are used and their impact
• fairness, including access for people from diverse and disadvantaged backgrounds
• accountability, including schools testing tools before they use them, and
• privacy, security and safety, including the use of "robust" cyber-security measures.

The framework will be reviewed every 12 months.

Caution is needed

The framework does important work acknowledging opportunities of this technology, while noting the importance of well-being, privacy, security and safety.

However, some of these concepts are much less straightforward than the framework suggests. As experts in generative AI in education, we have moved from optimism to a much more cautious stance about this technology over the past 12 months. As UNESCO has recently warned, "the speed at which generative AI technologies are being integrated into education systems in the absence of checks, rules or regulations, is astonishing."

The framework puts an extraordinary onus on schools and teachers to do high-stakes work for which they may not be qualified or do not have time or funding to complete.

For example, the framework calls for "explainability"—but even the developers of AI models struggle to fully explain how they work.

The framework also calls on schools to do risk assessments of algorithms, design appropriate learning experiences, revise assessments, consult with communities, learn about and apply intellectual property rights and copyright law and generally become expert in the use of generative AI.

It is not clear how this can possibly be achieved within existing workloads, which we know are already stretched. This is particularly so when the nature and ethics of generative AI are complex and contested. We also know the technology is not foolproof—it makes mistakes.

Here are five areas we think need to be included in any further version of this framework.

1. A more honest stance on generative AI

We need to be clear that generative AI is biased. This is because it reflects the biases of its training materials, including what is published on the internet.

Such limited datasets are created largely by those who are white, male and United States or Western-based.

For example, a current version of ChatGPT does not speak in or use Australian First Nations words. There may be valid reasons for this, such as not using cultural knowledges without permission. But this indicates the whiteness of its "voice" and the problems inherent in requiring students to use or rely on it.

2. More evidence

The use of technology does not automatically improve teaching and learning.

So far, there is little research demonstrating the benefits of generative AI use in education. In fact, (a recent UNESCO report confirmed there is little evidence of any improvement to learning from the use of digital technology in classrooms over decades.

But we do have research showing the the harms of algorithms. For example, AI-driven feedback narrows the kinds of writing students produce and privileges white voices.

Schools need support to develop processes and procedures to monitor and evaluate the use of generative AI by both staff and students.

3. Acknowledging dangers around bots

There is long-standing research demonstrating the dangers of chatbots and their capacity to harm human creativity and critical thinking. This happens because humans seem to automatically trust bots and their outputs.

The framework should aim to clarify which (low-stakes) tasks are and are not suitable for generative AI for both students and teachers. High stakes marking, for example, should be completed by humans.

4. Transparency

So far, the framework seems to focus on students and their activities,

All use of generative AI in schools needs to be disclosed. This should include teachers using generative AI to prepare teaching materials and plan lessons.

5. Acknowledging teachers' expertise

The global education technology ("edtech") market was estimated to be worth about US$300 billion (A$450 billion) as of 2022. Some companies argue edtech can be used to monitor students' progress and take over roles traditionally done by teachers.

Australia's national education policies need to ensure teachers' roles are not downgraded as AI use becomes more common. Teachers are experts in more than just subject matter. They are experts in how to teach various disciplines and in their students' and communities' needs.

This article is republished from The Conversation under a Creative Commons license. Read the original article.