AI tutors could be coming to the classroom—but who taught the tutor, and should you trust them?

The government recently announced ambitions to expand the use of artificial intelligence (AI) in New Zealand's classrooms. But as the technology rapidly changes, it is not clear how this would work or what it will mean for teachers and learners.

Science Minister Judith Collins' vision is for every student to have their own AI tutor. As Collins explained in a recent interview, "So instead of having to be wealthy enough to employ a tutor to help the children with the math or science questions, or something else that the parent doesn't know much about maybe, is to enable that child to have their own [AI] tutor."

But like AI itself, the concept of an AI tutor is still evolving. The idea of creating a "teaching machine" has been around for 100 years or so, and "intelligent tutoring systems" have been around since the 1980s with limited results.

The more recent advances of AI have rekindled the elusive promises of these systems. But while the technology has evolved, the basic concept of a machine taking over some of the responsibilities of the teacher has remained the same.

The risk of replacing human tutors

An AI tutor is a proxy for a human tutor—supporting and "scaffolding" a student's learning. Scaffolding is the space between what a learner can do without assistance and what they can learn next with the support of someone who is more knowledgeable.

In theory, an AI Tutor can play this role. But there are inherent dangers. What if your more knowledgeable tutor is not, in fact, more knowledgeable, but just makes things up? Or shows bias? Or favors uncritical, shallow material over more reliable resources?

The features that give generative AI its capabilities to interact with users also create its flaws. AI relies on the data it is trained on. However, this data can be wrong, and AI neither validates what goes into it, nor what comes out.

This issue has raised concerns about fairness. As AI tools consume quantities of unfiltered data, the risk is they will reinforce existing biases in this data, perpetuating gender stereotypes and other negative outcomes.

For people from Indigenous cultures, including Māori and Pacific peoples, AI provides both opportunities and threats.

If AI systems are trained on biased data or without considering diverse perspectives, there is a high likelihood decisions being made based on these systems will favor one group over others, reinforce stereotypes, and ignore or undervalue different ways of living and thinking.

The concern isn't just about the influence AI can have on us but also how AI consumes and processes data. AI systems are trained on vast amounts of data, often without properly acknowledging the sources or respecting creators' copyrights.

For Indigenous peoples, this can infringe upon their data sovereignty rights and exploit their cultural and knowledge heritage. This exploitation can perpetuate inequality and undermine the rights and contributions of Indigenous communities.

A 'walled garden' approach

A commonly proposed answer to this problem is to train AI systems on carefully curated data.

Book publisher Pearson, for example, has recently integrated AI in 50 of their textbooks. This allows students to use AI chatbots to engage with the texts.

According to Pearson, these tools are developed using a "walled garden" approach. The AI is trained only on the contents of these books. This, Pearson claims, reduces the risks of inaccuracies.

However, the walled garden approach also has major drawbacks, since it limits content to that selected and approved by the supplier. What does this mean for cultural knowledge and rights? Critical perspectives? Innovation in learning?

Pearson has, for example, been criticized for the content of some of its books. In 2017, the company apologized for a medical textbook considered "racist."

If a New Zealand AI tutor were to be created from local data, how could we ensure tikanga Māori protocols are safeguarded? As highlighted by Māori scholar Te Hurinui Clarke, there are significant challenges around the respectful and ethical handling of Māori knowledge.

Protecting knowledge

When it comes to AI tutors, policy makers need to ask who would be the custodians of this data, whose knowledge would be used and who has the rights to access?

If done well, a walled garden approach might provide a comprehensive, inclusive, culturally sustaining pathway to better learning. However, given the challenges of such an undertaking (never mind the expense), the chances of success in practice are extremely small.

Meanwhile, we can't just wait for AI tutors. AI is a reality in schools, and we need to prepare students for what they face now and in the future. Specific tools are important, but our focus should be on developing AI literacy across the educational sector.

This is why we are researching what it means to be AI literate and how this can empower critical evaluation and ethical use, ensuring AI complements rather than replaces human teaching.

We see the development of AI literacy, supported by suitable frameworks, as a priority, something all students, no matter their age, need to have. It is only through this that we can harness AI's benefits while safeguarding the core values of education.

This article is republished from The Conversation under a Creative Commons license. Read the original article.