The Australian government has finally come face to face fully with the elephant in the room it had tried mitigating or preventing early on, that the potentially destructive impacts of AI has expanded beyond the boundaries of safe image and video generation.
In a society already battling with revenge nude leaks and a 57% rate of reported digital sexual harassment (ABS, 2022), AI tools has opened the floodgates to previously unprecedented levels of technology-facilitated sexual violence, with a particular concern congregating about its impact on children and its proliferation of child abuse material.
Although we commend the recently introduced legislations such as the Federal Criminal Code Amendment (Deepfake Sexual Material) Act 2024, the fact that such responses are reactive rather than a preventative method of inhibition against undesirable outcomes means there will always be victims harmed by AI tools in sexual harassment.
We must fundamentally recognise that the root cause lay in a lack of education and moral grounding in what AI should and should not be used for. Our first step should always be education and information awareness, and current state and federal plans simply do not show much intention beyond a vague framework of promising to raise awareness generally.
The end user inputting potentially harmful prompts must be proactively both deterred by legislation and legal consequences whilst being given the opportunity to be advised on using these tools ethically before they become a bad actor in order to restrict this type of behaviour.
Sadly, such a rollout, if it happens, must place particular concern on protecting the wellbeing of children both as victims of sexual abuse material by AI as well as unwitting perpetrators if not given the correct guidance.
It is increasingly difficult to look away from what is happening in Australian schools, particularly those already with pre-existing cultures of bullying. With just a single photograph and a few taps on a screen, AI-powered nudify applications can generate non-consensual explicit images of real people, including children, in a matter of seconds.
Both public findings to the national eSafety Commissioner as well as our own research and surveys show the same thing: increasingly serious incidents of abuse.
Reports to eSafety's image-based abuse scheme involving digitally altered intimate images of those under 18 have more than doubled in the past 18 months alone, exceeding the total number received across the seven years prior. Four out of five of those reports involved young girls as the target.
Telling its findings to SBS, eSafety Commissioner Julie Inman Grant had been explicit: what is being reported is almost certainly not the full picture. While it is clear now to any federal or state inquiry into the level of harm targeting children within this abuse of AI tools, the amount underreported or omitted due to its capacity to occur at home beyond school environments could show a much more serious problem.
What makes this particularly confronting is the range of motivations driving it. Criminology researcher Asher Flynn from Monash University has noted that the creation of deepfake content is increasingly being normalised among young people, with motivations ranging from sexual gratification and deliberate harm to something as casual as building social status or satisfying curiosity.
The psychological and reputational consequences mirror those of other forms of technology-facilitated sexual violence, but remain far more dangerous in their replicability.
The technology itself is fast, free and widely accessible. It does not require technical expertise. It does not require anything more than a photograph already shared on a social media profile.
Australia has begun to respond, yet we reiterate the need for general education regarding ethical implications to be implemented within the national framework. New mandatory standards carry penalties of up to $49.5 million for platforms that breach their obligations, and eSafety has released updated toolkits urging schools to report potential criminal offences directly to police.
These are meaningful steps, but in any case mere suggestions and toolkits are useless if not implemented or executed.
As long as young people are given advanced tools without the moral framework to use them responsibly, the issue will persist, no matter how tight regulations seek to restrict their use.