The rapid developments and use of AI and generative AI technology, and the resultant rise of deepfakes, which are becoming increasingly difficult to detect both by the human eye and by existing detection technologies are raising concerns on the safety and trust of AI systems.

Given the evidence of the potential impact that generative AI content and deepfakes can have on society, they should not be overlooked. The use of AI to develop and propagate misinformation should also be addressed. Generative AI use in content creation could make it more challenging for creators to attest and defend ownership of their content. From a transparency perspective, the provenance of data used to train AI models is very important. Knowing where the data comes from helps to establish authenticity and reliability, and understand potential issues surrounding accuracy, bias, or licensing. Being able to verify the authenticity and owner of generative AI multimedia content is a necessity to protect the digital rights of the person or organization.

Governments are already working towards setting policy measures, codes of conduct and regulations to address the challenges of deepfakes and generative AI. Technical standards will play an important role in supporting the policy measures and regulations being introduced by governments.

This was highlighted during discussions at the AI Governance Day and in the Workshop on Deepfakes Detection and Generative AI during the AI for Good Global Summit 2024.

During the AI for Good Global Summit 2024, work and progress was discussed among a diverse range of organizations including Content Authenticity Initiative (CAI), Coalition for Content Provenance and Authenticity (C2PA), IETF. IEC, ISO, ITU and JPEG. In order to make progress as an industry as a whole it was agreed to set up a multistakeholder collaboration on global standards for AI watermarking, multimedia authenticity and deepfake detection technologies.

The standards collaboration objectives will be to:

1. Provide a global forum for dialogue on priority areas and requirements for technical standards for AI watermarking, multimedia authenticity and deepfake detection technologies.      
2. Map the landscape of technical standards for AI watermarking, multimedia authenticity, and deepfake detection and how they support the policy and regulatory requirements of governments, transparency and compliance with legal obligations such as protection of privacy of users, rights of consumers, digital copyright and intellectual property rights.
3. Facilitate sharing of knowledge on lessons learned from different stakeholders to disseminate information about technical standards and emerging technologies to address these issues and how they can be implemented to support government policy measures with regards to deepfakes and authenticating generative AI content.    
4. Identify gaps where new standards are required given the dynamic nature of generative AI and deepfake technologies.

The standards collaboration will be open to international, regional and national standards organizations, governments, tech companies, industry initiatives on AI watermarking, multimedia authenticity and deepfake detection and other relevant organizations involved in this space.