Login | October 22, 2024
NIST looking to assess Gen AI before it’s too late
RICHARD WEINER
Technology for Lawyers
Published: June 7, 2024
The National Institute of Standards and Technology (NIST) has just launched a new platform designed to assess generative AI technologies, including text- and image-generating artificial intelligence.
NIST is an agency of the Department of Commerce and, while it does not have enforcement powers, is the go-to standard-maker for creating the benchmarks used by the rest of government to evaluate technologies.
The plan is for the users of the platform to be able to create benchmarks that will catch people creating deepfakes through “content authenticity” detection.
In particular (most interesting) is the statement that the new platform promotes “the development of technologies for identifying the source of fake or misleading information.”
The goal there, presumably, is to develop benchmarks to help identify fakes for USPTO copyright purposes, and maybe for political purposes to identify AI-generated fake info from state actors designed to disrupt the US political system.
According to the press release of the launch, “The NIST GenAI program will issue a series of challenge problems [intended] to evaluate and measure the capabilities and limitations of generative AI technologies. These evaluations will be used to identify strategies to promote information integrity and guide the safe and responsible use of digital content.”
The idea is to create a pilot study to build systems that can detect AI-generated content.
To do this, NIST is inviting teams from academia, industry and research labs to submit either “generators,” which are AI systems to test, or “discriminators,” which are systems designed to ID AI.
NIST will provide the data to use for the generators and the discriminators, so that the systems will not be trained on publicly available data (as is the current AI-training data).
The platform is a tacit (or explicit) acknowledgment that current AI-testing systems are not working in real life, and that the government is trying to hurry up and create some before the AI apocalypse.
Deepfakes are up 900%, so the time is now.
The registration period for Round One submission will last through August.
Here is the new NIST GenAI website: https://ai-challenges.nist.gov/genai. Give it a try.