A 'ChatGPT'
...for
Satellite Photos already Exists
by Patrick Tucker
April
17, 2023
from
DefenseOne Website
PDF version
Using
advanced generative AI
and a
massive dataset of Earth images,
it's
possible to discover objects
almost
anywhere in just hours...
Scene:
A U.S.
adversary is at work on a new type of drone, ship, or aircraft
and it's your job to find it, wherever it is...
Not long ago, that task would take a massive effort of human,
signals, and open-source intelligence collection.
But a researcher from
AI company Synthetaic has created a tool that will allow users
to find virtually any large object that exists in any satellite
photo of the Earth within just one day.
It's also the sort of
capability the National Geospatial-Intelligence Agency (NGA) is also
looking to develop, and it could radically shift strategic
advantage on the battlefield.
Corey Jaskolski,
founder and CEO of
Synthetaic, dubbed his satellite image
scanning tool Rapid Automatic Image Categorization, or
RAIC.
After the
Chinese weather balloon incident caught the nation's
attention in January, Jaskolski applied RAIC to satellite photos
of the Earth's surface, as collected by geospatial satellite
imaging company
Planet.
He was able to
trace the balloon's origins to China in just a matter of days.
Now, Jaskolski
says, the company is using those lessons to further reduce the
time.
"Our goal
is to be able to ingest the entire Planet daily take [of
Earth images] and be able to process that all in less than
24 hours.
So if you
wanted to literally look for balloon launches around the
entire world, we could give you a daily update of that every
day.
Let you know if there was a balloon launched anywhere."
Interest around
new publicly available AI tools has been spiking, thanks to new
generative pretrained transformer - or
GPT - tools that allow
users to write essays, build business plans, and perform complex
tasks with a simple prompt.
The national
security community has a similar need, but for AI applications
for the vast expanse of satellite, surveillance, and other data
that could help uncover adversary activities and new
capabilities.
But it's not
necessarily a straightforward task, as
Jaskolski learned when he attempted to find the origin of
that Chinese balloon - a thing that had never been photographed
in the open, much less
labeled and inserted into a dataset readable by a
machine-learning algorithm.
"Normally
with an AI, you have to have a bunch of labeled examples for
the AI to learn, so, and it's not a small amount of data.
Like when
Facebook and Google train an AI, they commonly train on a
billion labeled images, not even, you know, thousands or
millions, but literally a billion labeled images," Jaskolski
said.
"The thing
that would normally stop an AI from finding this balloon is
we don't have any data. We don't have any labels.
We don't
know what it looks like from space."
The RAIC is
part of a new class of AI tools that don't require a massive,
labeled dataset to generate what
Jaskolski describes as an understanding of what to look for.
He was able to
teach it to look for the balloon based only on a single
hand-made drawing.
"We started
out with technologies that are used for generative
AI transformers and
GaNS.
[But]
instead of using that technology to generate images, we use
that technology in order to basically understand the data
domain," he said.
In essence, by
continuously looking at satellite images, the RAIC tool develops
a familiarity that comes close to expertise.
So when it
scans satellite imagery, it has a rudimentary understanding of
what's unusual, and can look for specific unusual objects.
And the input
doesn't have to be precise.
Jaskolski says his drawing depicted what a balloon might
look like in satellite data, and RAIC was able to find it.
Then, once they
found the actual balloon in one of the satellite datasets, RAIC
was able to look for that in other images.
"After a
couple days really searching for it in Alaska in Canada, we
decided to just bite the bullet and ingest that massive
amount of Earth across China, Japan, South Korea, North
Korea, and the ocean, open ocean and Aleutian islands," he
said.
They also used
wind modeling to narrow down where the balloon may have started
its flight.
That brought
them to islands 300 miles off the coast of China.
"At that
point we got really excited… And so from there, we find it
five or six more times, all the way back to the hidden
island."
At last week's
Planet conference in Washington, Microsoft President Brad
Smith described a future in which people could ask
image-based search tools to find objects, just as we ask search
engines for recommendations today.
Microsoft is a
major investor in
OpenAI, the best-known GPT platform.
"I do
believe that this next era of AI, you know, with GPT
based-technology, is a query-able Earth," Smith said.
NGA has already
taken control of
Project Maven, the Pentagon's flagship AI
program for image analysis.
At the Planet conference, NGA head
Vice Adm. Frank
Whitworth said the agency is trying to turn Maven from an
experimental effort into a program of record,
"which
means we will need to be very clear on the efficacy of every
dollar."
The agency is,
"experimenting with [geographical intelligence] AI programs
that integrate large language models to allow analysts to
ask and answer specific intelligence questions," an NGA
spokesperson told Defense One.
"We see a
future where these models can be trained with big spatial
data to answer questions in space and time."
