In what sounds like a scene from a comedy movie, a squad of Marines successfully outsmarted an advanced artificial intelligence system by employing tactics that would make any child playing hide-and-seek proud. The remarkable demonstration revealed both the impressive capabilities and surprising limitations of modern AI technology.
The experiment took place as part of DARPA’s Squad X program a while back. It aimed to develop advanced surveillance systems capable of identifying human threats in complex urban environments. The AI had undergone extensive training, spending six days learning to recognize Marines as they moved through various scenarios.
“What DARPA was working on was developing the ability to identify people in complex urban environments,” explained Paul Scharre, author of the book ‘Four Battlegrounds: Power in the Age of Artificial Intelligence,’ which documents this fascinating encounter. “And sense people approaching the squad.”
The system seemed sophisticated enough. As Phil Root, deputy director of the Defense Sciences Office at DARPA, noted the complexity of human detection: “A tank looks like a tank, even when it’s moving. A human when walking looks different than a human standing. A human with a weapon looks different.”
After days of training, the moment of truth arrived. Root set up the ultimate challenge: “If any Marines could get all the way in and touch this robot without being detected, they would win. I wanted to see, game on, what would happen.”
The results were both humbling and hilarious for the AI developers. “Eight Marines — not a single one got detected,” Root recounted.
The Marines’ tactics were as creative as they were effective. Two determined servicemembers somersaulted across 300 meters of terrain to approach the sensor undetected. Perhaps most memorably, another pair simply threw a cardboard box over themselves and walked right up to the system.
“You could hear them giggling the whole time,” Root remembered of the box-wearing Marines, capturing the almost playful nature of their successful deception.
Another Marine demonstrated remarkable ingenuity by stripping a fir tree and holding it in front of himself as camouflage while approaching the sensor. The improvised disguise worked perfectly.
The experiment exposed what experts call “distributional shift” – a phenomenon where AI systems trained on specific datasets struggle when confronted with scenarios outside their training parameters. The AI had learned to identify people walking normally, but it had never encountered somersaulting humans or individuals disguised as moving trees.
“An algorithm is brittle, and the takeaway from this is that there will always be these edge cases,” Scharre observed. “The real problem for the military is that it operates in an inherently adversarial environment, and people will always have the ability to evolve.”
While AI systems can process vast amounts of data at incredible speeds and excel at specific tasks, they lack the broader understanding that allows humans to think creatively and adapt to unexpected situations. “Humans tend to have a much richer understanding of the world,” Scharre noted, highlighting why the Marines could so easily outmaneuver a system designed to detect them.
It seems like impressive AI performance in controlled conditions doesn’t necessarily translate to competence in real-world scenarios. As Scharre explains, there’s a dangerous tendency for “mistaking performance for competence” when it comes to artificial intelligence capabilities.
The author emphasized that this experiment shouldn’t be viewed as a definitive judgment on AI’s current capabilities, noting that artificial intelligence continues to advance rapidly. The challenge for military organizations lies in “creating doctrine to rapidly spin in what AI technology can do” while understanding its limitations.
While artificial intelligence can be a powerful tool, human creativity, adaptability, and good old-fashioned ingenuity remain invaluable assets. The Marines’ victory wasn’t just about outsmarting a machine – it was about showcasing the uniquely human ability to think outside the box, sometimes quite literally.