On Tuesday, Niantic, the company behind Pokémon Go, announced the roll-out of a hyper-detailed mapping system they hope will pave the way for the expanded use of augmented reality (AR) in everyday life. For years, Niantic has been collecting spatial data from highly trafficked public spaces in cities across the world, and now it wants to give these immersive and precise maps to developers to create their own games, history lessons and audiovisual experiences. Some privacy experts, however, are concerned about the onset of AR and what it means for public surveillance.
AR consists of digital elements that appear within real-world environments, which can be seen either through one’s phone or smart glasses (like the defunct Google Glass). Instagram and Snapchat filters, for example, allow you to alter your appearance, voice, or the surrounding scenery.
Niantic CEO John Hanke believes that AR will continue to be incorporated into everyday life in increasingly sophisticated ways. For example, in the future, you might be able to see directions to your destination overlaid on the streets as you walk them, via smart glasses; or you could look at a building site and see its history and past architectural incarnations.
His company’s new 3D map will allow developers to see exactly how their users move through the world and view their surroundings, which Hanke says is a key step towards “opening this science fiction reality we dream about. We can start putting virtual things into the world that are attached to the right parts of that physical world,” he said in an interview.
The map was created by culling more than a hundred million video snippets from Niantic players (who opted into sharing their screens), developers and surveyors. The most developed parts of the map are in dense city centers where they’ve collected the most data, including San Francisco, London, Tokyo, and New York City.
Independent AR developers will now have access to the technology—starting free in beta—on a platform called Lightship. There they might create virtual animals that hide behind park benches; plaques for public statues delving into forgotten histories; or mobile characters “painted on the ground showing the path to your destination,” says Hanke.
One of the first projects created with the new map’s technology is a collaboration with the artist JR and the art company Superblue called JR Reality. JR is known for pasting giant photos of people in public places. Niantic’s technology allows users to add their own JR-style portraits or voice messages to specific public locations, adding to those of everyone else who have made the same pilgrimage.
“Have you ever passed somebody on the street and wondered what their story is?” JR wrote in a statement. “It’s time to go outside and explore, and reconnect with one another and show the world your face again. Together we can tell the world your story and meet the amazing people that live in your city.”
Hanke, meanwhile, is particularly excited about how the map might improve Niantic’s flagship product: Pokémon Go. A viral commercial that introduced the game in 2015 showed people watching the animated character Mewtwo soar over Times Square, but most people view the game through animated graphics on their phone. “This map paves the way for that giant Mewtwo above the crowd: to have the shared experience of many people watching the same thing from different vantage points,” Hanke says. “This is what we’ve been dreaming of since the beginning.”
Pokémon Go was arguably the first successful commercial use of AR. It was downloaded more than 500 million times in its first year, with aspiring trainers hunting Pokémon all over the world. Jacob Navok, the CEO of Genvid Technologies and one of the leading thinkers on the metaverse, says that Niantic’s pedigree means their new map will likely be a sought-after tool for AR developers. “Releasing more, better and cheaper tools to outside developers is key to the growth of the sector,” he wrote in an email to TIME. “Developers who choose Lightship will do so knowing that Niantic understands how to build product and not just APIs [programming code].”
As AR technology advances, many watchdogs have become worried about its impact on privacy. They worry that companies like Niantic will collect an astonishing amount of information that would be vulnerable to hacks, and that people’s movements could be tracked. “Every image that an AR device records is potentially one court order away from becoming a government tracking device,” says Albert Fox Cahn, the executive director of the Surveillance Technology Oversight Project. He points to search warrants that have collected recordings from Amazon’s Alexa and Ring cameras. Cahn also thinks that geofence warrants, in which police seek location data on every person within a specific location over a certain period of time, could be being particularly threatening to AR users. (One of those warrants was just ruled unconstitutional in Virginia, however.)
“It’s something we take very seriously,” Kjell Bronder, a product lead for Lightship, Niantic’s developer platform, says regarding privacy. “For the contributions that these players can upload [to the map], we do everything in the background to anonymize and make sure that a lot of this data is scrubbed.”
Niantic is also releasing its own social media platform: Campfire. The app will be map-based, with users able to see which other users are in a local area, message them, share content and organize events. “Our big thesis is that the metaverse is something that happens out in the real world,” Hanke says.
