Microsoft’s Flight Simulator 2020 is a technological masterpiece. Reviews are nearly uniformly glowing, adopting tones both confessional (“the most incredible experience I’ve ever had on a computer”) and epochal (a “once-in-a-generation wow moment”). The game’s devotion to realism begins in the mechanical and aesthetic accuracy of the aircraft you pilot but is also extended, in breathtaking fashion, to the terrain below. The earth appears as a vast global sandbox, fully navigable from corner to corner. So unbearably real is this earth, gushed New York Times journalist Farhad Manjoo, that “nearly all sense of abstraction falls away.” Any architect or urbanist is likely to find the simulation just as breathtaking.
As a player, you assume control of this sublime simulation, transporting yourself to nearly any airport worldwide and summoning up a vast array of planes on the spot. You are then let loose to joyride through the world, re-created down to the leaves on trees and up to continental mountain ranges, and from the dense cores of major cities to the remotest reaches of the planet. The magnitude of detail on display represents an enormous quantity of technical-industrial power and expertise that goes far beyond the capabilities of 3D modeling of space by humans. Microsoft and developer Asobo Studio turned to Austrian AI start-up Blackshark.ai, which is already engaged in the creation of an authentic “digital twin” (remember that phrase) of the planet.
Blackshark’s task entailed the creation of not just architecture—though, by the firm’s estimation, 99.8 percent of the world’s structures had not previously been modeled—but also swaths of vegetation and topography. The developers began with Bing satellite imagery and the collected photogrammetric 3D modeling data for about 400 cities that were already available on Bing Maps. This repository of raw material was then fed into a “deep learning neural network [that] segments and classifies buildings, vegetation, and roads globally, and a patented approach is used to reconstruct building attributes in highly detailed 3D.” This resulting world-data is then hosted on the Microsoft Azure cloud, where it is continually updated and refined. Blackshark’s demiurgic work is ceaseless, but already quite expansive; to date, the company maintains that it has re-created 1.5 billion structures and populated 30 million square kilometers (11.5 million square miles) of vegetation.
But before any of Blackshark’s worldbuilding can begin, the existing data must be stripped down and decontextualized. Information about buildings is categorized by typology and regional differences—that is, not as high-fidelity spatial artifacts on their own, but as mix-and-match parts that can be understood by the system. Typically, Blackshark’s algorithm may detect a building footprint, estimate its height based on the shadow, and then apply facade details, roof furniture, and so on—with the final product not necessarily corresponding to what exists but constituting a generic version of what likely would exist. Blackshark CEO Michael Putz admits that the actual process of this algorithmic “construction” is an arcane “black box,” unknowable and vague. This black box occasionally suffers from memorable hiccups that escape quality checks, such as the Washington Monument appearing as a pseudo–Seagram Building, the sudden existence of a slender monolith in the suburbs of Melbourne, Australia, and an abyssal pit in Brazil.
This process is less akin to a cartographer meticulously preparing a map than an automated factory churning out commodities. A slew of data and raw material inputs (point cloud data, satellite photos, etc.) go in, technological wizardry is applied, and finished objects (fake buildings to populate a virtual planet) emerge on the other side. The sheer scope of Blackshark’s project makes this inevitable; having human workers determine and build structures and their design elements directly is too costly and time-consuming. Flight Simulator 2020 is not about accuracy; it’s about instilling awe at the ability to approximate the planet in its entirety quickly and, thanks to the infallible logic that is often imputed to AI, mostly without complaint.
But why go to all this trouble in the first place? Doesn’t this seem awfully Promethean for a video game? The development team’s own answers point beyond the game as a finished product, even as the team itself remains cryptic: Jorg Neumann, head of the Flight Simulator franchise, notes anything can be done “once you have the entire earth.” Asobo head programmer David Dedeine has proposed that the technology behind the game can “democratize tourism,” making it accessible from home. Outside of Microsoft, Blackshark’s usual clients are developers of autonomous cars—so it’s easy to imagine that Flight Simulator’s engine and data will doubtlessly be introduced in this field very soon.
Flight Simulator 2020 is also a proof of concept for Microsoft’s Azure cloud computing service, as the marketing has attested. Azure makes truly monstrous amounts of processing power commercially available to corporate and governmental clients, essential for dealing with the petabytes of data necessary in serially modeling the planet. (Blackshark’s Fabian Schlager credits Azure with getting the processing time for “global feature detection and reconstruction” under the 72-hour mark.) But Azure is advertised as doing much more, beyond AI and game development, opening the door further to blockchain solutions, analytics, and the Internet of Things (IoT). Azure IoT enables Microsoft’s “smart cities” offerings, which it describes as “the epitome of human civilization.” (To everyone else, a smart city is the practice of using citizen data to inform and optimize urban governance.)
The past few years have seen Microsoft push hard to make Azure a market player in the exploding smart cities arena, through both its CityNext initiative and Azure IoT Central, which recognizes “the modern city” as “one of the most dynamic landscapes embracing [the] Internet of Things.” The end goal here, as with all smart city efforts, is to realize urban space as a source of profits by reengineering it as a quantitative field, open to the suitably equipped technological eye. Microsoft’s Azure Maps and Digital Twins, the former offering geospatial analysis and the latter a toolset to create “comprehensive digital models of entire environments,” fit the bill, promising profound insights in the supposed interest of bromides like efficiency and public safety. Sam George, director of Azure IoT, has stressed that Microsoft’s smart cities approach depends on Azure’s capabilities for mapping and geospatial asset tracking to “better connect smart cities, infrastructure and IoT solutions, and empower industrial transformation.” What better demonstration of Azure’s mapping capabilities than, say, a cadastral playset of the entire globe, universal in its scope and unsparing in its detail? Sound familiar?
The possible applications of a cloud-based global survey do not stop there. Not to be outdone by Amazon, Palantir, and other enfants terribles of the surveillance/smart cities circuit, Microsoft has been aggressive in supplying its technology to government clients. Azure has already found use in the Domain Awareness System, the facial recognition surveillance platform custom-built for the New York City Police Department, and is the commercial substrate of the U.S. Department of Defense’s Joint Enterprise Defense Infrastructure (JEDI) Cloud. The JEDI contract, awarded to Microsoft to the tune of $10 billion, employs Azure technology to power the U.S. military’s data transfer and storage “from the homefront to the tactical edge.” Of course, military involvement in geospatial technology has a long tradition, just as it does with gaming: It’s no coincidence that the enterprise version of the previous Flight Simulator is now being developed and distributed as Prepar3D by defense contractor Lockheed Martin. Whether coincidence is what brought Flight Simulator’s development in line with the JEDI contract process is a bit beyond the scope of this review. (Hey, I’m an architect by training, not a journalist.) Be that as it may, it would appear that Flight Simulator 2020 not only functions as a tech demo for Azure’s cloud and virtual machine capabilities but also as the ultimate “gift” from a corporate giant to its new military partner—that of a fully imaged planet.
Kevin Rogan is a writer, designer, student, and dilettante who lives in New York.
Top image: A fully navigable Rio de Janeiro. (Courtesy Microsoft)