Reconstructing San Francisco

Between 2019 and 2020 I photographed a series of San Francisco landmarks from the air and reconstructed them as textured 3D models using drone photogrammetry. Those environments became the foundation for a succession of experiments in virtual reality: first a speculative game world, then a narrated travel prototype. In 2026 I returned to the original photographs to see what Gaussian splatting, a newer way of representing a scene, could recover from them. This page is the record of that work so far. It is research, not a product.

Capturing the city

The source material is the thing I own outright: sixteen San Francisco sites flown with a DroneDeploy mapping drone in 2019 and 2020, each as a set of several hundred overlapping aerial frames at 5472 by 3648 pixels. Duboce Park, Buena Vista, Corona Heights, the Castro Theatre, Alamo Square, the Palace of Fine Arts, the Legion of Honor, the Music Concourse, the Botanical Garden, Dolores Park, Bernal Heights, City Hall and Civic Center. The processed models below were produced at the time and rendered in Blender for this page. The textures are as exported; the ragged edges are where the flights ended.

Textured photogrammetry model of the Palace of Fine Arts and its lagoon, viewed from above at an angle, with surrounding streets and houses — Palace of Fine Arts. Processed photogrammetry, oblique render.

Textured photogrammetry model of Bernal Heights hill and the surrounding grid of houses, viewed from above at an angle — Bernal Heights. Processed photogrammetry, oblique render. The sharpest set in the archive.

Textured photogrammetry model of San Francisco City Hall and the Civic Center plaza, viewed from above at an angle — City Hall and Civic Center. Processed photogrammetry, oblique render.

Three attempts at the same idea

The scans have been the constant. What I tried to do with them changed three times.

Wellspring, 2020 to 2024. A Unity VR game set in a magic-realm San Francisco built from the models, with twenty-five levels across three seasons. The environments were finished; the game around them was not, and it never shipped. In 2026 I shelved it as a game and kept the scans.

VIEWS, 2023 to 2025. A tour-guide prototype on the same scans, at six places inside and outside San Francisco. A named guide stood at each site and spoke a short script; film posters stood in the landscape as things to find. Quest builds shipped to a headset, and visionOS and iOS versions were explored. The strongest idea in it was the world view: the whole scan floating as an object in the room before you enter it.

Design mockup of a visionOS window in a living room showing a textured 3D scan of Halona Cove floating as an object, with a button reading Enter World — VIEWS, 2023. Design concept for visionOS: the place as an object before you enter it. Halona Cove, Oahu.

Design mockup of a fully immersive view of a scanned rocky cove with a floating glass panel naming the place and a one-line caption — VIEWS, 2023. Design concept for the immersive state, with a persistent panel for previous, next and exit.

The current direction, 2026. A voice instead of a guide with a body. Splats instead of meshes. San Francisco only. A design direction, with one pilot behind it.

The experiment: retraining Duboce Park

Photogrammetry produces a mesh with a texture: a surface. Gaussian splatting produces millions of small, soft, oriented blobs of colour that together reproduce what the cameras saw, including the way light and foliage change with viewpoint. The question was whether the 2020 flights, never intended for this, could be retrained as a splat on a laptop and whether the result held up as a place to stand.

The pilot used the Duboce Park flight from March 2020: 550 frames. Camera poses were solved with pycolmap and GLOMAP, open tools for recovering where each photograph was taken from. 545 of the 550 frames registered, with a mean reprojection error of 0.61 pixels, in under ten minutes. The splat was then trained in Brush, an open-source trainer, on an Apple M3 Max.

Two runs. The whole park at 1600 pixel resolution reached 5.0 million splats in 56 minutes. A cropped window around the meadow, trained at 2800 pixels, reached 7.6 million splats in an hour and three quarters, which is about five times the density per square metre of the first run on the same ground. The crop was the finding: train the window you will stand in, not the whole map.

Gaussian splat render of Duboce Park from drone height, looking across the lawn and surrounding houses toward a hill, with the sky and the far hill partly unresolved — Duboce Park Gaussian-splat pilot, 2026. Fly-over view from the trained scene. Convincing at flight height; the horizon frays where no frame saw it.

Gaussian splat render of Duboce Park at eye level, with sharp grass in the foreground and houses and trees dissolving into translucent streaks — The same scene at eye level. Aerial frames never saw the facades from here, so they stretch and go translucent. Street-level stops need ground capture.

The limits are as instructive as the result. Aerial capture holds up from flight height down to about thirty degrees above the horizon. Below that the facades were never photographed and the sky was never in frame, and the scene turns dreamlike. Bringing a visitor to the pavement needs a second kind of capture, on foot, registered into the same scene. Whether a scene like this can be pruned to a headset’s budget and still look like this is unproven; no frame-rate or memory figure exists yet.

The experience idea

The journey the pilot points toward is simple. Choose a place. See it as a miniature reconstructed world, the way VIEWS showed Halona Cove. Enter at the drone’s vantage, where the data is strongest. Follow a voice that tells you where you are and what happened here. Descend to street level only where ground capture exists. The six-beat script structure from VIEWS, a host rather than a lecturer, carries over; the avatar does not.

This is a design direction. Nothing described in this section exists as a working application.

Where it stands

Current status: archived research and an exploratory prototype. The scans are recovered at full resolution and catalogued. One site has been retrained as a splat and looked at from the air and from the ground.

If the work resumes, the next technical milestone is a single Quest 3 scene: the Duboce meadow compressed to a headset budget, one voice track, previous and next, and an honest measurement of how it performs on the device. Until then, this page is what the work has shown.