MICHAEL HASEY // AI & ARCHITECTURE

Neural Style Transfer // The City

year: 2020

type: research

GitHub: click for source code

“The rise of Artificial Intelligence in recent years have posed a challenge to the architecture community. How will this novel technology impact our profession?

…[This] might be the first genuinely 21st century design technique as [it] questions the role of the sole genius, perpetuated by the post-modern era, and proposes a conversation between the creativity and ingenuity of both, mind and machine.”

- Matias del Campo, architect and pioneer of contemporary technologies in architectural production

This research explores a recent breakthrough in A.I. technology called “neural style transfer”, a complex algorithm that allows any image to be re-created in an infinite number of new ways and styles. By taking two images - a content image and a style reference image (such as an artwork by a famous painter), the neural style transfer algorithm “blends” them together and produces a resultant output image that appears to be both the content image and the style reference image at the same time. Though the baseline content and underlying geometric organization of this new image matches the original content image, the re-styled output image appears to be created in its own unique style, allowing us to reinterpret images in ways that we may have never considered or imagined before through traditional means.

What does this mean for design?

By applying this interpretive approach to urban city scapes, neural style transfer algorithms give us the agency to accurately and rapidly visualize cities in multiple new and novel ways. Instead of taking days or weeks to manually create just a handful of new designs in the traditional manner, this tool could provide the means to produce hundreds within a fraction of time. This new ability to quickly re-interpret baseline geometries and designs will drastically change and improve the design process by allowing us to rapidly visualize our ideas and approach a satisfactory, and potentially better final design faster. By exploring these new tools, approaches to computer-human design thinking and craft, we may open the doors to new design approaches which will undoubtably improve our cities and futures in ways in which we could have never imagined.

Screen Shot 2020-07-19 at 12.25.38 PM.png

Visualizing the neural style transfer process

The core idea consists in optimizing a custom loss function by gradient descent, with this loss being computed based on features extracted from intermediate layers of the neural style transfer algorithm (shown left) , which is fed a content image, a style image and finally the image to optimize (output image), which is initially random noise.

Our research focus

Within this research project, we conducted a series of experiments to determine how Neural Style Transfer can be manipulated and applied to produce convincing new cityscapes in novel styles while maintaining the baseline geometries and composition of the original content image.

In particular, we tried to determine the best way to re-imagine downtown Manhattan as a neo-futuristic cityscape in the style of Syd Mead, an artist widely known for his techno-futuristic designs for science-fiction films such as Blade Runner, Aliens and Tron.

Screen Shot 2020-07-19 at 1.04.25 PM.png

Our results

By conducting a series of trial runs using various settings, filter weights, content and style images, we successfully determined the best method to allow this transformation to occur in a clear and convincing way. See below for the specific metrics, filter weights, and iteration counts that we’ve found to be most effective in creating successful output images.

Screen Shot 2020-07-19 at 10.47.16 AM.png

Screen Shot 2020-07-19 at 10.47.24 AM.png

Screen Shot 2020-07-19 at 12.52.33 PM.png

Screen Shot 2020-07-19 at 10.47.05 AM.png

Screen Shot 2020-07-19 at 11.16.26 AM.png

Screen Shot 2020-07-19 at 12.52.21 PM.png

Enhancing detail & complexity

Increased detail and complexity can be achieved by re-sampling the output image as content imagery within a subsequent round of the neural style transfer process. This was done by splitting up the output image into four equally sized quadrants, then re-inputting these quadrants as “new” content images into a new round of neural style transfer. The original style image used in the first round was also used in the second style transfer round. The subsequent new 4 output images were then digitally stitched together to make a larger and more detailed / complex version of the original.

Our enhancement process:

Screen Shot 2020-07-19 at 12.55.33 PM.png

Before & after detail enhancement:

Screen Shot 2020-07-19 at 12.56.12 PM.png

Before & after detail enhancement (zoomed in):

Screen Shot 2020-07-19 at 12.56.27 PM.png