MIT algorithm finds Secrets in Art: MosAlc is created with the help of art pieces in the metropolitan museum and Amsterdam’s Rijksmuseum. Mark Hamilton is the lead author of this paper alongside Bil Freeman from MIT. MosAlc compares not only similarities like color and style but also so in meaning and theme among the art pieces in the database. As per Mark Hamilton (MIT CSAIL Ph.D.), MosAlc helps retrieve art by different artists and cultures through the closest matches of the painting. 

Mark Hamilton also added, “These fields are rich with information that has never been processed with these techniques and can be a source for great inspiration for both computer scientists and domain experts.”  Which means he wants the technique to progress in every field possible. 

How does it work? - MIT algorithm finds Secrets in Art

The structure of this art search engine works as a 'KNN' tree-like structure in which groups and searches images with characteristics of the source image. Even an art expert or critique would fail to judge or assess so many pieces from different time and space on a go. 

Let us take an example of Francisco de Zurbarán’s, The Martyrdom of Saint Serapion and Jan Asselijn’s The Threatened Swan, “These two artists did not have a correspondence or meet each other during their lives, yet their paintings hinted at a rich, latent structure that underlies both of their works,” said Hamilton.


Another example could be when MosAlc presented an image of blue and white porcelain violin when asked about the closest image of a painting of a blue and white dress. This represented that MosAlc identified not only the similarities in color and style but also recognized the reference of broader cultural exchange between the Chinese and the Dutch.

Mark Hamilton and professor Bill freeman wrote this paper with the help of MIT undergraduates Stephanie fu and Minden fu, the website is built and designed by Felix Tran, Darius Bopp and Marina Rogers at Microsoft garage winter externship program.