UNIVERSITY PARK, Pa. — Elizabeth Mansfield, professor and head of the Department of Art History, and James Wang, distinguished professor of Information Sciences and Technology, have received a Digital Humanities Advancement Grant (DHAG) from the National Endowment for the Humanities (NEH) for phase 2 of a project that uses computer-aided image analysis to examine the depiction of clouds in the paintings of John Constable, a 19th-century European artist noted for the striking naturalism of his landscapes.
The NEH level 2 grant was awarded to support “After Constable’s Clouds,” a project that is advancing art historical research through the innovative application of computer vision (CV) and machine learning (ML).
The project promises to enhance scholarly understanding of aesthetic concepts and artistic techniques in 19th-century European landscape paintings, especially those responsive to the Realist movement. “After Constable’s Clouds” is Phase 2 of “Seeing Constable’s Clouds,” which was supported by a level 1 NEH grant.
“The opportunity to engage in collaborative research with internationally renowned colleagues in data science and artificial intelligence, meteorology and digital humanities has been incredibly rewarding,” Mansfield said. “I'm excited at the prospect of building on our results from the first phase of the project.”
History of the project
The first phase, “Seeing Constable’s Clouds,” was motivated by an art historical question: Was the remarkable realism of Constable’s clouds attributable to the artist’s exacting empiricism or was it a result of his bravura technique? In other words, do human viewers perceive Constable’s paintings of clouds to be realistic because they accurately document ephemeral meteorological phenomena or because they are aesthetically persuasive to viewers accustomed to the visual language of 19th-century European landscape painting?
To answer the question, the team’s lead student researcher, IST doctoral candidate Zhuomin Zhang, developed algorithms to detect the most significant features in Constable’s clouds and then designed and trained a Convolutional Neural Network (CNN) to compare the artist’s painted clouds to photographs of actual clouds.
“After Constable’s Clouds” will engage with longstanding art historical debates about originality and tradition in 19th-century French art while also seeking to develop further the application of CV for humanistic research, according the researchers. Among the questions the team now seeks to answer are: To what extent was Constable’s influence decisive for progressive painting in 19th-century France? To what extent was artistic precedent used to augment direct observations of nature? And to what extent are technically difficult but thematically subdued features like clouds reliable markers of artistic influence?
The research will not only address art historical questions but also help advance AI (artificial intelligence) according to Wang. Landscape paintings are abstracted from the real world, and different painters have their own ways of depicting the same subject. Modern techniques, such as neural networks, capture the pattern similarity among training examples at a relatively low level of information stored in the pixels, Wang explained. To answer art historical questions, however, the team needs to provide not only a reliable way to distinguish different groups of paintings, but also an explanation behind the computer-generated decisions with high-level information that can help advance art historical understanding. According to Wang, such challenges will help further develop AI methodology that may have broader applications.
The team
Mansfield, a scholar of 18th- and 19th-century European art, will provide art-historical expertise. Wang, an internationally recognized expert on image analysis, image modeling, image retrieval and their applications, will supervise all computational research related to the project.
Jia Li, a professor of statistics whose research areas include ML, AI, probabilistic graph models and image analysis, will provide additional computational expertise.
Experiments will be carried out by a doctoral student to be named in the College of Information Sciences and Technology and will work under the direction of Wang and Li in the Intelligent Information Systems Research Laboratory.
Project conceptualization, data management expertise and project management assistance will be provided by John Russell, digital humanities librarian and assistant professor.
George Young, professor of meteorology and a specialist in observational and predictive meteorology, as well as AI, will assist with dataset labeling and ML verification.
Data curation and structuring is overseen by Catherine Adams of Penn State’s Center for Virtual/Material Studies.
An Advisory Board composed of researchers outside Penn State will provide additional expertise and guidance. Members include Paul Messier, Emily Pugh and David G. Stork.
“After Constable’s Clouds” will culminate in a symposium devoted to the application of CV and ML to art historical research, with time, date and location to be determined.