Going Beyond the Punchlist: Why Architects Should Embrace Post-Occupancy Evaluations

On the eve of a massive building campaign for which universities planned to invest billions in new student housing, two architects undertook an unprecedented study of what students needed and wanted. Talking to residents and observing their habits in detail, they found that the typical new dormitory design was woefully inadequate. Public space was underutilized, personal space was inflexible, and housing was isolated from other functions on campus. Dorms were cold and impersonal—“as homey as a Greyhound bus depot,” according to one student—and surveys revealed that many inhabitants felt “stifled” and even “enraged.” Proposing an alternative, mixed-use approach that offered greater variety and more flexibility, the architects found that universities could significantly improve the student experience at two-thirds the cost.

That was 50 years ago. Sim Van der Ryn and Murray Silverstein’s groundbreaking research, published in 1967 as Dorms at Berkeley: An Environmental Analysis, was possibly the first formal example of what came to be called post-occupancy evaluation (POE), the process of assessing buildings after the fact. “There is no feedback channel between planning assumptions and building use,” they declared. “Our focus is on the silent partner in the design process—the user affected by design decisions.”

Today, POEs vary widely in scope, but generally they focus on two basic questions: Is the building behaving as intended? And are occupants happy with the results? Studies consistently show correlations between the two; a report last year indicated that 88 percent of building-performance attributes directly affect user satisfaction.

The benefits of these studies could not be clearer. Research by the U.S. Army’s Construction Engineering Research Laboratory estimates a nearly 80-fold return on investment: Every dollar spent can save as much as $77 in operating, maintenance, and renovation costs. This summer, the federal General Services Administration (GSA) released a three-year study of 200 buildings confirming that high-performance facilities have higher tenant satisfaction and significantly lower energy, water, and operating costs.

Yet, despite the proven results, POEs remain rare. “I’d be surprised if it’s more than 1 percent of projects nationally,” says Andrea Love, a principal and director of building science at Payette. What’s the holdup?

“One challenge is a lack of knowledge in the design industry about what this is and how it’s helpful,” explains Janice Barnes, director of resilience for Waggonner & Ball Architects. “There isn’t a widely held understanding about how to do them properly”—if POEs are done at all. Z Smith, director of sustainability and building performance at the firm Eskew+Dumez+Ripple, blames the “shocking innumeracy” of architects: “Illiteracy is about language, innumeracy is about numbers. We don’t like numbers. It’s not what people thought they were getting into when they went into architecture. But you have to do it if you want to make a good building.” Yet architects’ scope of work rarely extends much beyond the punch list. “By the time you get to occupancy, the team has moved on,” notes Love.

This is not a new problem. A quarter century ago, in How Buildings Learn: What Happens After They’re Built (1994), his classic study of the life cycle of buildings, Stewart Brand complained about the tendency of architects to conceive of buildings as fixed in time: “The inane but now standard term ‘post-occupancy evaluation’ (POE) shows what a divisive watershed the moment of occupancy is. One of architecture’s most adaptive devices is misnamed by the traumatic instant of letting users into a building.” By contrast, he remarked, space planners and interior designers often think in terms of “churn” and therefore must consider the cyclical nature of the built environment. To avoid the finality implied by the common language, some prefer the term “performance evaluation.”

Among those who are aware of the benefits of POEs, the most frequently mentioned barrier to conducting such evaluations is cost, since basic architectural service fees do not cover them. Yet a simple survey of building occupants can take only a few hours to prepare, distribute, and review once the results come in. With support from the GSA, UC Berkeley’s Center for the Built Environment (CBE) developed a template to gauge user satisfaction with indoor environmental quality, including daylight, temperature, air quality, and acoustics. “Believe it or not, in 2000 an online survey was considered revolutionary,” recalls David Lehrer, CBE’s communications director. The center now has a database of more than 1,200 projects with over 100,000 individual responses.

Nevertheless, for smaller firms and sole practitioners, the time and cost for even a simple survey can be challenging, and many experts insist that surveys alone are insufficient anyway. “If you just use a survey, you’re only capturing a person’s point of view at a single point in time,” Barnes explains. “You have to corroborate it with other data.” A more in-depth study, including detailed observation by teams on-site, thorough occupant interviews, and direct measurements, can cost hundreds of thousands of dollars, which few clients are willing to cover. “Clients ask, ‘Why would I pay you to verify that you’re doing what I’ve already paid you to do?’” notes Barnes. “The real question is how much it costs not to do this. If an organization isn’t maintaining an environment conducive to doing the work, what are the costs?”

“Nobody pays for these things yet,” says Smith. To overcome this, his firm began allocating 2 percent of project design fees to invest in POEs. “We just started doing it because we believed in it,” he says. “It also builds credibility with skeptical clients. We’re selling our buildings as doing something, but do they actually do it?” This question is essential for buildings advertised as achieving better performance. Of the safety science advisory firm UL’s “Seven Sins of Greenwashing,” the second is “The Sin of No Proof”—“an environmental claim that cannot be substantiated.” A decade ago, a landmark study by the New Buildings Institute found that even LEED Platinum buildings often consume far more energy than designers anticipated. This is why the industry is putting more emphasis on both modeled and measured performance. The U.S. Green Building Council’s “Dynamic Plaque,” for example, is intended to track building stats in real time, and last year the AIA Committee on the Environment reframed its Top Ten Measures of Sustainable Design to encourage POEs and provide proof in actual performance data.

Still, even when they have the time and money, many architects avoid POEs for fear of uncovering problems. “Every project we’ve looked at, we’ve found something wrong,” says Smith. “As my mom used to say, ‘If you look behind the refrigerator, you’ll be compelled to clean it.’” However, Lehrer points out that studies often reveal positive stories that would not otherwise come to light: “People can complain whenever they want, but a POE also can highlight what’s working well. People never pick up the phone and say, ‘The temperature is great in here!’”

POEs can also reveal feedback loops that the design process did not anticipate. When Payette looked at actual energy performance for its Milken Institute School of Public Health at the George Washington University, a 2017 AIA COTE Top Ten Award winner, the meters showed that consumption was 10 to 15 percent higher than predicted. Was the equipment not working? “The biggest thing was actually that there were more people in the space than expected,” says Love. “People loved the project so much, they were there more often. When we adjusted the model for higher occupancy, it fit perfectly.” Occupant surveys revealed that 87 percent of respondents feel the building supports their health more than other campus buildings, citing stairs and natural light as the primary reasons.

Beyond simple surveys, Payette often conducts what Love calls “shadow studies”—following users closely to see how they behave. This gives the architects a better understanding of the psychosocial aspects of buildings: “Even though we provide the recommended set points for thermal comfort,” she explains, “people often are colder than we thought they would be. We think of comfort as the temperature on the thermostat, but there are many factors influencing perceived comfort.” A high-tech approach to studying user habits is real-time location systems (RTLS), which employ tracking sensors on moving targets (like staff and mobile equipment) to monitor activity. With various clients, the health-care practice at the design firm CallisonRTKL has used RTLS to reduce space needs by a third, labor costs by $100,000, and construction costs by up to $5 million.

In the quest to understand how their buildings behave, other architects are deploying and developing new tools and techniques. Building scientists, now common at larger firms, typically keep a kit of devices to measure temperature, humidity, light levels, air quality, and sound. KieranTimberlake, for instance, has tinkered with wireless multivalent sensors (Pointelist) and the just-launched Roast, a cloud-based, user-friendly POE application. “We need basic, simple tools to do this kind of work,” says Billie Faircloth, a partner at KieranTimberlake and its director of research. “There are a number of gaps in the industry, but we can make the protocols easier, more accessible, less confusing, less specialized.” The firm is using its own office as a guinea pig to refine performance continually after occupation. “Our bodies get checkups,” Stephen Kieran told Metropolis in 2016. “So should our buildings.”

As continuous performance evaluation becomes integrated into spaces, buildings could soon adapt automatically to shifting user needs. Technology already exists to judge mood by monitoring facial expressions and eye movement. Will near-future architecture continually adjust light levels, temperature, airflow, and white noise to ensure physical and emotional comfort? “We talk about static POEs that should happen one or two years after the building is occupied,” muses Julie Hiromoto, a principal at HKS. “But wouldn’t a world of ongoing monitoring and continuous improvement be so cool?”

You may also enjoy “15 Products for Designing Sustainable, Healthy Buildings and Interiors.”

Would you like to comment on this article? Send your thoughts to: [email protected]