How Satisfied Are NYC Housing Residents? A New AI-Assisted Tool Helps Track the Answers

Researchers and data scientists at Columbia University’s SAFElab have used artificial intelligence and natural language processing to analyze more than 12.5 million Twitter posts as a way to track the satisfaction and well-being of residents in New York City Housing Authority (NYCHA) communities.

In a pilot program initiated by the Office of Neighborhood Safety of the Mayor’s Office of Criminal Justice, the team combined traditional interview techniques and focus groups with a study of 10 years’ worth of tweets in which residents tagged the housing authority (@NYCHA) or the New York City Mayor’s Office of Criminal Justice (@CrimJusticeNYC). The team also analyzed residents’ tweets related to national events such as the killings of George Floyd, Breonna Taylor, and Ahmaud Arbery and the mass shooting at Marjory Stoneman Douglas High School.

The program, called Neighborhood Navigator, was a collaboration among Columbia’s School of Social Work, School of Engineering and Applied Science, and Data Science Institute, with additional support from the John Jay Research and Evaluation Center. The research began in 2021 and concluded in September of this year.

To obtain an accurate read on residents’ attitudes, researchers interpreted tweets consisting of text only, of images only, and those that were “multimodal,” combining image with text. They also used sophisticated methods to pick up the social media user’s true intent — for instance, if the image and the tone of the text were at odds, the user might be speaking sarcastically.

The team observed peaks and valleys in the frequency of tweets on local and national topics, such as criminal activity and incarceration, police misconduct, healthcare costs, economic security and savings, and the city’s inaction toward chronic student absenteeism during the COVID-19 pandemic.

The Twitter findings were paired with the results of a traditional qualitative research phase, during which master’s and doctoral students from the Columbia School of Social Work performed one-on-one interviews with 69 individual residents in 10 NYCHA communities across all five boroughs, and conducted focus group interviews with 38 members of community advisory boards (CABs) in 9 communities. Interviewers used an empathetic information-gathering model to identify “lived experiences, pain points, challenges, and opportunities.”

The Social Work students asked residents, about life in their neighborhoods, the impacts of COVID-19, and their thoughts about the Black Lives Matter movement. Many of the surveyed residents were Black and Latinx New Yorkers who have lived in their communities for more than ten years. Residents were invited to comment on the quality of life offered by their housing and environment, how well served they felt by policing and the justice system, and their sense of personal safety and well-being, among other topics.

In the focus groups, CAB members spoke about the psychological effects of poor building maintenance, the difficulty of getting repairs, a lack of parks and trees, ambivalence about spending time outside (which offered opportunities for recreation and socializing but at the same time a risk of violence), and inequities in how the city provides technology access to young people.

Board members also expressed frustration with city policing, questioning whether a police presence is intended to protect the community and emphasizing that police need to know the culture of a neighborhood to determine what local behaviors are dangerous or not.

Overall, the one-on-one interviews revealed residents’ dissatisfaction with the maintenance and safety of their buildings, and concern about being pushed out by economic development in their neighborhoods. Residents expressed a desire for community patrols run by volunteers, a path to home ownership, and more mental and emotional support for young people.

After sifting together the tweet analysis with the face-to-face data gathering, the research team determined that social media is a reliable indicator of community sentiment. “We hypothesized and verified that amount of discussion is generally a good proxy for public concern about that indicator — as more discussion about a public issue (especially on Twitter) tends to skew towards complaints or raising issues.”

“Analysis of the social media posts from Twitter, coupled with geographical tagging and identification of socioeconomic wellness indicators, led us to finding some interesting patterns and discussion dynamics regarding the quality of life, wellbeing, community, and living conditions among residents of New York City,” said the SAFElab team in their final report.

Still, the team cautioned that if other researchers use this tool, they should be sure to involve housing residents in every step of the planning. “For an initiative like the Neighborhood Navigator to succeed, it has to be co-designed by community members who are the experts that understand the culture, needs, pain points, challenges, opportunities, and overall ecosystems of their respective communities.”

SAFElab’s director is Dr. Desmond Upton Patton, formerly of the Columbia School of Social Work and now a member of the faculty at the University of Pennsylvania. For more information about the Neighborhood Navigator, please see nycneighborhood.com. The full report is available at nycneighborhood.com/community-insights.