How Computer Scientists Are Using Twitter to Predict Gentrification

Gentrification, long a hot button issue, has become the subject of increasing—and increasingly angry—debate in recent years. Some argue that it is generally a force for good, raising incomes and opportunities for everyone in the neighborhood by generating jobs and tax revenue, and making the streets safer to boot. Others consider gentrification a scourge, a case of moneyed newcomers driving up prices and displacing long-term residents of historically disadvantaged neighborhoods. 

The term is a bad word in many quarters. In Los Angeles, anti-gentrification protesters in the historically Latino Boyle Heights neighborhood recently threatened walking tour groups and drowned out an opera performance in a local park. Last year in London’s Shoreditch neighborhood, protesters carrying torches and pigs' heads targeted a café selling breakfast cereal, calling it a “symbol of gentrification.”

Good or bad, gentrification can now be better predicted, thanks to a project from University of Cambridge computer scientists involving social media. By poring over some half a million Tweets, the researchers were able to come up with a metric for how human social behavior forecasts a neighborhood’s coming fortunes.

The researchers looked at Tweets from some 40,000 Londoners during a 10-month period beginning in 2010, parsing them for geotagged “check ins” via the social media app Foursquare. They analyzed the subjects' social networks to understand whether a specific location—a restaurant or a bar, say—was a place that attracted people from diverse groups—people that did not share friend networks on social media. This variable—whether or not a place brought together groups of friends or groups of strangers—was called the place’s “diversity.”

“We wanted to look at how a place brings together people who don’t otherwise know each other,” says Desislava Hristova, a Cambridge PhD candidate who led the research. “That’s when a place is playing this kind of brokerage role.”

Hristova and her team then looked at the data about a place’s diversity in combination with data about the surrounding neighborhood’s level of deprivation. The UK government publishes official indices of deprivation for cities and their neighborhoods every five years. These indices include housing prices and the level of health and education of local residents. The team then compared the 2010 indices of deprivation with ones from 2015 to see what had happened in the neighborhoods they studied.

“Basically what we found was when we looked at this diversity metric in relation to deprivation we can find an intersection of the two where some neighborhoods that have high diversity and high deprivation are in fact the ones that experience the most improvement over time,” Hristova says.

In other words, poor neighborhoods whose shops, restaurants and bars attract a diverse clientele are the ones likely to be gentrifying. Neighborhoods that are deprived but not diverse are less likely to gentrify, while neighborhoods that are neither deprived nor diverse probably already are.

The neighborhood that stood out the most in terms of both diversity and deprivation in 2010 was Hackney, a historically poor area in East London.

By studying the 2015 data, the researchers saw that Hackney has indeed gentrified dramatically over the past five years. It’s experienced the highest rise in housing prices of any London neighborhood over the past five years, and its crime rate has improved dramatically.

“When you go there you realize it’s kind of like the epitome of gentrification,” says Hristova, who presented her findings at a conference in Montreal earlier this month. “It has all those contrasts that describe the process, essentially a lot of hipster cafes and also a lot of council housing [public housing].”

Hristova and her colleagues are now looking into creating an application that could quickly and automatically analyze social media data to create a real-time look at how neighborhoods are shifting. Being able to predict how a neighborhood will change could help local authorities, urban planners and residents plan for the future. This might mean mandating more affordable housing or knowing the best time to sell property.

“Obviously, there are good things about gentrification, but there’s also negative as well,” Hristova says. “Maximizing the positive and minimizing the negative is something that local authorities can really think about doing.” 

Emily Matchar | | READ MORE

Emily Matchar is a writer from North Carolina. She's contributed to many publications, including the New York Times, the Washington Post, the Atlantic and many others. She's the author of the novel In the Shadow of the Greenbrier.