# Diversity beyond density: experienced social mixing of urban streets

Zhuangyuan Fan
Department of Geography
The University of Hong Kong
Senseable City Lab
MIT
<EMAIL_ADDRESS>
&Tianyu Su
Graduate School of Design
Harvard University
<EMAIL_ADDRESS>
&Maoran Sun
Senseable City Lab
MIT
<EMAIL_ADDRESS>
&Ariel Noyman
Media Lab
MIT
<EMAIL_ADDRESS>
&Fan Zhang
Senseable City Lab
MIT
<EMAIL_ADDRESS>
&Alex ‘Sandy’ Pentland
Institute for Data, Systems, and Society & Media Lab
MIT
<EMAIL_ADDRESS>
&Esteban Moro
Institute for Data, Systems, and Society & Media Lab
MIT
Departamento de Matemáticas & GISC
Universidad Carlos III de Madrid
<EMAIL_ADDRESS>

###### Abstract

Urban density, in the form of residents’ and visitors’ concentration, is long
considered to foster diverse exchanges of interpersonal knowledge and skills,
which are intrinsic to sustainable human settlements. However, with current
urban studies primarily devoted to city and district-level analysis, we cannot
unveil the elemental connection between urban density and diversity. Here we
use an anonymized and privacy-enhanced mobile data set of 0.5 million opted-in
users from three metropolitan areas in the U.S to show that at the scale of
urban streets, density is not the only path to diversity. We represent the
diversity of each street with the Experienced Social Mixing (ESM), which
describes the chances of people meeting diverse income groups throughout their
daily experience. We conduct multiple experiments and show that the
concentration of visitors only explains 26% of street-level ESM. However,
adjacent amenities, residential diversity, and income level account for 44% of
the ESM. Moreover, using longitudinal business data, we show that streets with
an increased number of food businesses have seen an increased ESM from 2016 to
2018. Lastly, although streets with more visitors are more likely to have
crime, diverse streets tend to have fewer crimes. These findings suggest that
cities can leverage many tools beyond density to curate a diverse and safe
street experience for people.

## Introduction

Diversity is intrinsic to a sustainable, resilient, and inclusive city [1, 2].
Within many forms of diversity, the diverse collection of people, socially and
economically, is one of the crucial preconditions of economic urban vitality
[3] and creativity [4, 5, 6]. On the contrary, the segregation of people was
shown to impact children’s economic outcomes [7], widening the digital gap and
hindering access to public health services [8]. Correspondingly, researchers
across the fields of economics, sociology, urban planning, and mobility have
devoted themselves to explaining the level of social mixing and segregation
across space and time.

Although most understanding of vitality and mixing in our cities has been done
using static, residential only, and sometimes outdated census data, recent
studies of human mobility further draw our attention to the activity space in
cities beyond where people live. It is clear that people do not only stay
where they live but also work, travel, and relax in places other than homes.
Therefore, people living in less diverse neighborhoods may still have chances
to encounter people with different knowledge and experience during their daily
life. Recent studies have measured how well people with different backgrounds
are mixed during their daily travel activities [9, 10, 11], online
communications, and purchase activities [12]. It is shown that the likelihood
of people meeting diverse others is related to an individual’s demographic
characteristics, lifestyle, and travel habits [9]. However, beyond an
individual’s behavior choices, what remains unanswered is how a city as a
system could build an environment that cultivates social mixing in the long
run.

Many urban theories and practices have voiced the importance of density, or
the concentration of people, in leading towards more urban vitality [1, 13, 3,
14], and thus favoring a more socially mixed urban environment [15, 4]. If we
view the desired outcome as the diverse admix of human knowledge, abilities,
preferences, interaction, and so forth, using density as the only tool has its
limitations - city blocks with a high density of office buildings could still
only see people with similar income levels and skill sets. With the
relationship between density and diversity inevitably being non-linear, we
should further unpack what other tools cities could leverage to curate a
socially mixed urban environment.

Building on the work that creates an activity-based measure of diversity and
segregation, this study is primarily concerned with the space of street
sidewalks. Theoretically, street sidewalks are critical urban open spaces
advocated by sociologist William H. (Holly) Whyte [16], journalist Jane Jacobs
[2], architect Jan Gehl [17], and New Urbanism scholars[18]. Practically, the
United Nations Sustainable Development Goals (SDGs)’s target 11 emphasizes the
vital role of urban public spaces in social and economic life. However, with
most studies on social mixing conducted at the city or district level, the
elemental contribution along the street network to social mixing is rarely
unveiled.

To understand the vital elements leading to socially mixed street experience,
we first create a measure of Experienced Social Mixing (ESM) that estimates
the income aspect of mixing in cities using a large collection of micro-scale
mobility data across 40 counties and three metropolitan areas in the U.S.
Then, we examine what factors beyond density could further contribute to
social mixing. Two main sets of factors connected with human mobility and
social interactions are examined in this study. We first discuss the
importance of socio-economic factors, including income and residential mixing.
These variables are learned from the human mobility and segregation literature
that people tend to visit places at a given income segregation level [9], and
residential segregation correlates with experienced segregation at a city
scale [19]. The second set of factors describes venues along the streets and
how safe the street looks. These variables resonate the city planning
literature that advocates the mixed-use development [20, 21, 22] and street
environment safety [23, 24, 1].

We present three main results. First, conditioning on density, ESM can still
be explained by adjacent neighborhoods’ residential mixing, income level, and
venues along the street. Density, in our measure, the number of visitors
visiting a street segment at any specified time, only explains around 26% of
the model estimation, while residential mixing, income level, and venues
contribute to 44% of the model estimation.

Second, ESM measured at different hours of a day is closely related to
different types of venues along a street segment, highlighting the importance
of a mixed-use environment. Among the venues, food-related businesses exert
the highest contribution to explaining the ESM at different times of the day.
Meanwhile, when controlling for the total number of visitors, the streets with
more coffee and tea venues can attract more diverse groups of people.
Moreover, we found that the street segments with an increase in food-related
business from 2016 to 2018 are likely to see an increase in the ESM. This
longitudinal effect holds conditioning on the increase of total visitors.

The well-being of urban dwellers is a multi-dimensional concept that goes
beyond diversity and economic status and involves health, crimes, and other
aspects of life. Beyond the daily experienced social mixing, a body of recent
literature indicates that mobility patterns also predict crimes [25, 26, 27]
\- where residents visit is also a source of neighborhood (dis)advantage [28].
To further understand the effect of the ESM, the last part of our study
analyzes the relationship between visitor volumes, ESM, and different types of
crime incidents around each street segment. We show that although denser
cities attract more crime incidents, conditioning the visitor volumes, street-
level ESM has a negative association with crime count.

This study highlights the importance of a high-resolution measure of social
mixing as a time-dynamic urban phenomenon - streets adjacent to each other
could present dramatically different levels of ESM at different times of the
day. Furthermore, we illustrate how cities could leverage the open space of
the street sidewalks to increase the chance for different people to meet each
other and thus mitigate the existing downfall of residential segregation.
Lastly, our result also shows that diverse visitor experience does not always
go in parallel with high volumes of crime incidents. Large cities can leverage
many policy tools beyond density to curate a diverse and safe street
experience for people.

Figure 1: Measure the Experienced Social Mixing. a. Street-level ESM is
calculated based on the time spent by each income group at each street
segment. b. Distribution of daily ESM of Boston. Only street segments with at
least 1 POI within the 100-meter buffer area are visualized on the map.
c.Regional map visualizing the ESM around Boston metropolitan area.
d.Distribution of the ESM by metropolitan area. e. Distribution of log-
transformed visitor counts during the study period by metropolitan area.

## Results

### Street-level ESM

We create a measure of ESM for three large metropolitan areas of Boston, New
York, and Philadelphia, which involves more than 40 counties across five
states (MA, NY, DL, NJ, PA). The privacy-enhanced mobility data is provided by
Cuebiq, which includes 3-month long records across two years of anonymized
device-level location pings for 0.5 million users who opted into data sharing
for research purposes under a GDPR and CCPA-compliant framework.

To construct the ESM, we pre-process the data to identify each device’s home
Census Block Group (CBG) and stay locations using the same method in a
previous work by Moro et al. [9]. We first associate each device from the
mobility dataset with an approximate socioeconomic status by their inferred
home CBG. Each individual’s home CBG is obtained from their most commonly
visited location between 10 pm to 6 am (See Methods). Then all individuals are
grouped into four quantiles of income groups according to their home CBG’s
median household income’s relation to the metropolitan area distribution of
median household income (See Methods). We then extract visits an individual
made to a given street segment for at least 5 minutes but a maximum of two
hours. This is to prioritize sidewalk activities that have the potential for
meaningful interaction among pedestrians. Activities such as visiting cafes,
restaurants, parks, or simply resting along the streets are emphasized. Other
activities such as working in an office building or watching movies, which
usually take a long time but offer little chances for people to meet each
other, are dropped. The post-stratification process reduces sample bias
regarding population and income level (See Supplementary Note 2).

Figure 2: Explain the Experienced Social Mixing. a. Street-level ESM
representation. Each dot represents 0.5% of time spent at a street segment by
a group of visitors. b. Street view examples of streets that are adjacent to
each other yet present different ESM levels. Each barplot presents the hour of
visits from each income group. The total number of unique visitors is shown
accordingly. c. Coefficient of all variables estimating street-level social
mixing (all shown variables have p-value<0.01). d. Compare the effect of
residential mixing on ESM at street and CT-level. Binned-scatter plot with 100
quantiles. e. Compare the effect of the number of POI on ESM at the street and
CT-level. f. A non-linear relationship between the Street Score and the
street-level ESM. (See the full table in the Supplementary Note Tab. S1)

.

We compute the street-level ESM with the proportion of total time spent at
that street by each income quartile (see Fig. 1a). The ESM is defined as the
Shannon entropy [29] of each income group’s activity time spent at a given
street segment (See Methods). ESM quantifies income mixing from 0 to 1. A
street segment that is fully mixed ($ESM_{s}=1$) when the total time across
all individuals spent at the street segment is split evenly among the four
income quartiles, while a street segment with $ESM_{s}=0$ indicates that the
street segment is only visited by one income group. Fig. 2b shows several
examples. The bar chart of each street represents the accumulated time spent
by each income group at the sample street. The dashed line demonstrates when
the street would be fully mixed ($ESM_{s}=1$). For example, 63 Cambridge
Street in Boston has a lower ESM (0.767) in comparison to 27 Tremont Street
(0.975), as the time spent by each income group along 27 Tremont Street is
more evenly distributed. We note that there are other choices of mixing and
segregation metrics. We also examine the robustness of our measure of social
mixing against other metrics (See Supplementary Note 3).

Street-level ESM presents spatial heterogeneity in each city (Fig. 1c).
However, it also has a very fine-grained spatial resolution. We illustrate
this observation in Fig. 2a and b. Fig. 2a presents the proportion of time
spent by each income group to each street segment in the Boston Downtown area.
Each dot represents 0.5% of time spent by each income group. Fig 2b
demonstrates the street view samples and their associated distribution of time
spent by each income group. Even two adjacent streets with the same
intersection could present drastically different levels of social mixing. This
finding shows that understanding vitality, density, or mixing in our cities at
larger scales (e.g. Census Tracts or districts [30]) miss the fine-grained
structure of how people interact and encounter in our cities and the
relationship of the diversity of encounters and urban environment.

### Explain the Street-level ESM

Figure 3: How much does the concentration of visitors explain social mixing?
a.Coefficients of all variables estimating the ESM (standardized) with and
without the number of visitors controlled. Context variables including the
geographical fixed effects and street segment length; X include the
residential income mixing, income level, population density, the number of
venues, and Street Score. b. Summary of importance of feature groups in the
regression model for social mixing by sequentially dropping street samples
visited by fewer count of unique users. c.Correlation of log-transformed
number of visitors and daily ESM by sequentially dropping street samples
visited by fewer count of unique users. d.Relationships between the log-
transformed visitor counts and daily ESM by different metropolitan areas.

To understand the relationship between the streets and ESM, we model the ESM
of each street using a simple linear regression model. The explanatory
variables include the street segment’s length, a series of residential factors
that describe the adjacent neighborhood in which the street segment is
located, and the number of venues (such as restaurants, cafes, grocery stores,
schools, etc.) located along the street, and how safe a street segment look.
The residential factors include population density, median household income,
and residential income mixing. In addition, considering the geographical
differences of all samples included in the study, we also add county-level
fixed effects. To compute the residential factors, we obtain a collection of
CBGs that fall within a street segment’s 800-meter buffer and use the median
household income and population size of each CBG accordingly (See Methods and
Supplementary Note 4.4). The residential income mixing mirrors the calculation
of ESM. It measures the level of income mixing of the four income groups,
where 0 indicates that all residents within the 800-meter buffer belong to the
same income group, whereas 1 implies a fully mixed street residential
neighborhood (See Methods).

The venues on each street are obtained from Foursquares’ point of interest
(POI) API. A collection of 0.1 million verified venues across the study areas
are used. Lastly, given that the sense of safety is one of the significant
concerns determining the street activity [1], we measure how safe each street
looks through the Street Score. The Street Score of each street segment is
predicted from the Google Street View images taken along the street segment.
The Street Score model was adapted from Zhang et al. [31], which is a
convolutional neural network trained with the data from Place Pulse [32]. We
collected 1.6 million street view images through Google API across the study
areas, and images taken from winter were dropped to avoid seasonal
inconsistency. A similar method was used by Naik [24] in describing the
inequality of urban safety perception.

Fig 2c summarizes the regression coefficients of the above-mentioned linear
regression. Besides the geographical fixed effect, street segments close to a
higher level of residential mixing and more venues tend to be more socially
mixed. We test if this relationship holds at the different spatial units by
repeating the experiment at Census Tract (CT) level. Fig.2d and e plot the
results in parallel. We find that Census Tracts with more mixed residential
composition and venues also tend to be more mixed.

We also find population density and Street Score are negatively associated
with the street-level EMS. The former indicates that neighborhoods with more
residents do not guarantee more chances of cross-group mixing during their
daily activities. To further unpack the negative association between the
Street Score and the street-level ESM, we included a quadratic term of the
safety score in the same model and identify a non-linear relationship between
the Street Score and the ESM (See Fig.2f). This could reflect a number of
forces, including gentrification and fear of crime in cities. People tend to
avoid places that look very unsafe [33], but in the meantime, streets with
luxury settings, well-planted groves of trees, and fine furniture also
indicate a sense of gentrification do not welcome social mixing [34]. It is
worth noting that the effect of Street Score diminishes at CT-level study,
implying that the perception of street view matters more at a scale of the
street segment rather than in larger spatial unit (See Supplementary Fig S3
and Table S1).

### ESM and Density

As many current urban theories indicate, urban density is one of the important
instruments that foster diversity [1, 13, 3, 14]. A street segment with more
visitors could naturally have a higher chance to be more socially mixed. Could
it be that the factors we discussed above are associated with the ESM through
the channel of density?

As we can see in Fig. 3 d, the log-transformed total number of visitors
visiting each street has a correlation with the street-level ESM in different
cities. We find this correlation dropped as we sequentially excluded street
segments with too few unique visitors to avoid the small sample bias (Fig. 3
c). This suggests that the ESM is not solely explained by density, albeit they
are correlated. Fig. 2b also shows specific street segment examples and
implies that streets with fewer total visitors can still be more mixed than
others.

Figure 4: Time-variant ESM and street venues. a. Mean time-variant ESM for
each metropolitan area. Note that we only present the streets with at least 20
unique visitors at each time period of time. $CI^{*}$ stands for the
confidence interval. 98% of the confidence interval is shaded. b. Time-variant
effect of the number of venues by their categories. c. Time-variant effect of
the number of venues by their categories conditioning on the visitor count.
Only coefficients with $p$ value smaller than 0.05 are shown as significant.

To further quantify the factors explaining ESM beyond density, we repeat the
regression model in the previous section by including the log-transformed
count of visitors as a variable. Fig 3 b shows the importance of variable
groups in predicting street-level ESM. By excluding street segments with too
few unique visitors (fewer than 20), the count of visitors accounts for around
26% of the variance in street-level ESM. Apart from the geographical fixed
effect and street segment length, the residential income mixing, income level,
population density, and venues account for around 44% of the model variance.

Again, Fig 3a plots the comparison of two regression models (See Supplementary
Table S1 for the full result). We show that by including the log-transformed
total visitor count as a variable, the effects of residential income mixing,
income level, and population density hold. The effect of the total count of
venues and the Street Score dropped drastically. These observations lead to
two insights: one is that the streets with more venues and look less safe tend
to have more visitors and therefore more socially mixed. The other lies in the
fact conditioning the ability to attract more visitors, street segments within
areas that have more mixed residential environments and higher income levels
are still more socially mixed. One might wonder if the mixed residential
environment contributes to the ESM directly as a street segment’s neighborhood
residents would visit the street segment very often. We calculate the average
travel distance from any given street to its visitors’ home CBG’s geometry
center. We found that more than 95% of the street segments’ visitors live more
than 800-meter away from the street. This result confirms that the street
segment with a diverse residential environment attracts visitors from
different income groups even though they don’t live nearby.

### Temporal Variation of ESM

Street life has a unique temporal characteristic [35, 1]. To better understand
the temporal variation of ESM throughout the day, we reconstruct the model by
estimating the ESM at four selected time periods of a day. Fig. 4a shows the
average street-level ESM for each metropolitan area throughout the day. We
observe that on average, the ESM is highest around noon and lowest in the
morning, reflecting the daily activities in cities. In addition, we group
venues by their categories to test how the category of venues might predict
ESM dynamically (See Supplementary Table S5. for venue summary by types).
Similarly, we compare the model results by excluding and including the count
of visitors at different times correspondingly (Fig. 4).

Fig. 4b shows that streets with more venues such as food, coffee, and tea are
more mixed throughout the day, while the streets with more shopping and
entertainment (bars and clubs) venues become more mixed in the late afternoon.
Streets with more health-related venues tend to be more mixed before 6 pm, and
a similar trend is also seen for streets with more work-related venues. We
interpret these results to be associated with both the schedule of the
business and people’s mobility patterns. While people mostly visit hospitals
and clinics during the day, streets with these venues tend to be more mixed
during their normal operating hours. Since people will more likely to visit
bars and go shopping after work, streets with these venues only start to see
mixed groups of people later in the afternoon. We also find that streets with
more grocery stores tend to have consistently lower levels of mixing
throughout the day. This is likely an effect of segregated residential
neighborhoods, where people tend to go to grocery stores closer to where they
live.

In parallel, we also add the number of visitors to each street segment to test
if the impact of density would change the model results. Fig 4c shows that
conditioning on the count of visitors, number of coffee, tea, food,
entertainment, and health care venues are still contributing to more mixed
streets throughout the day, while streets with more grocery stores are still
less diverse. We understand that health care facilities are naturally more
integrated given their unique service role in the city. However, the effect of
coffee, tea, food, entertainment, and grocery stores reveals that even on
streets with a similar amount of visitors, their level of social mixing can
still vary considering the different functions it provides. After controlling
the density, we also found the effect of shopping venues dropped and even
reversed. We interpret this observation as that shopping venues are more
successful in bringing in a high volume of people, yet the people group
attracted to these areas might not be as diverse.

### Changes of ESM from 2016 to 2018

The cross-sectional study above highlights the food-related venues in
predicting street-level ESM. We further design an experiment to test if the
relationship holds longitudinally. Here, we leverage a crowd-sourced dataset,
Boston’s Hidden Restaurant111https://www.hiddenboston.com/closings-
openings.html, contributed by local communities from the Boston region to test
if the open and close of food-related businesses from 2016 to 2018 cast any
impact on the changes of street-level ESM in the corresponding time,
controlling for the changes of residential features (Summary Stats included in
Supplementary Table S3). Understanding that streets with a very high ESM in
2016 would have less room to improve than streets that were less mixed, we
control for the ESM at 2016 for all models. In addition, the model also
includes the same social and geographical context features in 2016 to account
for the potential trend differences (See the full result in Supplementary
Table S4).

Table 1: Change of ESM 2016 - 2018 (Street-Level)

|  $\Delta$ ESM
---|---
| (1) | (2) | (3) | (4) | (5)
$\Delta$ Pop Den | 0.003 |  | 0.003 | 0.003 | 0.003∗
| (0.002) |  | (0.002) | (0.002) | (0.002)
$\Delta$ MH Income | -0.000 |  | -0.000 | -0.000 | -0.000
| (0.002) |  | (0.002) | (0.002) | (0.002)
$\Delta\%$ Bachelor | 0.010∗∗∗ |  | 0.010∗∗∗ | 0.010∗∗∗ | 0.009∗∗∗
| (0.002) |  | (0.002) | (0.002) | (0.002)
$\Delta$ Resi. Diversity | -0.001 |  | -0.001 | -0.001 | -0.001
| (0.002) |  | (0.002) | (0.002) | (0.002)
$\Delta$ Food Business |  | 0.004∗∗ | 0.004∗∗ | 0.006∗∗∗ | 0.003∗∗
|  | (0.001) | (0.001) | (0.002) | (0.001)
$\Delta$ Food Business $\times$ ESM. 2016 |  |  |  | -0.006∗∗ |
|  |  |  | (0.003) |
$\Delta$ Log(Visitors) |  |  |  |  | 0.017∗∗∗
|  |  |  |  | (0.002)
Observations | 3768 | 3768 | 3768 | 3768 | 3768
R-squared | 0.5480 | 0.5450 | 0.5484 | 0.5490 | 0.5579
Fixed effect (county) | Yes | Yes | Yes | Yes | Yes
Trend Control |  |  |  |  |
Log($\\#$ POI) | Yes | Yes | Yes | Yes | Yes
Resi. Controls | Yes | Yes | Yes | Yes | Yes
Seg. Length | Yes | Yes | Yes | Yes | Yes
ESM 2016 | Yes | Yes | Yes | Yes | Yes

  * •

OLS estimates on change of ESM from 2016 to 2018. Only streets with at least
20 unique visitors in both observation periods (84 days in each year) are
included. Standard errors in parentheses. $\\#$ refers to count. Each street
segment contains at least 3 POIs in the year 2016.

  * •

∗∗∗ denotes a coefficient significant at the ${0.5\%}$ level, ∗∗ at the
${5\%}$ level, and ∗ at the ${10\%}$ level.

Table 1 illustrates the results. Column 1 indicates that among all residential
variables, the change of proportion of residents with at least a bachelor’s
degree is the only feature that contributes to the change in ESM. With all
other features controlled, we found little relationship between the changes in
residential income diversity and changes of ESM. This is partly because the
residential income diversity only changes very subtly between the two years.

Columns 2 and 3 indicate that the absolute increase of the number of food
businesses positively correlates with the change of ESM. The coefficient of
change of education level still holds by including the change of food
business. Column 4 includes an interaction term to test the marginal effect of
the food business, considering that streets with different original ESM in
2016 might respond to the changes differently. We show that the interaction
term of ESM 2016$\times\Delta$ food businesses is negatively associated with
the change of ESM. It implies that with a similar increase of food businesses,
the streets with a lower ESM in 2016 tend to see more increase of ESM (Fig
4a). Fig 4b shows examples of streets with changes in the food business and
different levels of ESM in 2016. Lastly, we further include the log-
transformed changes of visitors to each street segment between 2016 to 2018 in
Column 5. As expected, the increase in the number of visitors contributes to
the increase of ESM. However, the effects of food business and education level
still hold with the changes in visitor volumes.

We also tested the effect of newly established businesses using a different
data source (Reference USA) and Street Score on the changes of ESM.
Supplementary Table S4 reports all results. The change of Street Score does
not have a significant connection with the change of ESM. This is also
potentially due to the fact that the change in urban appearance between the
two years is relatively subtle. Consistent with the result of the change of
the food business, the establishment of new business has a positive effect on
the change of ESM.

### ESM and Crime

One of the main concerns with dense cities is crime [15, 36]. If a higher ESM
indicates a higher chance for people with diverse backgrounds to meet each
other, will it lead to more crime incidents? To understand the potential
connection, we obtain crime reports from four sample cities within our study
areas: New York City, Boston, Cambridge, and Philadelphia. Fig 5 a and b plot
the relationships between crimes and the street-level ESM, conditioning on the
number of visitors. We found that the number of petty and violent crimes (see
Method for detailed definition) has a negative relationship with ESM.
Conditioning on residential population, income level, number of visitors,
residential diversity, and the number of POI, the street segments with one
standard deviation higher ESM is associated with 2.6% fewer violent crimes and
2.4% fewer pretty crime (See Supplemental Note 8 for the full results). This
result implies that social mixing does not need to come at the price of more
crimes. On the contrary, we can still create a socially mixed street
environment with fewer crimes.

Figure 5: ESM and Crime. a. Binned scatter plot between ESM and log-
transformed number of petty crimes (with number of visitors by street segment
controlled). b. Binned scatter plot between ESM and log-transformed number of
violent crimes (with number of visitors by street segment controlled). c.
Kernel density illustration of petty crime density in New York City. d. Kernel
density illustration of violent crime density in New York City.

## Discussion

Many forces like gentrification, redlining, housing, or income inequality tend
to segregate people in our cities. Curating a socially mixed urban environment
is a common challenge for cities that aim at the overall goal of
sustainability. Our study contributes to the current literature in this
context from three perspectives. First, we investigate how the street
sidewalk, as one of the most significant urban public spaces, can bring people
from different income backgrounds together. The social mixing measured at
street segment level rather than neighborhood or city represents the direct
environment people will encounter through their day of life in cities. A
street segment’s immediate areas’ residential mixing and the number of venues
it is adjacent to have a strong connection with how socially mixed it is.
Second, we show that social mixing can exist beyond density. Street segments
with a similar number of visitors could still present different levels of
social mixing, and the remaining variance could largely be explained by the
residential mixing and income level. In addition, by further modeling the
level of social mixing at different periods, we also present its temporal
trend that is connected to the types of venues a street contains. We
demonstrate that streets with more food, cafe, and tea places are still more
mixed conditioning on the number of total visitors, while the streets with
more grocery stores tend to be less mixed. Lastly, we found that conditioning
on the number of visitors, the ESM has a negative association with street-
level crime count.

Our results suggest venues for city governments and planning agencies to
foster multifarious hubs for their citizens through the street life they may
experience. While the street life in cities has mostly been described in
planning literature through “vitality” [20, 30], we show that high density, or
the large volume of people, is not the only path towards diversity. The
increase in education level and establishment of some type of business (e.g.,
food-related) around a street can further bring more people with diverse
income backgrounds together.

In addition, building upon the literature that has incorporated the mobility
pattern into crime prediction, we highlight that socially mixed street
environments are less likely to see violent or petty crimes compared to the
streets with similar visitor volumes but are less mixed. This result implies
future research to test the value of mixed-use development in crime reduction.

Our study has several limitations. First, this study only discusses social
mixing from the aspect of income. Other forms of social mixing, measured from
race occupation, might have a different presentation from income mixing. Our
measure of social mixing uses proximity as a proxy, thus cannot determine if
people staying at the same street segment are having meaningful interaction or
not.

## Methods

### Street Segment

All street segments are downloaded from the OpenStreetMap through the python
omnsx package. Each street segment is defined as a segment of a street between
two intersections. Each street segment contains an ID from the OpenStreetMap,
a pair of u, v values representing the intersection node, and the function
type of the street segment. Duplicated geometries are removed from the
original dataset. We also remove the major highway, primary link, secondary
links, trunks, services streets, footpaths, steps, and slopes from the
original dataset. (Noted here that although we focus on the pedestrian
network, footpaths are removed from the original dataset given its complexity
and potential duplicates in a small space.) Only street segments with at least
one POI within a 100-meter buffer radius are included in the study. A total of
151,680 street segments from three cities are included in the study.

### Mobility Data

#### Attribution of stays to streets

Each street segment is represented as a line in space. To attribute each stay
to a street segment, we find the closest street segment for each stay using
the python geopandas package. To avoid attributing a stay to a distant street
segment, we choose only the street segment within a $d_{max}=100$ meters from
each stay. If a stay is further than $d_{max}$ from any street segment, we
discard it from the dataset. 50% of stays are within 26.7-meter radius from
their closest street segment. The average distance of a stay to the closest
street segment is around 31.2 meters for all three cities. Distance is
calculated based on each state’s NAD83 state plane projection222New york:
EPSG:2263; Boston: EPSG:2250; Philadelphia: EPSG:2272.

Table 2: Summary statistics of the dataset per metropolitan area

Metropolitan Area | # Devices | # Stays | # Street Segment | # Census Tract | # POIs | # Street Images
---|---|---|---|---|---|---
Boston-Cambridge-Newton (2016) | 66k | 6.3M | 17k | 1007 | 14.4k | 0.23M
Boston-Cambridge-Newton (2018) | 174k | 12.2M | 17k | 1007 | - | 0.18M
New York-Jersey City | 210k | 42.2M | 77k | 4682 | 64.5k | 0.41M
Philadelphia-Camden-Wilmington | 112k | 14.8M | 26k | 1477 | 19k | 0.75M

  * •

Statistics summary for three cities included. Number of devices only include
those that are identified with a home census block group. Number of Stays only
include the stays that are not within 50-meter buffer of their associated home
location.

#### Street Level Activities

As we only focus on street-level activities, all stays that have a longer
duration than 2 hours are discarded from the dataset. Any stays within a
$d_{home}=50$ meters from the identified home locations are also dropped from
the study. A total number of stays and unique devices are shown in Table 2.

#### Identifying home and economic status

For each smartphone, we use its stays from 22:00 to 6:00 and spatially cluster
them using the Density-based spatial clustering of applications with noise
(DBSCAN) [37] algorithm to detect the most likely cluster of stays each
individual is located in during nighttime and early morning hours. We use 2 as
the minimum number of points per cluster and $\epsilon$ = 50 meters as the
neighborhood. Then we join all detected cluster centers with each CBG
geometry. We only consider individuals who were at the same CBG geometry for
more than 5 nights in the observation period (three months), and this CBG is
considered as the home for this user. We use this CBG’s median household
income during the associated year to estimate the user’s income level. This
process leaves us to consider only 0.5 million users. Post-stratification was
implemented to assure the representatives of the data in terms of income and
population.

### Measuring ESM

#### Create Income Groups

We compare the median household income inferred from each individual’s home
CBG with the distribution of income in the metropolitan area so each CBG is
assigned to a quartile of economic status within each metropolitan area. For
each metropolitan area, the intervals of median household income for each
economic group are different (See Supplementary Note Fig. S1)

#### Street-level Measure

To measure the ESM of each street $s$ in each city, we compute the proportion
of total time spent at that street $s$ by each income quartile $q$ during the
selected period $h$, $\tau_{qsh}$. Then we define $ESM_{hs}$ as the Shannon
entropy[29] of each income group’s activities at a given street segment $s$’s
during a given time frame $h$:

$ESM_{sh}=-\frac{1}{log4}\sum_{q=1}^{4}\tau_{qsh}log(\tau_{qsh}),$ (1)

where $ESM_{hs}$ equals 0 when all users visit the street $s$ in time period
$h$ are from the same income group, while a larger value of the $ESM_{hs}$
means users from all four income groups spend a more equal amount of time
visiting the street $s$ during period $h$. Only street segments with at least
20 users during a given period $h$ are included in the study to avoid severe
small-sample bias. The daily ESM only considers stays from 6 am to 10 pm. The
ESM at other periods is as specified in the paper.

#### Census Tract-Level Measure

Like street-level ESM, the CT level ESM is the entropy of each income group’s
activities within a CT during a given time frame. Each stay is attributed to a
CT through a spatial join process.

### POI data

#### POI to Street

POI data is from Foursquare. We assign each POI to a street segment if it
falls into a street segment’s 100-meter buffer. Each POI is also joint
spatially with a CT that it falls into. The POI distribution within each city
is shown in Supplementary Note Table S5.

#### Change of Business

The change in the food business is obtained from Boston’s Hidden Restaurant
website. The data contains the restaurants, cafes, bars, and other food-
related businesses that are closed or open in each month since 2007. For this
study, as the mobility data covers October to December in 2016 and October to
December in 2018, we only select the restaurants that are either open or
closed from Jan. 2017 to September 2018. The latitude and longitude of each
restaurant were verified through google geocoding API. The chain stores is
verified through Yelp. For the open and closed months for each recorded store,
we also found similar results through the date of yelp reviews.

### Street Score

To quantify the physical appearance of the built environment, we obtain 360
degrees panorama Google Street View (GSV) images of streetscapes through
Google Maps API in all three study areas. Each panorama is associated with a
unique identifier, latitude, longitude, month, and year of when the image was
captured. We specify four angles to capture the full panorama of each street
view location. To avoid the seasonal effect, we only keep images taken between
April and October. GSVs taken from 2015, and 2016 are used in the cross-
sectional study for the 2016 panel. GSV’s taken from 2018 and 2019 were used
for the 2018 panel. Moreover, images that were taken interior or highway only
were excluded from the dataset. A total number of 1.5 million GSVs were used
in the study (Table 2).

We measure the appearance of the built environment with a “Street Score,”
which indicates the perception of the safety of a GSV image. We use a Deep
Learning model [31] pre-trained with a crowd-sourced dataset called Place
Pulse, which contains millions of ratings on around 110,000 street view images
from all over the world [24]. The image diversity and rating consistency were
evaluated by previous works [24, 38], indicating no significant bias depends
on raters’ cultural backgrounds in the dataset. We predicted the perception of
safety for each image by ignoring the features of the sky, cars, and people in
the dataset to minimize effects from time of day and other dynamic events (See
Supplementary Note 6). The predicted continuous score ranges from 0 to 10,
with 0 being the least safe-looking and 10 being the most safe-looking view.
Then the “Street Score” for each CT and street segment is the average score of
all images associated with the CT and the street segment.

### Other Data

Demographic data at the level of CBG and CT were obtained from the 5-year
American Community Survey ACS (2012-2016, 2014-2018).

#### Residential Income Mixing

The residential income mixing is calculated using the same income group
quartile per metropolitan area. To be consistent with the ESM calculation, at
street level, we first buffer the street for 800 meters and extract all CBGs
that intersect with the street buffer. Then using the pre-assigned income
quartile based on each CBG’s median household income and population, we
calculate the street-level residential mixing as the equation below:

$R_{c}=-\frac{1}{log4}\sum_{q=1}^{4}n_{qc}log(n_{qc}),$ (2)

where $n_{qc}$ is the population with the median household income level
belonging to income quartile $q$. To test the robustness of this method, we
also repeat the calculation by buffering from the street at 400 and 1000
meters (See Supplementary Note 4.4).

#### Crime Incidents

The 2016 crime reports within the four sample cities are downloaded from each
city’s open data website. The original crime data comes with crime primary
types, crime incident date, and address. All four cities also provide crime
incident locations’ latitudes and longitudes except Cambridge, MA. We retrieve
the latitude and longitude of crimes in Cambridge using the Google Map API
geocoding service. Then we aggregated each crime incident to the street level
by associated crimes to a street segment within a 30-meter buffer distance.
Two main types of crimes are separated from the original data. Violent crimes
include rape, robbery, felony or aggravated assault, and homicide or murder.
Petty crimes include theft and larceny.

#### Other

The population density, percentage of people with at least a bachelor degree,
and median household income are derived from the same CBGs to calculate the
residential income diversity.

### Regression Specification

#### Cross-sectional Model

We specify the ordinary least square model to explain the ESM at street level
and CT level.

$Y=\\{Context\\}+\\{X\\}$ (3)

$Y=\\{Context\\}+\\{X\\}+\\{Density\\},$ (4)

where Y is the estimated ESM of each experiment. $\\{Context\\}$ is a set of
variables to control for the geographical context, including the segment
length for the street-level experiment, land area size for the CT-level
experiment, and county-level fixed effects. {X} includes the variables we are
interested to test: residential mixing, income, population density, venues
count, and Street Score predicted from street view image data. The median
household income comes from the ACS (5-year) 2011 - 2016 survey corresponding
to the mobility data’s associated year. To account for the effect of density,
we include a $\\{Density\\}$ term in equation 4, which stands for the total
number of visitors.

#### Difference-in-Difference Specification

To answer the question of which features contribute to the change of ESM, we
specify the following equation:

$\displaystyle\Delta Y=\\{\Delta R\\}+\\{\Delta B\\}+\\{X\\}$ (5)
$\displaystyle+Y_{2016}$ $\displaystyle+Y_{2016}\times\\{\Delta B\\},$

where $\Delta Y$ is the change of ESM from 2016 to 2018. $\\{\Delta R\\}$ is a
set of demographic variables that change values from 2016 to 2018. The
demographic data of 2018 is from ACS 2013 - 2018 survey. The $\\{\Delta B\\}$
is the change of food business aggregated at street level. $\\{X\\}$ includes
a set of demographic variables in 2016 to control for the trend. To test the
robustness, we also used the number of newly established businesses from the
Reference USA 2017 data as the $\\{\Delta B\\}$ in additional tests.

## Acknowledgements

We would like to thank Cuebiq who kindly provided us with the mobility data
set for this research through their Data for Good program. E.M. acknowledges
support by Ministerio de Ciencia e Innovación/Agencia Española de
Investigación (MCIN/AEI/10.13039/501100011033) through grant
PID2019-106811GB-C32. x

## Author contributions statement

Z.F and E.M. designed the research; Z.F. performed the analysis; T.S., M.S.,
F.Z. performed part of the analysis; Z.F. wrote the original manuscript; E.M,
A.N., T.S., M.S., and F.Z. revised the manuscript; A.P. edited the manuscript.
All authors reviewed the manuscript.

## Code Availability

The analysis was conducted using Python and Stata. Code to reproduce the main
results in the figures from the aggregated data is publicly available on
GitHub: https://github.com/brookefzy/social-mixing-street.

## Competing Interests

The authors declare no competing interests.

## References

  * [1] Jane Jacobs.  The uses of sidewalks: safety.  The city reader, pages 114–118, 1961.
  * [2] Allan B Jacobs.  Great streets.  ACCESS Magazine, 1(3), 1993.
  * [3] Edward Glaeser.  Triumph of the city: How our greatest invention makes us richer, smarter, greener, healthier, and happier.  Penguin, 2012.
  * [4] Richard Florida.  Cities and the creative class.  Routledge, 2005.
  * [5] Richard Florida.  The flight of the creative class: The new global competition for talent.  Liberal education, 92(3):22–29, 2006.
  * [6] Charles Landry.  The creative city: A toolkit for urban innovators.  Routledge, 2012.
  * [7] Susan E Mayer.  How economic segregation affects children’s educational attainment.  Social forces, 81(1):153–176, 2002.
  * [8] Michael R Kramer and Carol R Hogue.  Is segregation bad for your health?  Epidemiologic reviews, 31(1):178–194, 2009.
  * [9] Esteban Moro, Dan Calacci, Xiaowen Dong, and Alex Pentland.  Mobility patterns are associated with experienced income segregation in large us cities.  Nature communications, 12(1):1–10, 2021.
  * [10] Yang Xu, Alexander Belyi, Paolo Santi, and Carlo Ratti.  Quantifying segregation in an integrated urban physical-social space.  Journal of the Royal Society Interface, 16(160):20190536, 2019.
  * [11] Susan Athey, Billy Ferguson, Matthew Gentzkow, and Tobias Schmidt.  Estimating experienced racial segregation in us cities using large-scale gps data.  Proceedings of the National Academy of Sciences, 118(46), 2021.
  * [12] Xiaowen Dong, Alfredo J. Morales, Eaman Jahani, Esteban Moro, Bruno Lepri, Burcin Bozkaya, Carlos Sarraute, Yaneer Bar-Yam, and Alex Pentland.  Segregated interactions in urban and online space.  EPJ Data Sci., 9(1):20, December 2020.
  * [13] Louis Wirth.  Urbanism as a way of life.  American journal of sociology, 44(1):1–24, 1938.
  * [14] Erling Holden and Ingrid T Norland.  Three challenges for the compact city as a sustainable urban form: household consumption of energy and transport in eight residential areas in the greater oslo region.  Urban studies, 42(12):2145–2166, 2005.
  * [15] Stefano Moroni.  Urban density after jane jacobs: the crucial role of diversity and emergence.  City, Territory and Architecture, 3(1):1–8, 2016.
  * [16] William Hollingsworth Whyte et al.  The social life of small urban spaces.  Conservation Foundation Washington, DC, 1980.
  * [17] Jan Gehl.  Life between buildings: using public space.  Danish Architectural Press, 1971.
  * [18] Joseph F Cabrera and Jonathan C Najarian.  Can new urbanism create diverse communities?  Journal of Planning Education and Research, 33(4):427–441, 2013\.
  * [19] Susan Athey, Billy Ferguson, Matthew Gentzkow, and Tobias Schmidt.  Experienced Segregation.  Technical Report w27572, National Bureau of Economic Research, Cambridge, MA, July 2020.
  * [20] Jane Jacobs.  The Death and Life of Great American Cities.  Vintage Books, 1993.
  * [21] Peter Calthorpe and Sim Van der Ryn.  Sustainable communities: A new design synthesis for cities.  Suburbs and Towns, San Francisco: Sierra Club, 1986.
  * [22] Jill Grant.  Mixed use in theory and practice: Canadian experience with implementing a planning principle.  Journal of the American planning association, 68(1):71–84, 2002\.
  * [23] Charles C Branas, Eugenia South, Michelle C Kondo, Bernadette C Hohl, Philippe Bourgois, Douglas J Wiebe, and John M MacDonald.  Citywide cluster randomized trial to restore blighted vacant land and its effects on violence, crime, and fear.  Proceedings of the National Academy of Sciences, 115(12):2946–2951, 2018.
  * [24] Nikhil Naik, Jade Philipoom, Ramesh Raskar, and César Hidalgo.  Streetscore-predicting the perceived safety of one million streetscapes.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 779–785, 2014.
  * [25] Andrey Bogomolov, Bruno Lepri, Jacopo Staiano, Nuria Oliver, Fabio Pianesi, and Alex Pentland.  Once upon a crime: towards crime prediction from demographics and mobile data.  In Proceedings of the 16th international conference on multimodal interaction, pages 427–434, 2014.
  * [26] Fan Zhang, Zhuangyuan Fan, Yuhao Kang, Yujie Hu, and Carlo Ratti.  “perception bias”: Deciphering a mismatch between urban crime and perception of safety.  Landscape and Urban Planning, 207:104003, 2021.
  * [27] Andrey Bogomolov, Bruno Lepri, Jacopo Staiano, Emmanuel Letouzé, Nuria Oliver, Fabio Pianesi, and Alex Pentland.  Moves on the street: Classifying crime hotspots using aggregated anonymized data on people dynamics.  Big data, 3(3):148–158, 2015.
  * [28] Brian L Levy, Nolan E Phillips, and Robert J Sampson.  Triple disadvantage: Neighborhood networks of everyday urban mobility and violence in us cities.  American Sociological Review, 85(6):925–956, 2020.
  * [29] Claude E Shannon.  A mathematical theory of communication.  The Bell system technical journal, 27(3):379–423, 1948.
  * [30] Marco De Nadai, Jacopo Staiano, Roberto Larcher, Nicu Sebe, Daniele Quercia, and Bruno Lepri.  The death and life of great italian cities: a mobile phone data perspective.  In Proceedings of the 25th international conference on world wide web, pages 413–423, 2016.
  * [31] Fan Zhang, Bolei Zhou, Liu Liu, Yu Liu, Helene H. Fung, Hui Lin, and Carlo Ratti.  Measuring human perceptions of a large-scale urban region using machine learning.  Landscape and Urban Planning, 180:148–160, 2018.
  * [32] Mark Philip Salesses.  Place Pulse: Measuring the collaborative image of the city.  PhD thesis, Massachusetts Institute of Technology, 2012.
  * [33] Wesley Skogan.  Fear of crime and neighborhood change.  Crime and justice, 8:203–229, 1986.
  * [34] Jed Kolko.  The determinants of gentrification.  Available at SSRN 985714, 2007.
  * [35] Kevin Lynch.  The image of the city, volume 11.  MIT press, 1960.
  * [36] Edward L Glaeser and Bruce Sacerdote.  Why is there more crime in cities?  Journal of political economy, 107(S6):S225–S258, 1999.
  * [37] Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al.  A density-based algorithm for discovering clusters in large spatial databases with noise.  In kdd, volume 96, pages 226–231, 1996.
  * [38] Abhimanyu Dubey, Nikhil Naik, Devi Parikh, Ramesh Raskar, and César A Hidalgo.  Deep learning the city: Quantifying urban perception at a global scale.  In European conference on computer vision, pages 196–212. Springer, 2016.


# A Gap in Time: The Challenge of Processing Heterogeneous IoT Point Data in
Buildings

Xiachong Lin∗, Arian Prabowo∗, Imran Razzak, Hao Xue, Matthew Amos, Sam
Behrens,
Stephen White, Flora D. Salim X. Lin, A. Prabowo, I. Razzak, H. Xue, F. Salim
are with University of New South Wales, SydneyM. Amos, S. Behrens and S. White
are with CSIRO∗ These authors contributed equally to this work.

###### Abstract

The growing need for sustainable energy solutions has driven the integration
of digitalized buildings into the power grid, utilizing Internet-of-Things
technology to optimize building performance and energy efficiency. However,
incorporating IoT point data within deep-learning frameworks for energy
management presents a complex challenge, predominantly due to the inherent
data heterogeneity. This paper comprehensively analyzes the multifaceted
heterogeneity present in real-world building IoT data streams. We meticulously
dissect the heterogeneity across multiple dimensions, encompassing ontology,
etiology, temporal irregularity, spatial diversity, and their combined effects
on the IoT point data distribution. In addition, experiments using state-of-
the-art forecasting models are conducted to evaluate their impacts on the
performance of deep-learning models for forecasting tasks. By charting the
diversity along these dimensions, we illustrate the challenges and delineate
pathways for future research to leverage this heterogeneity as a resource
rather than a roadblock. This exploration sets the stage for advancing the
predictive abilities of deep-learning algorithms and catalyzing the evolution
of intelligent energy-efficient buildings.

###### Index Terms:

building IoT point data, Brick schema, building heterogeneity, building
integration, Internet of Things

##  I Introduction

In sustainable development, the energy production and consumption paradigm is
shifting towards a model prioritizing reliability, affordability, and
cleanliness. This transformation towards decarbonization, decentralization,
digitalization, and democratization is underpinned by a global consensus to
reduce carbon emissions and foster energy independence [1]. Nations are
channeling investments into renewable energy sources and innovative energy
storage solutions to create resilient digitally-enhanced infrastructures
capable of agile responses to fluctuations in energy transmission and market
volatility while aligning with strict environmental protocols.

Integrating buildings into the power grid is receiving rising attention to
achieve sustainable goals due to its multifaceted benefits and great potential
in energy optimization. On the one hand, buildings significantly impact human
well-being, influencing occupant comfort [2], health, and safety [3], while
also being major consumers of global primary energy and substantial
contributors to carbon dioxide emissions, comparable to the transportation
sector [4]. Improving building performance holds immense potential for
mitigating carbon emissions and addressing climate change. By sourcing
renewable energy upstream from the grid while optimizing operations and
implementing sustainable practices downstream, we can achieve energy savings
and enhance indoor air quality, reduce greenhouse gas emissions, and bolster
resilience to climate-related challenges. On the other hand, buildings possess
significant capacity to actively participate in virtual power plants (VPP),
where a VPP is the network of decentralized power generating or storage units.
Through building-to-grid (B2G) interactions, buildings can respond dynamically
to the grid’s announcement by supplying excess energy generated onsite or
decreasing their energy demand during peak times. Meanwhile, compared with
other bidirectional energy ecosystems like vehicle-to-grid (V2G), B2G provides
larger energy capacities and stable contributions without significantly
impacting the user experience. This dual role of consuming and contributing
energy supports grid stability and enables buildings to function as adaptive
energy units within a larger, sustainable energy ecosystem.

The fundamental support of this integration process is provided by the
Building Management System (BMS), a sophisticated assembly of hardware and
software that facilitates intelligent control over building operations. A
typical BMS system collects data from Internet-of-Things (IoT) devices
distributed throughout the building to monitor a network of equipment that
includes Heating, Ventilation, Air Conditioning (HVAC) systems, lighting,
photovoltaic (PV) panels, security systems, and electrical facilities. This
IoT-captured data is then processed to detect anomalies and is centrally
managed via a cloud server, as depicted in Figure 1. Subsequently, researchers
or building operators can access this information through a cloud-based
software platform, allowing them to address common data-related issues or to
continually enhance the efficiency of building operations.

Advancements in the Internet of Things (IoT) have led to a surge in data
generation within buildings, offering immense potential for improving their
efficiency and functionality. However, realizing this potential is hindered by
several challenges. Firstly, privacy concerns restrict the availability of
publicly open building data [5, 6]. Secondly, the unique characteristics of
each building pose difficulties in deploying software solutions universally
across different structures. Efforts such as Project Haystack https://project-
haystack.org/ and the Brick schema [7] have aimed to address these challenges
by standardizing building data models. Despite these initiatives, the
literature on leveraging these models within deep learning frameworks remains
scarce, highlighting a gap in research that warrants further exploration and
development.

Figure 1: The schematic graphic about data communication in a digitalized
building.

This work specifically concentrates on load forecasting methods. The
contribution of this paper can be summarized as follows:

  * •

In section II-IV, we provide a brief overview of our dataset and analyze it to
demonstrate that IoT data in buildings exhibit greater heterogeneity compared
to other common time series datasets in the literature, across multiple
dimensions: ontology, etiology, temporal, spatial, and periodicity.

  * •

Due to these heterogeneities, we show that popular deep learning architectures
perform inadequately on IoT data in buildings. This is done by a benchmarking
experiment, the methods, details, and results of which are elucidated in
sections V.

##  II How Heterogeneous is the Building Ontology and Etiology?

Building IoT data stands out from other time series datasets, such as those in
road traffic or electricity consumption domains, due to their significant
heterogeneity in the ontology. Unlike typical time series datasets, which
maintain ontology uniformity, such as all time series in electricity datasets
[8, 9] like ETT [10] and UCI Electricity [11] focusing on sensor ontology like
transformer temperature and electricity load, building point data exhibit
diverse ontologies beyond sensors. In the Brick point data analyzed by this
study, the building ontology includes 31.26% Point, 15.63% Sensor, 12.93%
Status, 11.63% Setpoint, 11.40% Command, 4.23% Parameter and 0.59% Alarm.

Additionally, while some time series within traffic datasets like METR-LA and
PeMS-D4 may feature multiple channels such as speed, flow, and occupancy [12],
building data often presents irregular structures, necessitating the
utilization of semantic graphs [7] for effective ontology representation.
Consequently, the varied ontology and irregular structures of building data
present unique challenges for data science analysis, setting them apart from
other types of time series data.

The heterogeneity in ontology observed in building IoT data also leads to
heterogeneity in etiology, where direct causal relationships between certain
variables, such as temperature setpoints influencing zone temperature within
the same Air Handling Unit (AHU), are less common in other time series data
such as traffic or electricity consumption. However, a purely data-driven
approach, typical in machine learning and deep learning, faces challenges in
this context. Firstly, datasets may lack sufficient samples; for instance, as
shown in Figure 2 the building dataset presented in this paper only has 44
temperature sensors and 54 maximum and minimum temperature setpoints, instead
of thousands, making it challenging to discern causal relationships
accurately. Secondly, gathering more data is hindered by the sensitive and
private nature of building data, resulting in a scarcity of openly available
datasets.

Figure 2: The statistics about the count number for each unique Brick
ontology. According to the Brick ontology name, we grouped all the ontologies
into Alarm, Parameter, Limit, Status, Setpoint, Point, Command, and Sensor.
The assignment of each unique ontology to the corresponding Brick type are
represented by multiple colors as legend.

##  III How Heterogeneous is the Building IoT Point Data from Temporal and
Spatial Aspects?

###  III-A Temporal Dimension

The provided building data includes four granularity ranging from 1-minute,
5-minute, 10-minute, to 1-day. According to our statistics, more than 81% of
IoT point data is collected by the 10-minute sampling frequency. Besides, 15%
of the point data is sampled per minute, and 3.14% is sampled every 5 minutes.
However, some points, typically Usage_Sensor, are sampled daily.

Meanwhile, unlike the well-cited public datasets, the starting and ending
dates for the building data are varied by IoT points. The first time-step
ranges from 01/01/2021 to 13/07/2021, and the last time-step ranges from
07/06/2022 to 31/12/2023.

The irregularity and varying granularity of IoT data pose challenges for DL
models that typically assume regular time intervals. Preprocessing steps like
resampling, while necessary for alignment, can distort the natural variability
and patterns in the data. Data imputation methods can fill the gaps for
periods where data is missing. Alternatively, models designed to handle
irregular intervals, such as those incorporating elapsed time as a feature or
attention mechanisms that prioritize relevant observations, can mitigate the
need for intensive preprocessing. However, these solutions require careful
implementation to fully leverage the intricate temporal dynamics of IoT data
without oversimplifying or misrepresenting the underlying processes.

###  III-B Spatial Dimension

The installation of IoT devices within a particular building is usually
reflected by the distinctive requirements and constraints inherent to that
environment. Initially, the varied architectural layouts of each building can
lead to variations in device placements since they are required to align with
the installation limitations. Secondly, the position of IoT devices depends on
wireless communication; they are usually installed within the range of the
wireless signal and avoid disruption by structural impediments, furnishings,
and competing electronic apparatuses. In addition, the device installation is
also impacted by the intended function of each IoT ontology to ensure efficacy
in its corresponding operational role. For instance, Damper_Position_Sensor
that are used to measure the percent of fully open should be placed where they
can best detect the changes of dampers. These individualized positioning of
IoT devices necessitated by specific functional requisites and structural
constraints can impact the spatial distribution of each IoT point data. This
tailored deployment can result in disparate and non-uniform data capture,
where zones with higher sensor diversity can yield high-resolution data while
others exhibit data sparsity. Meanwhile, the influence of physical
obstructions and electronic interference on wireless communication can further
affect the data quality and reliability variability across the IoT network.

Consequently, the data streams from IoT devices can show significant spatial
heterogeneity, reflecting the complex interplay of installed location,
environmental factors, and device functionality. Such heterogeneity can
present challenges in data analysis and model training, where assumptions of
uniform data distribution are invalidated. It necessitates the development of
sophisticated algorithms capable of integrating these diverse data streams to
form a coherent understanding of the monitored environment, as well as
advanced preprocessing techniques to normalize the data streams for consistent
analysis.

##  IV How Heterogeneous is the Distribution of Building IoT Point Data?

In Figure 3(a), we employ CopyNStep on the processed dataset to examine
periodicity within the whole building IoT point data. The processed procedure
is described in Section V-B. The dataset displays two distinct periodicities:
daily and weekly. This is evident from the consistent dips in error observed
whenever $\mathcal{N}$ represents a whole number of days, with even more
pronounced dips occurring at multiples of 7 days. Specifically, at
$\mathcal{N}=1$ timestep, the SMAPE stands around $10\%$, whereas at 1 day and
1 week intervals, it rises to approximately $35\%$. This indicates that $70\%$
of the variation stems from periodic effects, while another $20\%$ can be
attributed to temporary non-periodic effects, leaving the remaining $10\%$ to
be influenced by other factors.

Building point data can exhibit heterogeneous distribution attributed to the
diversity in ontology types. For instance, ontologies representing states or
events (e.g., Alarms or Setpoint) often contain fewer unique values due to
their finite set of conditions (e.g., On/Off, High/Low). Conversely, other
ontologies like Sensors feature more continuous variables such as temperature
or electricity usage and tend to display a wider range of values. This
contrast is depicted by the proportion of unique values per observation
($\tt{n\\_Unique}/\tt{n\\_Sample}\times 100\%$) in Figure 3(b). Additionally,
we conduct CopyLastStep on distinct Brick ontology as a naive benchmark to
illustrate the heterogeneity across different ontology types. As Figure 3(c)
demonstrates, the Sensor registers the highest SMAPE across 53 unique Brick
ontologies, while classes such as Setpoint, Alarm, and Parameter report
negligible errors. These discrepancies indicate a marked volatility in Sensor
data, potentially reflecting essential long-term dependencies within the data
streams. The CopyLastStep errors provide a potential modeling strategy for DL
models, suggesting models can pay more attention to recent observations for
Brick ontologies with lower errors as they appear to have less volatility and
shift to long-term dependencies learning for data streams with higher errors.

Figure 4 elaborates the heterogeneous distributions through visualizing time
series, Kernel Density Estimator (KDE) fittings, and spectral decomposition by
Fourier Transformation for 7 diverse building IoT point data. The analysis of
IoT points initially reveals a pronounced heterogeneity in distribution across
ontologies. The Position_Sensor shows a distinctly bimodal distribution,
indicative of a repeating, regular On/Off mechanism, likely reflecting the
binary operational state of the IoT equipment. In contrast, the Usage_Sensors
display varying behaviors, with some showing a cumulative pattern suggesting a
counting process and others exhibiting sparse activations that could indicate
infrequent or event-driven operations. The Electrical_Power_Sensor, with its
obvious Right-Skewed distribution and irregular spikes, indicates sporadic
consumption patterns caused by intermittent high-power usage. Meanwhile, the
Point_Sensor and Discharge_Air_Temperature_Sensor both present more continuous
distributions, where the former is unimodal with occasional value drops, and
the latter shows a more normal but skewed distribution, hinting at natural
variability influenced by daily temperature cycles.

In addition, within the same building ontology, there is notable diversity in
IoT point distributions. The three distinct Usage_Sensor points are analyzed
as an example for illustration. The first sensor within this category shows a
staircase accumulative trend, while another presents very low-frequency
activities with an almost flat time series, and yet another portrays
intermittent spikes. This intra-ontology variability emphasizes the complex
dynamics that a single ontology can encapsulate different usage patterns from
IoT devices, corroborating the premise of spatial heterogeneity. Such internal
heterogeneity within an ontology also poses unique challenges for DL model
training, as the models are required to generalize across a wide range of
behaviors and distributions from the same type of sensor or operational
process.

(a) Errors change with time for CopyNStep. The left y-axis represents MSE,
MAE, and RMSE values. The right y-axis represents the SMAPE range.

(b) The Unique Value ratio per Brick type. The ratio on the x-axis is computed
by the n_Unique divided by the n_Sample per Brick type.

(c) Error distribution per ontology on CopyLastStep sorted by SMAPE.
Ontologies that SMAPE $<1\%$ are ignored for observing convenience. The top
x-axis represents the SMAPE range, and the bottom x-axis is the MSE, MAE, and
RMSE ranges constrained in [0, 1].

Figure 3: Heterogeneity in building IoT point data distributions and
periodicity. Figure 4: Distribution analysis for data streams collected from
different ontology. Each row in the sub-figures indicates the original time
series, KDE fittings, and Fourier Transformation from top to bottom.

##  V How Does Heterogeneity Impact the Performance of Deep Learning Models?

We employ state-of-the-art forecasting models to quantitatively analyze the DL
model performance on heterogeneous building IoT point data.

###  V-A Problem Formulation

This subsection provides the problem formulation for multivariate forecasting
on 1) all IoT point data and 2) unseen IoT point data. Denote the given time
series containing $T$ time steps and $N$ variant IoT points as
$X\in\mathcal{R}^{N\times T}$, the historical window and forecasting horizon
as $H$ and $L$.

####  V-A1 Multivariate Forecasting

Traditional multivariate forecasting aims to predict the future values of
multiple IoT points by analyzing their historical records. The training data
is formed by copying $N$ IoT points’ last $H$ time-step values. The input
feature map $\mathcal{X}\in\mathcal{R}^{N\times H\times K}$, where $K<T$
represents the number of data points. The predictive model generates the
forecasted outcomes $\mathcal{Y}\in\mathcal{R}^{N\times L\times K}$.

####  V-A2 Forecasting on Unseen Nodes

Unlike typical multivariate forecasting, this experiment aims to train the
model in partial IoT points and then forecast on the others. The $N$ IoT
points are randomly split into two subsets, $N_{train}$ and $N_{test}$,
differentiated by training and testing purposes. Given the input feature
$\mathcal{X}\in\mathcal{R}^{N_{train}\times H\times K}$, the model output is
targeted by $\mathcal{Y}\in\mathcal{R}^{N_{test}\times L\times K}$ where
$N_{train}<N$ and $N_{test}<N$.

###  V-B Training Data

####  V-B1 Preprocessing Workflow

To construct the regular training data, we slice the data by date from
00:00:00 01/05/2022 to 00:00:00 01/10/2022 and set the granularity to 10 min.
Missing values caused by temporal irregularity are filled with the mean value.
To simplify the modeling process, we drop point data whose missing ratio
r_missing > 2% and n_Unique > 1,000. A Standard Normalization process is
conducted for scaling. Consequently, we obtained the 5-month training data
containing 22,033 time steps and 166 distinct IoT points.

####  V-B2 Input and Ground Truth Construction

Given the historical window $H$ and forecasting horizon $L$, the input feature
map is defined as $\mathcal{X}=\\{x_{N,0},x_{N,1},...x_{N,K-1}\\}$. Each data
point $x_{N,k}=X_{N,k:k+H}$ is constructed by copying the last $H$ values, its
corresponding forecasting values $y_{N,k}=X_{N,k:k+H+L}$. Our study sets the
$H$ and $L$ as 12.

We employed the ST-split methods proposed by Prabowo et al. [13] to split the
processed input features $\mathcal{X}$ as needed. This method enables a
sequential split on the temporal dimension and a random split on the spatial
dimension. The training data used for both experiments are split on the
temporal dimension, with splitting rates of 0.7, 0.1, and 0.2 for training,
validation, and testing, respectively. The spatial spit is further applied for
the unseen IoT points forecasting with the same splitting rates as the
temporal.

###  V-C Baseline Models

GWaveNet [IJCAI19] [14] Graph WaveNet comprises an initial MLP encoder, a
sequence of spatio-temporal layers, and a final MLP decoder. This architecture
effectively processes time-series data in a graph structure, leveraging both
temporal and spatial information and integrating the pre-trained encoder’s
output to enhance performance.

Informer [AAAI21] [10] Informer is an advanced transformer-based model
tailored for Long Short-Term Forecasting (LSTF). The introduction of
ProbSparse self-attention mechanism enables the reduction of time complexity
and memory usage while maintaining competitive performance in aligning
sequence dependencies for long sequence inputs. Unlike traditional step-by-
step prediction methods, Informer employs a generative-style decoder that
predicts long sequences in a single forward operation. It significantly
enhances inference speed for long-sequence predictions while maintaining
simplicity in concept.

DLinear [AAAI23] [15] DLinear is a novel one-layer linear model developed to
address the LTSF task. This approach challenges the recent trend of using
complex Transformer-based solutions for LTSF by questioning their
effectiveness in preserving temporal information due to the permutation-
invariant nature of the self-attention mechanism. The surprising efficiency of
DLinear highlights the potential for re-evaluating the application of
Transformers in various time series tasks. It suggests redirecting research
efforts towards simpler, potentially more effective models for capturing
temporal relationships in data.

TimesNet [ICLR23] [16] TimesNet is a task-general backbone model designed for
comprehensive time series analysis tasks, leveraging the inherent multi-
periodicity of time series data. Recognizing the limitations of 1D time series
representation, TimesNet innovatively transforms 1D sequences into 2D tensors,
enabling the separate modeling of intraperiod and interperiod variations
across the tensor. By employing 2D kernels, the TimesBlock component
efficiently captures complex temporal patterns and adapts to multiple
periodicities, enhancing performance across various time series applications.

PatchTST [ICLR23] [17] PatchTST is a Transformer-based model for multivariate
time series forecasting and self-supervised representation learning. Its
architecture hinges on two innovative approaches: firstly, the segmentation of
time series into smaller subseries-level patches, which serve as input tokens
to the Transformer, and secondly, a channel-independence feature, which
applies a single set of embeddings and Transformer weights across all channels
of a univariate time series. The patching technique enhances the model by
preserving local semantic information within the embedding, drastically
reducing the attention’s computational load and memory demands and enabling
the consideration of longer historical context.

SCPT [ECMLPKDD23] [13] SCPT (Spatial Contrastive Pre-Training) is an
innovative framework developed to enhance the generalization of deep
forecasting models to previously unseen roads. SCPT utilizes a spatial encoder
module pre-trained with contrastive learning, allowing it to extract
meaningful latent features from just two days of traffic data on new roads
without re-training. This encoder specializes in discerning complex Markovian
signals within the traffic data, separating them from more predictable
periodic signals. The SCPT approach incorporates a unique component, the
spatially gated addition layer, which effectively merges the encoder’s latent
features with those of established backbone models.

iTransformer [ICLR24] [18] iTransformer is an innovative adaption of the
Transformer architecture designed for LSTF tasks. It revolutionizes the
approach by inverting the dimensions and treating time points as variate
tokens. These tokens are processed through an attention mechanism that
captures the intricate correlations across multiple variables. Simultaneously,
a feed-forward network is employed on each variate token to extract complex,
non-linear representations. Its enhanced capabilities offer robust
generalization across different variables and efficient use of various look-
back windows, positioning the iTransformer as a robust foundational framework
for time series forecasting tasks.

###  V-D Experimental Settings

We employ the Adam optimizer with a 0.001 learning rate and the MSE loss to
guide the multivariate forecasting and transfer learning experiments. Training
epochs are default to 10 with an EarlyStopper, which patience=3. Besides, a
LearningRateScheduler with factor=3 reduces the learning rate by half at every
epoch, starting with the initial learning rate of 0.001. All experiments are
completed on an A100 GPU acceleration. In addition, we employ MSE, RMSE, MSE,
SMAPE, and R2 for model performance evaluation. Each metric score is computed
separately for each time step, followed by the average.

###  V-E Result Analysis

Exp1: All IoT Points Forecasting
---
| MAE | RMSE | MSE | SMAPE | R2
GWaveNet | mean | 0.3433 | 0.7451 | 0.5594 | 25.8277 | 0.4320
std | 0.0694 | 0.0978 | 0.0676 | 6.4912 | 0.1034
25% | 0.2998 | 0.4915 | 0.7010 | 21.9758 | 0.3585
50% | 0.3637 | 0.5874 | 0.7664 | 27.3963 | 0.4044
75% | 0.3924 | 0.6266 | 0.7916 | 30.5799 | 0.5026
Informer | mean | 0.3048 | 0.7378 | 0.5463 | 21.7450 | 0.4004
std | 0.0390 | 0.0684 | 0.0467 | 3.0093 | 0.0942
25% | 0.2774 | 0.4968 | 0.7048 | 19.6322 | 0.3283
50% | 0.3083 | 0.5506 | 0.7420 | 21.9284 | 0.4012
75% | 0.3351 | 0.5984 | 0.7736 | 24.1005 | 0.4704
DLinear | mean | 0.3276 | 0.7185 | 0.5196 | 21.2940 | 0.4523
std | 0.0694 | 0.0860 | 0.0606 | 4.7086 | 0.0931
25% | 0.2826 | 0.4592 | 0.6776 | 18.2835 | 0.3815
50% | 0.3373 | 0.5266 | 0.7256 | 21.9352 | 0.4437
75% | 0.3807 | 0.5844 | 0.7645 | 24.8691 | 0.5174
SCPT | mean | 0.3399 | 0.7347 | 0.5438 | 24.7975 | 0.4481
std | 0.0575 | 0.0951 | 0.0659 | 4.4296 | 0.0986
25% | 0.3027 | 0.4771 | 0.6907 | 21.8089 | 0.3681
50% | 0.3501 | 0.5544 | 0.7446 | 25.4500 | 0.4363
75% | 0.3866 | 0.6217 | 0.7884 | 28.1899 | 0.5160
TimesNet | mean | 0.3075 | 0.7458 | 0.5581 | 21.4314 | 0.4055
std | 0.0385 | 0.0685 | 0.0461 | 2.9658 | 0.0901
25% | 0.2804 | 0.5082 | 0.7129 | 19.3511 | 0.3367
50% | 0.3106 | 0.5601 | 0.7484 | 21.7264 | 0.4034
75% | 0.3375 | 0.6110 | 0.7817 | 23.7232 | 0.4727
PatchTST | mean | 0.2739 | 0.7105 | 0.5077 | 19.4184 | 0.4381
std | 0.0492 | 0.0778 | 0.0555 | 3.7578 | 0.1049
25% | 0.2427 | 0.4545 | 0.6742 | 17.0819 | 0.3594
50% | 0.2818 | 0.5157 | 0.7181 | 20.0105 | 0.4311
75% | 0.3102 | 0.5655 | 0.7520 | 22.2231 | 0.5119
iTransformer | mean | 0.3047 | 0.7379 | 0.5466 | 21.6993 | 0.4028
std | 0.0389 | 0.0686 | 0.0468 | 2.9710 | 0.0919
25% | 0.2774 | 0.4970 | 0.7049 | 19.6277 | 0.3322
50% | 0.3084 | 0.5500 | 0.7416 | 21.9680 | 0.3996
75% | 0.3350 | 0.5988 | 0.7738 | 24.0214 | 0.4705
Exp2: Unseen IoT Points Forecasting
| MAE | RMSE | MSE | SMAPE | R2
GWaveNet | mean | 0.2952 | 0.4724 | 0.6811 | 22.9095 | 0.5004
std | 0.0749 | 0.1263 | 0.0962 | 5.6375 | 0.1353
25% | 0.2511 | 0.3907 | 0.6249 | 19.5307 | 0.3964
50% | 0.3103 | 0.4892 | 0.6993 | 24.0671 | 0.4791
75% | 0.3533 | 0.5697 | 0.7548 | 27.2584 | 0.5839
DLinear | mean | 0.3096 | 0.4262 | 0.6499 | 22.2792 | 0.5314
std | 0.0692 | 0.0826 | 0.0650 | 4.4652 | 0.1007
25% | 0.2659 | 0.3726 | 0.6103 | 19.5910 | 0.4533
50% | 0.3205 | 0.4357 | 0.6600 | 23.1356 | 0.5173
75% | 0.3668 | 0.4914 | 0.7010 | 25.8554 | 0.5950
TABLE I: Empirical results for 1) multivariate forecasting on all IoT points
and 2) multivariate forecasting on unseen IoT points. The best score for each
metric is highlighted in Bold.

As the top suitable in Table I, PatchTST generally exhibits superior
performance with the lowest mean errors across MAE, RMSE, and MSE, alongside
the lowest SMAPE on 25%, 50%, and 75% levels, indicating it offers the most
accurate and consistent predictions relative to the actual values. PatchTST
outperforms GWaveNet by 20.20%, 9.25%, 4.64%, 24.82%, Informer by 10.12%,
7.07%, 3.69%, 10.70%, DLinear by 16.38%, 2.27%, 1.10%, 8.81%, SCPT by 19.42%,
6.63%, 3.29%, 21.69%, TimesNet by 10.91%, 9.03%, 4.72%, 9.39%, iTransformer by
10.11%, 7.11%, 3.71%, 10.51%, on MAE, RMSE, MSE, and SMAPE for the average of
the 12 time-step. DLinear, on the other hand, showcases its strength in
explaining the variance within the dataset, evidenced by its highest mean R2
score, suggesting it can capture a higher proportion of the variability in the
dependent variable. It outperforms GWaveNet, Informer, SCPT, TimesNet,
PatchTST and iTransformer by 4.69%, 12.96%, 0.93%, 11.53%, 3.24% and 12.28% on
R2. Meanwhile, TimesNet is notable for its consistency across metrics,
indicated by the lowest standard deviations in MAE, MSE, SMAPE, and R2,
suggesting stable and reliable forecasts. Compared to the evaluation scores
when these baseline models are applied on well-processed public datasets, as
M4 [19], the building IoT point data errors exhibit a higher average,
indicating the complexity and difficulties while processing this heterogeneous
data.

The discovery that is worth mentioning is the models that yield the highest
metric scores on the employed building IoT point dataset, which are PatchTST
and DLinear, are channel-independent (CI) models. This suggests CI models
exhibit better applicability than channel-dependent (CD) models (i.e.,
Informer, iTransformer) on heterogeneous building ontology data. However, CI
models are found to lack generalizability and robustness on unseen channels
and tend to overlook potential interactions by many studies, as [20]. Their
adaptivity to building heterogeneity requires more IoT point datasets for
further collaborative validation.

The bottom sub-table in Table I shows the metrics of GWaveNet and DLinear for
the new point data forecasting tasks. GWaveNet outperforms DLinear, but the
latter achieves better scores on RMSE, MSE, SMAPE, and R2. Another discovery
is that the metrics scores on the unseen IoT points forecasting are better
than those on all IoT points forecasting for either of the two adopted
methods. This may be caused by the random split on the spatial dimension not
being able to balance the distribution of heterogeneous data in training and
testing sets, leading to the result that IoT points in the testing set are
more predictable than those in the training set. This inspection requires more
building IoT point datasets from reality to verification.

Ultimately, the choice between these models should be guided by the specific
forecasting goals and the trade-offs between accuracy, interpretability, and
consistency that are most critical for the task.

##  VI What Challenges and Opportunities Lie Ahead?

According to our study, the heterogeneity in building IoT point data raises
challenges for deep learning applications by heterogeneous distributions
rooted in building ontology, etiology, temporal irregularity, and spatial
distribution. These challenges, at the same time, suggest opportunities and
potential directions for future research.

_Irregularity:_ Building IoT point data is accompanied by temporal
irregularity issues like diverse starting and ending dates and irregular
sampling frequency. The former issue can result in extensive gaps in the
training data, while the latter can introduce distortions when attempting to
downsample or upsample the time series. As the variety in data granularity
intensifies, these issues become increasingly complex. Identifying and
implementing effective data preprocessing methods, sophisticated imputation
techniques, or algorithms capable of handling irregular inputs is critical to
overcoming one of the principal obstacles in transforming scattered and
challenging-to-manage information into structured, model-ready data.

_Stochasticity:_ Heterogeneous data encompasses a wide range of data streams,
each varying in predictability. Some streams may exhibit clear, discernible
patterns that are relatively easy to forecast, while others may be erratic and
challenging to predict. Training DL models on such data poses significant
challenges, primarily due to the uneven distribution between training and
testing sets. This imbalance can cause the model to overly adapt to the more
majority or predictive streams while neglecting the less frequent or volatile
streams. The trained DL model thereby possibly performs well on common or easy
scenarios but fails to generalize effectively across the full spectrum of data
types in the actual environment. In addition, conventional evaluation metrics
cannot truly reflect the whole picture of the model’s effectiveness across the
heterogeneous data. Addressing these challenges requires careful dataset
preparation and the selection of appropriate evaluation metrics that consider
the diversity and specific characteristics of different data streams.
Strategies such as stratified sampling and advanced machine learning
techniques to handle imbalanced data can also be employed to ensure more
robust and equitable model training.

_Generalizability:_ The spatial heterogeneity inherent in building IoT point
data significantly affects the generalization capabilities of DL models, which
ideally should operate effectively across different buildings. Due to the
heterogeneity that arises from the varying architectural constraints, the
diversity of IoT device installations, or different usage patterns, models
trained on data from one building may perform poorly when applied to another.
Potential solutions can involve transfer learning and fine-tuning technologies
to facilitate the model for efficient generalization. Recognizing the
unpredictable nature of building configurations as a primary factor for domain
discrepancies, using a few samples from a new building can often be more
effective than applying a model without any specific data from that building.
Technologies capturing and highlighting important features from these samples
can be particularly valuable. Such technologies could be feature extractors in
DL models and help bridge the gap between different buildings, enabling the
model to adjust effectively to new settings with just a few samples.
Additionally, given that Brick-based data incorporates a knowledge graph
detailing engineering domain information for specific buildings, feeding the
model with the ’hints from physical connectivity of ontologies’ provided by
the Knowledge Graph (KG) is a potential solution. This may require utilizing
Large Language Models (LLMs) to interpret the KG to model-readable
representative graphs to achieve notable model adaptability.

_Scalability:_ Scalability remains a critical challenge as models are scaled
up to accommodate larger datasets or as their complexity increases, leading
directly to heightened computational costs. As model complexity grows,
additional memory and processing power demand can significantly strain
existing hardware resources. Furthermore, the heterogeneity in data means that
not all IoT point data contribute equally to the target outputs. In fact, some
data streams may not only fail to enhance the model’s performance but actively
confuse the model during the training process, unnecessarily increasing model
complexity. Future research may develop feature selection strategies to
isolate the most pertinent IoT point data that could significantly enhance
model efficiency and accuracy, ensuring maximum utilization of computing
resources while improving scalability.

##  VII Conclusion

Based on the investigation detailed in this paper, we have conducted an in-
depth exploration of Brick point data derived from real-world building IoT
devices. Our study reveals significant heterogeneity stemming from various
factors such as building ontology, etiology, temporal irregularity, and
spatial diversity, and we have analyzed how these aspects influence data
distribution. A comparative study was included to delineate the impacts of
these factors on deep-learning forecasting models. Moving forward, we plan to
make the IoT point datasets, structured using the Brick protocol, available
for public access. This endeavor aims to facilitate broader research
initiatives and promote transparency. We are optimistic that the insights
learned from our analysis will catalyze further research, aiming to integrate
building ontology data with real-world DL applications effectively. This could
potentially lead to enhanced predictive capabilities and smarter building
management systems, thereby advancing the field of building informatics.

## References

  * [1] U. Varshney, “Research impact framework,” _RACE for 2030 CRC_ , 2022.
  * [2] N. Gao, W. Shao, M. S. Rahaman, J. Zhai, K. David, and F. D. Salim, “Transfer learning for thermal comfort prediction in multiple cities,” _Building and Environment_ , vol. 195, p. 107725, 2021.
  * [3] B. Dong, Y. Liu, H. Fontenot, M. Ouf, M. Osman, A. Chong, S. Qin, F. Salim, H. Xue, D. Yan _et al._ , “Occupant behavior modeling methods for resilient building design, operation and policy at urban scale: A review,” _Applied Energy_ , vol. 293, p. 116856, 2021.
  * [4] I. E. A. E. in Buildings and C. T. C. Programme. (1999) The ebc research programme. [Online]. Available: https://www.iea-ebc.org/ebc
  * [5] X. Jin, C. Zhang, F. Xiao, A. Li, and C. Miller, “A review and reflection on open datasets of city-level building energy use and their applications,” _Energy and Buildings_ , vol. 285, p. 112911, 2023.
  * [6] B. Jin, J. Wang, P. Xu, H. Zhang, X. Sun, and H. Liu, “Accurate charging demand estimation of electric vehicle via multi-data sources,” in _2020 IEEE 3rd International Conference on Information Systems and Computer Aided Education (ICISCAE)_. IEEE, 2020, pp. 124–131.
  * [7] B. Balaji, A. Bhattacharya, G. Fierro, J. Gao, J. Gluck, D. Hong, A. Johansen, J. Koh, J. Ploennigs, Y. Agarwal _et al._ , “Brick: Metadata schema for portable smart building applications,” _Applied energy_ , vol. 226, pp. 1273–1292, 2018.
  * [8] A. Prabowo, K. Chen, H. Xue, S. Sethuvenkatraman, and F. D. Salim, “Navigating out-of-distribution electricity load forecasting during covid-19: A continual learning approach leveraging human mobility,” in _Proceedings of the 10th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation_ , 2023.
  * [9] A. Prabowo, K. Chen, H. Xue, S. Sethuvenkatraman, and F. Salim, “Continually learning out-of-distribution spatiotemporal data for robust energy forecasting,” in _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_. Springer, 2023, pp. 3–19.
  * [10] H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” 2021.
  * [11] A. Trindade, “ElectricityLoadDiagrams20112014,” UCI Machine Learning Repository, 2015, DOI: https://doi.org/10.24432/C58C86.
  * [12] A. Prabowo, W. Shao, H. Xue, P. Koniusz, and F. D. Salim, “Because every sensor is unique, so is every pair: Handling dynamicity in traffic forecasting,” ser. IoTDI ’23. New York, NY, USA: Association for Computing Machinery, 2023, p. 93–104. [Online]. Available: https://doi.org/10.1145/3576842.3582362
  * [13] A. Prabowo, H. Xue, W. Shao, P. Koniusz, and F. D. Salim, “Traffic forecasting on new roads using spatial contrastive pre-training (scpt),” _Data Mining and Knowledge Discovery_ , pp. 1–25, 2023.
  * [14] Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph wavenet for deep spatial-temporal graph modeling,” _arXiv preprint arXiv:1906.00121_ , 2019.
  * [15] A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” in _Proceedings of the AAAI conference on artificial intelligence_ , vol. 37, no. 9, 2023, pp. 11 121–11 128.
  * [16] H. Wu, T. Hu, Y. Liu, H. Zhou, J. Wang, and M. Long, “Timesnet: Temporal 2d-variation modeling for general time series analysis,” in _The eleventh international conference on learning representations_ , 2022.
  * [17] Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam, “A time series is worth 64 words: Long-term forecasting with transformers,” _arXiv preprint arXiv:2211.14730_ , 2022.
  * [18] Y. Liu, T. Hu, H. Zhang, H. Wu, S. Wang, L. Ma, and M. Long, “itransformer: Inverted transformers are effective for time series forecasting,” _arXiv preprint arXiv:2310.06625_ , 2023.
  * [19] S. Makridakis _et al._ , “M4 forecasting competition dataset,” Online, 2018, available: https://forecasters.org/resources/time-series-data/m4-competition/.
  * [20] J. Chen, J. E. Lenssen, A. Feng, W. Hu, M. Fey, L. Tassiulas, J. Leskovec, and R. Ying, “From similarity to superiority: Channel clustering for time series forecasting,” _arXiv preprint arXiv:2404.01340_ , 2024.