# Sensor Data and the City: Urban Visualisation and Aggregation of Well-Being
Data

Thomas Johnson<EMAIL_ADDRESS>Nottingham Trent
UniversityNottinghamUK ,  Eiman Kanjo<EMAIL_ADDRESS>Nottingham Trent
UniversityNottinghamUK and  Kieran Woodward<EMAIL_ADDRESS>Nottingham Trent UniversityNottinghamUK

(2020)

###### Abstract.

The growth of mobile sensor technologies have made it possible for city
councils to understand peoples’ behaviour in urban spaces which could help to
reduce stress around the city. We present a quantitative approach to convey a
collective sense of urban places. The data was collected at a high level of
granularity, navigating the space around a highly popular urban environment.
We capture people’s behaviour by leveraging continuous multi-model sensor data
from environmental and physiological sensors. The data is also tagged with
self-report, location coordinates as well as the duration in different
environments. The approach leverages an exploratory data visualisation along
with geometrical and spatial data analysis algorithms, allowing spatial and
temporal comparisons of data clusters in relation to people’s behaviour.
Deriving and quantifying such meaning allows us to observe how mobile sensing
unveils the emotional characteristics of places from such crowd-contributed
content.

Voronoi, Visualisation, Mental Well-Being, Sensors, Urban Environment

††copyright: acmcopyright††journalyear: 2020††doi:
10.1145/1122445.1122456††conference: UbiComp ’20; September 12–16, 2020;
Cancun, Mexico††price: 15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Human-
centered computing Geographic visualization

##  1\. Introduction

Stress-related illnesses are drastically increasing around the world. Stress
is often difficult to detect and is commonly associated with a stigma of
embarrassment and humiliation preventing people from receiving help or
treatment. However, the impact of stress is profound as it costs the UK
economy £2.4 billion per year due to work absence (Perkbox, 2018). To help
alleviate stress, many people restore their attention and seek relief through
meditation or outdoor recreation. Nature and urban environments offer a
restorative experience to improve an individuals’ well-being. However, some
environments are busy, hectic and stressful resulting in a negative impact in
well-being and not relieving stress or exhaustion. With recent technological
advances, several studies have emerged which can be utilized to assess the
effects of built environments on humans using physiological sensors. For
example, Heart Rate (HR) monitors and Electrodermal Activity (EDA) sensors,
have shown enhanced results following the exposure to restorative
environments.

When comparing walking in the countryside compared with urban environments the
results illustrated how places are constructed through different senses and
people’ s bodies impact whether the place is perceived as welcoming and
pleasant or hostile and aggressive (Thrift, 2003). Positive impressions about
places encourage people to visit, these places are most commonly quiet,
restful and tranquil allowing people to reduce their stress levels by
providing a palliative to the nonstop attentional demands of typical, city
streets. It is important to use objective sensor data to study the
relationship between different places and the mental states of people who
visit these places. Kaplan et al. (Kaplan, 1995), Kohlleppel et al.
(Kohlleppel et al., 2002) have studied the power of natural environments to
provide a restful experience encouraging a quick and strong recovery from any
stress encountered. In addition to natural settings, coffee shops, health
clubs, video arcades and some retail shops were shown to encourage the
restoration from stress and can promote positive emotions (Rosenbaum, 2009).

Many studies have been conducted to study the effects of environmental noise
on mental health and human well-being. The research results proved that noise
can impair productivity and cause serious health problems such as chronic
stress and heart diseases (Stansfeld and Matheson, 2003). Physiological
signals contain useful patterns that aim to help identify individual’s mental
state. For example, using physiological sensors such as Electrocardiogram,
Electromyogram, skin conductance and respiration has found to determine a
driver’s overall stress levels (Healey and Picard, 2005). Furthermore, recent
studies have begun to use physiological sensors to identify an individual’s
mood and emotions (Woodward et al., 2018) (Woodward et al., 2019) (Woodward et
al., 2020a).

There are many applications emerging in the area of environmental monitoring
and the impact the environment has on health and well-being (Kanjo et al.,
2018) (Donaire-Gonzalez et al., 2019). For instance, a wearable, low power,
air quality and environmental monitoring sensor sampling a range of air
pollutants (CO, NO2 and O3). The sensors in the system using real-time
information helped those suffering from health related problems such as asthma
which in turn helped those individual’s to avoid highly-polluted environments
(Zappi et al., 2012). Furthermore, the application NoiseSpy is a low-cost
sound measurement device that monitors environmental noise levels, allowing
users to explore an environment while visualising noise levels in real time
(Kanjo, 2010).

The relationship between emotional stability and an urban environment is
especially important and challenging in cities. This is because the
environments are typically highly dynamic and densely populated. The
ubiquitous nature of smartphones coupled with sensors and increased
computational power has allowed them to be considered as serious competitors
to dedicated sensor platforms. Mobile phones and physiological sensors present
many opportunities to observe human emotion in urban environments. (Alajmi et
al., 2013).

Previous research on visualising sensor-based data spatially is focused
primarily on mapping, heat-maps or grid overlay. The complexity of a spatial
interpolation has a signiﬁcant impact on how much sensor data can be placed
over a map. For example, Kriging can be used to visualise and map spatial data
(Krige, 1952). Alternatively, heat maps can be utilised for a quicker
visualisation. This has been used to track Electrodermal activity (EDA) in
supermarkets and around shopping centres to understand how these environments
have a negative impact on an individual’s well-being (Alajmi et al., 2013) (El
Mawass and Kanjo, 2013). Figure 1 presents a noise visualisation map whereby
noise was recorded over a two week period. The darker colours on the grid view
indicate a higher noise level in that particular location (Kanjo, 2010).

Figure 1. Visualising environmental noise data using grid overlay around
Cambridge City.

There has been limited research focused on understanding how human emotion
impacts when exploring an urban environment. A study (Matei et al., 2001) has
started to use this method by mapping human emotion around a place, by
visualising on a map individual feelings (fear and comfort) in Los Angeles.
However, although this is an early study in this area, the data was recorded
prior and did not report in real-time which would have given a deeper
understanding into the overall impact. The use of self-reporting to record
well-being is becoming increasingly popular (Woodward et al., 2020b)
especially through mobile systems such as Mappiness (MacKerron, 2012) and WiMO
(Mody et al., 2009) because of their ability to link the individual’s emotion
to a particular location.

We explore the use of physiological and environmental sensors in an urban city
context to monitor how the environment impacts stress. The built environments
considered in this work consist of both green spaces and urban built area
allowing us to understand how the exposure to natural green spaces may promote
greater attention restoration and stress recovery than visiting built
environments. We propose a new method of spatial exploration and visualisation
to help reduce stress within urban environments.

##  2\. Spatial Aggregation and Visualisation

To visualise the dynamic trends, the urban area needs to be divided in smaller
areas. Binning is a frequently used method to visualise geographical patterns
which use heat maps of the data. Figure 1 illustrates the heat map of the
physiological sensor data from one user data along within the city
environment. The heat map shows that stressful hotspots are scattered along
the path and not confined to one particular area.

While Figure 2 shows the level of intensity of different urban environments
based on GPS coordinates, the sensor data on these heat maps do not show the
real distribution of sensor data. One option to solve this challenge is to
divide the study area into grid cells (Kanjo, 2010); however, it is difficult
to allocate a cell for each sensor reading, moreover, it is not possible to
decide on the cell size, since the density of the sensor mobility traces can
be of different density distributions.

Figure 2. Map showing heat maps of various sensor data in urban settings.

The biggest challenge in metric layer visualisation lies in scalability. There
can be large number of records to map in a single city. Drawing all these
records at their accurate locations simultaneously will be impossible, due to
the large amount of rendering cost and the required visual bandwidth.

To address these issues, we use a combinatorial computational geometry
algorithm called “Voronoi”, which is a diagram partitioning a plane into
regions based on distance to points in a specific subset of the plane (Du et
al., 1999). Voronoi diagrams divide the space into a set of regions called
Voronoi cells, including the space that is closest to the object (location).
The size of these cells give an indication of the density of the area a
certain object is in or the size of an object.

The cell structure also shows the Delaunay triangulation, which enables the
calculation of an object’s immediate set of neighbours. The definition of a
Voronoi cell is given by the following equation, where x is a planar metric
space; p is the set of generator points in the metric space; and d is the
distance between all points in x and a specific generator point.

(1) $\text{Vori}=\\{x\mid d(x,pi)\leq d(x,pj),j\neq i\\}$

The creation of a Voronoi tessellations is a dynamic procedure that is
repeated until all of the points are represented in adjacent polygons. If
there are not a sufficient number of particles to satisfy Equation (1), then
the Voronoi diagram is only partially filled and the data is redistributed.
The Voronoi diagram is composed of a collection of tessellations defined as
Vor, where:

(2) $Vor=\left\\{Vor_{1},Vor_{2}\ldots\operatorname{Vor}_{n}\right\\}$

By giving each polygon a value that corresponds to the sensor value collected
in a particular GPS coordinate, it is possible to divide the space into
adjacent polygons with different sensor reading which are represented in
colours. This type of spatial analysis, helps in understanding the degree to
which a place is similar to other nearby places, the correlation among
different spatial objects needs to be deﬁned. We have mapped users to
different regions based on their mobility and form several user groups for
further exploration.

##  3\. Experimental setup

We use the “EnvBodySens” dataset consisting of various sensor and self-report
data from a smart phone application, and a Microsoft wristband 2. Collected
data is logged and stamped with the time and date and GPS location. In the
EnvBodySens application, an interface is implemented for continuous and quick
labelling of user emotions while walking and collecting data. When the user
launches the application, mobile interface appears with five iconic facial
expressions ranging from very negative to very positive. Users were asked to
constantly select one of the affective categories (in the form of buttons) as
they walk around the city centre. We adopted the 5-step SAM Scale for Valence
taken from (Banzhaf et al., 2014) to simplify the continuous labelling
process.

Forty participants took part in the study all female with an average age of
28. Participants were scheduled to take part in the study in order to collect
data around Nottingham city centre. The street we selected for the experiment
is a pedestrianised shopping street in the centre of Nottingham. The street
hosts some of the stylish shops and boutiques housed inside elegant buildings
along with a number of cafes for shoppers to rest in. Data was collected under
similar weather conditions (average 20∘ degrees), at around 11am.During the
data collection process 550,432 data lines of locations traces were collected
as well as 5,345 self-report responses.

##  4\. Discussion

We explore the use of Voronoi diagrams to map the impact physiological and
environmental sensor data have within an urban environment. Voronoi is a
computational geometry algorithm that allows the visualisation of data
(Dobrin, 2005). It works by defining a set of regions called cells. Once the
diagram has been created the size of the overall cells give an indication of
the density of the area an object is in or the size of the object itself.

Figure 3. Voronoi showing self-report emotions data within a fragrance and
Beauty Shop in Nottingham.

Figure 3 demonstrates self-reported emotions data using the EnvBodySens app
within a city environment. To represent the metric value of each record, we
apply a linear mapping from the value to the color hue, which forms a blue to
red rainbow palette. In particular, the red polygons highlight there was
negative sentiment reported by the user. The green polygons aim to highlight
where there was a positive sentiment.

Within the diagram, the negative sentiments, depicted in red polygons,
demonstrates where participants’ needed to respond to an assistant within a
shop. This correlation demonstrates the negative impact on an individual’s
well-being. We see these results being of particular importance to city
planners, to help them design and understand what individual’s needs in city
centres to help reduce stress and improve individual well-being. Our
visualisation tools have enabled the mapping of all collected variables
(physiological and environmental) as separate layers.

Figure 4. Voronoi presenting the heart rate of participants whilst visiting a
popular clothes shop. in Nottingham.

Figure 4 shows two Voronoi diagrams displaying the heart rate (HR) when
participants entered a range of different urban environments. The darker
colours used indicate higher levels of heat rate. Through analysis of sensor
data we have discovered a trend when shopping, in that heart rate frequently
increases when encountering discounted items in the shop.

Using Voronoi diagrams to express data visually offers many opportunities to
plot correlations and observe patterns. Figure 5 presents a map of a street in
Nottingham City Centre where Voronoi diagrams have been created using the
heart rate and location sensor data. By using location tracking and the
utilisation of Voronoi diagrams plotted on to maps has clearly shown the areas
where there is an increase in heart rate and stress.

Figure 5. Heart rate Voronoi diagram overlays of different shops in popular
Nottingham shopping street.

We have found that these Voronoi charts can be utilised by city planners to
explore areas that induce the most stress such as a busy shopping environment
and create calming spaces to develop a restorative experience. By enabling
people to reduce stress and improve their well-being will have positive
impacts on cities urban environments by creating a positive experience,
potentially helping to increase the number of visitors and improve shopping
experiences.

##  5\. Conclusion

We have shown how Voronoi diagrams demonstrate great potential to spatially
visualise and explore emotional and physiological sensor data. This spatial
exploration enables city planners to develop urban environments that can help
to reduce stress and improve well-being. It is hoped that city centres could
utilise this data to add calming areas such as green spaces in environments
which are considered as high stress. By understanding the environments which
induce stress and those that aid relaxation we hope future urban developments
can use this method of data visualisation to create restorative experiences in
high stress environments to improve mental well-being. In the future, we
envision the use of Voronoi charts should continue to be explored in urban
environments. With more participants, additional sensor data such as
additional environmental measurements (gases, particulate matter, noise) and
in more urban city environments will help to evaluate the full potential this
form of spatial exploration enables and further understand how urban
environments impact mental well-being.

## References

  * (1)
  * Alajmi et al. (2013) Nouf Alajmi, Eiman Kanjo, Nour El Mawass, and Alan Chamberlain. 2013.  Shopmobia: An Emotion-Based Shop Rating System. In _2013 Humaine Association Conference on Affective Computing and Intelligent Interaction_. IEEE, 745–750.
  * Banzhaf et al. (2014) Ellen Banzhaf, Francisco De La Barrera, Annegret Kindler, Sonia Reyes-Paecke, Uwe Schlink, Juliane Welz, and Sigrun Kabisch. 2014.  A conceptual framework for integrated analysis of environmental quality and quality of life.  _Ecological Indicators_ (2014).
  * Dobrin (2005) Adam Dobrin. 2005\.  A Review of Properties and Variations of Voronoi Diagrams.  _Whitman.Edu_ (2005).
  * Donaire-Gonzalez et al. (2019) David Donaire-Gonzalez, Antònia Valentín, Erik van Nunen, Ariadna Curto, Albert Rodriguez, Mario Fernandez-Nieto, Alessio Naccarati, Sonia Tarallo, Ming Yi Tsai, Nicole Probst-Hensch, Roel Vermeulen, Gerard Hoek, Paolo Vineis, John Gulliver, and Mark J. Nieuwenhuijsen. 2019.  ExpoApp: An integrated system to assess multiple personal environmental exposures.  _Environment International_ (2019).
  * Du et al. (1999) Qiang Du, Vance Faber, and Max Gunzburger. 1999.  Centroidal Voronoi tessellations: Applications and algorithms.  _SIAM Rev._ (1999).
  * El Mawass and Kanjo (2013) Nour El Mawass and Eiman Kanjo. 2013.  A supermarket stress map. In _UbiComp 2013 Adjunct - Adjunct Publication of the 2013 ACM Conference on Ubiquitous Computing_.
  * Healey and Picard (2005) J.A. Healey and R.W. Picard. 2005.  Detecting Stress During Real-World Driving Tasks Using Physiological Sensors.  _IEEE Transactions on Intelligent Transportation Systems_ 6, 2 (jun 2005), 156–166.
  * Kanjo (2010) Eiman Kanjo. 2010\.  NoiseSPY: A real-time mobile phone platform for urban noise monitoring and mapping.  _Mobile Networks and Applications_ (2010).
  * Kanjo et al. (2018) Eiman Kanjo, Eman M.G. Younis, and Nasser Sherkat. 2018\.  Towards unravelling the relationship between on-body, environmental and emotion data using sensor information fusion approach.  _Information Fusion_ (2018).
  * Kaplan (1995) Stephen Kaplan. 1995\.  The restorative benefits of nature: Toward an integrative framework.  _Journal of Environmental Psychology_ (1995).
  * Kohlleppel et al. (2002) Tammy Kohlleppel, Jennifer Campbell Bradley, and Steve Jacob. 2002.  A walk through the garden: Can a visit to a botanic garden reduce stress?  _HortTechnology_ (2002).
  * Krige (1952) D G Krige. 1952\.  A Statistical Approach to Some Basic Mine Valuation Problems on the Witwatersrand.  _Journal of the Chemical, Metallurgical and Mining Society of South Africa_ (1952).
  * MacKerron (2012) George MacKerron. 2012\.  Mappiness.
  * Matei et al. (2001) Sorin Matei, Sandra J. Ball-Rokeach, and Jack Linchuan Qiu. 2001\.  Fear and misperception of Los Angeles urban space: A spatial-statistical study of communication-shaped mental maps.  _Communication Research_ (2001).
  * Mody et al. (2009) Ruturaj N. Mody, Katharine S. Willis, and Roland Kerstein. 2009.  WiMo: Location-based emotion tagging. In _MUM 2009 - Proceedings of the 8th International Conference on Mobile and Ubiquitous Multimedia_.
  * Perkbox (2018) Perkbox. 2018.  THE 2O18 UK WORKPLACE STRESS SURVEY.  (2018).
  * Rosenbaum (2009) Mark S. Rosenbaum. 2009\.  Restorative servicescapes: Restoring directed attention in third places.  _Journal of Service Management_ (2009).
  * Stansfeld and Matheson (2003) Stephen A. Stansfeld and Mark P. Matheson. 2003.  Noise pollution: Non-auditory effects on health.
  * Thrift (2003) Nigel Thrift. 2003\.  Space: the fundamental stuff of geography.  _Key concepts in geography_ (2003).
  * Woodward et al. (2020a) Kieran Woodward, Eiman Kanjo, David Brown, and T.M. McGinnity. 2020a.  On-Device Transfer Learning for Personalising Psychological Stress Modelling Using a Convolutional Neural Network. In _On-device Intelligence Workshop, MLSys , Austin, Texas_.
  * Woodward et al. (2019) Kieran Woodward, Eiman Kanjo, David Brown, T. M. McGinnity, Becky Inkster, Donald J Macintyre, and Athanasios Tsanas. 2019.  Beyond Mobile Apps: A Survey of Technologies for Mental Well-being.  _IEEE Transactions on Affective Computing_ (2019).
  * Woodward et al. (2018) Kieran Woodward, Eiman Kanjo, Andreas Oikonomou, and Samuel Burton. 2018.  Emoecho: A tangible interface to convey and communicate emotions. In _UbiComp/ISWC 2018 - 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium on Wearable Computers_.
  * Woodward et al. (2020b) Kieran Woodward, Eiman Kanjo, Andreas Oikonomou, and Alan Chamberlain. 2020b.  LabelSens: enabling real-time sensor data labelling at the point of collection using an artificial intelligence-based approach.  _Personal and Ubiquitous Computing_ (jun 2020), 1–14.
  * Zappi et al. (2012) Piero Zappi, Elizabeth Bales, Jing Hong Park, William Griswold, and Tajana Šimuni. 2012\.  The CitiSense Air Quality Monitoring Mobile Sensor Node. In _Proceedings of the 11th ACM/IEEE Conference on Information Processing in Sensor Networks_.