# A Forensic Audit of the Tor Browser Bundle

Matt Muir, Petra Leimich and William J Buchanan  School of Computing,
Edinburgh Napier University

###### Abstract

The increasing use of encrypted data within file storage and in network
communications leaves investigators with many challenges. One of the most
challenging is the Tor protocol, as its main focus is to protect the privacy
of the user, in both its local footprint within a host and over a network
connection. The Tor browser, though, can leave behind digital artefacts which
can be used by an investigator. This paper outlines an experimental
methodology and provides results for evidence trails which can be used within
real-life investigations.

###### keywords:

Digital forensics , Tor , Windows 10

††journal: Digital Investigation

##  1 Introduction

Use of the Tor Browser and network became mainstream in 2013, after the arrest
and conviction of Ross Ulbricht - the alleged mastermind behind darknet
marketplace “Silk Road” (Dewey, 2013). Prior to Ulbricht’s arrest, a former
American National Security Agency (NSA) contractor, Edward Snowden, had leaked
classified files to The Guardian newspaper concerning the extent of mass
surveillance by global intelligence services (Greenwald et al., 2013). This
information, later known as “the Snowden revelations”, ignited a global debate
surrounding the lack of privacy offered by digital communication.

Motivated by fear of privacy-intrusion, members of the public and cyber
criminals alike increasingly turned to the Tor Browser, tempted by its
anonymising features. A new influx of users and increasing media attention
catalysed academic research into the effectiveness of Tor and its ability to
protect user privacy. The majority of this research concentrated on
improvements to the performance and scalability of the network, as well as the
investigation of remote attacks intended to unmask users (Nordberg, 2015).
However, the question is whether anonymity is maintained where an adversary is
local, rather than remote?

This paper assesses the effectiveness of the Tor Browser in protecting the
user against such an adversary by conducting a forensic analysis of the
software and its interaction with the host operating system (OS). Our
contribution is threefold:

  * 1.

We show that artefacts proving installation/use of the browser are generated
in memory and on disk in the form of default bookmarks. These artefacts are
attributable to a particular user, uniquely identify the Tor Browser, and
persist through uninstallation and logout.

  * 2.

Furthermore, user activity within Tor is written to the Windows Registry as a
consequence of recent updates to Windows 10. This allows a forensic adversary
to determine the titles of pages visited using the browser.

  * 3.

From the results of our forensic audit, we devise a forensic methodology.

Our approach simulates typical web browsing activity with Tor. Using
virtualisation and a pre-determined browsing protocol allows artefact recovery
with static and live forensic techniques, such as process monitoring, keyword
searching and file carving, with the aid of Autopsy and the Volatility
Framework.

Static analysis reveals significant leakage of user activity in the snapshots
of machines used to perform the testing. This includes HTTP header
information, web page titles and an instance of a URL. Further, live analysis
identified traces of Tor processes even after the user had closed and
uninstalled the browser and logged out. The absolute path to the browser
executable was seen in RAM on several occasions, including the username of the
user running the browser and the device from which it was run.

As a result, Tor is easily identified, cannot be securely deleted and activity
from within a browsing session is determinable, suggesting that defence
against a local adversary has not been fully addressed by the developers. This
finding significantly expands on previous research, where artefacts pertaining
to use of the browser, but not browsing history, were found.

##  2 Background

The novel approach to anonymous networking which Tor is (in)famous for has
attracted interest from the cybersecurity community, largely resulting in
research concerning de-anonymisation of users from a network perspective.
However, it is the intention of the Tor Project to protect the user from both
network and local adversaries. This is achieved through the implementation of
features and design choices intended to obfuscate network activity and employ
anti-forensics techniques to prevent browsing session data from being written
to disk (Perry et al., 2018).

###  2.1 Tor Overview

Tor, an acronym for _The Onion Router_ , refers to both a browser and a
networking protocol developed by The Tor Project. The purpose of the Tor
network is to mask the original source of Internet traffic by redirecting data
from source to destination through a series of randomly chosen, volunteer-run
servers (Torproject.org, 2018). Unlike regular web browsing, this obfuscates
the user’s IP address, ensuring that their identity is not disclosed to the
websites they visit. By redirecting the traffic through encrypted channels,
data is also protected from passive traffic analysis.

Use of the network alone does not guarantee anonymity, making the Tor Browser
Bundle (TBB) a necessary component in the privacy-oriented architecture
(Torproject.org, 2018). The TBB is an extended support release (ESR) of
Mozilla’s Firefox browser, bundled with mandatory add-ons which protect user
anonymity. The browser’s design philosophy states that the TBB should protect
the user from known web attacks designed to reveal a user’s identity and also
minimise the amount of browsing data written to disk (Perry et al., 2018). To
enable this, plugins such as NoScript, which protects the user from malicious
code on websites, are used by default, as is Firefox’s Private Browsing mode.
Forcing Private Browsing mode means that web history is not saved to disk by
the browser itself, which should make it difficult for a forensic adversary to
determine websites visited (Mozilla.org, 2019).

###  2.2 Motivation

Users of the Tor Browser and network rely on the perceived anonymity offered
by the Tor Project. This research is motivated by the idea that little
importance has been placed on protecting the user’s anonymity from a forensic
perspective, both by the Tor Project and by academic research in this area. If
user anonymity is easily compromised by a forensic adversary, the browser
could be considered a weak link in Tor’s privacy-focused architecture. Whilst
it is not currently illegal to use Tor in many states, shifts in attitudes
towards the browser could take place as a reaction to political events, or as
a wider drive for censorship. Moreover, users of Tor who see the browser as a
tool for the greater good, such as journalists and political dissidents, may
be easily identified if the browser leaves obvious traces of its use on
computing systems. This could lead to persecution and further surveillance,
regardless of legality. Understandably, the same features which make Tor
attractive to the aforementioned users, also make it attractive to those who
wish to cause harm. Therefore, a greater understanding of the forensic
implications of using Tor is required by law enforcement investigating a range
of digital crimes. It is this complex societal impact which has inspired the
subject of this paper.

###  2.3 Traditional Browser Forensics

Traditional web browsers record a significant amount of data from a browsing
session - particularly in the default browsing mode. Web browser forensics
involves the recovery of browsing artefacts which reveal information regarding
a suspect’s online activity. Browser forensics has become increasingly
important for investigators as search history, download activity and page
views can aid understanding of the criminal motive.

Like other popular browsers, Firefox, the TBB’s underlying browser engine,
stores history, download and cookie information in SQLite databases, generally
in clear text. This allows forensic investigators to easily retrieve browsing
information (Roussas, 2009; Shepherd, 2018; Noorulla, 2014). Acknowledging
this lack of user privacy, browser developers introduced a private browsing
mode, intended to limit or, ideally, eliminate browsing artefacts being
written to disk. In a comprehensive study, Montasari and Peltola (2015) found
that the extent to which this aim has been achieved varies widely between
browsers, with Firefox considered the second-most secure of the four
mainstream browsers tested.

Using a methodology which incorporated virtualised Windows platforms and the
use of Autopsy to analyse snapshots, Montasari and Peltola (2015) attempted to
recover cached web pages, web browsing history, download history, visited URLs
and search terms from the target computer after performing a pre-defined
browsing protocol using each browser’s version of private browsing. A variety
of typical activities, involving different types of web content, were covered:

  1. 1.

Searching for and viewing a YouTube video.

  2. 2.

Using Google to find a web page and then performing another search on the
page.

  3. 3.

Downloading a Facebook profile picture.

  4. 4.

Searching and viewing an Amazon item.

  5. 5.

Searching for and viewing a PDF document using Google.

The analysis was split into two stages; examining firstly “common” and
“uncommon” locations on a hard drive and secondly the contents of the
computer’s volatile memory (RAM). Of the four browsers tested, Firefox was
deemed relatively secure, although traces of search terms issued during the
browsing protocol were discovered in pagefile.sys and the profile picture
downloaded from Facebook was also discovered using file carving.

These results confirm earlier work which found artefacts including search
terms and URLs from private browsing sessions in pagefile.sys (David and
Hargreaves, 2012; Findlay and Leimich, 2014) and in RAM (Aggarwal et al.,
2010; Findlay and Leimich, 2014), and contradict Mozilla’s claims that Firefox
leaves no traces after termination of a private browsing session (Mozilla.org,
2019).

###  2.4 Overview of Tor Forensics Research

Any weaknesses in Mozilla’s implementation of private browsing suggest that
the TBB could be susceptible to similar exploitation unless the developers
have managed to mitigate these vulnerabilities. Acknowledging the need for a
forensic audit, Sandvik (2013) of the Tor Project conducted a cross-platform
forensic analysis of the TBB on OSX, Linux and Windows. The TBB ”aims to
ensure that the user is able to completely and safely remove the bundle
without leaving other traces on her computer” (Sandvik, 2013). However, the
analysis results showed that the default behaviour of the operating system
hosting the TBB could compromise user anonymity through its generation of
artefacts related to features such as prefetching, paging and index searching,
negating the anti-forensics measures implemented by the developers.

Darcie et al. (2014) found that analysis of the Windows Registry could
determine the presence of the TBB after uninstallation, while Epifani et al.
(2015) discovered during a real-life case study, where a rogue employee’s
laptop was seized, that Tor use on the target computer was proven by artefacts
residing in Tor’s install directory as well as the behaviour of Windows
prefetching. Hence the Tor Project either does not or cannot prevent a trail
of evidence being created by the host OS.

As the TBB was found to be successful in avoiding browsing data being written
to the hard disk (besides in low-RAM situations) (Darcie et al., 2014), a
static forensic analysis would likely yield evidence of installation and use.
This could be advantageous to an investigator wishing to prove the use of Tor
over other browsers, however, lacking artefacts of browsing history, it does
not offer a picture of a subject’s activity within Tor. A requirement for live
forensic analysis became evident.

Dayalamurthy (2013) evaluated the work of previous forensic research using RAM
dumps and suggested that this live forensics approach would be feasible when
analysing the use of the TBB and attempting to de-anonymise Tor users. Her
research proposed a forensic methodology which includes the use of the
Volatility framework for memory analysis, along with the recovery of web
graphics and analysis of the Windows Registry. However, this methodology was
never tested.

Darcie et al. (2014) discovered that by creating a RAM dump of a target
system, indexing it and searching for the keyword _Tor_ , numerous hits were
returned in a computer used to download Tor, a computer where Tor was used and
finally, a computer where Tor had been used and then uninstalled. This method
of performing string searches on RAM captures echoes the methodology used by
Montasari and Peltola (2015). Warren (2017) later expanded on this analysis of
RAM by using the Volatility Framework, with the addition of community-
developed plugins, to conduct an analysis of the RAM of a Windows 10 VMWare
virtual machine. In doing so, evidence of Tor was found through the
identification of Tor DLLs, environment variables, SIDs and command line
usage.

##  3 Methodology

Figure 1: Overview of the Experimental Methodology

###  3.1 Design considerations

Throughout this project, forensic analysis is intended to have two clear aims:

  1. 1.

Expanding on previous research to confirm that the existence and use of the
Tor Browser is still determinable in Windows 10, and

  2. 2.

Proving that artefacts from the Tor browsing protocol can be recovered from
the target computer.

Live analysis of an application during runtime is particularly beneficial in
order to understand how the host operating system and application interact. As
the design efforts of the Tor project have focused on preventing writes to
disk (Perry et al., 2018), a live analysis could potentially yield more
information pertaining to the browsing session. Using an earlier version of
Tor (3.6.1), Darcie et al. (2014) found evidence of web browsing in the form
of JPEG and HTML files in live forensics but dead-box (static) forensics was
unsuccessful. In a previous live forensics analysis of the Firefox browser,
artefacts from a private browsing session were recovered from memory whilst
the browsing session was open, and - to a lesser extent – after the browsing
session had closed (Findlay and Leimich, 2014). This showed that Firefox was
able to terminate running processes, effectively flushing memory of artefacts
of the browsing protocol when the user closed the Private Browsing window.
However, whether or not this is also true for the TBB has not been
established; this will be taken into account in our methodology.

To build upon previous research, our approach has been designed to answer the
following questions:

  * 1.

Does Tor manage to protect the user by flushing evidence of its use from RAM
when the browsing session is closed?

  * 2.

Can Tor use be detected at four key moments; whilst the browser window is
open, after closing the browsing window, after deleting the installation
directory/associated files, and after the user has logged out?

  * 3.

Are files from the browsing protocol still recoverable in live forensics using
Tor 7.5.2, the version current at the time of writing?

Answers to the above should protect the user’s anonymity from an adversary who
manages to seize a personal computer during or shortly after Tor use, as well
as after the seizure of a shared computer. Timing is a key parameter when
designing a live forensics methodology. Figure 1 gives an overview of the
experimental methodology used in this project, clearly showing the timing of
RAM captures for live forensics and virtual machine snapshots for static
forensics. The experiments were repeated using Tor in mobile mode, i.e. run
from a thumb drive, echoing Findlay and Leimich (2014) who found that mobile
browsing reduced the artefacts left behind on the host, and using private
browsing mode in native Firefox as a control.

Inspired by Montasari and Peltola (2015), our test protocol mimics a user’s
browsing session with a variety of actions (i.e. searching, interacting with
web forms, downloading, streaming media):

  1. 1.

Type theguardian.co.uk into the address bar and view the page.

  2. 2.

Go to https://support.mozilla.org and enter “profile folder” into the search
box (submission to a web form).

  3. 3.

Go to images.google.com, search for “moog mother 32” and download the first
image.

  4. 4.

Go to https://soundcloud.com/monolake/liveego1999 and stream the first three
minutes of the embedded audio.

  5. 5.

At ebay.co.uk, search for “nike air force 1” and view the first item.

The URLs and search terms were selected to include unusual strings to
facilitate later keyword searches without false positives.

###  3.2 RAM Capture and Analysis Tools

VirtualBox supports the exporting of the contents of RAM, provided the virtual
machine is started from the command line with debugging mode enabled. The
.pgmtophysfile filename command can then be issued at any point to dump the
contents of RAM. The resulting ELF 64 file can then be analysed by the
Volatility Framework (http://www.volatilityfoundation.org).

Volatility was chosen due to its versatility as a command line tool, the wide
range of plugins available for it and its effective use in previous Tor
forensics research (Warren, 2017). With its vast array of plugins, Volatility
is highly effective in analysing the state of memory at different points in
time, building a comprehensive picture of the use of physical memory by an
application. To address the aims stated in Section 3.1, the following
Volatility plugins were chosen (Volatility Foundation, 2017):

  * 1.

cmdline: displays command line arguments for currently running processes.

  * 2.

dlllist: displays loaded DLLs for a given process.

  * 3.

dumpfiles: extracts memory-mapped and cached files.

  * 4.

envars: displays the environment variables used by processes.

  * 5.

pslist: prints all running processes by following the EPROCESS kernel
structure.

  * 6.

pstree: prints the running processes as a tree, identifies parent and child
processes.

  * 7.

shellbags: displays shellbag information taken from appropriate registry keys.

  * 8.

timeliner: creates a timeline from various artefacts in memory.

A common practice in live forensics is to observe running processes through
the use of tools such as procmon (Darcie et al., 2014). Processes are assigned
unique IDs (PIDs) in Windows and can be used to confirm that an application
was running during the time of the analysis. Once the PID of a process is
discovered, the process can then be searched in the output of Volatility’s
plugins, allowing an investigator to map its behaviour at certain points in
time. Live analysis will take place first so that knowledge of the results
which identify Tor use (for example, the names of DLLs) can then be used in a
static analysis. Since it is considered likely that individually dumped RAM
captures will expose contents of the browsing session (i.e. through caching),
this was deemed an important inclusion to this project, particularly as Warren
(2017) was unable to do so due to time constraints. Volatility’s dumpfiles
plugin will be used to achieve this and the resulting files analysed manually.
This approach, inspired by Dayalamurthy (2013), should show changes to the
Windows Registry, paths to the Tor install directory and contents of the
browsing protocol.

##  4 Results

###  4.1 Live Forensics

RAM was captured at four key moments in the experiment:

  1. 1.

After the browsing scenario – with the browser window still open.

  2. 2.

After closing the browser window.

  3. 3.

After dragging the installation directory to the Recycle Bin and emptying
(simulating an uninstall).

  4. 4.

After the user logs out.

This should allow conclusions to be drawn which answer the questions asked in
Section 3.1. As we found the artefacts left behind to decrease after each of
the four stages, we present the results from the final stage.

After uninstalling the TBB and logging out, the Tor-related processes had been
ended, but outputs from parsing the volatile memory with cmdline, pslist and
dlllist plugins still showed the firefox.exe process with PID 4384; psscan and
timeliner additionally showed timestamps (Figure 22(a),2(b)), while envars and
shellbags produced nothing notable. While these findings could have been
caused by a vanilla installation of Firefox, the output of the pstree plugin
(Figure 2(c)) includes the absolute path to the Tor install directory, tying
the process to the Tor browser. Via the PID and PPID, the start and end times
from psscan and timeliner can then also be attributed to Tor.

(a) psscan

(b) timeliner

(c) pstree

Figure 2: Selected Volatility Plugin Outputs - After logout

###  4.2 Static Forensics

Snapshots of the VM were taken after three key stages in the experiment to
enable a static forensic analysis to assess Tor’s ability to prevent writing
data to disk. Again, we focus on the final snapshot, taken after the
uninstallation/deletion of Tor (Figure 1).

####  4.2.1 Keyword Searches

Figure 3: Results of _monolake_ string search, reflecting the title of the
webpage (<title>Monolake Live at Ego Düsseldorf June 5 1999 by monolake |Free
Listening on SoundCloud</title>) suffixed with _\- Tor Browser_

The first step in the analysis was to carry out keyword searches for
_monolake_ , _soundcloud_ , _the guardian_ and _moog mother 32_ to determine
if and where these strings could be found in the forensic image. All search
strings were found to appear in the same three files, with several occurrences
in each file: NTUSER.DAT, ntuser.dat.LOG1 (a log of changes to NTUSER.DAT) and
MEMORY.DMP. The only exception was _moog mother 32_ , which did not appear in
ntuser.dat.LOG1 but instead appeared in unallocated space, suggesting that the
log may have been flushed. Closer inspection of the contents of the three
files revealed that all had exactly the same content, suggesting that the
content of NTUSER.DAT was being logged and then replicated in MEMORY.DMP.
Surprisingly, the file contents included the entire page title of each website
visited suffixed with _\- Tor Browser_ as well as the absolute path to the Tor
install directory, clearly showing the username (40187070) as well as
referencing firefox.exe within the Tor Browser directory. Figure 3 shows the
_monolake_ results as an example.

Figure 4: String search for _moog mother 32_ in NTUSER.DAT (extract)

The keyword matches for the string “moog mother 32” reflect its use in a
Google search (Figure 4). Interestingly, the page title contained the German
word for search, “Suche”, suggesting that the Tor exit node was located in a
German-speaking country. The Downloads folder and explorer.exe also appear in
this context, reflecting the saving of the downloaded image. As with the other
keyword searches, the absolute path to the Tor directory is shown, with the
Firefox executable clearly identified.

####  4.2.2 Unallocated Space

Several artefacts were recovered from unallocated space using Autopsy’s carver
module. When searching for the string ’clot’ from the browsing protocol, six
.dll, .edb and .reg files were discovered in unallocated space. Closer
examination of these files showed considerable browsing data leakage: all
contained the Ebay URL visited; the .dll and .edb files also referenced
Private Browsing while the .reg file included remnants of HTTP header
information (Figure 55(a)-5(b)). Further searching of unallocated space
uncovered references to the Tor installation directory and the obfs4 bridging
IP addresses (Figure 55(c), 5(d)). The browsing data found in NTUSER.DAT was
also replicated in unallocated space. This may have occurred as a result of
the machine crashing, however further analysis would need to be conducted to
prove this.

(a) Ebay URL and Private Browsing evidence in .dll and .edb Files

(b) Ebay URL and HTTP Header Information in .reg Files

(c) References to Tor Install Directory (extract)

(d) Obfs4 Bridging IP Addresses (extract)

Figure 5: Artefacts in Unallocated Space - After Delete

###  4.3 Analysis of NTUSER.DAT

As static analysis revealed an abundance of artefacts in the user profile’s
NTUSER.DAT registry hive, but Autopsy’s parsing of registry is limited to
string searches, NTUSER.DAT hives from several VM snapshots were extracted for
further analysis.

Firstly, a string search for ’Tor Browser’ was carried out on the NTUSER.DAT
file from the Tor After Browse snapshot with the free forensic tool Registry
Explorer. Figure 6(a) shows the results of this search. Of particular interest
is the Key Path column which shows the paths of the six different Registry
keys where the string was found.

The browsing data strings previously seen in Autopsy occur in
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CloudStore\Store\Cache\DefaultAccount\$$windows.data.taskflow.shellactivities\Current
(the _shellactivities key_ for short), making it the most interesting. The
value, of REG_BINARY type, contains page titles from the pages visited during
the browsing protocol in Unicode as well as the absolute path to the
application generating the browsing tabs, C:\Users\40187070\Desktop\Tor
Browser\Browser\firefox.exe (Figure 7(a)).

(a) Tor, Standard Mode

(b) TOR, Portable Mode

(c) Firefox, Private Mode

Figure 6: NTUSER.DAT Keys containing Search Strings

To test whether writing of data to the Registry still occurs when the browser
is run from an external device (in portable mode), the same analysis was
performed on the NTUSER.DAT file extracted from a Tor Portable snapshot. While
the string ’Tor Browser’ was found in only two keys (Figure 6(b)), the
shellactivities key again contained significant evidence from the browsing
protocol including page titles and the absolute path to the Firefox executable
within Tor’s installation directory (Figure 7(b)). The latter, E:\Tor
Browser\Browser\firefox.exe, reflects the use of an external device, more
details about which can be found by correlating the drive letter with registry
keys such as USBSTOR.

(a) TOR, Standard Mode

(b) TOR, Portable Mode

(c) Firefox, Private Browsing

Figure 7: Extracts of the shellactivities registry key showing titles of
browsing tabs

In addition to the shellactivities key, the key
HKCU\Software\Microsoft\InternetExplorer\LowRegistry\
Audio\PolicyConfig\PropertyStore\<value> contained the ’Tor Browser’ string in
both the snapshot of Tor being run locally and in portable mode. Closer
examination of the value data showed that it contains part of the path to the
Firefox executable within the TOR browser installation directory (Figure 8).
This key is likely used in the handling of audio by the host computer and the
data would have been written to it when audio was streamed from the Internet
as part of the browsing protocol. While the shellactivities key is a more
comprehensive record of user activity, this audio key could corroborate the
evidence, providing the user has used it to stream audio.

Figure 8: “Tor Browser” String in \Audio\PolicyConfig\PropertyStore - TOR
Portable

As a control, the same experiment was performed using the ’vanilla’ Firefox
browser in private browsing mode on a fresh VM, with a snapshot taken after
executing the pre-determined browsing activities. As this control could not
contain any references to the Tor browser, ’monolake’ was used as the search
string. The only match was in the shellactivities key (Figure 6(c)). Further
inspection of this key in a hex editor revealed the same data leakage seen in
the two Tor snapshots (Figure 7(c)).

###  4.4 Decoding the shellactivities key contents

It is evident that, under Windows 10, browsing data from user sessions is
written to non-volatile storage, specifically the shellactivities registry
key. This data leakage occurs regardless whether Tor, built upon the Extended
Support Release (ESR) of Firefox, is used or Firefox’s default Private
Browsing mode, and also when a portable browser is used. This consistency
suggests that the observed data leakage is a result of how the Operating
System handles applications rather than some anomaly in the operation of Tor
or Firefox. As a result, the anonymity-preserving intentions of both the Tor
Project and Mozilla’s private browsing mode are negated. Conversely, the
shellactivities registry key is likely to contain valuable forensic artefacts
and thus warrants further investigation.

The parent of the shellactivities registry key, Cloudstore, does not exist in
Windows 7, nor in the 1511 update of Windows 10. It first appears in the
Windows 10 with the Anniversary Update, (1607 / 1603 in Win Education), but at
that point it was not populated, having no subkeys or values. In Windows 10
machines running the Creator’s Update 1703, the CloudStore key exists and has
numerous subkeys that are populated with values, including the shellactivities
key.

The data within the shellactivities is of REG_BINARY type, i.e. hexadecimal.
It appears to have an internal structure, which warrants decoding fully as
this would allow plugins for forensic tools such as Regripper, Encase etc to
be created.

In Figure 9, we have colour-coded the start of the contents of this key after
TOR was run in standard mode, uninstalled and the user logged out, in order to
expose the internal structure.

  * 1.

The key header starts with 0x02000000; this is followed by an eight-byte
Windows Filetime Timestamp (highlighted grey, here representing 3 April 2018
14:09:47), then 4 bytes of zero and 8 bytes of values (here
0x43420100CB0A0A14).

  * 2.

The first record entry starts at offset 0x18.

  * 3.

Every record starts with a record header signature of 0xD214 (buff), followed
by one byte (here consistently 0x39, but varies in other examples; this could
be a type, length or offset indicator of some sort).

  * 4.

This is followed by the absolute path to executable (blue-grey, variable
length), then 8 bytes which are consistently 0xC61FD28310D2230B (pink), the
name of the executable (green, variable length), 0xD228 (purple), 1 variable
byte.

  * 5.

After the above, Page title (turquoise) + 0xC632 (orange) + 5 bytes +
0xEAF2E901 + 0xC63C (yellow) + 5 bytes + 0xEAF2E901 + 0xCA500000 (dark gold).

  * 6.

The underlined could be 1 byte + 8 byte timestamp of some sort - but see
conversion attempts above.

  * 7.

This structure looks to be consistent for the actual web pages viewed, but the
final few records vary somewhat.

  * 8.

If this is correct, the individual entries are delimited by 0xD21439 header
and 0xCA500000 footer, and internal components separated by specific values as
well, so size of entry would not need to be stored.

  * 9.

This entry header and footer might then be searchable, e.g in registry
entries, or possibly unallocated space if deleted? (but I don’t know how
registry deals with deletion - possibly internally).

Figure 9: Internal structure of the shellactivities Key contents

##  5 Findings

###  5.1 Proposed Forensic Methodology

The interactions between Tor and Windows described in Section 4 can be used to
design a generalised forensic methodology for identifying the use of TOR and
Web pages visited using TOR or Private Browsing.

  * 1.

Where possible, take a RAM dump. Analyse with Volatility’s psscan, pstree and
timeliner plugins to (a) establish the use of TOR and (b) find the username.
This will also reveal timestamps and can be carried out even after the user
has uninstalled TOR and logged out. Where they exist, pagefile.sys or
hiberfile.sys can be used instead of a RAM dump. The analysis of all three of
these data sources from the same system could result in the recovery of
different, but nonetheless relevant - and corroborative or complimentary -
evidence.

  * 2.

From non-volatile storage, extract the NTUSER.DAT registry hive of the
previously identified user, or, failing this, of all users.

  * 3.

Extract the contents of the shellactivities key. Search for Tor and/or firefox
to find the titles of web pages visited. Similarly, a search for other browser
executables will reveal visited pages even if private browsing was used.

  * 4.

A keyword search for ’obfs4’ in unallocated space can reveal bridging IP
addresses that may have been used by TOR.

###  5.2 Live Forensics

The live forensics methodology in this project showed that by using the
Volatility Framework and associated plugins, four processes which can be
attributed to Tor are created at runtime. These processes are as follows:

  * 1.

firefox.exe: two occurrences of this process were seen.

  * 2.

tor.exe.

  * 3.

obfs4proxy.exe: it is unlikely this would have appeared had the obfs4 proxy
not been used.

Whilst the names of these processes are obvious, the output of several of the
Volatility Framework plugins also revealed the absolute path to the
installation directory where the browser executable resides - providing
further indication of Tor use. In doing so, the disk from which the browser
was run, the username of the user running the browser, and the name of the
browser can be clearly seen. This means that by using the Volatility Framework
on a RAM dump seized whilst the browsing session is still active, a forensic
adversary can positively identify the use of Tor.

After termination of the session, The Tor Browser manages to end tor.exe,
obfs4proxy.exe and one of the firefox.exe processes. This makes it
significantly more difficult for a forensic adversary to confirm that Tor was
running on the machine at the time of seizure and coincides with the behaviour
of Firefox observed by Findlay and Leimich (2014). Despite this, one
firefox.exe process persisted. Although the process can still be seen, the
environment variables, command line arguments, and DLLs which were
attributable to this and the other three processes could no longer be
identified after session termination. This suggests that the persisting
firefox.exe process could not be fully terminated by closing the browser
window and now exists in a traceable but inactive state. Regardless, the
output of Volatility’s pstree plugin still managed to attribute it to Tor by
identifying the absolute path to the install directory. Without this finding,
it would be impossible to determine that this process had originated from Tor
and not a regular installation of Firefox.

This behaviour continued even after the user logged-out, proving the pstree
plugin to be invaluable when attempting to prove the use of Tor on a shared
computer.

In summary, Tor use can be easily detected using live forensics, particularly
when the browsing session is still active. Ensuring that the browsing session
is closed after use helps to conceal the fact that Tor was used. However, an
artefact (firefox.exe) remains detectable in RAM after closure, deletion, and
log-out. It is likely that the traceable artefact is the result of an anomaly
in Firefox’s handling of running processes. This belief is strengthened by the
fact that Tor manages to remove all evidence of the processes directly
attributed to its browser, yet one Firefox process remains. Perhaps this
abnormality was introduced in an update of Firefox’s Extended Support Release,
or it may even be an unforeseen result of the interaction between Tor’s
plugins and the underlying browser. Nonetheless, it shows that reliance on a
third-party browser can introduce problems which undermine user anonymity.

To expand on this, an examination of the files dumped from the contents of RAM
will need to be performed. This will determine whether evidence of the
browsing protocol is also present in RAM, a significantly greater finding than
the identification of running processes. Unfortunately, time-constraints made
this impossible in this project, an evaluation of which follows in a later
section.

###  5.3 Static Forensics

####  5.3.1 Browsing Data Leakage

Perhaps the most interesting and unexpected conclusion of this project is that
Tor does, in fact, write browsing data to disk. Artefacts from the browsing
protocol, including HTTP header information, titles of web pages, and an
instance of a typed URL were found in several Registry files. This means that
use of static forensics by forensic investigators is potentially more
worthwhile than examining the contents of RAM, at least for this version of
the browser. Epifani et al. (2015) and Darcie et al. (2014) both mention the
Registry as an area where Tor evidence could be recovered. In the Registry,
Darcie et al. (2014) identified evidence of Tor presence, but not of browsing
data. However, their searches were limited in only including the keyword
“Tor”. Similarly, Epifani et al. (2015) showed that Tor could be identified in
the NTUSER.DAT Registry hive, within the User Assist Key. Yet, they determined
that only the number and time of execution could be found. Therefore, the
findings of this research which show that the majority of the browsing
protocol can be recovered from NTUSER.DAT are particularly significant. This
suggests that in the versions between 4.0.2 (Darcie et al., 2014) and 7.5.0, a
design change occurred which has led to this data leakage.

The vast majority of the browsing protocol was found in the NTUSER.DAT Windows
Registry file. This makes it possible for a forensic adversary to reconstruct
the activities of the user in relation to the browser. It was unclear what
caused this information to be written to that area of the Registry, however
multiple.reg files which are used to add changes to the Registry were found in
the snapshot taken after the browser had been deleted. These files contained
an Ebay URL visited during the browsing protocol, making it likely that some
aspect of Tor was writing the contents of memory to the Registry. This is
backed-up by the discovery of the ntuser.dat.LOG1 file, this file logs changes
to the NTUSER.DAT Registry file and contained the same browsing data-leakage.

####  5.3.2 Firefox Artefacts

The two SQLite databases used by Firefox to track cookies and history
(cookies.sqlite & places.sqlite) were both recoverable from the file system
after deletion. Cookies.sqlite contained nothing of relevance but its
existence could indicate that Tor was installed on the computer - if it could
be proven that it did not come from a regular Firefox installation.
Places.sqlite contained no information about the browsing protocol, however,
it did contain entries for the Tor Project’s blog and a Learn About Tor page.
This seems counter-productive when designing a browser which is designed to be
anonymous. The developers of the Tor Project may see this as a trade-off
between security and usability, yet the fact that it can still be recovered
after deletion is further proof that the browser is incapable of being
securely deleted. Since the use of Tor is not currently illegal in most
countries, mere identification of the browser may not be considered
significant. However, these artefacts may have forensic value when trying to
prove that Tor was used over another browser or as part of a wider
investigation.

##  6 Key Findings

This section summarizes some of the key findings.

###  6.1 Live Forensics

  * 1.

Four processes can be attributed to Tor whilst the browser is open.

  * 2.

Use of the bundled obfs4 proxy (used to bridge connections to the Tor network)
can be detected.

  * 3.

After the user terminates the browsing session, only one Firefox process
remains.

  * 4.

Using the Volatility Framework, the path to the browser and the run time can
be identified. Even after the user has closed the browsing window, deleted the
browser and logged-out.

  * 5.

As a result of finding the absolute path to the browser, the username,
location of the browser and use of Tor versus Firefox can be confirmed.

###  6.2 Static Forensics

The following are observations for static forensics:

  * 1.

The design aim of preventing Tor from writing to disk (Perry et al., 2018) is
not achieved in this version.

    * (a)

Configuration files, downloaded files, and browser-related data are
recoverable from the file system.

    * (b)

Significant data-leakage from the browsing session occurred: HTTP header
information, titles of web pages and an instance of a URL were found in
registry files, system files, and unallocated space.

    * (c)

The data-leakage contained the German word for ’search’ in reference to a
Google search. This hints at the locale of the Tor server used to exit the
network (exit relay).

  * 2.

The Tor Project’s design aim of enabling secure deletion of the browser
(Sandvik, 2013) is not achieved in this version.

    * (a)

References to: the installation directory, Firefox SQLite files, bridging
IPs/ports, default bookmarks, Tor-related DLLs and Tor product information
were all recovered after the browser was deleted.

    * (b)

In a scenario where the operating system paged memory, an instance of a URL
from the browsing protocol along with references to private browsing was
found.

    * (c)

The file “state” used by Tor to track successful launches of the application
was found after browser deletion. Prior to being deleted, this file contained
the last run time, however, in the recovered file this timestamp was
overwritten. This suggests that Tor manages to overwrite this data.

###  6.3 Forensic Implications of Using Tor

Acknowledging the above results, a forensic investigator could prove:

  * 1.

The use of Tor even after a user has attempted to delete the browser.

  * 2.

The last execution time/date.

  * 3.

Evidence of pages visited during a browsing session.

  * 4.

The language of the country where the server used to exit the Tor network
resides.

  * 5.

That Tor was used over Firefox.

###  6.4 Limitations

Some limitations in the methodology of this paper can be observed after
completion. One such limitation is that the networking aspect of Tor was not
used to aid the forensic examination. For example, Warren (2017) uses
Volatility’s netscan plugin to observe open network sockets, determining the
TCP ports used by Tor in live forensics. This could be used to further prove
the use of Tor. Equally, time constraints in this project meant that the
individual files dumped from memory could not be analysed. This is likely to
have resulted in further evidence of the browsing protocol. Furthermore, to
find the browsing data-leakage seen in this project, a knowledge of unique
strings related to the browsing protocol was required. This would be unlikely
in a real-life static analysis, although if it was known that data from Tor is
leaked to areas of the Registry such as NTUSER.DAT, the same data-leakage
could be found. Finally, the experiments performed suffered from a lack of
control. With more time, the same browsing protocol and analyses could be
performed on a system running Firefox. This would allow conclusions to be
drawn about which artefacts were attributable to Tor and which were
attributable to Firefox. From this, an assessment of whether Firefox’s
implementation of private browsing or Tor’s privacy-enhancing mechanisms led
to the data-leakage observed in the results could be made.

##  7 Conclusions

The limited research in the area of Tor Forensics discovered during the
Literature Review suggested that a need for live forensics had become
increasingly important. This was largely due to recent papers concluding that
Tor did not write browsing data to disk (Warren, 2017). Prior to this, Darcie
et al. (2014) had successfully managed to retrieve files residing in RAM from
their browsing protocol, even after the browser was deleted. This coincided
with the forensic methodology proposed by Dayalamurthy (2013), which also
placed emphasis on live forensics. A new methodology was proposed, which
favoured a live forensic analysis followed by a static analysis of the
virtualised test machine. This resulted in a more thorough forensic process,
allowing the maximum level of knowledge about the browser to be gained in the
limited timeframe available.

Although previous researchers had successfully recovered artefacts of Tor-use,
a comprehensive analysis of both volatile and non-volatile memory had yet to
be completed – with most of the existing research concentrating on a singular
aspect of computer memory. Likewise, the idea of the Tor Browser being used in
a shared-computing scenario had yet to be fully explored. Sandvik (2013) had
contemplated this, yet it seemed inadequately addressed in her results.
Therefore, it became obvious that the design of an experiment which
incorporated the action of a user logging-out should be devised. It was
determined that in such a scenario, a forensic adversary would likely conduct
live forensics on the shared computer (Tsalis et al., 2017) and the purpose of
this experiment would be to determine whether Tor could be identified running
on the system and whether evidence from the browsing protocol could be
extracted from live memory.

Due to the volatile nature of RAM, acquisition of live memory is rarely
possible in the field. This is applicable even in shared-computing
environments, as often the user can power cycle a shared computer without
consequence. Considering that the intended audience for this project was both
users of the browser and forensic investigators wishing to analyse it, the
omission of a subsequent static analysis would constitute an incomplete
methodology. This is especially true as a large number of Tor users will
likely use their personal computer which could be subject to seizure by a
forensic adversary. Therefore, the multi-faceted experimental design was
required. This proved successful in the end as many unexpected results were
born from the static analysis, an aspect which may have been omitted if too
much reliance had been placed on the results of previous research.

The technique of indexing the hard drive and applying keyword searches based
on known Tor artefacts and the browsing protocol was simple yet is something
that the browser should protect against. In particular, Perry et al. (2018)
state explicitly in their design philosophy that browsing data leakage should
not happen. This indicates that the Tor Browser does not adequately protect
the user from a forensic adversary.

## References

  * Aggarwal et al. (2010) Aggarwal, G., Bursztein, E., Jackson, C., Boneh, D., 2010\.  An analysis of private browsing modes in modern browsers, in: Proceedings of the 19th USENIX Security Symposium, USENIX Association. pp. 1–8.  doi:10.1109/ISSA.2013.6641058.
  * Darcie et al. (2014) Darcie, W., Boggs, R.J., Sammons, J., Fenger, T., 2014\.  Online Anonymity: Forensic Analysis of the Tor Browser Bundle.  Technical Report. Marshall University.  URL: http://www.marshall.edu/forensics/files/WinklerDarcie_ResearchPaper_8-6-141.pdf.
  * David and Hargreaves (2012) David, A., Hargreaves, C., 2012\.  An Analysis of Hotmail Artefacts in Firefox, in: 6th Cybercrime Forensics Education and Training Conference, Canterbury, UK. pp. 1–20.  URL: http://dspace.lib.cranfield.ac.uk/handle/1826/8085.
  * Dayalamurthy (2013) Dayalamurthy, D., 2013.  Forensic Memory Dump Analysis and Recovery of the Artefacts of Using Tor Bundle Browser: The Need, in: Australian Digital Forensics Conference, pp. 71–83.  doi:10.4225/75/57b3c7f3fb86e.
  * Dewey (2013) Dewey, C., 2013.  Everything We Know about Ross Ulbricht, the Outdoorsy Libertarian behind Silk Road. The Washington Post.  URL: https://www.washingtonpost.com/news/the-switch/wp/2013/10/03/everything-we-know-about-ross-ulbricht-the-outdoorsy-libertarian-behind-silk-road.
  * Epifani et al. (2015) Epifani, M., Scarito, M., Picasso, F., 2015.  Tor Forensics on Windows OS, in: DFRWS EU, Dublin.  URL: https://www.dfrws.org/sites/default/files/session-files/pres-tor_forensics_on_windows_os.pdf.
  * Findlay and Leimich (2014) Findlay, C., Leimich, P., 2014\.  An Assessment of Data Leakage in Firefox under Different Conditions, in: 7th International Conference on Cybercrime Forensics Education & Training (CFET 2014), Canterbury, UK.  URL: https://www.researchgate.net/publication/330925976_An_Assessment_of_Data_Leakage_in_Firefox_under_Different_Conditions.
  * Greenwald et al. (2013) Greenwald, G., MacAskill, E., Poitras, L., 2013.  Edward Snowden: The Whistleblower behind the NSA Surveillance Revelations. The Guardian.  URL: https://www.theguardian.com/world/2013/jun/09/edward-snowden-nsa-whistleblower-surveillance.
  * Montasari and Peltola (2015) Montasari, R., Peltola, P., 2015\.  Computer Forensic Analysis of Private Browsing Modes, in: Global Security, Safety and Sustainability: Tomorrow’s Challenges of Cyber Security. ICGS3 2015. Communications in Computer and Information Science, vol 534. Springer, pp. 96–109.  doi:10.1007/978-3-319-23276-8_9.
  * Mozilla.org (2019) Mozilla.org, 2019.  Private Browsing: Use Firefox without Saving History.  URL: https://support.mozilla.org/en-US/kb/private-browsing-use-firefox-without-history.
  * Noorulla (2014) Noorulla, E.S., 2014.  Web Browser Private Mode Forensics Analysis.  Msc. Rochester Institute of Technology.  URL: https://scholarworks.rit.edu/theses/8451/.
  * Nordberg (2015) Nordberg, L., 2015.  The State of Academic Research in Tor, IFIP Summer School, Karlstad, Sweden.  URL: https://www.frisc.no/wp-content/uploads/2015/05/tor-finse15.pdf.
  * Perry et al. (2018) Perry, M., Clark, E., Murdoch, S., Koppen, G., 2018\.  Design and Implementation of the Tor Browser.  Technical Report. The Tor Project.  URL: https://www.torproject.org/projects/torbrowser/design/.
  * Roussas (2009) Roussas, G., 2009.  Visualization of Client-Side Web Browsing and Email Activity.  Msc. Naval Postgraduate School, Monterey, CA.  URL: https://core.ac.uk/download/pdf/36698452.pdf.
  * Sandvik (2013) Sandvik, R.A., 2013.  Forensic Analysis of the Tor Browser Bundle on OS X, Linux, and Windows.  Technical Report. The Tor Project.  URL: https://research.torproject.org/techreports/tbb-forensic-analysis-2013-06-28.pdf.
  * Shepherd (2018) Shepherd, E., 2018.  The Places Database. Mozilla Developer Network Web Docs.  URL: https://developer.mozilla.org/en-US/docs/Mozilla/Tech/Places/Database.
  * Torproject.org (2018) Torproject.org, 2018.  Tor: Overview.  URL: https://www.torproject.org/about/overview.html.en.
  * Tsalis et al. (2017) Tsalis, N., Mylonas, A., Nisioti, A., Gritzalis, D., Katos, V., 2017.  Exploring the Protection of Private Browsing in Desktop Browsers.  Computers & Security 67, 181–197.  URL: https://www-sciencedirect-com.ezproxy.napier.ac.uk/science/article/pii/S0167404817300597, doi:10.1016/J.COSE.2017.03.006.
  * Volatility Foundation (2017) Volatility Foundation, 2017.  Command Reference.  URL: https://github.com/volatilityfoundation/volatility/wiki/Command-Reference.
  * Warren (2017) Warren, A., 2017.  Tor Browser Artifacts in Windows 10.  Technical Report. SANS.  URL: https://www.sans.org/reading-room/whitepapers/forensics/tor-browser-artifacts-windows-10-37642.


# Active Mass Distribution Estimation from Tactile Feedback

Jiacheng Yuan1, Changhyun Choi1, Ellad B. Tadmor2 and Volkan Isler3 1 are with
Department of Electrical and Computer Engineering, University of Minnesota,
Minneapolis, MN, 55455, USA yuanx320<EMAIL_ADDRESS>is with the Department of
Aerospace Engineering and Mechanics, University of Minnesota, Minneapolis, MN,
55455, USA<EMAIL_ADDRESS>is with the Department of Computer Science and
Engineering, University of Minnesota, Minneapolis, MN, 55455, USA
<EMAIL_ADDRESS>
##  I Introduction

In order to manipulate an object or to use it as a tool, it is important to
estimate its mechanical properties such as its mass distribution. Take the
hammer in Fig. 1 as an example: the center of mass of the hammer, which is
critical for grasping it stably, depends on its shape and the specific
material it is made of. The mass distribution also affects the friction
interactions between the hammer and the tabletop, which acts on the dynamic
behavior of the hammer during non-prehensile manipulation.

Previous studies have shown that robots can estimate such mechanical
properties of objects through feedback from various sensor modalities
including vision [1], audio [2], or touch [3]. Object properties such as shape
and pose can be measured without the need for physical interaction with the
object [4, 5, 6]. However, mechanical properties usually require forceful
interactions and tactile or force feedback measurements. The object mass
distribution is no exception. In this work, we focus on how to actively
estimate the mass distribution of a rigid object through tactile feedback
during object manipulation.

There are a few unique challenges involved in this task. For one, the friction
interaction is determined by contact parameters such as contact pressure and
friction coefficient. These parameters are usually unknown which complicates
the problem. Second, it is not clear whether there are actions that excite the
unknown parameters. If they exist, how can we choose robot actions that are
more informative? Lastly, modeling the dynamics of both the robot end-effector
and the object means dealing with rigid object contact modeling, which
requires complex approximation methods such as Linear Complementarity Problem
[LCP]. These approximation methods might introduce artifacts to estimation
results as well.

Figure 1: The robot manipulator actively interacts with a rigid object
(hammer) on the tabletop by swinging while grasping it. The 6-axis
force/torque sensor (marked by a blue ellipse) on the wrist provides tactile
feedback from the interaction.

To overcome these challenges, we choose to actively apply mechanical
perturbations to an object by first grasping it. Thus we restrict the dynamics
modeling to only the object itself and its sliding friction interaction with
the tabletop. We then adopt a particle-based representation and a
differentiable discrete-time dynamics model to describe the physical process
where the tactile feedback acts as the control input. Through the analytical
formulation, we show how to sequentially estimate the _hidden state_
parameters which can then be used to select robot actions to efficiently
estimate the object mass distribution.

Our main contributions are summarized as follows:

  * •

We show how to conduct active object mass distribution estimation from tactile
feedback on a real robot setup.

  * •

We design a strategy to select actions to excite various parameters based on
the analytical formulation.

  * •

We evaluate our approach against multiple baselines and show that ours
outperforms them by a large margin.

##  II Related Work

We now present an overview of related work. System identification [7] can
benefit robotic manipulation by providing information on the object dynamics
model. Researchers have studied various tasks that require estimating object
properties and physical states during manipulation [8, 9]. In [10], the
authors proposed to estimate an object’s center of mass by tracking a history
of push interactions with an LSTM module and training an auxiliary objective
function. On the other hand, object property parameters can also be
incorporated into the dynamics model and be estimated accordingly. Zhang et
al. [11] showed that by adopting a probabilistic model, tactile feedback can
be used to overcome occlusion to perform object pose estimation. Gaussian
Process is one of the widely used approaches to model unknown object dynamics
[12, 13, 14]. Additionally, Model-based Reinforcement Learning can also be
applied to learn the dynamics from observing the state action pairs [15, 16,
17]. Works have also been proposed to estimate the object by either IMU [18]
or pure force sensors [19].

However, choosing the estimation method is only part of the problem. Action
selection also affects the efficiency of the algorithms. A good action will
reveal as much information as possible, while poorly selected actions may not
excite the target parameters at all. In [20], it is shown that the robot can
efficiently infer the composite structure of its environment by quantifying
the expected informativeness of its own actions. In [21], authors show that an
optimal action selection policy could obtain more information about the
environment with uncertainties and bring the system to the manipulation goal
more efficiently. In our work, we analyze the dynamics model and present an
efficient strategy to actively select more informative actions.

In robotic manipulation on the tabletop, the interactions between the robot
and objects or between objects and the table require modeling for the contact
behavior of rigid bodies. Bauza et al. [22] shown that it is necessary to
model the sliding or sticky contact for a planar pushing controller. Adding
these components could further complicate the model. Approximation algorithms
such as LCP have been proposed to handle the contact modeling between rigid
bodies [23] and later implemented in auto-differentiation frameworks as a
differentiable physics engine in [24]. The differentiable LCP model was used
to compute the gradient of the distance between the prediction and the actual
observations, and the gradient is then applied to search for values of the
model parameters to reduce the reality gap. The same approach was later
adopted by Song et al. [25] to estimate object mass distribution with random
pushing actions. In our work, we show that LCP as an approximation can only
close the reality gap to a certain extent, and action selection rules based on
the analytical formulation are more efficient for parameter estimation.

Besides differentiable LCP, many research other works also focus on
differentiation through the dynamics model. Degrave et al. [26] proposed a
differentiable physics engine based on impulse-based velocity stepping methods
[27] to differentiate control parameters in robotics applications. In [28],
the authors combined analytical dynamics models and neural networks and used
end-to-end training to predict the effects of robot actions. New neural
network structures were also proposed to directly implement fluid dynamics
inside a deep network in [29] and [30]. Differentiable physics was also
applied to solve the problem of sequential manipulation and tool-use planning
in domains that include physical interactions such as hitting and throwing
[31]. In our work, we also implemented the dynamics model as a differentiable
physic engine using Taichi Lang [32].

##  III Background

In this section, we introduce our assumptions for the objects and their
motion, followed by the discrete time dynamics model we adopt based on these
assumptions. The object is placed on a horizontal planar surface such as
tabletop. Thus its motion can be defined by $2D$ rigid body transformation
$SE(2)$ within the horizontal plane while maintaining frictional contact with
the underlying surface. We also assume the object motion is caused by external
perturbation from a robot manipulator. Specifically, the manipulator grasps
the object and apply force or torque on the object.

TABLE I: Summary of Notations

Notation | Description
---|---
$\mathbf{p}_{t}$ $\mathbf{v}_{t}$ $\mathbf{a}_{t}$ |  Object pose, velocity and acceleration in $2D$ at time $t$, $\mathbf{p}=[p_{x}\ p_{y}\ p_{w}]$ where $x,y$ denote translation and $w$ denotes rotation. Same for velocity and acceleration
$M\ I_{cm}\ I_{i}$ |  Inertial mass and planar moment of inertia with respect to the center of mass or particle $i$
$\mathbf{c}$ |  Position of the center of mass $\mathbf{c}=[c_{x}\ c_{y}]$
$\mathbf{p}_{i}\ \mathbf{v}_{i}\ \hat{\mathbf{v}}_{i}$ |  Position, velocity and unit velocity of particle $i$ in $2D$, $\mathbf{p}_{i}=[p_{i,x}\ \ p_{i,y}]$ and $\mathbf{v}_{i}=[v_{i,x}\ \ v_{i,y}]$
$n_{p}$ |  Number of particles in the model
$\mathbf{m}\ \ \bm{\mu}$ |  Vector of particle mass $m_{i}$ and friction coefficient $\mu_{i}$ in each group, with $dim(\mathbf{m})=n_{m}$, $dim(\bm{\mu})=n_{\mu}$
$\mathbf{u}_{i,t}$ |  $\mathbf{u}_{i,t}=\begin{bmatrix}u_{i,x}\ u_{i,y}\ u_{i,w}\end{bmatrix}_{t}$ the external perturbation applied on particle $i$

_Notes:_ bold uppercase letters for matrix, bold lowercase for column vector,
normal ones are scalar

###  III-A Object Parameters

There are two crucial aspects to consider when modeling the object properties:
the mass distribution and the bottom contact surfaces. In Table I, we list the
parameters we use to model the object properties and dynamics. In Fig. 2, we
use a hammer as an example to illustrate the modeling process. First, a
particle model is obtained by grid partition based on the 3D object model.
Then the particles can be grouped into different regions in the mass group and
contact surface group. In each region, the particles have the same parameter.
For the simplicity of our model, we only consider sliding friction. We use
$\mu mg$, where $\mu$ is friction coefficient, $m$ is the particle mass, and
$g$ is gravity, as the magnitude for the sliding friction force on each
particle. In Fig. 2(d) only the colored regions (blue and green) are in
contact. For particles not in contact with the bottom surface, $\mu$ will be
$0$.

In addition, we introduce the group mapping matrix $\mathbf{G}_{m}$ and
$\mathbf{G}_{\mu}$ that maps the low dimension $\mathbf{m}$ and $\bm{\mu}$ to
each particle. Specifically,

$\mathbf{G}_{m}=\begin{bmatrix}\mathbf{e}_{1}&\ldots&\mathbf{e}_{n_{p}}\end{bmatrix}\
\text{and}\
\mathbf{G}_{\mu}=\begin{bmatrix}\mathbf{g}_{1}&\ldots&\mathbf{g}_{n_{p}}\end{bmatrix}$
(1)

Here, $\mathbf{e}_{i}\in\mathbb{R}^{n_{m}}$ is a column vector of zeros except
for the entry that corresponds to the index of the mass group. In the example
of Fig. 2, $\mathbf{e}_{i}$ can be either $[1,0]^{T}$ or $[0,1]^{T}$.
Similarly, $\mathbf{g}_{i}\in\mathbb{R}^{n_{\mu}}$ is computed the same way
for particles with contact. But for non-contacting particles (white ones in
Fig. 2d), the corresponding column is all zeros.

Figure 2: An example of how we model the object properties. Based on the image
(a) of the hammer, a particle model (b) is used to capture the object shape.
We may also make assumptions about the mass grouping (c) where particles in
the same group have the same mass (marked as red and orange). For the bottom
contact (d), particles may have no contact (white) or in contact with
different sliding friction parameter (green and blue).

###  III-B Object Dynamics

Since the object is manipulated while being grasped, we consider one object in
the system dynamics. This helps significantly simplify our model as it is no
longer necessary to model the contact dynamics between multiple objects.
According to Newton’s second law, the translational acceleration of the object
is determined by the force acting on it and its inertial mass. While the
rotational acceleration is determined by the torque and its moment of inertia.
The concept of inertial mass $M$ is similar to the moment of inertia from
Newton’s Law but the moment of inertia $I$ of a body also depends on the shape
of the body, and the location and orientation of the axis of rotation relative
to the body, making it a tensor. For rotation about a fixed axis, only a
single scalar number from the moment of inertia tensor is required. In our 2D
planar setting, the moment of inertia only depends on the point of the
rotation center.

As a result, we propose to use the following discrete update functions to
describe the dynamics of the object from time $t$ to $t+1$:

$\displaystyle\mathbf{p}_{t+1}$ $\displaystyle=\mathbf{p}_{t}+\mathbf{v}_{t}\
dt$ (2) $\displaystyle\mathbf{v}_{t+1}$
$\displaystyle=\mathbf{v}_{t}+\mathbf{a}_{t}\ dt$ (3)

The acceleration $\mathbf{a}$ at each time step is a function of the object
mass $M$, the moment of inertia $I_{cm}$, the center of mass $\mathbf{c}$, the
combined effective force/torque from the friction force $\mathbf{f}_{i,f}$ and
the external perturbation $\mathbf{f}_{i,u}$ on each particle $i$. The
effective torque is a combination of the torque input and the cross product of
the force input with the lever arm vector. In our work, we use
$\mathbf{u}_{i}$ for the external perturbation applied on particle $i$ and
assume the external perturbation is only applied to a single particle.
Therefore on the other particles, both $\mathbf{u}_{i}$ and $\mathbf{f}_{i,u}$
become zero.

Let $\otimes$ denote vector cross product. We may write $M$, $I_{cm}$,
$\mathbf{c}$, $\mathbf{f}_{i,f}$ and $\mathbf{f}_{i,u}$ as follows:

$\displaystyle M$ $\displaystyle=\sum\mathbf{G}^{T}\mathbf{m}$ (4)
$\displaystyle\mathbf{c}$
$\displaystyle=\sum_{i}(\mathbf{e}_{i}^{T}\mathbf{m})\mathbf{p}_{i}/M$ (5)
$\displaystyle I_{cm}$ $\displaystyle=\sum_{i}(\mathbf{e}_{i}^{T}\mathbf{m})\
\lVert\mathbf{p}_{i}-\mathbf{c}\lVert^{2}$ (6) $\displaystyle\mathbf{f}_{i,u}$
$\displaystyle=\begin{bmatrix}u_{i,x}\\\ u_{i,y}\\\
u_{i,w}+\begin{bmatrix}u_{i,x}\\\
u_{i,y}\end{bmatrix}\otimes(\mathbf{p}_{i}-\mathbf{c})\end{bmatrix}$ (7)
$\displaystyle\mathbf{f}_{i,f}$
$\displaystyle=-(\mathbf{g}_{i}^{T}\bm{\mu})(\mathbf{e}_{i}^{T}\mathbf{m})g\begin{bmatrix}\hat{v}_{i,x}\\\
\hat{v}_{i,y}\\\
\hat{\mathbf{v}}_{i}\otimes\left(\mathbf{p}_{i}-\mathbf{c}\right)\end{bmatrix}$
(8)

Then the acceleration vector $\mathbf{a}$ can be written as:

$\mathbf{a}=\mathbf{M}\sum_{i}(\mathbf{f}_{i,u}+\mathbf{f}_{i,f})\
\text{where, }\mathbf{M}=\begin{bmatrix}M&&\\\ &M&\\\
&&I_{cm}\end{bmatrix}^{-1}$ (9)

Figure 3: An example of the robot actions: ”grasp-and-slide” and ”grasp-and-
rotate”. The pink rectangles illustrate the two fingers of the robot gripper.
The black arrows and blue arrows correspond to object velocity and particle
velocity. (a) In translational motion, all particles have the same velocity as
the object. (b,c) In rotational motion, particle velocities are dependent on
the rotation axis.

###  III-C Robot Actions

The $SE(2)$ motion of the object consists of translation and rotation and we
classify the robot actions accordingly. In Fig. 3, we use an L-shaped object
to illustrate the idea. The object with 10 particles is grasped by the
parallel jaw gripper (pink rectangles for two fingers) at different locations.
The black arrows show the object velocity, where Fig. 3a moves towards upper
right, and Fig. 3b and Fig. 3c rotate about the particle centers of rotation.
It is worth noticing that, in Fig. 3a, all particle velocities (blue arrows)
are the same as the object velocity. But in Fig. 3b, and Fig. 3c, the particle
velocity depends on the rotation center and the velocity of the particle at
the rotation center is zero. Thus, we name the robot actions as “grasp-and-
slide” or “grasp-and-rotate” for pure translational or pure rotational
motions.

##  IV Problem Formulation

We aim to estimate the 2D mass distribution of an object by applying
mechanical perturbation and observing the motion of the object. Given an
object on a 2D plane with its grid partition and mass/friction grouping, a
robot manipulator with a gripper, and a force/torque sensor that can measure
the mechanical perturbation applied to the object, the goal is to estimate the
object mass distribution $m_{i}$ by measuring the tactile feedback and the
object trajectory while the robot grasps and moves the object around.

Let $\Phi$ be the object forward dynamics function and $\mathbf{u}_{i,t}$ be
the external perturbation on particle $i$, we may write $\Phi$ as:

$\mathbf{v}_{t+1}=\Phi\left(\mathbf{p}_{t},\mathbf{v}_{t},\mathbf{m},\bm{\mu},\mathbf{u}_{i,t}\right)$
(10)

we want to find the optimal $\mathbf{m}$ and $\bm{\mu}$ that minimize the
error of the object trajectory predicted by the dynamics model and the one
observed as ground truth. So our problem can be formulated as computing:

$\left(\mathbf{m}^{*},\bm{\mu}^{*}\right)=\operatorname*{argmin}_{\mathbf{m},\bm{\mu}}\sum_{t=0}^{T}\lVert\Phi\left(\mathbf{p}_{t},\mathbf{v}_{t},\mathbf{m},\bm{\mu},\mathbf{u}_{i,t}\right)-\mathbf{v}_{t+1}\lVert^{2}$
(11)

##  V Our Approach

Our approach to solving the mass distribution estimation problem is built on
top of the analytical formulation and the dependencies of the dynamics model
parameters on the external perturbations as well as the object trajectory. In
this section, we first introduce a new set of derived parameters called
“Hidden States”. Then we talk about the algorithms we designed to estimate the
Hidden States parameters.

###  V-A Hidden States

The discrete-time update equations of system dynamics show that the model has
high non-linearity with respect to the model parameters $m_{i}$. It is
challenging to apply traditional nonlinear system identification methods such
as the Volterra series methods because the data requirement scales
exponentially with respect to the order of polynomials. Additionally, the
friction parameter $\mu_{i}$ is also unknown and $m_{i}$ is multiplied by
$\mu_{i}$ in Eq. 8, further complicating the problem.

However, by introducing the Hidden States $\mathbf{H}$ as follows, we will
show how to reduce the model complexity to linear or quadratic models and
observe the Hidden State parameters sequentially:

$\mathbf{H}=\begin{bmatrix}M&I_{cm}&\mathbf{c}&\mathbf{s}\end{bmatrix}$ (12)

where, $\mathbf{s}$ is a vector for the set of $\mu mg$ for all particles with
contact, and we may adopt the group mapping matrix
$\mathbf{G}_{s}=\begin{bmatrix}\mathbf{h}_{1}\ ...\
\mathbf{h}_{n_{p}}\end{bmatrix}$ for $\mathbf{s}$, similar to $\mathbf{m}$ and
$\bm{\mu}$.

The Hidden States $\mathbf{H}$ consist two types of parameters:

  * •

object-level parameters including the inertial mass $M$, moment of inertia
$I_{cm}$ and center of mass $\mathbf{c}$

  * •

particle-level parameters $s_{i}$ due to the friction interaction between the
bottom surface with the table

Our approach is to first estimate the object level parameters in the Hidden
States followed by the particle level parameters. Then $m_{i}$ can be derived
arithmetically based on Eq. 4-6.

Initialize with uniform mass $\mathbf{m}$ and zero friction $\bm{\mu}$

Compute robot action set $S$

Sample $k$ grasp-and-rotate actions from $S$ to $S_{1}$ with $k\geq 3$

for  _each action $\in S_{1}$_ do

Grasp the object at $\mathbf{p}_{i}$

Apply various torque $u_{i,w}$ and record $a_{w}$

Compute $I_{i}$ by linear fitting using Eq.13

end for

Solve Eq. 15 for $\mathbf{c}$ and $I_{cm}$

Sample $n_{s}/3$ grasp-and-rotate actions from $S$ to $S_{2}$

Compute $rank(Q)$

while  _$rank(Q) <n_{s}$_ do

Add one more sample to $S_{2}$

Compute $rank(Q)$

end while

while  _not timeout nor convergence of $\mathbf{s}$_ do

for  _each action $\in S_{2}$_ do

Compute forward dynamics

end for

Compute $\mathcal{L}(\mathbf{s})$ and
$\frac{\partial\mathcal{L}}{\partial\mathbf{s}}$

$s=s-\alpha_{rate}*\frac{\partial\mathcal{L}}{\partial\mathbf{s}}$

end while

Solve for $\mathbf{m}$ from Hidden States $\mathbf{H}$

Algorithm 1 Multi-stage Hidden States Estimation

###  V-B Parameter Estimation

In the object-level parameters, one significant difference between the
inertial mass and the moment of inertia is that gravitational mass is
equivalent to inertial mass. So the mass can be determined by weighing rather
than from Newton’s second law. The moment of inertia, however, can only be
measured by observing the force/torque and angular acceleration. So the
inertial mass $M$ can be directly observed by weighing the object without
having the robot move it. For the rest of the object level parameters, i.e.
$[c_{x}\ c_{y}]$ and $I_{cm}$, they can be jointly estimated by first
observing $I_{j}$.

During a “grasp-and-rotate” action around the particle $j$. The angular
acceleration of the object $a_{w}$ can be described by replacing $\mathbf{c}$
with $\mathbf{p}_{j}$ in Eq. 6 and Eq. 8. Therefore, $a_{w}$ in Eq. 9 can be
rewritten as:

$a_{w}=I_{j}^{-1}\left(u_{j,w}-\sum_{i}(\mathbf{h}_{i}^{T}\mathbf{s})\hat{\mathbf{v}}_{i}\otimes\left(\mathbf{p}_{i}-\mathbf{p}_{j}\right)\
\right)$ $\text{where, }I_{j}=\sum_{i}(\mathbf{e}_{i}^{T}\mathbf{m})\
\lVert\mathbf{p}_{i}-\mathbf{p}_{j}\lVert^{2}$ (13)

From Eq. 13, we can see that the output $a_{w}$ is linearly dependent on the
input torque $u_{j,w}$. Therefore, $I_{j}$ can be observed by applying
different torque in such grasp-and-rotate motion and observing the angular
acceleration of the object. Furthermore, we know from the parallel axis
theorem that $I_{j}$ is a function of $I_{cm}\ \text{and}\ \mathbf{c}$:

$I_{j}=I_{cm}+M\lVert\mathbf{p}_{j}-\mathbf{c}\lVert^{2}$ (14)

Let $S$ be the set of the index $j$ for which $I_{j}$ is observed, the
estimation of $\mathbf{c}$ and $I_{cm}$ can be formulated as computing:

$\left(\mathbf{c}^{*},I_{cm}^{*}\right)=\operatorname*{argmin}_{\mathbf{c},I_{cm}}\sum_{j\in
S}\lVert I_{cm}+M\lVert\mathbf{p}_{j}-\mathbf{c}\lVert^{2}-I_{j}\lVert^{2}$
(15)

Once we compute the object-level parameters, we can treat them as known
variables in Eq. 9. As a result, both $\mathbf{v}_{t+1}$ and $\mathbf{a}_{t}$
in Eq. 3 are linear with respect to $\mathbf{s}$, and Eq. 3 can be rewritten
in vector product form as:

$\displaystyle\mathbf{v}_{t+1}$ $\displaystyle=\mathbf{v}_{t}+\mathbf{A}_{t}\
\mathbf{G}_{s}^{T}\ \mathbf{s}+\mathbf{B}_{t}\quad\text{where, }$
$\displaystyle\mathbf{A}$ $\displaystyle=dt\
\mathbf{M}\begin{bmatrix}\hat{v}_{0,x}&...&\hat{v}_{i,x}&...\\\
\hat{v}_{0,y}&...&\hat{v}_{i,y}&...\\\
\hat{\mathbf{v}}_{0}\otimes\left(\mathbf{p}_{0}-\mathbf{c}\right)&...&\hat{\mathbf{v}}_{i}\otimes\left(\mathbf{p}_{i}-\mathbf{c}\right)&...\end{bmatrix}$
$\displaystyle\mathbf{B}$ $\displaystyle=dt\
\mathbf{M}\sum_{i}\mathbf{f}_{i,u}$ (16)

Therefore, we could adopt the least square method to estimate $s$ by
populating observations from different trajectories of the object. If we use
superscript $k$ to denote the parameters in the $k^{th}$ trajectory, the
problem in Eq. 11 reduces to:

$\displaystyle\mathbf{s}^{*}$
$\displaystyle=\operatorname*{argmin}_{\mathbf{s}}\mathcal{L}(\mathbf{s})\quad\text{where,
}$ $\displaystyle\mathcal{L}(\mathbf{s})$
$\displaystyle=\sum_{t=0}^{T}\lVert\begin{bmatrix}\mathbf{A}^{0}\\\ \vdots\\\
\mathbf{A}^{k}\\\
\vdots\end{bmatrix}_{t}\mathbf{G}_{s}^{T}\mathbf{s}+\begin{bmatrix}\mathbf{B}^{0}\\\
\vdots\\\ \mathbf{B}^{k}\\\
\vdots\end{bmatrix}_{t}+\begin{bmatrix}\mathbf{v}^{0}\\\ \vdots\\\
\mathbf{v}^{k}\\\ \vdots\end{bmatrix}_{t}-\begin{bmatrix}\mathbf{v}^{0}\\\
\vdots\\\ \mathbf{v}^{k}\\\ \vdots\end{bmatrix}_{t+1}\lVert^{2}$ (17)

To be able to solve for Eq. V-B, the matrix
$\mathbf{Q}=[\mathbf{A}^{0}...\mathbf{A}^{k}...]^{T}\mathbf{G}_{s}^{T}\in\mathbb{R}^{3k\times
n_{s}}$ must be full column rank, i.e. $rank(\mathbf{Q})=n_{s}$. Based on
Section. III-C, we only need to analyze two special cases of the trajectories,
which corresponds to two special robot actions: constant speed “grasp-and-
slide” and constant speed “grasp-and-rotate”.

For “grasp-and-slide” actions, all particles have the same velocity as the
rigid object: $[v_{i,x}\ v_{i,y}]=[v_{x}\ v_{y}]^{T}$ This means the first two
rows of $\mathbf{A}$ become:
$M^{-1}\hat{\mathbf{v}}\begin{bmatrix}1&...&1&...\end{bmatrix}$. As a result,
the first two rows of all $\mathbf{A}^{k}$ in $\mathbf{Q}$ are linearly
dependent on each other, giving $k+1$ as the upper bound to
$rank(\mathbf{Q})$.

However, for “grasp-and-rotate” actions with rotation center around particle
$j$, the first two rows of $\mathbf{A}$ become:
$M^{-1}\begin{bmatrix}\hat{\mathbf{v}}_{0}&...&\hat{\mathbf{v}}_{j-1}&\mathbf{0}&\hat{\mathbf{v}}_{j+1}&...\end{bmatrix}$,
where only the $j^{th}$ entry is zero. This means the vectors from two
different “grasp-and-rotate” actions are guaranteed to be linearly
independent, making $\mathbf{Q}$ less degenerated compared to the $\mathbf{Q}$
constructed from “grasp-and-slide” actions. Therefore, we adopt a random
sampling method with higher priorities for “grasp-and-rotate” actions to
observe the particle-level parameters.

Algorithm. 1 summarize the multi-stage estimation process to actively estimate
the parameters in the Hidden States.

###  V-C Differentiable Physics

In order to efficiently solve the problem in Eq. V-B, we propose to use a
differentiable physics engine for the forward dynamics with the Hidden States.
The jacobian $\frac{\partial\mathcal{L}}{\partial\mathbf{s}}$ in Algorithm. 1
can be obtained through auto-differentiation, and then a gradient descent on
the loss function in Eq. V-B can be performed. In our work, instead of the
popular auto-differentiation frameworks like PyTorch or TensorFlow, we adopted
Taichi Lang [32] to implement a computationally efficient differentiable
physics model.

##  VI Evaluation

In this section, we first introduce our real robot setup and block object
design followed by our evaluation metrics for the algorithms. Then we continue
our section with the baseline methods used for comparisons and evaluation.

Figure 4: The objects we used for our experiments. (a-f) shows different
configurations of the block object. (g) a 3 lb hammer (f) a wrench

###  VI-A Real Robot Implementation Details

For the real robot setup, we use the tabletop manipulator setting shown in
Fig. 1. We mount the Robotiq FT-300S 6-axis force/torque sensor on the wrist
of a Kinova Gen 3 7-DOF manipulator. The force/torque feedback is measured at
100 Hz while the manipulator performs servoing to follow the trajectories
defined by the selected actions. For “grasp-and-slide” actions, the robot end-
effector moves at a constant speed of $5~{}cm/s$ along a straight line for $4$
seconds. For “grasp-and-rotate” actions, the robot rotates its end-effector at
a constant angular speed of $10~{}deg/s$ for $18$ seconds. Instead of taking
the measured raw tactile data directly, we take advantage of the physics
behind the constant speed robot actions we designed. In constant-speed “grasp-
and-rotate” actions, the object has no change in the rotational kinetic energy
and the center of mass is circulating with respect to the rotation axis.
Therefore the measured tactile feedback should have constant torque and
sinusoidal force. In constant-speed “grasp-and-rotate” actions, the object has
no change in both the translation and rotation kinetic energy, so the measured
tactile feedback should have constant force and torque. We utilize such
properties to pre-process and filter our data and ground truth trajectories,
which allowed us to reduce the noise from the data collection process.

As for the objects for the experiments, we designed and fabricated a re-
configurable block object set besides regular objects such as a hammer and a
wrench. The block object is made from blocks of aluminum or steel and can
change shape or mass distribution. This design allows us to capture the ground
truth mass distribution for better evaluation. Fig. 4 shows the objects we
used for the experiments. We name the block objects in Fig. 4 (a-f) as “I1”,
“I2”, “L1”, “L2”, “F1”, and “F2” accordingly.

TABLE II: Normalized Absolute Difference of Mass Distribution ($\%$) of each
block object (column) and each approach (row)

I1 I2 L1 L2 F1 F2 Our Approach 7.4 9.2 10.1 7.3 8.5 14.2 Random Search 56.5
32.6 31.1 46.4 24.2 42.9 Weighted Sampling Search 30.1 25.1 44.4 32.9 27.9
25.0 Explicit State 18.2 22.7 35.0 29.4 22.8 34.9 Diff-LCP 44.2 51.7 32.2 50.4
36.3 68.5

###  VI-B Evaluation Metrics

Although we could capture the ground truth mass distribution for the block
objects, it is difficult to do so for the hammer and the wrench. Therefore we
propose two evaluation metrics to evaluate the estimated mass distribution.

For the estimation results from the block objects, let $\mathbf{m}$ and
$\mathbf{m}_{g}$ be the estimated and ground truth mass distributions. The
Normalized Absolute Difference (NAD) $\eta_{1}$ is computed by summing the
absolute error for each particle and then normalizing by the total mass:

$\eta_{1}=\frac{\sum\mathbf{G}_{m}^{T}\
\lvert\mathbf{m}-\mathbf{m}_{g}\rvert}{\sum\mathbf{G}_{m}^{T}\mathbf{m}_{g}}$
(18)

For further evaluation, we also run the forward simulation with the estimated
results to compare against the ground truth trajectories observed on the real
robot setup. Instead of computing the difference for all time steps along the
trajectories, we only compare the mean particle distance at the final time
step. Let $\mathbf{p}_{i}$ and $\mathbf{p}_{i,g}$ be the simulated and ground
truth particle position at the last time step, we define the Mean Particle
Distance (MPD) $\eta_{2}$ as:

$\eta_{2}=\frac{\sum_{i}\lVert\mathbf{p}_{i}-\mathbf{p}_{i,g}\rVert}{n_{p}}$
(19)

###  VI-C Baseline Methods

Next, we go through the methods we used as baselines to compare against our
approach.

Random Search [33]randomly samples $\mathbf{m}$ and $\bm{\mu}$ repeatedly and
runs the forward simulation to search for the sample that returns the minimal
loss $\mathcal{L}$.

Weighted Sampling Search[34] is initialized with grid sampling method for
$\mathbf{m}$ and $\bm{\mu}$. And then iterative by re-sampling from a Gaussian
distribution around the best sample in the last iteration. The sampling
deviation is decreased over time to focus on the most promising sampling
region.

Explicit State uses the same analytical formulation for the dynamics as our
approach. But instead of first estimating the Hidden States, this method
directly computes the gradient of $\mathcal{L}$ with respect to $\mathbf{m}$
and $\bm{\mu}$ and run gradient descent.

Diff-LCP [25] adopts the same particle-based object model and uses a
differentiable physics engine based on LCP as the forward simulation for the
dynamics. Diff-LCP requires a smaller update time step so we do linear
interpolation on the tactile feedback data before feeding them into the
forward simulation. Diff-LCP also computes the gradient $\mathcal{L}$ with
respect to $\mathbf{m}$ and $\bm{\mu}$ and runs gradient descent to optimize
the estimation.

Figure 5: Heatmap of the Absolute Difference of mass (in kilograms) for each
block in the block objects. Each column corresponds to different methods and
each row corresponds to different block object configurations.

###  VI-D Results

TABLE III: Mean Particle Distance from Forward Simulation ($cm$) of each block
object (column) and each approach (row)

I1 I2 L1 L2 F1 F2 hammer wrench Our Approach 1.2 1.5 1.1 0.8 1.6 2.2 1.9 1.3
Random Search 9.1 13.8 7.6 8.2 9.3 9.6 3.4 6.1 Weighted Sampling Search 5.4
9.8 5.8 5.9 4.6 7.5 4.0 4.2 Explicit State 1.7 2.6 1.8 1.5 2.9 3.2 3.3 3.6
Diff-LCP 6.2 5.3 6.8 4.8 3.5 7.3 4.6 3.7

Since we do not know the ground truth mass distribution for the hammer and
wrench, we only compare NAD for the block object set. For MPD, we could
compare across all the objects. We run our approach with 500 gradient steps.
For fair comparisons, we also run each of the baseline methods for 500
iterations. Table II shows the NAD of the mass distribution after 500
iterations for each object and each method. All gradient-based methods are
tested with various optimizers such as Adam [35] and RMSProp [36], and the
best result is picked for evaluation. To evaluate the forward predictions on
the estimated mass distribution, we take the “grasp-and-rotate” actions that
are not sampled during the data collection stage. Table III shows the MPD at
the final time step for each object and each method.

Our approach outperforms all the baselines in both evaluation metrics. Both
Random Search and Weighted Sampling Search perform poorly. Specifically,
Random Search suffers from the high dimensional sampling space from the block
objects. Weighted Sampling Search shows high sensitivity to the sampling
deviation decay rate. The reason for the sensitivity could be aligned with
what prevents the Explicit State method perform well. In the Explicit State
method, we always observe significantly slow updates after the initial few
iterations. It is likely that the estimation is trapped by “local minimum” or
“valley” in the joint space of mass and friction. As a comparison, our
approach avoids the estimation within the joint space by first estimating the
Hidden States sequentially and computing the mass distribution analytically.
Diff-LCP adopts a different dynamics model from all the methods above. The
results show that although it could reduce the reality gap (MPD) to a certain
extent, the forward model is physically inaccurate, making the estimation
results off by a large margin.

##  VII Conclusion

We presented a method to actively estimate the planar mass distribution of a
rigid object through tactile feedback during object manipulation. We combined
a sequential estimation strategy with a set of robot action selection rules
based on the analytical formulation of a discrete-time dynamics model. Our
evaluation results on objects with various shapes and mass distributions show
that our approach can solve this challenging task with high accuracy. Exciting
future research directions include extensions to more complex objects such as
articulated ones and analysis of action selection rules that will guarantee
the uniqueness of the estimation result.

## References

  * [1] R. Martín-Martín, S. Höfer, and O. Brock, “An integrated approach to visual perception of articulated objects,” in _2016 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2016, pp. 5091–5097.
  * [2] E. Torres-Jara, L. Natale, and P. Fitzpatrick, “Tapping into touch,” 2005.
  * [3] M. J. Kim, M. Choi, Y. B. Kim, F. Liu, H. Moon, J. C. Koo, and H. R. Choi, “Exploration of unknown object by active touch of robot hand,” _International Journal of Control, Automation and Systems_ , vol. 12, pp. 406–414, 2014.
  * [4] X. Li, H. Wang, L. Yi, L. J. Guibas, A. L. Abbott, and S. Song, “Category-level articulated object pose estimation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 3706–3715.
  * [5] K. Mo, L. J. Guibas, M. Mukadam, A. Gupta, and S. Tulsiani, “Where2act: From pixels to actions for articulated 3d objects,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 6813–6823.
  * [6] T. Standley, O. Sener, D. Chen, and S. Savarese, “image2mass: Estimating the mass of an object from its image,” in _Conference on Robot Learning_. PMLR, 2017, pp. 324–333.
  * [7] J. Swevers, C. Ganseman, D. B. Tukel, J. De Schutter, and H. Van Brussel, “Optimal robot excitation and identification,” _IEEE transactions on robotics and automation_ , vol. 13, no. 5, pp. 730–740, 1997.
  * [8] R. Bayrleithner and K. Komoriya, “Static friction coefficient determination by force sensing and its application,” in _Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’94)_ , vol. 3. IEEE, 1994, pp. 1639–1646.
  * [9] M. C. Koval, N. S. Pollard, and S. S. Srinivasa, “Pose estimation for planar contact manipulation with manifold particle filters,” _The International Journal of Robotics Research_ , vol. 34, no. 7, pp. 922–945, 2015\.
  * [10] J. K. Li, W. S. Lee, and D. Hsu, “Push-net: Deep planar pushing for objects with unknown physical properties.” in _Robotics: Science and Systems_ , vol. 14, 2018, pp. 1–9.
  * [11] L. Zhang and J. C. Trinkle, “The application of particle filtering to grasping acquisition with visual occlusion and tactile sensing,” in _2012 IEEE International Conference on Robotics and Automation_. IEEE, 2012, pp. 3805–3812.
  * [12] M. P. Deisenroth, C. E. Rasmussen, and D. Fox, “Learning to control a low-cost manipulator using data-efficient reinforcement learning,” _Robotics: Science and Systems VII_ , vol. 7, pp. 57–64, 2011.
  * [13] S. Bansal, R. Calandra, T. Xiao, S. Levine, and C. J. Tomlin, “Goal-driven dynamics learning via bayesian optimization,” in _2017 IEEE 56th Annual Conference on Decision and Control (CDC)_. IEEE, 2017, pp. 5168–5173.
  * [14] S. Dragiev, M. Toussaint, and M. Gienger, “Gaussian process implicit surfaces for shape estimation and grasping,” in _2011 IEEE International Conference on Robotics and Automation_. IEEE, 2011, pp. 2845–2850.
  * [15] P. Agarwal, S. Kumar, J. Ryde, J. Corso, V. Krovi, N. Ahmed, J. Schoenberg, M. Campbell, M. Bloesch, M. Hutter, _et al._ , “Physics-based grasp planning through clutter,” 2013.
  * [16] J. Scholz, M. Levihn, C. Isbell, and D. Wingate, “A physics-based model prior for object-oriented mdps,” in _International Conference on Machine Learning_. PMLR, 2014, pp. 1089–1097.
  * [17] P. Abbeel, M. Quigley, and A. Y. Ng, “Using inaccurate models in reinforcement learning,” in _Proceedings of the 23rd international conference on Machine learning_ , 2006, pp. 1–8.
  * [18] M. Bednarek, P. Kicki, J. Bednarek, and K. Walas, “Gaining a sense of touch object stiffness estimation using a soft gripper and neural networks,” _Electronics_ , vol. 10, no. 1, p. 96, 2021.
  * [19] F. Coutinho and R. Cortesão, “Online stiffness estimation for robotic tasks with force observers,” _Control Engineering Practice_ , vol. 24, pp. 92–105, 2014.
  * [20] H. Van Hoof, O. Kroemer, H. B. Amor, and J. Peters, “Maximally informative interaction learning for scene exploration,” in _2012 IEEE/RSJ International Conference on Intelligent Robots and Systems_. IEEE, 2012, pp. 5152–5158.
  * [21] J. Yuan, N. Häni, and V. Isler, “Multi-step recurrent q-learning for robotic velcro peeling,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021, pp. 6657–6663.
  * [22] M. Bauza, F. R. Hogan, and A. Rodriguez, “A data-efficient approach to precise and controlled pushing,” in _Conference on Robot Learning_. PMLR, 2018, pp. 336–345.
  * [23] M. B. Cline, “Rigid body simulation with contact and constraints,” Ph.D. dissertation, University of British Columbia, 2002.
  * [24] F. de Avila Belbute-Peres, K. Smith, K. Allen, J. Tenenbaum, and J. Z. Kolter, “End-to-end differentiable physics for learning and control,” _Advances in neural information processing systems_ , vol. 31, 2018.
  * [25] C. Song and A. Boularias, “Identifying mechanical models through differentiable simulations,” _arXiv preprint arXiv:2005.05410_ , 2020.
  * [26] J. Degrave, M. Hermans, J. Dambre, _et al._ , “A differentiable physics engine for deep learning in robotics,” _Frontiers in neurorobotics_ , p. 6, 2019.
  * [27] T. Erez, Y. Tassa, and E. Todorov, “Simulation tools for model-based robotics: Comparison of bullet, havok, mujoco, ode and physx,” in _2015 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2015, pp. 4397–4404.
  * [28] A. Kloss, S. Schaal, and J. Bohg, “Combining learned and analytical models for predicting action effects,” _arXiv preprint arXiv:1710.04102_ , vol. 11, 2017\.
  * [29] C. Schenck and D. Fox, “Spnets: Differentiable fluid dynamics for deep neural networks,” in _Conference on Robot Learning_. PMLR, 2018, pp. 317–335.
  * [30] Y. Li, J. Wu, R. Tedrake, J. B. Tenenbaum, and A. Torralba, “Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids,” _arXiv preprint arXiv:1810.01566_ , 2018.
  * [31] M. A. Toussaint, K. R. Allen, K. A. Smith, and J. B. Tenenbaum, “Differentiable physics and stable modes for tool-use and manipulation planning,” 2018.
  * [32] Y. Hu, L. Anderson, T.-M. Li, Q. Sun, N. Carr, J. Ragan-Kelley, and F. Durand, “Difftaichi: Differentiable programming for physical simulation,” _arXiv preprint arXiv:1910.00935_ , 2019.
  * [33] Z. B. Zabinsky _et al._ , “Random search algorithms,” _Department of Industrial and Systems Engineering, University of Washington, USA_ , 2009.
  * [34] K. Forbes and E. Fiume, “An efficient search algorithm for motion data using weighted pca,” in _Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation_ , 2005, pp. 67–76.
  * [35] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [36] G. Hinton, “Neural networks for machine learning,” https://www.cs.toronto.edu/t̃ijmen/csc321/slides/lecture_s
lides_lec6.pdf, 2014.