# CLARINET: A RISC-V Based Framework for Posit Arithmetic Empiricism

Niraj N. Sharma  0000-0003-3311-3442 Indian Institute of Technology, Bombay4th
Floor, Department of Electrical EngineeringMumbaiIndia<EMAIL_ADDRESS>,
Riya Jain<EMAIL_ADDRESS>Indian Institute of Technology,
BombayMumbaiIndia ,  Madhumita Mohan<EMAIL_ADDRESS>Indian Institute
of Technology, BombayMumbaiIndia ,  Sachin Patkar  Indian Institute of
Technology, Bombay4th Floor, Department of Electrical Engineering MumbaiIndia
,  Rainer Leupers  RWTH Aachen UniversityKopernikusstraße 16AachenNRWGermany ,
Nikhil Rishiyur  1234-5678-9012 Bluespec Inc.FraminghamMAUSA and  Farhad
Merchant  RWTH Aachen UniversityAachenGermany

###### Abstract.

Many engineering and scientific applications require high precision
arithmetic. IEEE 754-2008 compliant (floating-point) arithmetic is the de
facto standard for performing these computations. Recently, _posit arithmetic_
has been proposed as a drop-in replacement for floating-point arithmetic. The
posit™data representation and arithmetic claim several absolute advantages
over the floating-point format and arithmetic, including higher dynamic range,
better accuracy, and superior performance-area trade-offs. However, there does
not exist any accessible, holistic framework that facilitates the validation
of these claims of posit arithmetic, especially when the claims involve long
accumulations (_quire_).

In this paper, we present a consolidated general-purpose processor-based
framework to support posit arithmetic empiricism. The end-users of the
framework have the liberty to seamlessly experiment with their applications
using posit and floating-point arithmetic since the framework is designed for
the two number systems to coexist. Melodica is a posit arithmetic core that
implements parametric fused operations that uniquely involve the quire data
type. Clarinet is a Melodica-enabled processor based on the RISC-V ISA. To the
best of our knowledge, this is the first-ever integration of quire with a
RISC-V core. To show the effectiveness of the Clarinet platform, we perform an
extensive application study and benchmark some of the common linear algebra
and computer vision kernels. We emulate Clarinet on a Xilinx FPGA and present
utilization and timing data. Clarinet and Melodica remain actively under
development and is available in open-source for posit arithmetic empiricism.

posit arithmetic, RISC-V, open-source hardware, custom instructions

††journal: TACO††ccs: Hardware Arithmetic and datapath circuits††ccs: Hardware
Application specific instruction set processors

##  1\. Introduction

High precision arithmetic is necessary for several scientific and engineering
applications (Higham, 2002). These applications require high precision
arithmetic hardware for accurate and efficient execution. Floating-point
arithmetic hardware has been challenging to design due to the intricacies
involved in staying within the limits of the desired area, and power budget
(Leeser et al., 2014)(Volkova et al., 2019). Specifically, reproducing results
across floating-point unit (FPU) implementations has been a challenge due to
the requirements posed by the IEEE 754-2008 (floating-point) standard.
Recently, researchers and computer architects have either compromised on
compliance to the standard or devised alternate formats to overcome the design
challenges (Gustafson and Yonemoto, 2017) (Burgess et al., 2019). Posit is one
such data representation proposed by John L. Gustafson in the year 2017, which
aims to overcome shortcomings of the floating-point format (Gustafson and
Yonemoto, 2017).

The posit arithmetic and data representation claim simpler hardware to
implement, higher dynamic range and numerical accuracy with comparable area
and energy footprints as advantages over the floating-point arithmetic and
format. In general, $n$-bit posit has a higher dynamic range compared to
$n$-bit floating-point, and $n$-bit floating-point arithmetic can be replaced
by $m$-bit posit arithmetic units where $m<n$ (Chaurasiya et al., 2018). The
posit representation is a super-set of the floating-point format and can serve
as a drop-in replacement. It is also shown that the posit format is robust and
reliable compared to its floating-point counterpart for single and double bit-
flips (Alouani et al., 2021).

Due to the advantages of the posit number system, several academic and
industrial research labs have started exploring and studying applications that
can benefit from posits. The _SoftPosit_ library emulates posit arithmetic and
supports early-stage numerical investigation of applications in software
(Leong, 2018). However, no such framework exists to evaluate posits from a
latency, resources and operating frequency perspectives on actual hardware.
There is a need for an easily reconfigurable hardware platform for early-stage
design space exploration of posit arithmetic for various applications. With
its ever-increasing popularity and a conducive open-source ecosystem, we
believe that RISC-V (Waterman and Asanovic, 2019) is an excellent vehicle to
have such a framework supporting posit arithmetic empiricism. We chose the BSV
high-level HDL (Bluespec Inc., 2020a) as the implementation language to enable
rapid design space exploration through an easy reconfiguration of the hardware
platform. The major contributions of this paper are:

  * •

We present _Clarinet_ ; a floating-point and posit arithmetic enabled CPU-
based framework for numeric empiricism. Clarinet is based on the RISC-V ISA
(with custom instructions for posit arithmetic), and is derived from the open-
source _Flute_ core developed by _Bluespec Inc_ (Bluespec Inc., 2020b). The
Clarinet framework also features a customized RISC-V gcc tool-chain to support
the new instructions.

  * •

We present _Melodica_ , a parameterizable posit arithmetic core which supports
fused operations that involve the quire data type, and type-converters between
floating-point, posit and quire data representations.

  * •

Through Clarinet, we also present a new usage model where posits and floating-
point can coexist as independent types cleanly, allowing applications to be
ported more easily to posits when they offer an advantage.

  * •

Finally, we investigate applications in the domain of linear algebra and
computer vision to show the effectiveness of Clarinet as an experimental
platform.

We support fused multiply accumulate (and subtract), and fused divide
accumulate (and subtract) with quire to carry-out experimental studies for our
applications. To the best of our knowledge, this is the _first-ever_ quire
enabled RISC-V CPU. The organization of the paper is as follows. In Section 2,
we discuss posit, quire and float formats, the Flute core, and some of the
recent implementations of posit arithmetic. Clarinet is described in Section
3, and Melodica in Section 4. Application analyses and benchmarks are
presented in Section 5. Experimental setup and results are discussed in
Section 5.4. We conclude our work in Section 6.

##  2\. Background and Related Work

###  2.1. Background

####  2.1.1. Posits

A posit number is defined by two parameters: the width of the posit number, N,
and the maximum width of the exponent field, es. One of the important
advantages of the posit number format is that we can vary es to trade-off
between greater dynamic range (larger es) and greater precision (smaller es).
The posit format has four fields:

Figure 1. IEEE 754-2008 floating-point, posit, and quire register format for
N-bit posits

  * •

Sign (s): The MSB of the number. If the bit is set, the posit value is
negative. In this case all remaining fields are represented in two’s
complement notation.

  * •

Regime Field (r): The regime is used to compute the scale factor, k. In a
posit number this field starts just after the sign bit and is terminated by a
bit opposite to its leading bits. The computation of k is as per the equation
1, where r is the number of leading bits in the regime.

  * •

Exponent Field (exp): The exponent begins after the regime field and the
maximum width of the exponent field is es.

  * •

Fraction Field (f): The remaining number of bits after the exponent make up
the fraction. The fractional field is preceded by an implied hidden bit which
is always 1.

For a number represented in the posit format, its value is as per the equation
2.

(1) $k=\begin{cases}r-1,&\text{if regime has leading ones}\\\ -r,&\text{if
regime has leading zeros}\end{cases}$

(2) $value=(-1)^{s}*(2^{2^{es}})^{k}*2^{exp}*1.f$

Posits do not have a representation for NaNs, or separate representations for
$\pm 0$ and $\pm\infty$. Posits recognize only two special cases – zero and
not-a-real (NaR), and support one rounding mode Round-to-Nearest-Even (RNE).
Posit number system shows better accuracy around 1 than floating-point of the
same size (de Dinechin et al., 2019). Using 8-bit posits as an example.
0000_0000 represents 0, 1000_0000 represents $\pm\infty$, and the rest of the
combinations can be derived from the equation 2. Posit and floating-point
formats are depicted in Fig. 1.

The _quire_ is a fixed-point register that serves the purpose of accumulation
like a Kulisch accumulator (Kulisch, 2002). The quire for a given posit-width
is sized to represent the smallest posit squared, and the largest posit
squared without any overflow. When the quire is used as an accumulator for a
series of steps, it allows computation without intermediate rounding. The size
of an N-bit quire is determined by $\frac{N^{2}}{2}$ where N is the posit
number width (Fig. 1).

####  2.1.2. Flute - A RISC-V CPU

_The Flute_ is an in-order open-source CPU based on the RISC-V ISA,
implemented using the BSV HL-HDL. The Flute pipeline is nominally 5-stages but
longer for instructions like memory load-stores, integer-multiply, or
floating-point operations. The core supports high-level parameters that allow
it to be configured to operate at 32-bit (RV32) or 64-bit (RV64) and support
different variants of the RISC-V ISA (Waterman and Asanovic, 2019). The Flute
core also supports a memory management unit (MMU) and is capable of booting
the Linux operating system. The pipeline stages in Flute are:

F::

Issue fetch requests to the instruction memory. The fetch stage can also
handle compressed instructions.

D::

Decode the fetched instruction. Checks for illegal instructions.

E1::

The first execution stage. Reads the register files or accept forwarded values
from earlier instructions. Execute all single-cycle opcodes meant for the
integer ALU. Branches are resolved here. Discard speculative instructions.

E2::

Execute multi-cycle operations, including floating-point operations. Multi-
cycle operations are dispatched to their individual pipelines from this stage.
If the instruction was executed in E1, this stage is just a pass-through.

WB::

Collects responses from various multi-cycle pipelines, handle exceptions and
asynchronous events like interrupts, and commit the instruction.

###  2.2. Related work

Table 1. Posit Arithmetic Implementations in the Literature Impl. |  Fully parametric |  Application
specific |  Application
study |  RISC-V
integration |  Posit custom
instruction support |  Open source |  SoftPosit
porting |  Quire
support
---|---|---|---|---|---|---|---|---
Uguen (Uguen et al., 2019) |  ✓ |  ✗ |  ✗ |  ✗ |  NA |  ✓ |  NA |  ✓
Jaiswal (Jaiswal and So, 2019) |  ✗ |  ✗ |  ✗ |  ✗ |  NA |  ✓ |  NA |  ✗
PAU (Chaurasiya et al., 2018) |  ✓ |  ✗ |  ✗ |  ✓ |  NA |  ✗ |  NA |  ✗
Lu (Lu et al., 2020) |  ✗ |  ✓ |  ✓ |  ✗ |  NA |  ✓ |  NA |  ✗
Adaptive Posit (Langroudi et al., 2020) |  ✓ |  ✓ |  ✓ |  ✗ |  NA |  ✗ |  NA |  ✗
SmallPositHDL (SmallPositHDL, 2021) |  ✗ |  ✓ |  ✗ |  ✗ |  NA |  ✓ |  NA |  ✓
Deep PeNSieve (Murillo et al., 2020) |  ✓ |  ✓ |  ✓ |  ✗ |  NA |  ✓ |  NA |  ✓
Jaiswal (Jaiswal and So, 2018b) |  ✓ |  ✗ |  ✗ |  ✗ |  NA |  ✓ |  NA |  ✗
Cheetah (Langroudi et al., 2019) |  ✗ |  ✓ |  ✓ |  ✗ |  NA |  ✗ |  NA |  ✓
Deep Positron (Carmichael et al., 2019) |  ✗ |  ✓ |  ✓ |  ✗ |  NA |  ✓ |  NA |  ✓
ExPAN(N)D (Nambi et al., 2020) |  ✗ |  ✓ |  ✓ |  ✗ |  ✗ |  ✓ |  ✗ |  ✗
Melodica |  ✓ |  ✗ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓
RISC-V Integration
PERI (Tiwari et al., 2021) |  ✓ |  ✗ |  ✗ |  ✓ |  ✗ |  ✗ |  ✗ |  ✗
PERC (ThoughtWorks, 2020) |  ✓ |  ✗ |  ✗ |  ✓ |  ✗ |  ✓ |  ✗ |  ✗
CRISP111Based on publicly available data (Technologies, 2020) |  ✗ |  ✗ |  ✗ |  ✓ |  ✗ |  ✗ |  ✗ |  ✗
Saxena (Saxena et al., 2021) |  ✓ |  ✗ |  ✓ |  ✓ |  ✗ |  ✗ |  ✗ |  ✗
POSAR (Ciocirlan et al., 2021) |  ✓ |  ✗ |  ✓ |  ✓ |  ✗ |  ✓ |  ✗ |  ✗
Clarinet |  ✓ |  ✗ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓

Since the inception of posit data representation and arithmetic, there have
been several implementations of posit arithmetic in the literature. In
(Jaiswal and So, 2018a), the authors have covered the design of a parametric
posit adder/subtractor, while in (Jaiswal and So, 2018b), the authors have
presented parametric designs of float-to-posit and posit-to-float converters,
and multiplier along with the design of adder/subtractor. The _PACoGen_ open-
source framework in (Jaiswal and So, 2019) generates pipelined
adder/subtractor, multiplier, and divider. PACoGen is capable of generating
the hardware units that adapts precision at run-time. A more reliable
implementation of a parametric posit adder and multiplier generator is
presented in (Chaurasiya et al., 2018). However, the implementation in
(Chaurasiya et al., 2018) is not pipelined resulting in low operating
frequency for large bit-widths. _Cheetah_ presented in (Langroudi et al.,
2019) discusses the training of deep neural network (DNN) using posits. We
believe that the architecture presented in (Langroudi et al., 2019) is
promising and some of the features can be incorporated in Melodica in the
future. Apart from the mentioned efforts, there have been several other
implementations of posit hardware units (Zhang et al., 2019)(Lu et al., 2019).

More recently, there has been an interest to integrate posit numeric units
with RISC-V processors and demonstrate applications with posit arithmetic.
_PERI_ (Tiwari et al., 2021) integrates a posit numeric unit as a functional
unit with the Shakti C-Class RISC-V processor, and _POSAR_ (Ciocirlan et al.,
2021) integrates a parameterized posit arithmetic unit with a Rocket Chip-
based RISC-V core. These implementations do not support quire and redirect
floating-point instructions and operands to the new posit numeric unit.
Coupled with the hardware changes, a gcc back-end function which represents
real number variables in posit notation, makes computations using posits
transparent to the programmer. PERI, POSAR, PERC, CRISP, and Saxena (Saxena et
al., 2021) cannot support the breadth of experimentation possible in Clarinet
because (a) only 32-bit and 64-bit posits may be used, (b) floats and posits
may not co-exist in the same program, and (c) quire computing is not
supported.

None of the previous efforts have resulted in a complete computing environment
that allows hardware-software posit arithmetic empiricism. Further, they do
not include an easy-to-use software framework that allows floating-point and
posit types to cohabit in an application. Clarinet has been created to serve
as an open-source test-bed for hardware-software experimentation around
posits. By allowing floating-point and posits to coexist in an application,
Clarinet uniquely allows researchers to make trade-offs between hardware
costs, latency and precision. Table 1 summarizes the posit arithmetic
implementations and features such as parameterization, application
specificity, RISC-V integration, quire support, and SoftPosit porting. The
only feature that is not supported in Clarinet is application specificity.
Since the Melodica unit is fully parametric, it is practicable to generate the
application-specific instances of the unit for domain-specific computing
platforms which is true for the other literature works that are parametric.

##  3\. Clarinet

The system comprises two main components – _Melodica_ , a _parameterizable_
posit arithmetic unit that supports quire-based arithmetic described in
Section 4, and _Clarinet_ , a RISC-V CPU that is enhanced with special
instructions for posit arithmetic and a dedicated Posit Register File (PRF).

###  3.1. Clarinet organization – a type-centric approach

In addition to the existing floating-point types (float and double) which
exist in the RISC-V architecture, Clarinet introduces two new types for
computation with real numbers – posit and quire. The posit type is
parameterized by two numeric values – N and es as described in Section 2.1.1.
The Posit Register File (PRF) holds values of posit type. Values of posit type
may be read from and written to memory, and may be accessed directly by the
programmer. The quire register is an accumulator inside Melodica, which
accumulates values of the quire type. The quire type is paramterized by N as
described in Section 2.1.1. The quire values reside only inside the quire
register and may not be accessed directly by software. A quire value however
may be converted to a posit value and made available to the programmer via the
PRF.

(a) Clarinet pipeline

(b) New Clarinet instructions

Figure 2. Clarinet

Clarinet’s organization is illustrated in Fig. 2a. The starting point for
Clarinet was a Flute CPU core, configured with the RV32IMAFC variant of the
RISC-V ISA (Waterman and Asanovic, 2019). Clarinet integrates Melodica as a
functional unit parallel to the existing floating point unit. The module
mkFBox_Core, integrates the existing floating point core, and the new Melodica
core. Decode logic directs the new instructions described in Section 3.2 to
Melodica, while all other floating point instructions continue to be serviced
by the FPU, responses from Melodica are routed back to the Clarinet pipeline.

###  3.2. New instructions

####  3.2.1. Quire-based computing

Computation using the quire accumulator in Melodica can be visualized as
comprising of the following three operations: (i) initialize the quire
register (accumulator), (ii) compute into the quire, and (iii) read the result
from the quire by converting into a posit value. Six new instructions have
been been added to support these operations. The instruction FCVT.R.P
initializes the quire with a posit value from the PRF. The instructions FMA.P,
FMS.P, FDA.P, and FDS.P perform fused multiply (M) or divide (D) addition (A)
and subtraction (S) computations using two posit values from the PRF. The
result accumulates into the quire register. The instruction FCVT.P.R converts
the value in the quire register into a posit value destined for the PRF.

####  3.2.2. Interfacing with memory

Two instructions – PLW and PSW provide memory access to load and store posits.
These instructions use the GPR to compute the memory address, and the PRF for
the data. In addition, two new move instructions, PMV.X.W and PMV.W.X move
data between the GPR and PRF. Unlike conversion instructions, the move
instructions do not interpret the data present in the source register. The PMV
instructions were introduced to allow referencing of posit data structures
using integer types well-supported by the C compilers like int, short and
char. As the move instructions do not reinterpret the source data, these
instructions have low overhead in terms of computation and implementation
costs.

####  3.2.3. Type converters

One of the design considerations for Clarinet was to create a CPU which
allowed computing using both floating-point and posit values. For legacy
applications, programmers may want to enable posit arithmetic for loops or
kernels to benefit from the advantages of posits, while retaining floating
point arithmetic for the overall application. Two type-converter instructions,
FCVT.P.S and FCVT.S.P, convert between posit and floating-point values, using
the PRF and FPR respectively. These instructions can be visualized to be book-
ends for a posit computation kernel as described in 3.2.1.

####  3.2.4. Instruction encoding

Bit representations of the new instructions are in Fig. 2b. All the
instructions except the load-store instructions belong to the R-format type of
the RISC-V ISA. The PLW instruction is in the I-format, while the PSW
instruction is in the S-format. The instructions reuse existing major opcode
values as defined in (Waterman and Asanovic, 2019) (bits 6:0). In order to
handle the new posit types, a binary encoding 10 was introduced for the two-
bit fmt field. In R-format instructions, these bits occupy bits 26:25 of the
instruction. In the four compute instructions, this field indicates the type
of the source operands. The type converter instructions and quire
initialize/read instructions require both source and result encodings. The fmt
serves as the encoding for the result’s type, while the five-bit rs2 field
serves as the encoding for the source’s type. New rs2 encodings have been
introduced for posit values (0x10) and for quire values (0x11). Due to a lack
of free encoding space in the fmt field, a two-bit code was not assigned for
quire values. Differentiation between fcvt.r.p and fcvt.p.r instructions is
done by looking at a combination of fmt and rs2 fields. If the source operand
is a quire value (rs2 equals 0x11), it is interpreted as a fcvt.p.r
instruction, and if the source and destination operands are posit values (rs2
equals 0x10 and fmt equals 10), it is interpreted as a fcvt.r.p instruction.
For the load and store instructions, the Rm fields differentiates a posit load
or store from the floating-point loads and stores. The Rm encoding of 110
indicates that the load or store uses the PRF. For the move instructions, the
source register file is encoded in the rs2 field while the destination
register file is encoded in the fmt field.

The decision to add new instructions instead of reusing existing opcodes
belonging to the F subset of the RISC-V ISA was driven by two requirements –
integrating quire functionality (whose equivalent does not exist in the RISC-V
floating-point ISA), and type-converter instructions that would allow posit
and floating point values to coexist in an application as independent types.

####  3.2.5. Coexisting posits with floating-point

The type-converter instructions allow existing programs which use floating-
point arithmetic to benefit from the use of posits and quire (greater dynamic
range or accuracy) by converting certain computation kernels to use the quire,
while retaining the rest of the computation in floating-point. From our
experiments as illustrated in 5.2.1, we observe that applications see
significant reductions in normalized error through the introduction of quire-
based accumulation even when most of the computation remains in floating-
point. When an application can benefit from the use of posits (be it greater
dynamic range or accuracy), the type-converter instructions allow the user to
convert a part of the computation to posits and accumulate into the quire
register. In order to do so, they would first need to convert their
intermediate floating-point data to posits using the type-converter
instructions, before executing one of the compute instruction that accumulates
into the quire. Eventually, the results are converted back to the floating-
point format for further processing.

Using type-converter instructions to insert posit and quire-based computation
into an existing application, introduces an overhead in terms of the number of
instructions that need to be executed to complete the overall computation as
illustrated in the code example on section 5.1. Since this overhead is present
in each iteration of the inner compute loop, for a loop with N iterations,
there are $2N$ extra instructions for type conversion from floating-point to
posit. However, for new application which are built using posits, there is no
overhead in terms of the number of instructions executed as illustrated in
section 5.1.

####  3.2.6. Modifying the RISC-V gcc compiler

In order to compile programs that uses the new instructions, and work with the
newly introduced PRF, we made modifications to the RISC-V gcc assembler. This
allowed us to compose assembly programs that uses the posit new instructions.
However, we did not extend these changes into gcc’s C front-end. Due to this
limitation, C programs that rely on the new instructions have to use inline
assembly to access posit functionality on Clarinet. Further, since the posit
data type is not recognized by the C compiler, direct use of the PLW and PSW
instructions are not possible from C code. In order to circumvent these
limitations we have experimented with different approaches to work with real-
number data as described in section 5.1.

###  3.3. Hardware considerations to integrate the quire

As indicated in Fig. 1 the recommended size of the quire can grow very rapidly
with increasing posit-width. This implies that treating the quire register
similar to an entry in one of the register files would be quite expensive as
far as hardware resources are concerned. For instance, using 32-bit posits
would mean making a 512-bit quire value available on the forwarding paths and
from the register files. Further, providing a path from quire to memory (via
modified load and store instructions) would require extensive modifications of
the memory pipeline. Clarinet takes an alternate approach to integrating the
quire – treating it as an accumulator. Quire values can only be manipulated
indirectly using the FCVT.R.P and FCVT.P.R to initialize and read the quire
register respectively. Additionally, the four compute instructions: FMA.P,
FMS.P, FDA.P, and FDS.P, compute results into the quire register. These
decisions allow us to contain the cost of integrating the quire to just the
actual storage for the quire register. Section 4.1 discusses the design
challenges in designing the quire datapath.

##  4\. Melodica

Melodica is a posit arithmetic unit implemented using BSV HL-HDL. Melodica is
configured using three parameters: the posit-width (N), the maximum width of
the exponent field (es), and float-width. While Melodica supports any size
float input but for Clarinet float-width has been set to 32. In Clarinet we
have experimented with (8,0), (16,1) and (32,2) configurations for the N and
es fields. For an N-bit Melodica architecture $\frac{N^{2}}{2}$ sized quire is
integrated with the operation pipelines as an accumulator register. Depending
on the size of N, it is possible that the quire may not be sized to a multiple
of byte. All fused operations accumulate into the quire. The four supported
fused operators can also be used for basic arithmetic operations. In addition
to the fused operations Melodica implements a set of converters between
floating-point and posit formats, and operations to initialize the quire from
a poist value and to read the quire as a poist value.

Melodica’s organization is illustrated in Fig. 3a. Meodica’s input from the
pipeline comprises upto two operands, and an opcode. The CPU pipline issues a
command to Melodica only when the pipline is executing one of the special
instructions described in section 3.2 that operate on posits. The Table 2 maps
Clarinet special instructions to Melodica commands. There are four stages
involved in Melodica’s operation:

Table 2. Mapping Clarinet instructions to Melodica commands Clarinet Instruction | Melodica Input | Melodica Output |  Stages |  Pipelines
---|---|---|---|---
Operand 1 | Operand 2 | Command
FMA.P | Posit | Posit | FMA_P | ✗ |  3 |  ext1, ext2, multiply, quire.accumulate
FMS.P | Posit | Posit | FMS_P | ✗ |  3 |  ext1, ext2, multiply, quire.accumulate
FDA.P | Posit | Posit | FDA_P | ✗ |  3 |  ext1, ext2, divide, quire.accumulate
FDS.P | Posit | Posit | FDS_P | ✗ |  3 |  ext1, ext2, divide, quire.accumulate
FCVT.R.P | Posit | ✗ | FCVT_R_P | ✗ |  2 |  ext1, quire.init
FCVT.P.R | ✗ | ✗ | FCVT_P_R | Posit |  2 |  quire.read, norm
FCVT.P.S | Float | ✗ | FCVT_P_S | Posit |  2 |  FtoP, norm
FCVT.S.P | Posit | ✗ | FCVT_S_P | Float |  2 |  ext1, PtoF

(a) Melodica block diagram

(b) Quire internals

Figure 3. Melodica block diagram and Quire internals

  1. (1)

Extraction:: posit operands have variable length fields. In the extraction
stage the fields in the posit operands are decoded by the two extractor blocks
(ext1 and ext2), and broken up into their component sign, regime, exponent and
fraction fields. This stage also classifies if the posit operands represent
special cases (zero or infinity). If the input operand is a float (float-to-
posit operations), no extraction is required and the operand is forwarded to
the request interface of the FtoP block.

  2. (2)

Computation: performs the appropriate (multiply or divide) operation in the
case of a fused command from the pipeline (FMA.P, FMS.P, FDA.P, FDS.P. The
input to this stage is the output of the extractor blocks in the previous
stage. If the command is FCVT_S_P (posit to float conversion), the output of
the extractor from the previous stage is fed to the PtoF block.

  3. (3)

Quire: involves the quire module and depending on the command being executed
by Melodica, different commands are issued to the quire as listed in Table 2.
For FMA.P, FMS.P, FDA.P, FDS.P commands, the output of the multiply or divide
operations in the previous stage connect into the quire using its accumulate
interface. If the command is FCVT_R_P (initialize the quire), the output of
ext1 in stage one is used to initialize the quire using its init interface.
For commands to read the quire (FCVT_P_R), the read request is presented to
the quire at its read.request interface in this stage as well.

  4. (4)

Normalization: is the final stage where the output is prepared for the CPU
pipeline. The output from the read.response interface of the quire is used as
input to the normalizer. The normalizer may also receive input from the FtoP
output of stage 1. If the output is of type float (output of PtoF), it is
directly multiplexed with the output of this stage before forwarding to the
CPU.

###  4.1. The Quire Module

The quire module implements the fixed-point $\frac{N^{2}}{2}$-bit wide
accumulator in Melodica. Three operations are permitted on the quire:
initialization, accumulation and read-out (as posit). A block diagram of the
quire’s internals is illustrated in Fig 3b.

The width of the quire has a quadratic relationship with posit width.
Consequently in order to scale to wider posits, it becomes necessary to
segment the quire and pipeline the accumulate operation (Kulisch, 2002). In
this implementation the segment size (which is a high-level parameter) has
been fixed at $32$. This number was chosen as the 32-bit segment adders
comprising the pipelined adder would be no wider than the integer adders
already in the Clarinet processor core. A side-effect of this choice is that
when operating with 8-bit posits, accumulates complete in a single cycle. A
zero flag accompanies each segment of the quire, and denotes if the segment is
zero or not. The zero flag facilitates a fast (single cycle, fixed latency)
count of the leading zeros in the quire which is needed to complete the read
operation that converts the fixed-point value to fraction and scale fields for
the normalization stage.

##  5\. Case Studies

We cover case studies using some of the linear algebra kernels and from
optical flow in computer vision. Firstly, we discuss application building in
the Clarinet framework using both, assembly programming and later with C
programs. Secondly, we look into application kernels that are rich in
floating-point arithmetic operations - matrix operations and optical flow. For
matrix operations, we develop a subset of basic linear algebra subprograms
(BLAS) and linear algebra packages (LAPACK) using SoftPosit calls for the
error analyses. Based on this investigation, we arrive at a suitable
arithmetic size for each of the kernels in BLAS and LAPACK, and optical flow
estimation using Lucas-Kanade method. We use this information to set HDL
parameters to arrive at customized Melodica and Clarinet instances. Finally,
we execute these applications on the Clarinet CPU using RTL simulations and
report number of cycles.

Table 3 summarizes the scope of empirical studies using BLAS, LAPACK, and
optical flow as applications. The applications were executed using SoftPosit
calls for error analysis and later simulated on Clarinet RTL for latency
analysis. In addition all hardware configurations used in the study were
synthesized for ASIC and FPGA. The RTL simulation infrastructure, modified
RISC-V compiler toolchain, HDL source code for Melodica and Clarinet along
with reference SoCs and applications are available at (Sharma, 2021) and
(Bluespec Inc, 2021).

Table 3. Empirical Studies on Clarinet. NR = Not Required. Data types |  p8 |  p16 |  p24 |  p32 |  p8-q8 |  p16-q16 |  p24-q24 |  p32-q32 |  f32 |  f64
---|---|---|---|---|---|---|---|---|---|---
SoftPosit & SoftFloat |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓
Clarinet (synthesis) |  NR |  NR |  NR |  NR |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓
Clarinet (latency) |  NR |  NR |  NR |  NR |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓

###  5.1. Building Posit applications using Clarinet

We begin by illustrating how a programmer can create applications which use
posits in Clarinet. We look at two scenarios – an application where the
operand data is represented as Posits in the memory (typically a new
application), and a legacy application which operates on float-point operands
where a developer might want to introduce posits for their computational
advantages.

####  5.1.1. Vector dot product

We illustrate how to program Clarinet to implement a simple vector dot product
loop using two assembly code snippets. In the assembly code samples in 5.1.1
and 5.1.1, register names starting with p belong to the PRF, while register
names starting with x belong to the GPR. Floating-point registers from the FPR
have names starting with f.

1; ------------------------------------------------

2; x1, x2 : Vectors A and B base addresses

3; x3 : Vector length

4; q : the quire register

5; f2p,p2f: float-to-posit, posit-to-float

6

7initq: fcvt.r.p p0 ; initialize quire

8

9loop: flw f1, 0 (x1) ; f1 <\- load float A

10 flw f2, 0 (x2) ; f1 <\- load float B

11 fcvt.p.s p1, f1 ; p1 <\- f2p (f1)

12 fcvt.p.s p2, f2 ; p2 <\- f2p (f2)

13 fma.p p1, p2 ; q <\- q + (p1 * p2)

14 add x1, x1, 4 ; increment A index

15 add x2, x2, 4 ; increment B index

16 sub x3, x3, 1 ; decrement loop ctr

17 bnez x3, loop

18

19readq: fcvt.p.r p0 ; p0 <\- read (q)

20 fcvt.s.p f0, p0 ; f0 <\- p2f (p0)

1; ------------------------------------------------

2; x1, x2 : Vectors A and B base addresses

3; x3 : Vector length

4; q : the quire register

5

6

7initq: fcvt.r.p p0 ; initialize quire

8

9loop: plw p1, 0 (x1) ; p1 <\- load posit A

10 plw p2, 0 (x2) ; p2 <\- load posit B

11

12

13 fma.p p1, p2 ; q <\- q + (p1 * p2)

14 add x1, x1, 4 ; increment A index

15 add x2, x2, 4 ; increment B index

16 sub x3, x3, 1 ; decrement loop ctr

17 bnez x3, loop

18

19readq: fcvt.p.r p0 ; p0 <\- read (q)

20

The code sample on the right in 5.1.1 illustrates the core loop of the vector
dot product computation where the operands are represented as posits in
memory. Due to the quire-based computation model, any computation with posits
in Clarinet involves three steps: i) initializing the quire, ii) computing
into the quire using one or more of the four compute instructions, and iii)
reading the quire. Initializing and reading the quire represent a fixed
overhead of two instructions. Since these do not form part of the core
computation loop, their effect can be ignored for loops with a large number of
iterations, but may become substantial when these operations are frequent or
for loops with few iterations as illustrated in 5.2.

The code sample on the left in 5.1.1 illustrates the same dot-product program
with float operands. The data being operated on is stored as floats in memory
however the computations continues to use posits. The three steps involved in
quire-based computation remain unchanged. However, the computation step
involves first converting the float operands into posits. This leads to an
overhead of two instruction per iteration of the compute loop, and represents
a significant overhead compared to pure float (or posit) based computation
flow. The mixed mode of computation may find favour in legacy implementations
of applications where a transition of the working data from floats to posits
may prove problemmatic. In such cases, the mixed mode of computation would
allow the application developer to insert posit based computation where
necessary into their algorithm to take advantages of greater accuracy or
dynamic range while retaining the working data in its original format.

####  5.1.2. Abstracting away from assembly instructions

For the larger BLAS, LAPACK and optical flow examples, we packaged the three
steps involved in quire-based computation into C functions using inline
assembly for ease of use and debug. The functions in 5.1.2 are examples of
such helper functions. Since our present work does not include supporting the
posit data type as a primitive data type in C, the compilers do not recognize
the posit type. In order to circumvent this limitation we reference posit
values as unsigned char (8-bit posits) or unsigned short (16-bit posits) or
unsigned int (24 and 32-bit posits). These variables are read from or written
to memory using integer load and store instructions, followed by the pmv
instructions to move the posit values into the PRF for further processing.

1// ------------------------------------------------

2// Initialize the quire, given a 32-bit posit

3// initVal (stored as unsigned int in memory)

4void fn_init_p_quire (unsigned int initVal) {

5

6 // Move the GPR value to the PPR

7 volatile asm ("pmv.w.x fa4, %[inA]\n"

8 "fcvt.r.p ft0, fa4"

9 : : [inA] "r" (initVal) : "fa4");

10 return;

11}

1// ------------------------------------------------

2// Read the quire into a posit register and return

3// the value as unsigned int

4unsigned int fn_read_p_quire (void) {

5 unsigned int res;

6 // Move the posit value from PPR to GPR.

7 asm volatile("fcvt.p.r ft0, ft0 \n"

8 "pmv.x.w %[o_res], ft0"

9 : [o_res] "=r" (res) : : "ft0");

10 return (res);

11}

1// ------------------------------------------------

2// Perform a fused multiply accumulate into the

3// quire with two 32-bit posit values as inputs

4// (the posits are represented as unsigned int)

5void fn_positFMA (

6 unsigned int a, unsigned int b) {

7

8 // Move input values from GPR into the PPR

9 asm volatile (

10 "pmv.w.x fa4, %[inA]\n"

11 "pmv.w.x fa5, %[inB]\n"

12 "fma.p ft0, fa4, fa5"

13 : : [inA] "r" (a), [inB] "r" (b)

14 : "ft0", "fa4", "fa5"); // clobber

15

16 return;

17}

1// ------------------------------------------------

2// Dot product of two vectors of 32-bit posits. The

3// arrays are stored as unsigned int in memory

4unsigned int fn_positVDP (int r, // array size

5 unsigned int p_a[], unsigned int p_b[]) {

6

7 unsigned int acc = 0; // posit-32 output

8 int i = 0;

9 fn_init_p_quire (acc); // quire <\- 0

10

11 // The dot product loop, using quire

12 for (i=0; i<r; i++) fn_positFMA (p_a[i], p_b[i]);

13

14 // Read result from quire as posit-32

15 acc = fn_read_p_quire ();

16 return (acc);

17}

###  5.2. BLAS and LAPACK

####  5.2.1. Error Analysis using SoftPosit

Figure 4. Accurate number of digits with different data types in BLAS and
LAPACK routines, and RMS error in optical flow

The BLAS and LAPACK are encountered in a wide range of engineering and
scientific applications. In BLAS, we consider dot product (xDot), matrix-
vector (xGemv), matrix-matrix operations (xGemm), and in LAPACK, we consider
Givens rotation (xGivens) where x denotes the data type used for the
implementations. For all the matrix operations, we implement nine different
versions using different data types for comparison and use 64-bit floating-
point implementation as a reference. We randomly generate numbers using rand()
function. Since, Clarinet supports quire and FMA, we emphasize more on quire
based implementations with using SoftPosit for our analyses. To calculate the
error in xDot, we average the relative error over 100K runs. To calculate
error in xGemv and xGemm, we use
$\frac{\parallel\hat{x}-x\parallel_{2}}{\parallel x\parallel_{2}}$ and
$\frac{\parallel\hat{A}-A\parallel_{2}}{\parallel x\parallel_{2}}$,
respectively where $x$ and $A$ are the operations computed in 64-bit floating-
point and $\hat{x}$ and $\hat{A}$ are the operations computed using SoftPosit.

The accurate digits in the different implementations are shown in Fig. 4. In
dot product, the 32-bit quire (q32Dot) results in 8.8 accurate digits for
small ($<$10) input vector sizes in range of 0 to 1 (Fig. 4a). For large
vectors ($<$10000) in the same range, the number of accurate digits drop to
8.2 which is a drop of 6.8%. In the same input rage, we observe a drop of
12.3% in fDot, 17.4% in p32Dot, and 9.3% in q24Dot. For the input vector range
of 0 to 10 and the sizes of 10 to 10000, we observe a similar trend (Fig. 4b).
Varying the range of input vectors impacts the accuracy heavily, specifically
for large vectors. We observe a drop in the number of accurate digits by 55.6%
in q32Dot, 53.94% in p32Dot, and 36.3% in q24Dot while in fDot it is 18.63%
(Fig. 4c). The drop in accuracy is due to the fact that the posit and quire
are more accurate for the values around 1.0 while as the input range shifts
from 1.0 the accuracy deteriorates.

A similar trend is observed in xGemv, xGemm, and xGivens routines for
increasing matrix sizes and varying ranges (Fig. 4d). A key observation here
is that in p32Givens and q32Givens routines where we observe the number of
accurate digits are significantly higher (8.2 and 8.8 respectively) compared
to fGivens (6.79). The shaded region in Fig. 4 represents the routines that
can be executed on the current version of Clarinet due to absence of posit
addition, multiplication, division hardware. For implementation of routines in
software we have used floating-point in conjunction with quire. For example,
q32-f32Givens is implementation of Givens rotation using combination of 32-bit
quire and 32-bit floating-point arithmetic. The implementation yields similar
accuracy as q32Givens since the majority of the operations are dominated by
quire. In BLAS routines, the 100% of the arithmetic operations can be
implemented using only quire.

####  5.2.2. Simulation on Clarinet RTL

The intent of simulating applications on Clarinet RTL was to study the effects
that different real number types, namely floating-point and posits had on
latency of computation, and also to come with best practices to use posit
hardware more effectively in a real RISC-V CPU. Our test rig consisted of RTL
simulators for the different RTL configurations tabulated in table 6, elf
files generated by compiling the applications and kernels using the modified
RISC-V compiler described in section 3.2.6, and tools to extract timing
information from instruction traces generated during RTL simulations.

Applications for Clarinet can adopt any one of three approaches with respect
to real-number types: (i) single or double-precision floating point where the
data in memory resides as floats and the computation is carried out using
floating-point arithmetic pipelines, (ii) float-posit where the data in memory
resides as floats but the computation is carried out using posit arithmetic
pipelines, and (iii) posit where the data in memory resides as posits and the
compution is carried out using posit arithmetic pipelines. We implemented the
BLAS kernels and LAPACK routines using all three approaches, and across four
different posit widths – 8, 16, 24 and 32. These implementation configurations
are summarized in the Table 4.

Table 4. Implementation Variations (wrt real number types) Configuration |  Input Data |  Output Result |  Computation |  FPR-PRF Conversion |  GPR-PRF Move
---|---|---|---|---|---
f32 |  FPR loads (flw) |  FPR stores (fsw) |  FP fused multiply-add (fmadd.s) |  ✗ |  ✗
f32-p8 |  FPR loads (flw) |  FPR stores (fsw) |  8-bit Quire fused multiply-add (fma.p) |  ✓ |  ✗
f32-p16 |  FPR loads (flw) |  FPR stores (fsw) |  16-bit Quire fused multiply-add (fma.p) |  ✓ |  ✗
f32-p32 |  FPR loads (flw) |  FPR stores (fsw) |  32-bit Quire fused multiply-add (fma.p) |  ✓ |  ✗
p8 |  GPR byte loads (lbu) |  GPR byte stores (sbu) |  8-bit Quire fused multiply-add (fma.p) |  ✗ |  ✓
p16 |  GPR half-word loads (lhu) |  GPR half-word stores (shu) |  16-bit Quire fused multiply-add (fma.p) |  ✗ |  ✓
p32 |  GPR word loads (lw) |  GPR word stores (sw) |  32-bit Quire fused multiply-add (fma.p) |  ✗ |  ✓

(a) BLAS Cycles (absolute)

(b) BLAS Cycles (%)

Figure 5. BLAS kernels (xDot, xGemv, xGemm) and xGivens cycle counts

(a) BLAS Cycles (absolute)

(b) BLAS Cycles (%)

Figure 6. BLAS kernels (xDot, xGemv, xGemm) and xGivens cycle counts
(estimated with posits as a primitive data type)

Fig 5a represents the number of clock cycles consumed by different groups of
instructions while executing the xDOT, xGEMV, xGEMM, and xGivens kernels. Fig
5b breaks up the same information into percentages of the total clocks cycles
consumed to execute the kernels. In order to present data across the three
BLAS kernels, the vector and matrix sizes input to the kernels were chosen to
require the same number of floating-point or posit (multiply-accumulate)
operations. Consequently, xDOT uses 4096 element vectors, xGEMV uses 64
element vectors and 64x64 matrices, and xGEMM uses 16x16 matrices. Normalizing
across number of computations however implies that the amount of data required
for each kernel reduces with the increasing order of the kernel. All kernels
were run with warm caches and the -O3 optimization flag was used to compile
the C code. In the case of xGivens, the matrix size was reduced to 8x8. The
cycles required to execute xGivens on a 16x16 matrix was more than double the
cycles required for the BLAS kernels - halving the size allowed us to fit the
data from xGivens in the same scale. For the analysis, instructions were
grouped as per the following categories:

  1. (1)

float compute instructions directly involved with floating-point computation.
In the case of the BLAS kernels these are fmadd.s, fmul.s, and fadd.s. In
other applications like givens, this category would also include other
floating-point instructions like fsqrt.s and fdiv.s

  2. (2)

float ld/st instructions to load and store floating-point data from and to
memory. For single-precision operation these are the flw and fsw instructions.

  3. (3)

posit compute instructions that perform computations into the quire. These are
fma.p, fms.p, fda.p, and fds.p. Instructions to initialize the quire
(fcvt.r.p) and read the final result from the quire (fcvt.p.r) are also
included in this category as initializing the quire and reading out its value
as a posit value form the bookends of any posit computation in Clarinet.

  4. (4)

float-posit interop instructions that are present to allow interoperability
between floating-point and posit values. These include instructions like
fcvt.s.p and fcvt.p.s to convert between floating-point and posit values. This
category also includes the instructions to move posit values between the GPR
and PRF (pmv.x.w and pmv.w.x). The pmv instructions are not used strictly for
interoperability between floats and posits, however their presence is needed
because of the lack of high-level compiler support for the posit data type.

  5. (5)

posit ld/st instructions to load and store posit data from and to memory. Due
to the lack of higher-level compiler support for the posit data type, these
appear as 8, 16 and 32 bit unsigned loads and stores in the trace (lbu and
sbu, lhu and shu, and lw and sw).

  6. (6)

others all other instructions in the trace not categorized as one of the
above.

The absolute number of cycles spent on posit compute in the BLAS kernels is
consistently lower than those spent on float compute. The main reason for more
efficient posit compute is better use of pipelining in the posit core. As
instructions that update the quire do not have a side-effect on the Clarinet’s
register files, the posit core can queue up several such instructions leading
to more effective use of the deep posit pipeline. Compared to an individual
FMADD.S instruction that takes 12 cycles, a FMA.P instruction may take 12, 20,
36 cycles for 8-bit, 16-bit and 32-bit posits respectively. However, unlike a
FMADD.S that responds with the answer after 14 cycles, a FMA.P instruction
responds immediately since the answer is going to accumulate into the quire
and the pipeline can proceed. Subsequent FMA.P instructions queue into the
pipeline, effectively hiding the long latencies associated with the deeper
posit pipelines. The concealment of the posit fused operation latency is more
effective for narrower posits as these take fewer cycles to compute the result
into the quire and subsequently require fewer operations to be queued in to
optimally use the pipeline. Since the operating frequency of the FPU and
Melodica are very similar for the FPGA devices considered here, the fewer
cycles takes for posit compute translate to shorter latencies for posit
compute.

The cycles in yellow in Fig 5a can be considered as an overhead to the use of
posits in the clarinet system. When operating in one of the interop modes -
floating-point data, posit compute configurations (f32-p*) - this overhead is
significantly larger than the posit compute cycles themselves as it includes
the cycles spent in converting the floating point data into posit values for
each iteration of the computation loop as described in 5.1. These operations
are not pipelined as they return a value to the Clarinet’s register files.
However, when operating with posit data directly (configurations p*), this
overhead is smaller (Fig 5b and is an artefact of the lack of higher-level
compiler support for the posit data type in C. This lack of support requires
us to save posit data as unsigned integer types, read/write them via the GPR
and then move them over to the PRF to initiate the posit computation. If posit
data types were to be treated as a primitive data type in C, the overhead when
working with posit data directly would disappear as illustrated in Fig 6a.
However, there is no change seen for the interop modes (f32-p*) as the
overhead in these modes are due to switching between floats and posits - this
overhead would remain even if posits were to be a primitive data type.

When operating with 8 and 16-bit posit data (p8 and p16), memory access
latencies are lower due to the more efficient utilization of Clarinet’s 8KB
level one caches. In the case of xDOT the data set completely fits in the
cache only when operating with 8-bit posits, and in all other cases memory
latency is dominated by capacity misses. However in higher-order kernels like
xGEMV and xGEMM the effect is less pronounced as the data sets are smaller. In
the case of xGEMM, the complete data set fits into the cache across all data
types. Consequently, the memory accesss latency is near constant across all
configurations. For xGEMV the complete data set fits in to the cache when
operating with 8 and 16-bit posits. This results in a small advantage in
overall cycle counts for 8 and 16-bit posit widths when compared to f32.

Higher-order kernels involve more bookkeeping in the form of cycle counters
and function calls to lower-order kernels, and these reflect in the larger
number of cycles spent in the unaccounted (others) category. When operating
with floating-point data (f32) or in one of the interop modes (f32-p*), higher
order kernels take fewer number of overall cycles to execute. This is due to a
greater proportion of the input data fitting into the caches resulting in
fewer capacity misses and consequently a lower cycle count for floating point
loads and stores. Across all configurations that use posits for computation
(f32-p* and p*), higher order kernels result in an increase in the cycle spent
in computation. This effect is especially pronounced in Fig 6a and Fig 6b due
to larger percentage of overall cycles being spent on posit compute. While the
number of multiply-accumulate operations are constant across all
configurations, higher-order kernels require repeated initialization and
reading of the quire register (once per call of the xDot kernel). In the case
of xGEMV, this works out to $O(N)$ quire accesses, and $O(N^{2})$ quire
accesses for xGEMM. Requests to initialize the quire complete immediately, as
there is no value returned to the Clarinet register files and do not
contribute much to these increase in cycles. However, reading the quire is an
expensive operation from a latency point of view as the pipeline has to wait
for all outstanding fused operations to complete before a value can be
returned to Clarinet’s register files.

Table 5. Accumulations versus Quire Reads across kernels |  xDot (4096) |  xGEMV (64) |  xGEMM (16) |  xGivens (8)
---|---|---|---|---
Quire initializations and reads |  1 |  64 |  256 |  64
Accumulations between quire reads |  4096 |  64 |  16 |  1

The xGivens application represents a case where all posit operations cannot be
realised in hardware due to limitations in the current Melodica
implementation, namely the unavailability of the square root operator, and
interoperability with floats is necessary to complete the $O(N^{3})$ square
root operations in this implementation. As Figure 5a indicates, running
xGivens on floating-point outperforms all posit and float-posit
configurations. The main reason for the poorer performance of posit based
configurations is the inability to effectively pipeline posit compute
operations in this xGivens implementation. Table 5 tabulates the number of
accumulations between initializing the quire and its subsequent read at the
end of a set of computations. The longer the string of uninterrupted
accumulations between two quire reads, the more effective is the use of the
posit pipeline. Further, narrower posits are more forgiving of shorter
accumulation sequences as the pipeline depth for quire accumulation in
melodica grows quadratically with posit width (one-stage, four-stage, and
16-stage for 8-bit, 16-bit and 32-bit posits respectively). In the case of the
xGivens implementation, the number of accumulations is independent of the
matrix size, each posit computation sees an effective latency of 36 cycles for
32-bit posits and the effect is amplified. Even in the case of xGEMM the poor
ration of accumulation depth to quire reads leads to a steep increase in the
number of computation cycles, especially for 32-bit posits (Fig. 5b).

###  5.3. Lucas-Kanade optical flow

####  5.3.1. Error Analysis using SoftPosit

Lucas-Kanade is a differential method of tracking features given a sequence of
frames. Given I as brightness-per-pixel at (x,y), the local optical flow
(velocity) vector ($\vec{u}$,$\vec{v}$) is given by $\frac{\partial
I}{\partial x}u+\frac{\partial I}{\partial y}v+\frac{\partial I}{\partial
t}=0$. The Lucas-Kanade method is used to calculate the optical flow for
consecutive frames of rotating objects which are given in Fig. 7. We compare
the different posit and single-precision floating-point configuration
combinations with 64-bit floating-point values using SoftPosits, and generate
heat maps of the absolute error for both u and v. The three configurations
that are being compared are: i) 32-bit single-precision floating-point
arithmetic (f32), ii) 32-bit single-precision float arithmetic combined with
N-bit quire arithmetic (f32-qN), iii) N-bit posit arithmetic and N-bit quire
arithmetic (pN-qN). Furthermore, owing to the better accuracy of posits around
1.0 we have normalized (norm) grey-scale pixel values (0 to 255) to (0.0 to
16.0).

(a) Rubik’s cube and sphere

(b) Maximum error and RMS error with respect to 64-bit floating-point

Figure 7. Dataset for Lucas-Kanade and error analysis using SoftPosit

(a) Error heat-maps of Rubik’s cube for pixels between 0-255

(b) Error heat-maps of Rubik’s cube normalized pixels between 0.0-16.0

(c) Error heat-maps of sphere pixels between 0-255

(d) Error heat-maps of sphere normalized pixels between 0.0-16.0

Figure 8. Error heat-maps for Rubik’s cube and sphere

From the heat-maps in Fig. 8a the effects of normalization and q32 on error
become obvious. When working with normalized data, the configuration p32-q32
clearly outperforms all other configurations. For data which is not
normalized, the performance of p32-q32 depends on whether the data naturally
falls around 1.0. However, even in the non normalized case, f32-q32 performs
consistently better than f32. The general trend in maximum and RMS error for
Rubik’s cube and sphere object frames for different configurations are shown
in Fig. 7b. The y-axis value for RMS error in Fig. 7b gives the number
accurate digits for the configurations compared to 64-bit floating-point.
Allowing one decimal places of tolerance to error, p24-q24-norm configuration
can give accurate results close to f32. With a penalty of 2 more decimal place
p16-q16-norm can be a feasible alternative. When optical flow is computed for
posit configurations for values not around 1.0 the accuracy falls. As
summarised in the satellite Table in Fig. 4, p32-q32-norm configuration
results in an order improvement in accuracy as compared to f32 for the sphere
dataset. The f32-q32 configuration for gray-scale pixel values (0-255)
improves the accuracy by 23% and 32% for Rubik’s cube and sphere dataset
respectively.

####  5.3.2. Simulating on Clarinet RTL

Adapting the Lucas-Kanade optical flow application to run on Clarinet is a
good example showcasing the co-existence of floating-point and posit types on
Clarinet hardware. In order to take advantage of the quire accumulator in
Clarinet only the inner multiply-accumulate loop to compute the velocity is
converted to use posits. This corresponds to the f32-q32, f32-q24, and f32-q16
cases in the earlier experiments with SoftPosits.

1// Original (with floating point)

2float AtA_f[2][2]; int r, c, k;

3for (r = 0; r < 2; r++) {

4 for (c = 0; c < 2; c++) {

5 AtA_f[r][c] = 0.0;

6 // accumulation loop (using float in memory)

7 for (k = 0; k < Arows; k++)

8 AtA_f[r][c] += (A[k][r] * A[k][c]);

9

10 }

11}

1// With posit based accumulation

2float AtA_f[2][2]; int r, c, k;

3for (r = 0; r < 2; r++) {

4 for (c = 0; c < 2; c++) {

5 fn_init_p_quire (0); // init quire

6 // accumulation loop (using quire)

7 for (k = 0; k < Arows; k++)

8 fn_posit_fma (A[k][r], A[k][c]);

9 AtA_f[r][c] = fn_read_quire (); // read quire

10 }

11}

The code sample in 5.3.2 illustrates the modifications required to change a
loop to use the quire instead of accumulating into a floating-point variable.
The code on the left is the original loop that accumulates into float
variables in memory and in the code on the right, the same loop has been
rewritten to use the quire. Such modifications could be the first step for an
application developer to evaluate if posit based accumulation is suitable for
their application. As explained in section 5.1 this approach does introduce
the overhead of converting between floating point and posit types.

In our simulation of optical flow on Clarinet we are using a set of three
images, each 240x240 with a kernel size of 5x5 for the velocity computation.
The cycle measurements focus on the velocity calculation routines which
accounts for about 70% of the total cycles run in the simulation. There are
three dot-product loops as part of the velocity computation. For the first two
loops, the size of the vectors depends on the kernel size, and for a 5x5
kernel, the dot product operates on a vector of length 25. From simulations we
observe that each dot product takes 548 cycles with posits versus 495 cycles
with floats representing a 16% overhead in cycles due to the use of posits,
where each dot product involves 25 accumulations. However, the third dot-
product operates on vectors of length 2, and while computing with float
requires 25 cycles, this increases to 95 cycles with posits. Over the entire
duration of the simulation, posits require 25.6 million cycles while float
complete the same computation in 21 million cycles. This corroborates with our
observations in 5.2 where posits outperform floats in terms of latency when
the number of accumulations in a loop is large enough to effectively pipeline
the posit computations in the Melodica pipeline.

###  5.4. Emulation

Table 6. Clarinet Experimental Configurations Sl. |  Configuration |  Floating Point Unit |  FP Registers |  FP Divider & Sqrt |  Melodica |  Posit Registers |  Posit Divider
---|---|---|---|---|---|---|---
1 |  Flute-F |  Single Precision |  32 x 32-bit |  ✗ |  ✗ |  ✗ |  ✗
2 |  Flute-F-DIV |  Single Precision |  32 x 32-bit |  ✓ |  ✗ |  ✗ |  ✗
3 |  Clarinet-F-P8.0 |  Single Precision |  32 x 32-bit |  ✗ |  N=8, es=0 |  32 x 8-bit |  ✗
4 |  Clarinet-F-P16.1 |  Single Precision |  32 x 32-bit |  ✗ |  N=16, es=1 |  32 x 16-bit |  ✗
6 |  Clarinet-F-P32.2 |  Single Precision |  32 x 32-bit |  ✗ |  N=32, es=2 |  32 x 32-bit |  ✗
7 |  Clarinet-F-P8.0-DIV |  Single Precision |  32 x 32-bit |  ✓ |  N=8, es=0 |  32 x 8-bit |  ✓
8 |  Clarinet-F-P16.1-DIV |  Single Precision |  32 x 32-bit |  ✓ |  N=16, es=1 |  32 x 16-bit |  ✓
10 |  Clarinet-F-P32.2-DIV |  Single Precision |  32 x 32-bit |  ✓ |  N=32, es=2 |  32 x 32-bit |  ✓
11 |  Clarinet-P8.0 |  ✗ |  ✗ |  ✗ |  N=8, es=0 |  32 x 8-bit |  ✗
12 |  Clarinet-P16.1 |  ✗ |  ✗ |  ✗ |  N=16, es=1 |  32 x 16-bit |  ✗
14 |  Clarinet-P32.2 |  ✗ |  ✗ |  ✗ |  N=32, es=2 |  32 x 32-bit |  ✗
15 |  Clarinet-P8.0-DIV |  ✗ |  ✗ |  ✗ |  N=8, es=0 |  32 x 8-bit |  ✓
16 |  Clarinet-P16.1-DIV |  ✗ |  ✗ |  ✗ |  N=16, es=1 |  32 x 16-bit |  ✓
18 |  Clarinet-P32.2-DIV |  ✗ |  ✗ |  ✗ |  N=32, es=2 |  32 x 32-bit |  ✓
19 |  Flute-D |  Double Precision |  32 x 64-bit |  ✗ |  ✗ |  ✗ |  ✗
20 |  Flute-D-DIV |  Double Precision |  32 x 64-bit |  ✓ |  ✗ |  ✗ |  ✗

(a) LUT utilization by sub-units in Melodica

(b) LUT distribution across sub-units in Melodica

Figure 9. Implementation Results for Melodica (100 MHz, Zed Board)

(a) LUT Utilization (absolute) by sub-units in Clarient CPU

(b) LUT distribution (%) across sub-units in Clarinet CPU

Figure 10. Implementation Results for Clarinet CPU (25 MHz, Zed Board)

Clarinet was emulated on a Zed board with a xc7z020-clg484-1 FPGA. We used
Vivado 2020.2 for synthesis. The CPU configurations chosen for emulation have
been summarized in table 6 and correspond to the configurations used to
execute applications in sections 5.2 and 5.3. Detailed implementation analysis
was also done for the Melodica posit arithmetic unit as well. For analyzing
Melodica implementation, all the configurations in table 6 with a posit
arithmetic unit (configurations 3 to 18) were considered.

The bars in figure 9a represents the number of LUTs in the xc7z020-clg484-1
FPGA required to implement the different sub-unis comprising Melodicas across
the configurations of interest. The sub-units considered were (two)
extractors, the normalizer, float-posit converters (in selected
configurations), the multiplier, the divider (in selected configurations), and
the quire. The quire is easily the most resource intensive sub-unit across all
configurations as seen clearly from figure 9b where the LUT utilization data
is represented as a percentage of the total utilization for a particular
Melodica configuration. The size of the quire (in terms of LUTs utilized)
grows non-linearly with posit-width (approaching $O(N^{2})$, and this reflects
in the sizes of the melodicas as well. At the CPU level all the different
clarinet configurations listed in the Table 6 were synthesized. While the
complete configuration space of Clarinet is too large to exhaustively discuss
here, a subset of configurations that were used to execute the applications
listed in sections 5.2 and 5.3 were selected as candidates for implementation
analysis. The baseline for comparisons is configuration 1, a 32-bit RISC-V
Clarinet processor with support for single-precision (32-bit) floating-point
arithmetic (without FP divide and square root). No support for posits exist in
this configuration.

##  6\. Conclusion

We presented Clarinet – an open-source, hardware-software framework for posit
arithmetic empiricism. By integrating a parameterized posit arithmetic unit
(Melodica) that supports quire-based fused operations with a RISC-V processor
extended with special instructions for posit and quire operations, we created
the Clarinet platform that enables experimentation around multiple axes
related to posit arithmetic. Floating-point and posit data types can coexist
in applications that run on Clarinet, allowing researchers to use it as a
platform to gauge the impact of posits in real-world floating-point
applications. We illustrated these capabilities through case studies on BLAS
and LAPACK kernels and Lucas-Kanade optical flow estimation. We characterized
Clarinet for latency, operating frequency, and resource utilization on Xilinx
FPGAs across several configurations. In the future, we plan to explore
Melodica’s use as a posit-enabled accelerator.

## References

  * (1)
  * Alouani et al. (2021) Ihsen Alouani, Anouar BEN KHALIFA, Farhad Merchant, and Rainer Leupers. 2021.  An Investigation on Inherent Robustness of Posit Data Representation. In _Proceedings of the International Conference on VLSI Design (VLSID)_.
  * Bluespec Inc. (2020a) Bluespec Inc. 2020a.  BSV HL-HDL.  https://github.com/B-Lang-org/bsc.
  * Bluespec Inc. (2020b) Bluespec Inc. 2020b.  FLUTE RISC-V Core.  https://github.com/bluespec/Flute.
  * Bluespec Inc (2021) Niraj Nayan Sharma, Bluespec Inc. 2021\.  Clarinet.  https://github.com/HPC-Lab-IITB/Clarinet/.  [Online; accessed 21-Jan-2021].
  * Burgess et al. (2019) N. Burgess, J. Milanovic, N. Stephens, K. Monachopoulos, and D. Mansell. 2019\.  Bfloat16 Processing for Neural Networks. In _2019 IEEE 26th ARITH_. 88–91.  https://doi.org/10.1109/ARITH.2019.00022
  * Carmichael et al. (2019) Z. Carmichael, H. F. Langroudi, C. Khazanov, J. Lillie, J. L. Gustafson, and D. Kudithipudi. 2019.  Deep Positron: A Deep Neural Network Using the Posit Number System. In _2019 Design, Automation Test in Europe Conference Exhibition (DATE)_. 1421–1426.  https://doi.org/10.23919/DATE.2019.8715262
  * Chaurasiya et al. (2018) R. Chaurasiya, J. Gustafson, R. Shrestha, J. Neudorfer, S. Nambiar, K. Niyogi, F. Merchant, and R. Leupers. 2018\.  Parameterized Posit Arithmetic Hardware Generator. In _2018 IEEE 36th ICCD_. 334–341.  https://doi.org/10.1109/ICCD.2018.00057
  * Ciocirlan et al. (2021) Stefan Dan Ciocirlan, Dumitrel Loghin, Lavanya Ramapantulu, Nicolae Tapus, and Yong Meng Teo. 2021\.  The Accuracy and Efficiency of Posit Arithmetic.  arXiv:2109.08225 [cs.AR]
  * de Dinechin et al. (2019) Florent de Dinechin, Luc Forget, Jean-Michel Muller, and Yohann Uguen. 2019.  Posits: The Good, the Bad and the Ugly. In _Proceedings of the CoNGA 2019_ (Singapore, Singapore) _(CoNGA’19)_. ACM, New York, NY, USA, Article 6, 10 pages.  https://doi.org/10.1145/3316279.3316285
  * Gustafson and Yonemoto (2017) Gustafson and Yonemoto. 2017.  Beating Floating Point at Its Own Game: Posit Arithmetic.  _Supercomput. Front. Innov.: Int. J._ 4, 2 (June 2017), 71–86.  https://doi.org/10.14529/jsfi170206
  * Higham (2002) Nicholas J. Higham. 2002\.  _Accuracy and Stability of Numerical Algorithms_ (2nd ed.).  Society for Industrial and Applied Mathematics, USA.
  * Jaiswal and So (2018a) M. K. Jaiswal and H. K. . So. 2018a.  Architecture Generator for Type-3 Unum Posit Adder/Subtractor. In _ISCAS 2018_. 1–5.  https://doi.org/10.1109/ISCAS.2018.8351142
  * Jaiswal and So (2018b) M. K. Jaiswal and H. K. . So. 2018b.  Universal number posit arithmetic generator on FPGA. In _DATE 2018_. 1159–1162.  https://doi.org/10.23919/DATE.2018.8342187
  * Jaiswal and So (2019) M. K. Jaiswal and H. K. . So. 2019.  PACoGen: A Hardware Posit Arithmetic Core Generator.  _IEEE Access_ 7 (2019), 74586–74601.  https://doi.org/10.1109/ACCESS.2019.2920936
  * Kulisch (2002) Ulrich W. Kulisch. 2002\.  _Advanced Arithmetic for the Digital Computer: Design of Arithmetic Units_.  Springer-Verlag, Berlin, Heidelberg.
  * Langroudi et al. (2019) Hamed F. Langroudi, Zachariah Carmichael, David Pastuch, and Dhireesha Kudithipudi. 2019\.  Cheetah: Mixed Low-Precision Hardware & Software Co-Design Framework for DNNs on the Edge.  arXiv:1908.02386 [cs.LG]
  * Langroudi et al. (2020) H. F. Langroudi, V. Karia, J. L. Gustafson, and D. Kudithipudi. 2020.  Adaptive Posit: Parameter aware numerical format for deep learning inference on the edge. In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_. 3123–3131.  https://doi.org/10.1109/CVPRW50498.2020.00371
  * Leeser et al. (2014) M. Leeser, S. Mukherjee, J. Ramachandran, and T. Wahl. 2014.  Make it real: Effective floating-point reasoning via exact arithmetic. In _DATE 2014_. 1–4.  https://doi.org/10.7873/DATE.2014.130
  * Leong (2018) Cerlane Leong. 2018\.  SoftPosit Version 0.4.1rc.  https://gitlab.com/cerlane/SoftPosit
  * Lu et al. (2020) J. Lu, C. Fang, M. Xu, J. Lin, and Z. Wang. 2020.  Evaluations on Deep Neural Networks Training Using Posit Number System.  _IEEE Trans. Comput._ (2020), 1–1.  https://doi.org/10.1109/TC.2020.2985971
  * Lu et al. (2019) Jinming Lu, Siyuan Lu, Zhisheng Wang, Chao Fang, Jun Lin, Zhongfeng Wang, and Li Du. 2019.  Training Deep Neural Networks Using Posit Number System.  arXiv:1909.03831 [cs.LG]
  * Murillo et al. (2020) Raul Murillo, Alberto A. Del Barrio, and Guillermo Botella. 2020\.  Deep PeNSieve: A deep learning framework based on the posit number system.  _Digital Signal Processing_ 102 (2020), 102762.  https://doi.org/10.1016/j.dsp.2020.102762
  * Nambi et al. (2020) Suresh Nambi, Salim Ullah, Aditya Lohana, Siva Satyendra Sahoo, Farhad Merchant, and Akash Kumar. 2020.  ExPAN(N)D: Exploring Posits for Efficient Artificial Neural Network Design in FPGA-based Systems.  arXiv:2010.12869 [cs.AR]
  * Saxena et al. (2021) Vinay Saxena, Farhad Merchant, Ankitha Reddy, John Gustafson, Jonathan Neodorfer, Sangeeth Nambiar, and Rainer Leupers. 2021.  Brightening the Optical Flow through Posit Arithmetic. In _International Symposium on Quality Electronic Design (ISQED)_. IEEE.
  * Sharma (2021) Riya Jain, Niraj Nayan Sharma. 2021\.  Melodica.  https://github.com/HPC-Lab-IITB/Melodica.  [Online; accessed 21-Jan-2021].
  * SmallPositHDL (2021) SmallPositHDL. 2021\.  Chisel-based SmallPositHDL.  https://github.com/starbrilliance/SmallPositHDL/.  [Online; accessed 21-Jan-2021].
  * Technologies (2020) Calligo Technologies. 2020\.  Posit Numeric Unit (PNU-IP).  https://calligotech.com/posit-numeric-unit-pnu-ip/.  [Online; accessed 17-Dec-2020].
  * ThoughtWorks (2020) ThoughtWorks. 2020\.  Posit Enhanced Rocket Chip (PERC).  https://www.thoughtworks.com/engineering-research/perc.  [Online; accessed 17-Dec-2020].
  * Tiwari et al. (2021) Sugandha Tiwari, Neel Gala, Chester Rebeiro, and V. Kamakoti. 2021.  PERI: A Configurable Posit Enabled RISC-V Core.  _ACM Trans. Archit. Code Optim._ 18, 3, Article 25 (April 2021), 26 pages.  https://doi.org/10.1145/3446210
  * Uguen et al. (2019) Y. Uguen, L. Forget, and F. de Dinechin. 2019.  Evaluating the Hardware Cost of the Posit Number System. In _2019 29th International Conference on Field Programmable Logic and Applications (FPL)_. 106–113.  https://doi.org/10.1109/FPL.2019.00026
  * Volkova et al. (2019) A. Volkova, M. Istoan, F. De Dinechin, and T. Hilaire. 2019.  Towards Hardware IIR Filters Computing Just Right: Direct Form I Case Study.  _IEEE Trans. Comput._ 68, 4 (April 2019), 597–608.  https://doi.org/10.1109/TC.2018.2879432
  * Waterman and Asanovic (2019) Andrew Waterman and Krste Asanovic. 2019.  _The RISC-V Instruction Set Manual, Volume I: User-Level ISA, Document Version 20191213_.
  * Zhang et al. (2019) H. Zhang, J. He, and S. Ko. 2019.  Efficient Posit Multiply-Accumulate Unit Generator for Deep Learning Applications. In _ISCAS 2019_. 1–5.  https://doi.org/10.1109/ISCAS.2019.8702349


* Turcan and McKeown (2019) Elsbeth Turcan and Kathleen McKeown. 2019.  Dreaddit: A Reddit Dataset for Stress Analysis in Social Media.  http://arxiv.org/abs/1911.00133 arXiv:1911.00133 [cs].
  * Wang et al. (2020) Dakuo Wang, Elizabeth Churchill, Pattie Maes, Xiangmin Fan, Ben Shneiderman, Yuanchun Shi, and Qianying Wang. 2020.  From human-human collaboration to Human-AI collaboration: Designing AI systems that can work together with people. In _Extended abstracts of the 2020 CHI conference on human factors in computing systems_. 1–6.
  * Wei et al. (2021) Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021.  Finetuned language models are zero-shot learners.  _arXiv preprint arXiv:2109.01652_ (2021).
  * Wei et al. (2022) Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022.  Finetuned Language Models Are Zero-Shot Learners.  http://arxiv.org/abs/2109.01652 arXiv:2109.01652 [cs].
  * Wei et al. (2023) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023.  Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.  http://arxiv.org/abs/2201.11903 arXiv:2201.11903 [cs].
  * Wu et al. (2023) Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023.  PMC-LLaMA: Further Finetuning LLaMA on Medical Papers.  http://arxiv.org/abs/2304.14454 arXiv:2304.14454 [cs].
  * Xu et al. (2021b) Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, and Fei Huang. 2021b.  Raise a child in large language model: Towards effective and generalizable fine-tuning.  _arXiv preprint arXiv:2109.05687_ (2021).
  * Xu et al. (2019) Xuhai Xu, Prerna Chikersal, Afsaneh Doryab, Daniella K. Villalba, Janine M. Dutcher, Michael J. Tumminia, Tim Althoff, Sheldon Cohen, Kasey G. Creswell, J. David Creswell, Jennifer Mankoff, and Anind K. Dey. 2019\.  Leveraging Routine Behavior and Contextually-Filtered Features for Depression Detection among College Students.  _Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies_ 3, 3 (Sept. 2019), 1–33.  https://doi.org/10.1145/3351274
  * Xu et al. (2021a) Xuhai Xu, Prerna Chikersal, Janine M. Dutcher, Yasaman S. Sefidgar, Woosuk Seo, Michael J. Tumminia, Daniella K. Villalba, Sheldon Cohen, Kasey G. Creswell, J. David Creswell, Afsaneh Doryab, Paula S. Nurius, Eve Riskin, Anind K. Dey, and Jennifer Mankoff. 2021a.  Leveraging Collaborative-Filtering for Personalized Behavior Modeling: A Case Study of Depression Detection among College Students.  _Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies_ 5, 1 (March 2021), 1–27.  https://doi.org/10.1145/3448107
  * Xu et al. (2023) Xuhai Xu, Xin Liu, Han Zhang, Weichen Wang, Subigya Nepal, Yasaman Sefidgar, Woosuk Seo, Kevin S. Kuehn, Jeremy F. Huckins, Margaret E. Morris, Paula S. Nurius, Eve A. Riskin, Shwetak Patel, Tim Althoff, Andrew Campbell, Anind K. Dey, and Jennifer Mankoff. 2023.  GLOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior Modeling.  _Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies_ 6, 4 (2023), 1–34.  https://doi.org/10.1145/3569485
  * Xu et al. (2021c) Xuhai Xu, Jennifer Mankoff, and Anind K. Dey. 2021c.  Understanding practices and needs of researchers in human state modeling by passive mobile sensing.  _CCF Transactions on Pervasive Computing and Interaction_ (July 2021).  https://doi.org/10.1007/s42486-021-00072-4
  * Xu et al. (2022) Xuhai Xu, Han Zhang, Yasaman Sefidgar, Yiyi Ren, Xin Liu, Woosuk Seo, Jennifer Brown, Kevin Kuehn, Mike Merrill, Paula Nurius, Shwetak Patel, Tim Althoff, Margaret E Morris, Eve Riskin, Jennifer Mankoff, and Anind K Dey. 2022\.  GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization. In _Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track_. 18\.
  * Yang et al. (2023a) Kailai Yang, Shaoxiong Ji, Tianlin Zhang, Qianqian Xie, and Sophia Ananiadou. 2023a.  On the Evaluations of ChatGPT and Emotion-enhanced Prompting for Mental Health Analysis.  http://arxiv.org/abs/2304.03347
  * Yang et al. (2023b) Kailai Yang, Tianlin Zhang, Ziyan Kuang, Qianqian Xie, Sophia Ananiadou, and Jimin Huang. 2023b.  MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models.  http://arxiv.org/abs/2309.13567 arXiv:2309.13567 [cs].
  * Zhong et al. (2023) Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, and Dacheng Tao. 2023.  Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert.  _arXiv preprint arXiv:2302.10198_ (2023).
  * Zhou et al. (2023) Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. 2023.  Least-to-Most Prompting Enables Complex Reasoning in Large Language Models.  http://arxiv.org/abs/2205.10625 arXiv:2205.10625 [cs].

## Appendix: Detailed Results Tables

Table 10.  Balanced Accuracy Performance Summary of Zero-shot, Few-shot and
Instruction Finetuning on LLMs. $context$, $mh$, and $both$ indicate the
prompt design strategies of context enhancement, mental health enhancement,
and their combination (see Table. 2). Small numbers represent standard
deviation across different designs of Prompt${}_{\textit{Part1-S}}$ and
Prompt${}_{\textit{Part2-Q}}$. The baselines at the top rows do not have
standard deviation as the task-specific output is static, and prompt designs
do not apply. Due to token limit, computation cost, and resource constraints,
some infeasible experiments are marked as “–”. For each column, the best
result is bolded, and the second best is underlined.

|  Dataset | Dreaddit | DepSeverity | SDCNL | CSSRS-Suicide | Red-Sam | Twt-60Users | SAD
---|---|---|---|---|---|---|---|---
Category | Model | Task #1 | Task #2 | Task #3 | Task #4 | Task #5 | Task #6 | Task #2 | Task #2 | Task #1
Zero-shot Prompting | AlpacaZS | 0.593±0.039 | 0.522±0.022 | 0.431±0.050 | 0.493±0.007 | 0.518±0.037 | 0.232±0.076 | 0.524±0.014 | 0.521±0.022 | 0.503±0.004
AlpacaZS-context | 0.612±0.065 | 0.567±0.077 | 0.454±0.143 | 0.497±0.006 | 0.532±0.033 | 0.250±0.060 | 0.525±0.019 | 0.559±0.064 | 0.501±0.004
AlpacaZS_mh | 0.593±0.031 | 0.577±0.028 | 0.444±0.090 | 0.482±0.015 | 0.523±0.013 | 0.235±0.033 | 0.527±0.006 | 0.569±0.017 | 0.522±0.027
AlpacaZS_both | 0.540±0.029 | 0.559±0.040 | 0.421±0.095 | 0.532±0.005 | 0.511±0.011 | 0.221±0.030 | 0.495±0.016 | 0.499±0.004 | 0.557±0.041
2-11 | Alpaca-LoRAZS | 0.571±0.043 | 0.548±0.027 | 0.437±0.044 | 0.502±0.011 | 0.540±0.012 | 0.187±0.053 | 0.577±0.004 | 0.607±0.046 | 0.477±0.016
| Alpaca-LoRA${}_{ZS_{c}ontext}$ | 0.537±0.047 | 0.501±0.001 | 0.343±0.152 | 0.472±0.020 | 0.567±0.038 | 0.214±0.059 | 0.535±0.017 | 0.649±0.021 | 0.443±0.047
| Alpaca-LoRAZS_mh | 0.500±0.000 | 0.500±0.000 | 0.331±0.145 | 0.497±0.025 | 0.557±0.023 | 0.216±0.022 | 0.541±0.016 | 0.569±0.019 | 0.471±0.033
| Alpaca-LoRAZS_both | 0.500±0.000 | 0.500±0.000 | 0.386±0.059 | 0.499±0.023 | 0.517±0.031 | 0.224±0.049 | 0.507±0.009 | 0.535±0.025 | 0.420±0.019
2-11 | FLAN-T5ZS | 0.659±0.086 | 0.664±0.011 | 0.396±0.006 | 0.643±0.021 | 0.667±0.023 | 0.418±0.012 | 0.554±0.034 | 0.613±0.040 | 0.692±0.093
| FLAN-T5${}_{ZS_{c}ontext}$ | 0.663±0.079 | 0.674±0.014 | 0.378±0.013 | 0.653±0.011 | 0.649±0.026 | 0.378±0.029 | 0.563±0.029 | 0.613±0.046 | 0.738±0.056
| FLAN-T5ZS_mh | 0.616±0.070 | 0.666±0.009 | 0.366±0.012 | 0.648±0.010 | 0.653±0.018 | 0.372±0.033 | 0.547±0.035 | 0.613±0.033 | 0.739±0.039
| FLAN-T5ZS_both | 0.604±0.074 | 0.661±0.004 | 0.389±0.051 | 0.645±0.005 | 0.657±0.019 | 0.382±0.048 | 0.536±0.027 | 0.606±0.040 | 0.767±0.050
2-11 | LLaMA2ZS | 0.720±0.012 | 0.693±0.034 | 0.429±0.013 | 0.589±0.010 | 0.691±0.014 | 0.261±0.018 | 0.574±0.008 | 0.735±0.017 | 0.704±0.026
| LLaMA2${}_{ZS_{c}ontext}$ | 0.658±0.025 | 0.707±0.056 | 0.410±0.019 | 0.588±0.026 | 0.722±0.039 | 0.367±0.043 | 0.562±0.011 |  0.736±0.019 | 0.650±0.027
| LLaMA2ZS_mh | 0.617±0.012 | 0.711±0.033 | 0.395±0.017 | 0.642±0.008 | 0.696±0.021 | 0.291±0.038 | 0.572±0.012 | 0.689±0.056 | 0.567±0.021
| LLaMA2ZS_both | 0.584±0.017 | 0.704±0.036 | 0.444±0.021 | 0.643±0.014 | 0.689±0.043 | 0.328±0.058 | 0.559±0.012 | 0.692±0.069 | 0.560±0.009
2-11 | GPT-3.5ZS | 0.685±0.024 | 0.642±0.017 | 0.603±0.017 | 0.460±0.163 | 0.570±0.118 | 0.233±0.009 | 0.454±0.007 | 0.536±0.024 | 0.717±0.017
| GPT-3.5${}_{ZS_{c}ontext}$ | 0.688±0.045 | 0.653±0.020 | 0.543±0.047 | 0.618±0.008 | 0.577±0.090 | 0.265±0.048 | 0.473±0.001 | 0.560±0.002 | 0.723±0.003
| GPT-3.5ZS_mh | 0.679±0.017 | 0.636±0.021 | 0.642±0.034 | 0.576±0.001 | 0.477±0.014 | 0.310±0.015 | 0.467±0.004 | 0.571±0.000 | 0.664±0.061
| GPT-3.5ZS_both | 0.681±0.010 | 0.627±0.022 | 0.617±0.014 | 0.632±0.020 | 0.617±0.033 | 0.254±0.009 | 0.506±0.004 | 0.570±0.007 | 0.750±0.027
2-11 | GPT-4ZS | 0.700±0.001 | 0.719±0.013 | 0.588±0.010 | 0.644±0.007 | 0.760±0.009 | 0.418±0.009 | 0.434±0.005 | 0.566±0.017 | 0.854±0.006
| GPT-4${}_{ZS_{c}ontext}$ | 0.706±0.009 | 0.719±0.009 | 0.590±0.011 | 0.644±0.011 | 0.753±0.028 |  0.441±0.057 | 0.465±0.010 | 0.565±0.006 |  0.848±0.001
| GPT-4ZS_mh | 0.725±0.009 | 0.684±0.004 | 0.656±0.001 | 0.645±0.012 | 0.737±0.005 | 0.396±0.020 | 0.496±0.005 | 0.527±0.007 | 0.840±0.003
| GPT-4ZS_both | 0.719±0.021 | 0.689±0.000 | 0.650±0.011 | 0.647±0.014 | 0.697±0.005 | 0.411±0.009 | 0.511±0.000 | 0.546±0.014 | 0.837±0.002
Few-shot Prompting | AlpacaFS | 0.632±0.030 | 0.529±0.017 | 0.628±0.005 | — | — | — | — | — | —
FLAN-T5FS | 0.786±0.006 | 0.678±0.009 | 0.432±0.009 | — | — | — | — | — | —
GPT-3.5FS | 0.721±0.010 | 0.665±0.015 | 0.580±0.002 | — | — | — | — | — | —
GPT-4FS | 0.698±0.009 | 0.724±0.005 | 0.613±0.001 | — | — | — | — | — | —
Instructional Finetuning | Mental-Alpaca |  0.816±0.006 |  0.775±0.006 |  0.746±0.005 | 0.724±0.004 | 0.730±0.048 | 0.403±0.029 | 0.604±0.012 | 0.718±0.011 | 0.819±0.006
Mental-FLAN-T5 | 0.802±0.002 | 0.759±0.003 | 0.756±0.001 | 0.677±0.005 | 0.868±0.006 | 0.481±0.006 |  0.582±0.002 | 0.736±0.003 | 0.779±0.002
Baseline | Majority | 0.500±— | 0.500±— | 0.250±— | 0.500±— | 0.500±— | 0.200±— | — | — | —
BERT | 0.783±— | 0.763±— | 0.690±— | 0.678±— | 0.500±— | 0.332±— | — | — | —
Mental-RoBERTa | 0.831±— | 0.790±— | 0.736±— |  0.723±— |  0.853±— | 0.373±— | — | — | —
|  |  |  |  |  |  |  |  |  |