# Analog or Digital In-memory Computing? Benchmarking through Quantitative
Modeling

Jiacong Sun  MICAS, KU Leuven
Leuven, Belgium
<EMAIL_ADDRESS>Pouya Houshmand  MICAS, KU Leuven
Leuven, Belgium
<EMAIL_ADDRESS>Marian Verhelst  MICAS, KU Leuven
Leuven, Belgium
<EMAIL_ADDRESS>
###### Abstract

In-Memory Computing (IMC) has emerged as a promising paradigm for energy-
efficient, throughput-efficient and area-efficient machine learning at the
edge. However, the differences in hardware architectures, array dimensions,
and fabrication technologies among published IMC realizations have made it
difficult to grasp their relative strengths. Moreover, previous studies have
primarily focused on exploring and benchmarking the peak performance of a
single IMC macro rather than full system performance on real workloads. This
paper aims to address the lack of a quantitative comparison of Analog In-
Memory Computing (AIMC) and Digital In-Memory Computing (DIMC) processor
architectures. We propose an analytical IMC performance model that is
validated against published implementations and integrated into a system-level
exploration framework for comprehensive performance assessments on different
workloads with varying IMC configurations. Our experiments show that while
DIMC generally has higher computational density than AIMC, AIMC with large
macro sizes may have better energy efficiency than DIMC on convolutional-
layers and pointwise-layers, which can exploit high spatial unrolling. On the
other hand, DIMC with small macro size outperforms AIMC on depthwise-layers,
which feature limited spatial unrolling opportunities inside a macro.

###### Index Terms:

Machine learning, quantitative modeling, analog in-memory computing, digital
in-memory computing

©2023 IEEE. Personal use of this material is permitted. Permission from IEEE
must be obtained for all other uses, in any current or future media, including
reprinting/republishing this material for advertising or promotional purposes,
creating new collective works, for resale or redistribution to servers or
lists, or reuse of any copyrighted component of this work in other works. DOI:
10.1109/ICCAD57390.2023.10323763.

##  I Introduction

Figure 1: 8-nested loop DNN layer representation and workloads representation
Figure 2: Fundamental architecture for AIMC/DIMC and their mapping paradigm.
”u” in the subscript represents spatial unrolling.

Recent developments of ultra-low power machine learning models have enabled
the deployment of artificial intelligence on extreme edge devices. However
typical embedded digital accelerators suffer from high data movement costs and
low computational densities, degrading the energy efficiencies by up to
2$\times$-1000$\times$ with respect to the ideal baseline of digital
computations. To minimize the data transfer overhead, in-memory computing
(IMC) has recently emerged as a promising alternative to conventional
accelerators based on arrays of digital processing elements (PEs). By directly
performing the operations near/in the memory cells, these architectures allow
to greatly reduce access overheads and enable massive parallelization
opportunities, with potential orders of magnitude improvements in energy
efficiency and throughput [1]. Most of the initial IMC designs published in
the literature are focused on analog IMC (AIMC), where the computation is
carried out in the analog domain [2, 3, 4]. While this approach ensures
extreme energy efficiencies and massive parallelization, the analog nature of
the computation and the presence of intrinsic circuit noise and mismatches
compromises the output accuracy. To avoid the hurdles of AIMC, digital in
memory computing (DIMC) is lately gaining more interest as a valid alternative
[5, 6, 7, 8]. Thanks to its noise-free computation and more flexible spatial
mapping possibilities, DIMC allows to realize additional flexibility and
accurate computation at the expense of reduced peak energy efficiency. These
new opportunities stemming from AIMC and DIMC resulted in many recent
implementations and publications in the literature. However, these works vary
strongly in terms of hardware architecture, array dimensions, and silicon
technology. This makes it difficult to grasp their relative strengths. While
several works assess and discuss IMC trends qualitatively, only few aim at
quantitatively modeling or benchmarking architectural strategies. This paper
tries to fill this gap by:

  * •

providing a unified analytical performance model for AIMC and DIMC, which is
validated against several design points from the literature.

  * •

performing quantitative performance benchmarking for various configurations of
AIMC and DIMC, in terms of macro-level peak performance as well as system-
level workload performance, by integrating the proposed model into ZigZag [9],
a system-level hardware design space exploration (DSE) framework.

##  II Background and related works

###  II-A Dataflow concepts for IMC

Deep neural network (DNN) workloads consist of a sequence of layers. The
operations in the most common layers can be described as a combination of
8-nested for loops, which iterate over the indices of an input feature map
tensor $I$, a weight tensor $W$ and that generate an output tensor $O$, as
shown in Fig. 1. The tensor operations can be decomposed in a sequence of
matrix-vector multiplications (MVM) by tiling the suitable loops. These MVM
operations offer a great opportunity for in-memory acceleration, as their
dense 2D array structure aligns well with the array structure of the memory
macros. Due to the physical properties of the hardware template, the spatial
unrolling dimensions of the weight matrix are chosen so as to maximize spatial
reuse of the activations along the columns of the memory, and accumulation of
the partial sums along the rows of the IMC array, as shown in Fig. 2: The $K$
and $OX$ loops – (partially) irrelevant for the inputs – are typically
unrolled across the columns, while the $C$, $FX$ and $FY$ loops – irrelevant
for the outputs – are parallelized across the rows. To further speed up
computation and increase the reuse of operands at chip-level, the $OX$, $OY$
or $G$ loops can be parallelized across multiple IMC macros on the same die,
requiring, however, duplication of the weights [9]. It is nevertheless
necessary to carefully align the array dimensions and the workload parallelism
to minimize underutilization of the computational resources.

###  II-B SRAM-based IMC

SRAM is largely deployed for IMC, thanks to its robustness, large scale
integration in CMOS, high endurance, and reliability, when compared to NVM-
based solutions. SRAM-based IMC designs can be categorized into AIMC and DIMC
based on whether the MAC computation is carried out in the analog or digital
domain respectively.

####  II-B1 AIMC architectures

The AIMC architecture template is depicted in Fig. 2.a. To minimize data
transfers between the compute array and the external memory levels and to
maximize the array utilization, a weight stationary dataflow is commonly
adopted for IMC designs. This dataflow aims at minimizing the data movement
related to weights, maximizing their reuse at the computational array level:
in IMC architectures this translates in pre-loading $B_{w}$-bit weights into
the array, where $M$ memory cells are grouped and attached on the local
bitline to increase memory density. To execute an MVM, a digital input vector
is provided along the wordlines of the memory array; the values of each
element of the vector are converted to the analog domain with DACs – either as
a pulse width [2] or a voltage level [3, 4] – and the resulting signals
propagated along all wordlines in parallel. The analog signal on the wordlines
is then combined with the value stored in the activated SRAM cells performing
one multiplication per cell. The result of each multiplication will be
transmitted onto the bitlines, where accumulation occurs in the analog domain
across all cells connected to the same bitline. The final value on the
bitlines is then converted back to the digital domain by means of ADCs, stored
in output registers, after which it flows back to the higher level memories.

####  II-B2 DIMC architectures

In contrast to AIMC, the multiplication and accumulation operation in DIMC [5,
6, 7, 8] are implemented with digital logic gate-based multiplier and adder
circuitry, as shown in Fig. 2.b. IMC multiplication is done at SRAM cell
level, where $M$ memory cells share one multiplying NAND gate. The digital MAC
results can immediately be offloaded to output registers after accumulating
the full-precision results in digital accumulators. The presence of extra
logic at cell level and the adder trees in DIMC give rise to area overheads
and lower peak energy efficiencies compared to its analog counterpart; yet,
increased dataflow flexibility can be attained: the spatial loop unrolling at
macro level can be made reconfigurable by means of multiplexers in the
accumulation trees with low energy and area overheads. A further benefit of
DIMC lies in its robustness against noise and variability, enabling high
operand precision. Thanks to its purely digital signal processing, DIMC
computes deterministically, while the energy and area cost of DIMC scale
proportionally with the operand precision. However, in literature the trade-
off between efficiency and increased flexibility is not quantified on actual
workloads and a back-to-back comparison of these two IMC paradigms is
necessary to identify the optimal design choice.

Figure 3: Peak performance benchmarking on AIMC/DIMC architectures. Each point
reports the used technology node and weight bit precision.

###  II-C IMC design benchmarking

Numerous implementations for AIMC and DIMC exist in literature, however they
differ in their hardware architectures, array dimensions, and fabrication
technologies, making it challenging to grasp their relative strengths [10, 11,
12, 1, 13, 14]. More importantly, a fundamental concern in the reported
performance metrics is that IMC chips are mostly benchmarked according to
their peak performance for a single IMC macro, as summarized in Fig. 3.
However, these performance assessments unrealistically suppose that the (in-
memory) computing logic is fully utilized at all times: to evaluate the
performances on actual workloads, DSE frameworks and cost models have been
implemented to explore the design space and design the hardware based on the
target network model.

###  II-D DSE framework and modeling tools

In order to evaluate the two IMC design paradigms at system level on target
workloads design space exploration at the system level can be done through
analytical models for IMC. Published models however primarily focus on AIMC
designs [15, 16, 1, 17, 18, 19, 20, 21], while there is a lack of DIMC
modeling efforts. Similarly, for mapping space explorations, most of the focus
has been dedicated to AIMC designs, while lacking DIMC assessment [22, 18, 19,
23]. In particular, the work in [15] offers analytical energy and area
modeling of AIMC ADCs and DACs, but the model is only validated on standalone
ADCs and DACs rather than actual IMC implementations. An alternative energy
model is proposed in [16], however, the energy cost of each component is
separately extracted from different papers, and has not been validated against
a complete taped-out chip.

Fully digital ML processors, in contrast, have been the subject of many
system-level design space explorations: dedicated DSE frameworks supporting
hardware-mapping co-optimization have emerged over the last years [9, 24, 25],
with different levels of supported parametrization of the hardware
architecture space. Yet, so far none of these frameworks have been deployed to
extensively assess AIMC as well as DIMC topologies, due to a lack of dedicated
cost modeling efforts. To address this gap, we present an analytical IMC cost
model in the following section.

##  III Unified analytical IMC performance model

TABLE I: Cost summary for components in a single IMC macro

Component | Energy | Delay | Area ($\text{\,}\mathrm{\SIUnitSymbolMicro m}$2)
---|---|---|---
| AIMC | DIMC | AIMC | DIMC | AIMC | DIMC
Cell array | $0.5C_{\text{gate}}V_{\text{DD}}^{2}B_{w}\cdot D_{i}\cdot D_{o}$ | 0 | 0 | CACTI [26]
ADC | $(k_{1}\text{ADC}_{\text{res}}+k_{2}4^{\text{ADC}_{\text{res}}})V_{\text{DD}}^{2}$ | – | $(k_{3}\cdot D_{i}+k_{4})\text{ADC}_{\text{res}}$ | – | $10^{-k_{5}\text{ADC}_{\text{res}}+k_{6}}\cdot 2^{\text{ADC}_{\text{res}}}$ | –
DAC | $k_{7}\cdot\text{DAC}_{\text{res}}V_{\text{DD}}^{2}$ | – | 0 | – | 0 | –
1b multiplier | $0.5C_{\text{gate}}V_{\text{DD}}^{2}$ | $D_{\text{gate}}$ | $A_{\text{gate}}$
Adder tree | $E_{\text{FA}}\cdot\\#\text{FA}$ | $D_{\text{FA}_{S}}log_{2}D_{i}+D_{\text{FA}_{C}}B_{\text{adds\\_out}}$ | $A_{\text{FA}}\cdot\\#FA$
Accumulator | $(E_{\text{FA}}+E_{\text{DFF}})B_{\text{acc}}$ | $D_{\text{FA}_{C}}(B_{\text{acc}}-B_{\text{adds\\_out}})$ | $(A_{\text{FA}}+A_{\text{DFF}})B_{\text{acc}}$
1b register | $E_{\text{DFF}}$ | $0$ | $A_{\text{DFF}}$

$C_{\text{gate}},D_{\text{gate}},A_{\text{gate}}$: capacitance, delay, area of single NAND2 gate |  $B_{i},B_{w},B_{o}$: input, weight, output precision in bit
---|---
$V_{\text{DD}},D_{i},D_{o}$: supply voltage, size of input dimension, output dimension |  $\text{ADC}_{\text{res}},\text{DAC}_{\text{res}}$: ADC resolution, DAC resolution
$B_{\text{adds\\_in}}$, $B_{\text{adds\\_out}}$, $B_{\text{acc}}$: input precision of an adder tree, output precision of an adder tree, output precision of an accumulator |  $E_{\text{FA}},D_{\text{FA}_{S}},D_{\text{FA}_{C}},A_{\text{FA}},\\#$FA: energy/sum-delay/carry-delay/area of FA, number of FA in an adder tree

TABLE II: Parameters in proposed cost model @ 0.9V, 28nm Parameter | Value | Parameter | Value
---|---|---|---
$C_{\text{gate}}$ | 0.7 fF | k6 | 1.206
$D_{\text{gate}}$ | 47.8 ps | k7 | 50 fF
$A_{\text{gate}}$ |  $0.614\text{\,}\mathrm{\SIUnitSymbolMicro m}$2 | $E_{\text{FA}}$ | 6$C_{\text{gate}}V_{\text{DD}}^{2}$
k1 | 100 fF [1] | $D_{\text{FA}_{S}}$ | 4.8$D_{\text{gate}}$
k2 | 1 aF [1] | $D_{\text{FA}_{C}}$ | 2$D_{\text{gate}}$
k3 | 6.53 ps | $A_{\text{FA}}$ | 7.8$A_{\text{gate}}$
k4 | 640 ps | $E_{\text{DFF}}$ | 3$C_{\text{gate}}V_{\text{DD}}^{2}$
k5 | 0.0369 | $A_{\text{DFF}}$ | 6$A_{\text{gate}}$

The fundamental structure of an IMC accelerator is constant across different
implementations: a computational memory array with an interface to read and
write data from. However, large variability exists in the implementation
details of each work. While it is a daunting task to develop an accurate
unified analytical cost model that matches all designs from literature, it is
possible to create a model within an acceptable mismatch for system-level
evaluation, enabling us to identify the main bottlenecks and the benefits of
various architectural choices.

To describe the granularity of the computational parallelization options in
the IMC array, the presented model represents the array along a set of
dimensions, as depicted in Fig. 2: $D_{i}$ refers to the parallelism of the
input activation vector, while $D_{o}$ refers to the parallelism of the output
vector generated after each MVM operation. Each element along $D_{o}$ is
further split into separate cells based on the weight bitwidth $B_{w}$;
similarly, each element along the $D_{i}$ dimension corresponds to $M$ cells,
both in AIMC [3, 27, 28] and DIMC [6, 7, 8]. Moreover, the MVM operation can
be temporally split in multiple cycles, in which $B_{\text{cycle}}$ bits are
processed per cycle per input.

To estimate area, energy and delay for both DIMC and AIMC macros, a unified
analytical cost model is developed for these architectures, to enable
performance evaluation at the system level for different design points. The
model takes into account memory instances, analog peripherals, and digital
peripherals for the energy, area and latency estimation, while the cost of
each component is shown in Table I, together with its calibration parameters
for 28nm. The cost models for AIMC and DIMC for each of these components will
be further detailed in the next subsections.

Figure 4: Equivalent worst case scenario for ADC in AIMC macro. The transistor
or transmission gate in each IMC cell connected to the bitline can be treated
as a resistor and a capacitor. The worst case happens when only one cell is
charging or discharging all capacitors.

###  III-A Memory instances: IMC cell array and registers

Memory instances refer to the set of storage elements in the IMC macro: these
are the weight memory cells in the computational array, as well as the input
and output registers. Assuming that the weight memory elements in the macro
are based on 6T SRAM cells, CACTI [26] is utilized to estimate the area, delay
and energy per access of the IMC cell array. The CACTI model, based on an RC
model, is further validated with synthesized memory macros via a memory
compiler. However the CACTI model does not correctly cover the MVM computation
in IMCs since multiple wordlines activate simultaneously for AIMC and there is
no significant bitline activity for DIMC during the computation, behaviours
that do not correspond to normal SRAM operation. Given this, a separate model
is adopted for IMC MVM array computation. For AIMC, assuming that energy is
mostly consumed in the charge and discharge phases of the bitlines, the total
energy cost on bitline switching (E${}_{\text{cell array}}$) is expressed as:

$E_{\text{cell array}}=C_{\text{cell}}V_{\text{DD}}^{2}B_{w}\cdot D_{i}\cdot
D_{o}$ (1)

Two transistors seen by the bitline for each bitcell, $C_{\text{cell}}$ is
normalized to 0.5$C_{\text{gate}}$, where $C_{\text{gate}}$ represents the
gate capacitance of a four-transistor NAND2/NOR2 gate. In DIMC, with its very
short bitlines, this cost is negligible.

Figure 5: $B_{\text{adds\\_in}}$-bit adder tree topology for A/DIMC (right).
$B_{\text{adds\\_in}}=\text{ADC}_{\text{res}}$ for AIMC and
$B_{\text{adds\\_in}}=B_{w}$ for bit-serial DIMC. The structure of each node
(single-stage adder) assuming RCA type is shown in the top left. The gate-
level circuit for 1b FA is shown in the bottom left.

The delay of the signal traversal on the bitline is merged into the ADC delay
for AIMC. As DIMC has negligible delay on the local bitline, the delay cost
attached to the cell array can be approximated to 0 for the MAC computation.
The energy and area cost of input and output registers are modeled as D Flip-
Flop (DFF) standard cells, using the relative cost ratio of a DFF to a NAND2
gate (Table I). The clock-to-Q delay $D_{\text{DFF}}$ is negligible compared
to other peripheral contributions.

###  III-B Analog peripherals: ADCs and DACs

AIMC requires DACs and ADCs to interface with the computational array to
initiate/terminate the array operation. Their contributions are described in
this section.

####  III-B1 ADC model

The largest cost in AIMC designs are usually associated with ADCs and is
strongly dependent on the ADC resolution. Following [1] and assuming
uncorrelated and uniformly distributed activation and weights, the required
ADC resolution is determined by the activation precision $B_{i}$ and the
amount of accumulation terms $D_{i}$:

$\text{ADC}_{\text{res}}=B_{i}+\log_{2}(k\cdot\text{FS}\cdot\sqrt{D_{i}})$ (2)

with FS$=0.5$ (as in [1]) representing the ADC’s normalized full-scale range
occupying 50% of the entire voltage range; the empirical constant $k=2$
represents the ADC noise deviation at the output to be $k$ times smaller than
the accumulation error variance on the bitline. Using the resolution of Eq.
(2) an energy, latency, and area model is developed assuming a SAR-ADC
topology, the most common ADC topology deployed for AIMC designs thanks to its
small area footprint and low energy consumption. The energy model, based on
[1], is defined as:

$E_{\text{ADC}}=(k_{1}\text{ADC}_{\text{res}}+k_{2}4^{\text{ADC}_{\text{res}}})V_{\text{DD}}^{2}$
(3)

with constants $k_{1}$ and $k_{2}$ obtained by fitting to the most energy
efficient ADC designs, dominated by SAR-ADCs [29]. The ADC delay cost is
expressed as the sum of 1) the settling time of the analog value on the
bitline $t_{\text{set}}$ and 2) the time required for the ADC conversion
$t_{\text{conv}}$. The first contribution can be described as the delay of a
parallel RC network at the ADC input node (Fig. 4). The time required for the
signal on the bitline to settle within 1 LSB of the ADC is given in Eq. (4),
with the RC time constant $\tau=D_{i}\cdot R_{\text{cell}}C_{\text{cell}}$.
The model assumes the worst case scenario for $t_{\text{set}}$, where only one
cell is active on the entire bitline.

$\begin{split}&V_{\text{DD}}(1-e^{-t_{\text{set}}/\tau})<V_{\text{DD}}(1/2^{\text{ADC}_{\text{res}}})\\\
&\Rightarrow t_{\text{set}}>0.69\tau\text{ADC}_{\text{res}}\end{split}$ (4)

The second contribution, the time required for the ADC conversion, is defined
as in Eq. (5), where the delay for a single bit conversion
$t_{\text{conv/bit}}$ is a constant that depends on the technology node.

$t_{\text{conv}}=t_{\text{conv/bit}}\cdot\text{ADC}_{\text{res}}$ (5)

By combining Eq. (4) and Eq. (5), the ADC delay can be modeled in Eq. (6),
where $k_{3}=0.69R_{\text{cell}}C_{\text{cell}}$ reflects the impact of the
number of cells attached to the bitline and $k_{4}=t_{\text{conv/bit}}$
represents the intrinsic conversion delay per bit for SAR ADC. Table I lists
the used parameters $k$, fitted for 28nm technology.

$D_{\text{ADC}}=(k_{3}\cdot D_{i}+k_{4})\cdot\text{ADC}_{\text{res}}\ $ (6)

Based on [15], the ADC area is estimated as:

$A_{\text{ADC}}=10^{-k_{5}\text{ADC}_{\text{res}}+k_{6}}\cdot
2^{\text{ADC}_{\text{res}}}$ (7)

####  III-B2 DAC model

The delay and area costs of the DACs are neglected, as they have relatively
small latency overheads and area [2]. The DAC energy cost is modeled using Eq.
(8), where $k_{7}$ is a technology constant that represents the capacitance
associated with each bit conversion.

$E_{\text{DAC}}=k_{7}\text{DAC}_{\text{res}}V_{\text{DD}}^{2}\ $ (8)

###  III-C Digital peripherals

####  III-C1 Digital multiplication

In both AIMC and DIMC, the multiplication operation is modeled at the cell
level as (one or more) 1-bit operation(s) using a NAND gate between the 1-bit
input and 1-bit weight stored in the cell. Hence, the cost of such 1-bit
multiplier is equivalent to the cost of a single gate, as listed in Table I.

####  III-C2 Digital accumulation

Both AIMC and DIMC use adder trees to combine digital outputs and get
eventually partial sums across separate cycles. In the proposed model, these
partial sum adder trees and accumulators are modeled assuming ripple-carry
adders (RCA) (Fig. 5). The input precision $B_{\text{adds\\_in}}$ equals
weight resolution $B_{w}$, $\text{ADC}_{\text{res}}$ for DIMC, AIMC
respectively (Fig. 2). The cost of the adder trees is expressed as a function
of its basic unit, the Full Adder (FA). The circuit of a standard FA is shown
in Fig. 5, consisting of three NAND2 gates and two XOR2 gates, with its cost
computed as listed in Table I. The total energy and area of an adder tree
depends on the number of FA, which can be expressed as below:

$\\#\text{FA}=\sum^{\log_{2}D_{i}}_{n=1}(B_{adds\\_in}+n-1)\frac{D_{i}}{2^{n}}$
(9)

The delay of the adder tree $D_{\text{adds}}$ depends on the number of FA on
the critical path, as shown in Fig. 5. This component can be modeled as in Eq.
(10), where the output precision of the adder tree
$B_{\text{adds\\_out}}=B_{\text{adds\\_in}}+\log_{2}D_{i}$.

$D_{\text{adds}}=D_{\text{FA}_{S}}\log_{2}D_{i}+D_{\text{FA}_{C}}B_{\text{adds\\_out}}$
(10)

Finally, the cost of the accumulator can be derived from the cost of the FA
and a group of DFFs. The costs of the RCA-based adder trees and accumulators
for both AIMC and DIMC macros are all summarized in Table I.

##  IV Macro-level model validation

The model presented in Section III is validated against the area, delay, and
energy consumption reported of a collection of taped-out IMC chips for both
AIMC and DIMC architectures (Fig. 6). To decouple the impact of the chosen
technology node from the performance and efficiency metrics, only published
IMC implementations in 22nm and 28nm technology, have been considered. Table
III lists the considered implementations, together with their hardware
configuration. The following paragraphs will analyze the validation results in
more detail.

\- [2]: reported area includes repeaters, decap, et.al.
---
\- [2, 4]: area and energy estimation has been linearly scaled from 28nm to
22nm.
\- [3, 6, 8]: pipeline inserted before or in the middle of the adder tree,
which decreases the delay by 0.5$\times$.
\- 50% input toggle rate and 50% weight sparsity are assumed in model
validation. These values are clearly reported in [2, 4, 5, 7], yet sparsity is
not reported in other papers.

Figure 6: Macro-level validation of proposed IMC cost model against 28nm/22nm
implementations. Gray bars: published metrics; Colored bar: model estimation.

###  IV-A AIMC macro-level model validation

The validation of the model against 22-28nm AIMC designs was carried out on
[2, 3, 4]. The IMC arrays of these works all consist of 6T SRAM cells followed
by SAR ADCs.

For the area validation the model shows a good match, except for [2], which
extensively uses repeaters in their memory banks. This results in a
significantly larger(4.5$\times$) SRAM array compared to the CACTI estimation.
With the extracted Eq. (6), the delay validations match well. And finally, for
the energy estimation a maximum mismatch of 11% is achieved, except for [3],
which is believed to be caused by difference in sparsity used in model and in
their chip measurements. Unfortunately, the used sparsity is not reported in
[3].

TABLE III: Hardware configurations of the validation works  Index | IMC | $\bm{B_{i}/B_{w}/B_{cycle}}$ | $\bm{D_{i}}$ | $\bm{D_{o}}$ | $\bm{M}$ | $\bm{\\#\text{macros}}$
---|---|---|---|---|---|---
[2] | AIMC | 7/2/7 | 1024 | 512 | 1 | 1
[3] | AIMC | 8/8/2 | 16 | 12 | 32 | 1
[4] | AIMC | 8/8/1 | 64 | 256 | 1 | 8
[5] | DIMC | 8/8/2 | 32 | 6 | 1 | 64
[6] | DIMC | 8/8/1 | 32 | 1 | 16 | 2
[7] | DIMC | 8/8/2 | 128 | 8 | 8 | 8
[8] | DIMC | 8/8/1 | 128 | 8 | 2 | 4

###  IV-B DIMC macro-level model validation

The validation of the model against DIMC designs was carried out on [5, 6, 7,
8], which are all implemented in 28nm technology with 6T SRAM cells and all
employ the digital addition operation with conventional RCA structures.
Although different configurations are adopted in these implementations (i.e.,
different $M$ and $\\#$macros), the model shows a good area estimation for [5]
and [7], while it underestimates [6], which contains additional logic to
support adder tree configurability, and overestimate [8], which introduces
approximate multipliers for increased efficiency. In terms of delay, the model
matches well, with some mismatch for [5], due to delay in its unmodeled Booth
Encoder, and for [6], due to its configurable adder tree topology. For the
energy validation, the model estimation excellently fits the design of [5, 7]
which clearly reported their sparsity in the publication. [6, 8] did not
report the used sparsity in their measurements. Our 50% sparsity assumption
seems to be significantly off from their unreported measurement settings.

Although mismatches can be observed whenever special approximate computing or
extreme sparsity techniques are used, the mismatch is limited to 20% for
standard implementations. It can therefore serve as an excellent vehicle to
assess the impact of various architectural parameters on system-level
performance under peak/actual workloads in the next section.

##  V System-level benchmarking for IMC

The developed IMC analytical cost model can now be used to perform a set of
architecture explorations to aid in design decisions. In this section, this
will be pursued in the context of IMC architectures for edge devices. The
focus will first be at macro level, where peak IMC efficiency will be
assessed, together with its scalability with array dimensions. This will be
contrasted with peak system performance – i.e. also taking data transfers and
activation memory area into account – causing a shift in the optimal hardware
configurations (Subsection V.A). In the second subsection, actual workload
system performance will be analyzed on MLPerf Tiny benchmark models [30],
studying how it differs from peak system performance across various workloads
and the resulting optimal hardware configurations (Subsection V.B). For these
system-level assessments, the derived cost models are integrated into a
modified version of the system-level design space exploration tool ZigZag [9].
We perform this study based on a single IMC macro as shown in Fig. 2, with
$M=1$ and assuming a 256 KB on-chip cache, which is equipped with a bandwidth
fitting with IMC dimension and whose cost is estimated by CACTI [26]. All
assessments are carried out assuming INT8 operations, being the preferred
quantization scheme in MLPerf Tiny models [30] with zero sparsity.
$B_{\text{cycle}}$ is fixed to 2 and 1 for AIMC and DIMC, respectively, in
light of their popularity in literature.

###  V-A Peak performance assessment for A/DIMC designs

Peak performance is the starting metric for evaluation as it provides a
roofline for the computational capabilities of the design. In literature this
is often the preferred way to quantify the efficiency of an IMC architecture,
even though it does not provide a clear overview of the actual workload
efficiency of the design [31] (see Subsection V.B). All assessments in this
section assume dense GeMM workloads filling the complete array with infinite
weight reuse (weights remain always stationary).

##### Macro efficiency

Fig.7 shows the peak macro performance (excluding the cost of on-chip memories
and off-chip DRAM) in function of IMC array size. The peak energy efficiency
of AIMC is clearly depending on the amortization of its peripherals. An order
of magnitude improvement is achieved by going from a 32$\times$32 array to a
1024$\times$1024 array, thanks to the reduction of the contribution of ADC/DAC
conversions (Fig.8.a). However, to preserve SNR at the analog output for these
larger array sizes, ADCs with higher resolution are required, which in turn
require longer conversion times and larger area overheads. This leads to
degraded throughput and area, which affect the computational density in terms
of TOP/s/mm2 (Fig.8.b). DIMC energy efficiency, on the other hand, does not
benefit from larger array sizes, since the cost of the adder trees and
multipliers scale proportionally with the array size. While DIMC with a
smaller size ($\leq 64\times 64$) outperforms AIMC in terms of TOP/s/W this is
no longer true for larger IMC arrays, in which the ADC overhead is better
amortized. The peak macro computational density for DIMC marginally decrease
with array size due to the logarithmic increase of the depth of the adder
trees. In general, DIMC consistently outperforms AIMC in terms of
computational density, as the delay of the ADCs is worse than that of digital
adder trees, enabling it to operate at a higher clock speed.

##### System efficiency

Fig.8 displays, next the the peak macro efficiency, also the peak system
efficiency, including the cost of feeding data to/from on-chip feature
memories and off-chip DRAM. For the 32$\times$32 scenario, the system energy
efficiency is at least 2$\times$ below the macro efficiency. The gap between
system and macro efficiency, however, diminishes when the array size increases
thanks to the memory access cost amortization from multicasting the input and
output operands across a larger array. Computational intensity (TOP/s/mm2) is
even more affected: for smaller IMC sizes, the cache area dominates the total
system area and reduces the TOP/s/mm2 by more than one order of magnitude.
These trends can also clearly be observed from the energy, delay and area
breakdowns at system level in Fig. 8.

The takeaway might seem that larger arrays lead to overall better system
efficiency. Yet, a massive array is useless if only a small portion of it is
effectively utilized by actual workloads, since it leads to the nullification
of any peripheral amortization. It is therefore necessary to study the
efficiency under actual workload scenarios, as done in the next subsection.

Figure 7: Peak macro performance and peak system performance with scaling IMC
sizes  Figure 8: Peak system cost breakdown with scaling IMC sizes

###  V-B Actual workload performance for A/DIMC

Efficiency and performance metrics are estimated at system level on the MLPerf
Tiny benchmark models, which exhibit a nice range of different layer
topologies (Fig. 9).

Figure 9: Operator breakdown of MLPerf Tiny [30] benchmark models  Figure 10: Single layer system performance with scaling IMC sizes TABLE IV: Layer parameters Type (source) | $G$ | $OX$ | $OY$ | $K$ | $C$ | $FX$ | $FY$
---|---|---|---|---|---|---|---
fc (AutoEncoder) | 1 | 1 | 1 | 128 | 640 | 1 | 1
pw (MobileNet_v1) | 1 | 12 | 12 | 64 | 64 | 1 | 1
dw (DS-CNN) | 64 | 25 | 5 | 1 | 1 | 3 | 3
conv (ResNet8) | 1 | 32 | 32 | 16 | 16 | 3 | 3
Figure 11: Single layer energy breakdown with scaling IMC sizes

While for peak performance the main concern is to amortize the overhead of the
peripherals, here the efficiency will strongly depend on 1) the effective
workload utilization of the IMC array; 2) the amount of spatial reuse
(multicasting) of the activations; and 3) the cost of loading the weights into
the weight stationary array (from DRAM @3.7 pJ/bit [32]), which is no longer
neglected in this workload study. All these factors depend on the applied
spatial and temporal mapping. The ZigZag mapper [9] is utilized to optimize
the deployed dataflow for each assessed architecture and workload
combinations.

To dissect the actual workload efficiencies, the assessment starts by
evaluating how individual layer types impact hardware efficiency. Fig. 10
compares the energy and throughput efficiencies of various layer types,
specified in Table IV, against peak system efficiency. Energy efficiency is
higher for those layers that exhibit layer shapes that can best be spatially
unrolled along the array dimensions and do not require frequent weight
updates. This is the case for pointwise (pw) and convolutional layers (conv).
Depthwise (dw) and fully connected (fc) layers, in contrast, are less well
suited for IMC operation, as visualized in the energy breakdowns of Fig. 11:
fc layers lack weight reuse, requiring a costly weight update after each MVM
operation. dw layers operate with a single kernel on a single input channel,
resulting in limited unrolling possibilities, even when assuming OX unrolling
along the Do dimension [33]. To identify the most energy efficient array size
on a layer-basis, we can look at the in/out unrolling ratio, which indicates
the fraction of the spatial reuse of the in/outputs of a layer, compared to
the maximum reuse possible for that layer. This is visualized in Fig. 11,
clearly illustrating the most efficient configuration as the first point where
the maximum ratio is achieved. Larger array sizes exceed the requirement of
the workload, increasing the fJ/MAC, Tclk and area without increasing
effective throughput.

Figure 12: Workload system performance with scaling IMC sizes

Finally, the evaluation is broadened to cover entire networks from the MLPerf
Tiny suite, as shown in Fig. 12. The results reflect the previous conclusions:
those models that majorly consist of conv and pw layers perform better than
those dominated by dw and fc layers. A distribution of the workload types for
each model can be found in Fig. 9. Resnet8, entirely consisting of
convolutional layers, has the highest weight reuse opportunity and therefore
achieves a TOP/s/W closest to the peak TOP/s/W. The fc-heavy Autoencoder
suffers from frequent weight reloading, while MobileNet and DS-CNN suffer from
severe macro under-utilization for big macro sizes. Ultimately, a geometric
mean of the performance across all 4 MLPerf Tiny models is provided. Compared
to AIMC, DIMC has a higher workload TOP/s/mm2 on average, while its energy
efficiency is similar at the system level.

##  VI Conclusion

The wide diversity in AIMC and DIMC architectures and silicon technologies
published in literature makes it hard to fairly assess their optimal
configurations and relative strengths. To this end, this paper proposed a
modeling approach for SRAM-based IMC and conducted a comprehensive performance
evaluation across various array sizes. The performance was assessed at macro
level and at system level, both for peak and actual workloads. Though sparsity
impact on the energy is not considered, we clearly show the importance of
evaluating IMC system-level performance on real workloads, as higher peak
performance does not always lead to higher effective performance. Our model
allows further quantitative comparisons across a broad set of AIMC and DIMC
processor system architectures, aiding designers in making more informed
decisions during the early phase of IMC architecture selection for their
applications. The derived cost models and system level benchmarking framework
are provided open source at https://github.com/KULeuven-MICAS/zigzag-imc.

##  VII Acknowledgements

We want to thank Boris Murmann for his suggestions on AIMC modeling. This
project has been partly funded by the European Research Council (ERC) under
grant agreement No. 101088865, the European Union’s Horizon 2020 programme
under grant agreement No. 101070374, the Flanders AI Research Program and KU
Leuven.

## References

  * [1] B. Murmann, “Mixed-signal computing for deep neural network inference,” _IEEE Transactions on Very Large Scale Integration (VLSI) Systems_ , vol. 29, no. 1, pp. 3–13, 2021.
  * [2] I. A. Papistas, S. Cosemans, B. Rooseleer, J. Doevenspeck, M.-H. Na, A. Mallik, P. Debacker, and D. Verkest, “A 22 nm, 1540 top/s/w, 12.1 top/s/mm2 in-memory analog matrix-vector-multiplier for dnn acceleration,” in _2021 IEEE Custom Integrated Circuits Conference (CICC)_ , 2021, pp. 1–2.
  * [3] J.-W. Su, Y.-C. Chou, R. Liu, T.-W. Liu, P.-J. Lu, P.-C. Wu, Y.-L. Chung, L.-Y. Hong, J.-S. Ren, T. Pan, C.-J. Jhang, W.-H. Huang, C.-H. Chien, P.-I. Mei, S.-H. Li, S.-S. Sheu, S.-C. Chang, W.-C. Lo, C.-I. Wu, X. Si, C.-C. Lo, R.-S. Liu, C.-C. Hsieh, K.-T. Tang, and M.-F. Chang, “A 8-b-precision 6t sram computing-in-memory macro using segmented-bitline charge-sharing scheme for ai edge chips,” _IEEE Journal of Solid-State Circuits_ , vol. 58, no. 3, pp. 877–892, 2023.
  * [4] P. Chen, M. Wu, W. Zhao, J. Cui, Z. Wang, Y. Zhang, Q. Wang, J. Ru, L. Shen, T. Jia, Y. Ma, L. Ye, and R. Huang, “7.8 a 22nm delta-sigma computing-in-memory ($\delta\sigma$cim) sram macro with near-zero-mean outputs and lsb-first adcs achieving 21.38tops/w for 8b-mac edge ai processing,” in _2023 IEEE International Solid- State Circuits Conference (ISSCC)_ , 2023, pp. 140–142.
  * [5] F. Tu, Y. Wang, Z. Wu, L. Liang, Y. Ding, B. Kim, L. Liu, S. Wei, Y. Xie, and S. Yin, “A 28nm 29.2tflops/w bf16 and 36.5tops/w int8 reconfigurable digital cim processor with unified fp/int pipeline and bitwise in-memory booth multiplication for cloud deep learning acceleration,” in _2022 IEEE International Solid- State Circuits Conference (ISSCC)_ , vol. 65, 2022, pp. 1–3.
  * [6] B. Yan, J.-L. Hsu, P.-C. Yu, C.-C. Lee, Y. Zhang, W. Yue, G. Mei, Y. Yang, Y. Yang, H. Li, Y. Chen, and R. Huang, “A 1.041-mb/mm2 27.38-tops/w signed-int8 dynamic-logic-based adc-less sram compute-in-memory macro in 28nm with reconfigurable bitwise operation for ai and embedded applications,” in _2022 IEEE International Solid- State Circuits Conference (ISSCC)_ , vol. 65, 2022, pp. 188–190.
  * [7] A. Guo, X. Si, X. Chen, F. Dong, X. Pu, D. Li, Y. Zhou, L. Ren, Y. Xue, X. Dong, H. Gao, Y. Zhang, J. Zhang, Y. Kong, T. Xiong, B. Wang, H. Cai, W. Shan, and J. Yang, “A 28nm 64-kb 31.6-tflops/w digital-domain floating-point-computing-unit and double-bit 6t-sram computing-in-memory macro for floating-point cnns,” in _2023 IEEE International Solid- State Circuits Conference (ISSCC)_ , 2023, pp. 128–130.
  * [8] J. Yue, C. He, Z. Wang, Z. Cong, Y. He, M. Zhou, W. Sun, X. Li, C. Dou, F. Zhang, H. Yang, Y. Liu, and M. Liu, “A 28nm 16.9-300tops/w computing-in-memory processor supporting floating-point nn inference/training with intensive-cim sparse-digital architecture,” in _2023 IEEE International Solid- State Circuits Conference (ISSCC)_ , 2023, pp. 1–3.
  * [9] L. Mei, P. Houshmand, V. Jain, S. Giraldo, and M. Verhelst, “Zigzag: Enlarging joint architecture-mapping design space exploration for dnn accelerators,” _IEEE Transactions on Computers_ , vol. 70, no. 8, pp. 1160–1174, Aug 2021\.
  * [10] N. R. Shanbhag and S. K. Roy, “Comprehending in-memory computing trends via proper benchmarking,” in _2022 IEEE Custom Integrated Circuits Conference (CICC)_ , 2022, pp. 01–07.
  * [11] R. Sehgal and J. P. Kulkarni, “Trends in analog and digital intensive compute-in-sram designs,” in _2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS)_ , 2021, pp. 1–4.
  * [12] G. W. Burr, S. Lim, B. Murmann, R. Venkatesan, and M. Verhelst, “Fair and comprehensive benchmarking of machine learning processing chips,” _IEEE Design & Test_, vol. 39, no. 3, pp. 18–27, 2022.
  * [13] S. Rai, M. Liu, A. Gebregiorgis, D. Bhattacharjee, K. Chakrabarty, S. Hamdioui, A. Chattopadhyay, J. Trommer, and A. Kumar, “Perspectives on emerging computation-in-memory paradigms,” in _2021 Design, Automation & Test in Europe Conference & Exhibition (DATE)_, 2021, pp. 1925–1934.
  * [14] B. Taylor, Q. Zheng, Z. Li, S. Li, and Y. Chen, “Processing-in-memory technology for machine learning: From basic to asic,” _IEEE Transactions on Circuits and Systems II: Express Briefs_ , vol. 69, no. 6, pp. 2598–2603, 2022.
  * [15] H. Kim, Y. Kim, S. Ryu, and J.-J. Kim, “Algorithm/hardware co-design for in-memory neural network computing with minimal peripheral circuit overhead,” in _2020 57th ACM/IEEE Design Automation Conference (DAC)_ , 2020, pp. 1–6.
  * [16] B. Patrick, R. Guy, R. Nir, P. Bruno, H. Edward, and C. Yiran, “Analog, in-memory compute architectures for artificial intelligence,” _cs_ , vol. cs/2302.06417, 2023, arXiv: 2302.06417. [Online]. Available: https://arxiv.org/abs/2302.06417
  * [17] G. Fernando, B. Ali, V. Kanishkan, C. Henk, and D. Shidhartha, “Saca: System-level analog cim accelerators simulation framework: Accurate simulation of non-ideal components,” in _2022 37th Conference on Design of Circuits and Integrated Circuits (DCIS)_ , 2022, pp. 01–06.
  * [18] D. Bhattacharjee, N. Laubeuf, S. Cosemans, I. Papistas, A. Maliik, P. Debacker, M. H. Na, and D. Verkest, “Design-technology space exploration for energy efficient aimc-based inference acceleration,” in _2021 IEEE International Symposium on Circuits and Systems (ISCAS)_ , 2021, pp. 1–5.
  * [19] S. Spetalnick and A. Raychowdhury, “A practical design-space analysis of compute-in-memory with sram,” _IEEE Transactions on Circuits and Systems I: Regular Papers_ , vol. 69, no. 4, pp. 1466–1479, 2022.
  * [20] C.-J. Jhang, C.-X. Xue, J.-M. Hung, F.-C. Chang, and M.-F. Chang, “Challenges and trends of sram-based computing-in-memory for ai edge devices,” _IEEE Transactions on Circuits and Systems I: Regular Papers_ , vol. 68, no. 5, pp. 1773–1786, 2021.
  * [21] U. Saxena, I. Chakraborty, and K. Roy, “Towards adc-less compute-in-memory accelerators for energy efficient deep learning,” in _2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)_, 2022, pp. 624–627.
  * [22] S. Yang, D. Bhattacharjee, V. B. Y. Kumar, S. Chatterjee, S. De, P. Debacker, D. Verkest, A. Mallik, and F. Catthoor, “Aero: Design space exploration framework for resource-constrained cnn mapping on tile-based accelerators,” _IEEE Journal on Emerging and Selected Topics in Circuits and Systems_ , vol. 12, no. 2, pp. 508–521, 2022.
  * [23] X. Peng, R. Liu, and S. Yu, “Optimizing weight mapping and data flow for convolutional neural networks on rram based processing-in-memory architecture,” in _2019 IEEE International Symposium on Circuits and Systems (ISCAS)_ , 2019, pp. 1–5.
  * [24] A. Parashar, P. Raina, Y. S. Shao, Y.-H. Chen, V. A. Ying, A. Mukkara, R. Venkatesan, B. Khailany, S. W. Keckler, and J. Emer, “Timeloop: A systematic approach to dnn accelerator evaluation,” in _2019 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)_ , 2019, pp. 304–315.
  * [25] H. Kwon, P. Chatarasi, V. Sarkar, T. Krishna, M. Pellauer, and A. Parashar, “Maestro: A data-centric approach to understand reuse, performance, and hardware cost of dnn mappings,” _IEEE Micro_ , vol. 40, no. 3, pp. 20–29, 2020.
  * [26] R. Balasubramonian, A. B. Kahng, N. Muralimanohar, A. Shafiee, and V. Srinivas, “Cacti 7: New tools for interconnect exploration in innovative off-chip memories,” _ACM Trans. Archit. Code Optim._ , vol. 14, no. 2, jun 2017. [Online]. Available: https://doi.org/10.1145/3085572
  * [27] P.-C. Wu, J.-W. Su, Y.-L. Chung, L.-Y. Hong, J.-S. Ren, F.-C. Chang, Y. Wu, H.-Y. Chen, C.-H. Lin, H.-M. Hsiao, S.-H. Li, S.-S. Sheu, S.-C. Chang, W.-C. Lo, C.-C. Lo, R.-S. Liu, C.-C. Hsieh, K.-T. Tang, C.-I. Wu, and M.-F. Chang, “A 28nm 1mb time-domain computing-in-memory 6t-sram macro with a 6.6ns latency, 1241gops and 37.01tops/w for 8b-mac operations for edge-ai devices,” in _2022 IEEE International Solid- State Circuits Conference (ISSCC)_ , vol. 65, 2022, pp. 1–3.
  * [28] B. Wang, C. Xue, Z. Feng, Z. Zhang, H. Liu, L. Ren, X. Li, A. Yin, T. Xiong, Y. Xue, S. He, Y. Kong, Y. Zhou, A. Guo, X. Si, and J. Yang, “A 28nm horizontal-weight-shift and vertical-feature-shift-based separate-wl 6t-sram computation-in-memory unit-macro for edge depthwise neural-networks,” in _2023 IEEE International Solid- State Circuits Conference (ISSCC)_ , 2023, pp. 134–136.
  * [29] B. Murmann, “Adc performance survey 1997-2022.” [Online]. Available: https://web.stanford.edu/ murmann/adcsurvey.html
  * [30] C. Banbury, V. J. Reddi, P. Torelli, J. Holleman, N. Jeffries, C. Kiraly, P. Montino, D. Kanter, S. Ahmed, D. Pau _et al._ , “Mlperf tiny benchmark,” _Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks_ , 2021.
  * [31] V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer, “How to evaluate deep neural network processors: Tops/w (alone) considered harmful,” _IEEE Solid-State Circuits Magazine_ , vol. 12, no. 3, pp. 28–41, 2020.
  * [32] “Memory energy,” https://my.eng.utah.edu/ cs7810/pres/14-7810-02.pdf.
  * [33] P. Houshmand, G. M. Sarda, V. Jain, K. Ueyoshi, I. A. Papistas, M. Shi, Q. Zheng, D. Bhattacharjee, A. Mallik, P. Debacker, D. Verkest, and M. Verhelst, “Diana: An end-to-end hybrid digital and analog neural network soc for the edge,” _IEEE Journal of Solid-State Circuits_ , vol. 58, no. 1, pp. 203–215, 2023.


|  MET report |  No. 07/2020
ISSN 2387-4201
Free
---|---|---

_seNorge_ observational gridded datasets

seNorge_2018, version 20.05

Cristian Lussana
The Norwegian Meteorological Institute, Oslo, Norway

|  MET report
---|---

Title |  Date
---|---
seNorge observational gridded dataset. seNorge_2018, version 20.05. |  August 27, 2024
Section |  Report no.
Division for Climate Services |  07/2020
Author(s) |  Classification
Cristian Lussana |  FreeRestricted

Abstract
---
seNorge_2018 is an observational gridded dataset for daily aggregated
temperatures and precipitation data over the Norwegian mainland. The time
interval covered spans more than 60 years, from the 1st of January 1957 to the
present day. Version 20.05 of the historical archive (1957-2019) is described
in this document, with special emphasis on the variations in the methods with
respect to the previous version of the archive, which is version 18.12.
Keywords
observational gridded datasets, temperature, precipitation, climatology,
hydrology, statistical interpolation

|
---|---
Disciplinary signature | Responsible signature
Hans Olav Hygen | Cecilie Stenersen

###### Contents

  1. 1 Introduction
  2. 2 Data and Definitions
  3. 3 Methods
    1. 3.1 Precipitation
    2. 3.2 Temperatures
  4. 4 Results
  5. 5 Conclusions

##  1 Introduction

seNorge indicates a family of observational gridded datasets over Norway. The
Norwegian Meteorological Institute (MET Norway) started to produce the seNorge
datasets versions 1.0 and 1.1 for the Norwegian mainland in the nineties
(Tveito and Førland, 1999; Tveito et al., 2000, 2002, 2005; Mohr, 2008). The
version 2.0 has been produced after 2015 (Lussana et al., 2018b, a). The most
recent member of this family is seNorge_2018 and this document describes
version 20.05. In particular, we focus on the modifications of methods and
data used with respect to version 18.12, which has been documented by Lussana
et al. (2019).

The main motivation behind the development of seNorge is to provide gridded
datasets of a few near-surface atmospheric key-variables for climatological
and hydrological applications. Observational gridded datasets exploit the
information provided by networks of traditional weather stations over a region
and they return estimates of the selected variables potentially anywhere
within the region.

The tag ”2018” in seNorge_2018 indicates that the statistical models used for
spatial analysis and the code have been developed in 2018. An important
component of the spatial analysis procedure is Optimal Interpolation (OI
Gandin and Hardin, 1965), and the reader may refer to Uboldi et al. (2008) for
the description of the modified OI scheme used. The versioning of seNorge_2018
indicates the production date of the datasets, such that version 20.05 (or
ver. 20.05) refers to May 2020. As stated above, the first version of
seNorge_2018 was ver. 18.12. A new version may include both updated data
sources and further developments of the methods. Note that the new
developments are not considered significant enough to change the name of
seNorge_2018, because the Bayesian inference behind the statistical methods
remains unchanged.

seNorge_2018 includes gridded fields of daily mean, maximum and minimum
temperature, and daily total precipitation amounts. The time period covered by
the datasets begins the 1st of January 1957 (1957-01-01) and continues to the
present day. The dataset is updated on a daily basis. seNorge_2018 datasets
can be divided into two archives: an historical archive and an operational
archive. The historical archive includes data from 1957-01-01 to a date in the
recent past, which is 2019-12-31 for ver. 20.05. The historical archive is re-
built periodically. Then, a new version of the archive is created and new
Digital Object Identifiers (DOIs) are assigned to the corresponding datasets.
The operational archive is updated every day based on the data of the last few
days. In addition, each time a new historical archive is built, the
operational archive is also modified, such that there is a seamless transition
between the most recent version of the historical archive and the operational
archive. In this sense, the operational archive is a provisional archive. In
the case of ver. 20.05, the operational archive starts at 2020-01-01.

The motivations that led us to change parts of the methods used for spatial
analysis in ver. 20.05 with respect to ver. 18.12 are different for
temperature and precipitation. In the case of precipitations, the gridded
fields of a regional climate model are used as input data to scale
precipitation in the spatial analysis. Since a climate model covering the same
region but spanning a longer time period is available, the gridded fields from
this model are used to replace those previously used for ver. 18.12. In the
case of temperature, the OI in ver. 18.12 was configured to achieve the best
agreement -on average- between (independent) observations and cross-validation
analyses. However, the optimization procedure favoured some regions and
penalized others, depending on the spatial distributions of the observational
network. In particular, since most of the stations used are located at low
elevations, the optimization procedure was biased towards ensuring the optimal
performances for those regions, while the uncertainties in mountainous regions
and in data sparse regions was considered only marginally in the optimization.
As a consequence, the variability of ver. 18.12 analyses fields in data sparse
regions is rather large, larger than in seNorge2 (or seNorge version 2.0) for
instance. The ideal situation would be to rely on the OI performances of
seNorge_2018 ver. 18.12 in data dense regions and, at the same time, reduce
the uncertainty of the temperature fields in data sparse regions down to the
level of seNorge2. seNorge_2018 ver. 20.05 has exactly this objective. The
variability of the gridded analysis field in data sparse regions is reduced by
blending together the results of several different OIs based on slightly
different configurations.

The document is organized as follows. Sec. 2 defines seNorge_2018 variables
and presents the data used. Sec. 3 describes the variations in the spatial
analysis with respect to the methods implemented for ver. 18.12. Sec. 4
compares the gridded fields of ver. 20.05 with those of ver. 18.12 with the
aim of studying the effects of variations in the input data and methods over
the results. Sec. 5 summarizes the main messages of the document. The data
access is then described in the Appendix. After the Appendix, a section with
supplementary material is included, in order to make the reading of the main
text more fluent by avoiding the interruptions caused by pages and pages of
figures.

##  2 Data and Definitions

seNorge covers the Norwegian mainland and it includes all the catchments of
rivers flowing in the Norwegian territory, as a consequence seNorge extends
also into parts of Sweden and Finland bordering Norway. The domain of
seNorge_2018 ver. 20.05 is the same of ver. 18.12. More details and figures on
the domain can be found in previous papers, such as Lussana et al. (2018b, a,
2019). The gridded data are presented on a high-resolution terrain-following
grid with 1 km spacing. The coordinate reference system is Universal
Transverse Mercator, zone 33.

The data sources used are: MET Norway’s climatological archive (frost.met.no),
which includes also observations from stations managed by several public
Norwegian institutions, such as The Norwegian Water Resources and Energy
Directorate (NVE), the Norwegian Public Roads Administration (Statens
vegvesen, SVV) and the Norwegian Institute of Bioeconomy Research (NIBIO); the
Swedish Meteorological and Hydrological Institute (SMHI) climatological
archive; Finnish Meteorological Institute (FMI) climatological archive. In
addition, the daily datasets provided by the European Climate Assessment &
Dataset project (ECA&D, https://www.ecad.eu, Klein Tank et al., 2002) are also
used.

The definitions of the variables available in seNorge_2018 are:

  * •

RR at a given date is the total amount of precipitation accumulated from 06
UTC of the day previous to that date to 06 UTC of the day in the date. RR is
also referred to as daily precipitation. Note that RR at a station location
does not correspond to the value observed by the corresponding ombrometer, not
only because of the adjustment made by the spatial analysis but also because
observations are adjusted for the wind-induced under-catch according to the
relationships reported by Lussana et al. (2019) and based on Wolff et al.
(2015).

  * •

TG at a given date is the mean averaged temperature from 06 UTC of the day
previous to that date to 06 UTC of the day in the date. TG is also referred to
as daily mean temperature. In detail, it is the arithmetic mean of 24 hourly
values or a formula based mean value computed from fewer observations (Førland
and Tveito, 1997).

  * •

TX at a given date is the maximum temperature from 18 UTC of the day previous
to that date to 18 UTC of the day in the date. TX is also referred to as daily
maximum temperature.

  * •

TN at a given date is the minimum temperature from 18 UTC of the day previous
to that date to 18 UTC of the day in the date. TX is also referred to as daily
minimum temperature.

The 63-year time period covered by the historical archive of seNorge_2018 ver.
20.05 ranges from January 1957 to December 2019. In such a long period of time
the observational network has continuously changed. The data availability is
shown in Fig. 1. There are more observations for precipitation than for
temperature. In the case of temperature, there are no significant differences
between TG, TX and TN. The area of seNorge_2018 spatial domain measures
$606579\;\mathrm{km}^{2}$. In the idealized case of a completely uniform
spatial distribution of the observational network, the ratio (area) / (number
of observations) corresponds to the area around each observation where that
observation is the closest data source. In this sense, that area can be
referred to as a sort of ”area of influence” of an observation. The radius of
the circle equivalent to the area of influence is a measure of the average
distance we need to travel in every direction before meeting another
observation. This radius is a rough indicator of the observation spatial
density. Suppose the number of available observations is 100, then the radius
is equal to $44$ km. With numbers of available observations equal to 200, 300,
600 and 900, then we get circle radii of, respectively: $31$ km, $25$ km, $18$
km and $15$ km. For all variables, it is possible to identify three different
regimes in the time series. From 1957 to 1975, there was an increase in the
number of available observations. From 1975 to 2000-2005, there was a gradual,
slow decrease in the number of observations. The decrease is rather sharp for
RR in the period from 2000 to 2005. From 2005 to 2020, the situation is
different between the two quantities. In the case of precipitation, there is
an oscillatory trend with an average of approximately 700 observations, while
for temperature a sharp increase from 300 to 500 observations is shown.

Figure 1: Time series of number of observations used for spatial analysis that
are in the seNorge_2018 domain after the data quality control. The left panel
shows RR observations. The right panel shows TG (black dots) and TX, TN (gray
dots) observations. Note the different scales on the y-axes between the two
panels.

The in-situ data shown in Fig. 1 has been quality controlled according to MET
Norway’s quality assurance system specifications. In addition, for
temperature, a spatial consistency test such as those described by Lussana et
al. (2010) has been applied. For precipitation, the time series have been
visually inspected to identify anomalies with respect to neighbouring
stations.

By comparing Fig. 1 with Fig. 1 of Lussana et al. (2019), it can be noted that
on average ver. 20.05 is based on more observations than ver. 18.12. However,
similarly to ver. 18.12, the in-situ data used in ver. 20.05 are not evenly
distributed across the altitude range. More stations are located in the
lowlands, while in the mountains the observational network is quite sparse.
This should have an influence on the uncertainties of the spatially
interpolated fields of ver. 20.05 similar to what reported for ver. 18.12 and
documented by Lussana et al. (2019).

##  3 Methods

The steps of the procedure used for the production of daily gridded datasets
for an arbitrary day are listed in the Algorithm 1. The mathematical notation
and the symbols used are defined in Tab. 1. The objective of this Section is
to describe the modifications in the methods of seNorge_2018 ver. 20.05
compared to ver. 18.12, while the general description of the methods
documented by Lussana et al. (2019) is still the reference for seNorge_2018.

In the Algorithm 1, we made it clear that a number of data are required for
the production of seNorge_2018. The fixed datasets used are not reported in
the algorithm and those are the digital elevation model and the land area
fraction over the 1 km regular grid. The gridded fields of wind speed are
derived from numerical model output, in this way it is possible to associate a
value of wind speed to each observation, independently of the actual
availability of an observed value of wind speed. In fact, observed wind speed
values would have been more accurate data but we would have inevitably lost
some of the precious observations of temperature and precipitation. We
estimated that data loss would have been a worse solution than the less
accuracy for the wind speed. The numerical models used are described in the
paper Lussana et al. (2019) and they are based on MET Norway’s products
(Reistad et al., 2011; Müller et al., 2017; Frogner et al., 2019).

###  3.1 Precipitation

seNorge_2018 statistical interpolation of precipitation is based on an
iterative OI scheme that uses the relative anomalies between observed and
reference precipitation values. The references are long-term monthly averages
of precipitation obtained through numerical models. In the case of ver. 18.12,
the reference datasets used for precipitation are based on hourly
precipitation provided by the climate model version of HARMONIE (version
cy38h1.2), a seamless NWP model framework developed and used by several
national meteorological services. For ver. 20.05, the reference dataset has
changed. The long-term monthly averages of precipitation have been derived by
the "NorCP" simulation, which provides high-resolution (3 km) climate model
data covering the 21-year time period from 1998-2018 by an updated climate
model version of HARMONIE (Lind et al., 2020).

In Fig. 2, the typical annual total precipitation fields for the references of
ver. 20.05 and ver. 18.12 are compared. First thing to notice is that the
domain of the reference for ver. 20.05 is wider than that of ver. 18.12. As a
consequence, the Norwegian mainland in ver. 20.05 reference is less likely to
be affected by those border effects that may affect ver. 18.12, especially in
Southern Norway. Besides, the wider domain of ver. 20.05 should also allow for
a better development of the high-resolution dynamics of the dynamical
downscaling procedure. As shown in the right panel, the reference of ver.
20.05 has -on average- larger values of precipitation, though the situation
can vary significantly between months and regions, as shown in Figs. 11\- 22.

Figure 2: Comparison of typical annual total precipitation fields of the
seNorge_2018 reference fields. Left panel: ver. 20.05 reference field. Middle
panel: ver. 18.12 reference field. Right panel: relative deviations as
percentages of the ver. 18.12 reference field (i.e., ref(ver 20.05) / ref(ver
18.12)).

###  3.2 Temperatures

seNorge_2018 statistical interpolations of temperature are all based on a two-
step spatial analysis scheme. The first step aims at representing the
temperature field at a point as the average over a spatial support larger than
the local station density, where this is characterized as the average distance
between a station and its closest neighbouring stations in the surroundings of
that point. The second step refines the temperature representation at a point
by including those effects that are observed only by a small number of
neighbouring stations. The first step is implemented through the elaboration
of sub-regional pseudo-background fields. This part has remained unchanged
from ver. 18.12. The second step is implemented by means of an Optimal
Interpolation (OI) scheme and this part has been significantly changed, as it
is described in the following of this Section.

For each of the three variables TG, TX and TN, the OI runs with more than a
single configuration, as it is instead for ver. 18.12. In particular, four
different OI are performed. The parameter that is varying within the different
OI runs is $D^{z}$, which is the vertical decorrelation length used in the
definition of the background error covariance matrix. For ver. 18.12, $D^{z}$
was set to 210 m. In the case of ver. 20.05 the four values used are: 210 m;
400 m; 600 m and 800 m. For each combination of the pair variable/OI
configuration, a dataset of gridded fields is produced.

There are two more steps that must be performed in order to obtain the final
dataset of TG, TX and TN. The first step is the merging of the four datasets
into a single dataset for each variable. The second step is the check for
consistency between the three variables, specifically to ensure that TX is
greater or equal to TG and, at the same time, TN is smaller or equal to TG.
Note that such a check is needed, because the spatial analysis is performed
for each variable independently from the others, therefore in regions where
the analysis uncertainty at a given time is greater than the difference
between variables, the consistency between TG, TX and TN may be violated. The
check for consistency was implemented already in ver. 18.12 and it first
replaces TX with TG for those grid points where TG is greater than TX, then it
replaces TN with TG for those grid points where TG is smaller than TX. This
simple and practical solution might not always yield satisfactory results, for
this reason it is one of the points that should be improved in future versions
of seNorge_2018.

The merging of datasets into a single one is a new feature of ver. 20.05
compared to ver. 18.12. Statistical interpolation schemes produce fields where
each value is an average over a spatial support and the sizes of those spatial
supports vary across the domain. The OI settings influence the sizes of the
three-dimensional spatial supports. The vector of OI settings
$\boldsymbol{\lambda}_{1}$ includes $D^{z}=210\,\mathrm{m}$ (same as ver.
18.12), then: $\boldsymbol{\lambda}_{2}$ includes $D^{z}=400\,\mathrm{m}$;
$\boldsymbol{\lambda}_{3}$ includes $D^{z}=600\,\mathrm{m}$ (same as
seNorge2); $\boldsymbol{\lambda}_{4}$ includes $D^{z}=800\,\mathrm{m}$. Apart
from $D^{z}$, all the parameters in the four OI settings do share the same
values. The variables considered are TG, TX and TN, they are further
abbreviated as $g$, $x$ and $n$, respectively. For example, the notation for
the analysis of TG at the $j$th grid point for the OI configuration with
$D^{z}=800\,\mathrm{m}$ is $\mathbf{x}^{g,a}_{j}(\boldsymbol{\lambda}_{4})$
(see also Tab. 1). In addition to the analysis, the Integral Data Influence
(IDI, Uboldi et al., 2008) is available for each point in each field. IDI is
an indicator ranging from 0 to 1 that summarizes the amount of information
used in the OI that is derived from nearby observations. At the $j$th grid
point, if we consider for instance TX, the notation
$\mathbf{x}^{x,\mathrm{IDI}}_{j}(\boldsymbol{\lambda}_{2})$ indicates IDI
obtained with $D^{z}=400\,\mathrm{m}$. Given a particular set of
$\boldsymbol{\lambda}$ values, grid points where the IDI values are close to 0
will have the analysis almost identical to the large-scale pseudo-background,
since no information from nearby observations has been used in the analysis.
Then, the spatial support of the analysis is mostly determined by the pseudo-
background characteristics. On the other hand, grid points where the IDI
values are close to 1 will use nearby observations to adjust the analysis. In
this case, the spatial support of the analysis is mostly determined by the OI
settings and the spatial support of the analysis temperature field is smaller
for smaller values of $D^{z}$.

The merging is done such that more weight is given to the field with smaller
spatial support (i.e. $\boldsymbol{\lambda}_{1}$) and weight has been given to
the other (coarser) fields only where they provide additional information. In
this way, we make use of the observational network in an optimal way by
providing a higher resolution field where the network is denser. The relative
content of information of a field with respect to another field is evaluated
through a function inspired by Shannon’s measure of information content,
similarly to what reported for the probability density functions in Tarantola
(2005), Sec. 1.2.5. In our equations, IDI will replace probability density
functions. The fields are ordered from the one with the smallest spatial
supports, which is that obtained with $\boldsymbol{\lambda}_{1}$, to the one
with the largest spatial supports, when the OI is performed with
$\boldsymbol{\lambda}_{4}$. Then, relative content of information in one field
with respect to its immediate antecedent field with smaller spatial supports
is computed. The first field, the one with $\boldsymbol{\lambda}_{1}$, is
assigned the complementary of the relative content of information of the
second field. The weights are the normalized relative contents of information.
The procedure is described in mathematical terms in the equations that
follows.

The TG analysis at the $j$th grid point is obtained as a weighted average of
the four OI analysis:

$\mathbf{x}^{g,a}_{j}=\sum_{l=1}^{4}\mathbf{w}^{l}_{j}\cdot\mathbf{x}^{g,a}_{j}(\boldsymbol{\lambda}_{l})$
(1)

where the weights with $l=2,\ldots,4$ are obtained as:

$\mathbf{w}^{l}_{j}=\boldsymbol{\alpha}^{-1}_{j}\cdot\mathbf{x}^{g,\mathrm{IDI}}_{j}(\boldsymbol{\lambda}_{l})\cdot\mathrm{log}\left(\frac{\mathbf{x}^{g,\mathrm{IDI}}_{j}(\boldsymbol{\lambda}_{l})}{\mathbf{x}^{g,\mathrm{IDI}}_{j}(\boldsymbol{\lambda}_{l-1})}\right)$
(2)

The weight for $l=1$ is set to:

$\mathbf{w}^{1}=\boldsymbol{\alpha}^{-1}\cdot(1-\mathbf{w}^{2})$ (3)

and $\alpha$ is the vector of normalization factors used to enforce that
$\sum_{l=1}^{4}\mathbf{w}^{l}_{j}=1,\;\forall j=1,\ldots,m$. Note that to
avoid problems in Eq. (2), all the IDI values smaller than 0.0000001 have been
replaced with 0.0000001.

In Figs. 3\- 6 examples of the normalized weights of Eqs. (2)- (3) are shown.
The figures show the averaged weights over the days within year 2019 for TG.
In the right panels, the fields on the original seNorge_2018 grid are shown.
In the left panels, the fields have been aggregated on a coarse resolution
grid with square boxes of side length 50 km. In this way, we focus on averages
over regions and not on small-scale details. The weight $\mathbf{w}^{1}$ in
Fig. 3 is by far the largest one and for a few boxes on the coarser grid its
value is equal to 1. The OI analysis obtained with $\boldsymbol{\lambda}_{1}$
should have more weight than the others, especially close to an observation.
In mountainous regions where the observational network is sparse,
$\mathbf{w}^{1}$ reaches its smallest values. The weight $\mathbf{w}^{2}$ in
Fig. 4 has maximum values in the range 0.25-0.5 exactly in those regions where
$\mathbf{w}^{1}$ has its smallest values. The other two weights,
$\mathbf{w}^{2}$ and $\mathbf{w}^{3}$, are shown in Figs. 5\- 6 and the
information of the corresponding OIs are used to integrate the higher-
resolutions OI only in mountainous regions where the observational network is
sparse. The maximum value of $\mathbf{w}^{3}$ is 0.25, while the maximum value
for $\mathbf{w}^{4}$ is 0.16.

Figure 3: Normalized weights $\mathbf{w}^{1}$ ($D^{z}=210\,\mathrm{m}$) for
TG, averaged over 2019. The left panel shows the aggregated field over square
boxes with sides of 50 km. The right panel shows the original field on the 1
km grid. Figure 4: Same as Fig. 3 but for $\mathbf{w}^{2}$
($D^{z}=400\,\mathrm{m}$). Figure 5: Same as Fig. 3 but for $\mathbf{w}^{3}$
($D^{z}=600\,\mathrm{m}$). Figure 6: Same as Fig. 3 but for $\mathbf{w}^{4}$
($D^{z}=800\,\mathrm{m}$).

##  4 Results

A comparison between seNorge_2018 ver. 20.05 and ver. 18.12 is reported in
this section for the four variables RR, TG, TX and TN. In particular, the
comparison is based on the most recent 30-year period common to both datasets,
which is the time period 1988-2017. Note that a complete study to characterize
estimation uncertainties based on cross-validation experiments has not been
performed yet for ver. 20.05, while it is available in the paper by Lussana et
al. (2019) for ver. 18.12.

The comparison of total precipitation amounts is shown in Fig. 7 for winter
months and in Figs. 23\- 25 for the other seasons. The comparison is performed
on monthly precipitation totals averaged over the period 1988-2017. The
results show the amounts of ver. 20.05 as percentages of the amounts of ver.
18.12. For example, where the monthly total precipitation of ver. 20.05 is
approximately equal to the amount of ver. 18-12 ($\pm$5%), then the
corresponding square box will be filled with the white color. Green/blue
colors indicate boxes where ver. 20.05 have larger precipitation amounts than
ver. 18.12. Yellow/brown colors indicate boxes where ver. 20.05 have smaller
precipitation amounts than ver. 18.12. For the majority of the boxes the two
versions are rather similar, that is $\pm$5% or $\pm$10%. However, for some
boxes in the mountains or along the coast the differences are significant (up
to 300% for two boxes in northern Sweden, close to the boundary with Norway).
The reasons are the different observational dataset used and the variations in
the reference field used for the OI. If compared to the other seasons, the
summer months shown in Fig. 24 display a different pattern. Compared to ver.
18.12, ver. 20.05 has less precipitation along the coast and larger
precipitation amounts inland.

Figure 7: Monthly total precipitation amounts for seNorge_2018 ver. 20.05, as
percentage of ver. 18.12, averaged over the 30-year period from 1988 to 2017
for winter months: December, in the left panel; January, in the middle;
February, in the right panel. The fields of averaged values over square boxes
with sides of 50 km are shown in the panels.

The comparison of daily mean temperatures is shown in Fig. 8 for winter months
and in Figs. 26\- 28 for the other seasons. The quantity considered is the
monthly mean temperature averaged over the period 1988-2017 and the
elaboration is based on daily mean temperatures. Then, the differences between
ver. 20.05 and ver. 18.12 are shown in the figures. The only significant
differences occur during the winter, while for the other season the regional
averages are almost everywhere $\pm 0.1^{\circ}\mathrm{C}$. Fig. 8 shows that
during winter: ver. 20.05 is colder than ver. 18.12 on the Scandinavian
mountains in the Northern part of Sweden; ver. 20.05 is warmer than ver. 18.12
on the northernmost part of Norway, in the region of East-Finnmark.

Figure 8: Differences between monthly mean daily temperature of seNorge_2018
ver. 20.05 and ver. 18.12, as ${}^{\circ}\mathrm{C}$, averaged over the
30-year period from 1988 to 2017 for winter months: December, in the left
panel; January, in the middle; February, in the right panel. The fields of
averaged values over square boxes with sides of 50 km are shown in the panels.

The comparison of the daily maximum temperature is shown in Fig. 9 for winter
months and Figs. 29\- 31 for the other months. The quantity considered is the
monthly mean of daily maximum temperatures. The monthly mean is based on the
period 1988-2017. Then, the differences between ver. 20.05 and ver. 18.12 are
shown in the figures. In general, the two seNorge_2018 versions are rather
similar. The situation is similar to that of daily mean temperature, but the
differences between the two versions are even less pronounced.

Figure 9: Differences between monthly average of maximum daily temperature of
seNorge_2018 ver. 20.05 and ver. 18.12, as ${}^{\circ}\mathrm{C}$, averaged
over the 30-year period from 1988 to 2017 for winter months: December, in the
left panel; January, in the middle; February, in the right panel. The fields
of averaged values over square boxes with sides of 50 km are shown in the
panels.

The comparison of the daily minimum temperature is shown in Fig. 10 for winter
months and Figs. 32\- 34 for the other months. The quantity considered is the
monthly mean of daily maximum temperatures. The monthly mean is based on the
period 1988-2017. Then, the differences between ver. 20.05 and ver. 18.12 are
shown in the figures. As for the other two temperature variables, the largest
differences between the two versions are observed for the winter months, even
though in the case of minimum temperature there are still significant
differences in the northern part of the domain also for September, October and
March. The patterns of the deviations resemble also those observed for the
daily mean temperature, with a region in northern Norway where ver. 20.05 is
warmer (more than $1^{\circ}\mathrm{C}$) than ver. 18.12 and a region of the
Scandinavian mountains in northern Sweden where ver. 20.05 is colder than ver.
18.12. In addition, the deviations between minimum temperatures over South-
Norway in winter are also significant for a few square boxes, up to $\pm
0.3^{\circ}\mathrm{C}$.

Figure 10: Differences between monthly average of maximum daily temperature of
seNorge_2018 ver. 20.05 and ver. 18.12, as ${}^{\circ}\mathrm{C}$, averaged
over the 30-year period from 1988 to 2017 for winter months: December, in the
left panel; January, in the middle; February, in the right panel. The fields
of averaged values over square boxes with sides of 50 km are shown in the
panels.

##  5 Conclusions

seNorge_2018 version 20.05 has been released and it is described in this
document. The observational gridded datasets include daily mean, minimum and
maximum temperatures and daily total precipitation amounts over the Norwegian
mainland for the time period 1957-2019. The gridded fields are made available
over a regular grid with 1 km of spacing between grid nodes in both easting
and northing directions.

Version 20.05 is an update of ver. 18.12, which covers the same spatial domain
but spans a smaller time period, from 1957 to 2017. The data sources for the
in-situ observations are the same for the two versions, however the
observational datasets for ver. 20.05 have been collected from scratch, that
is independently from ver. 18.12. The in-situ observations have been quality
controlled based on both visual inspection of the time series and the
application of automatic procedures. The set of in-situ observations used for
ver. 20.05 is rather similar to the one used for ver. 18.12. A bit more
observations have been used in ver. 20.05.

Version 20.05 includes some changes in the methods with respect to ver. 18.12.
In the case of precipitation, the reference fields used to scale the daily
precipitation amounts in the statistical interpolation have been changed. In
the case of temperature, the statistical interpolation has been modified such
that the pseudo-background, large-scale, field is adjusted locally on the
basis of the surrounding observations over wider regions in ver. 20.05 than in
ver. 18.12.

The combined impacts over the results of the updated input observational
datasets and the modifications in the methods have been evaluated for the
period 1988-2017 by considering monthly averaged deviations between ver. 20.05
and ver. 18.12. Instead of considering grid points, we have aggregated
deviations over square boxes with sides of 50 km. In general, the deviations
are rather small, especially in data dense regions, and this fact indicates a
stability of the methods with respect to variations in some procedures.
However, in data sparse regions or where some new observations have been used
in ver. 20.05, the differences between the two versions are evident even over
30-year averages. For all variables, the largest differences occur during
winter and they are more pronounced over few square boxes that are
characterized by complex terrain and a sparse observational network. Future
plans include a complete evaluation of seNorge_2018 ver. 20.05 based on cross-
validation to estimate the improvements in accuracy and precision with respect
to ver. 18.12.

##  Appendix: Data Access

seNorge_2018 version 20.05 datasets are available for public download under
the Norwegian Licence for Open Government Data (NLOD,
https://data.norge.no/nlod/en/). The data can be accessed from two distinct
sources: Zenodo zenodo.org and MET’s Norway THREDDS data server
thredds.met.no. A detailed description on how to access the data is available
at the seNorge wiki-pages:

  * •

https://github.com/metno/seNorge_docs/wiki/seNorge_2018

In the wiki, it is also possible to get the latest news on the datasets and
contact the developers by opening issues about specific topics.

The digital object identifiers (DOIs) of the datasets are:

  * •

TG, daily mean temperature (Lussana, 2020b). DOI is 10.5281/zenodo.3923706.

  * •

RR, daily total precipitation amount (Lussana, 2020a). DOI is
10.5281/zenodo.3923703. In addition to RR, the archive also provides RRa (RR
alternative), which is the field of daily total precipitation without the
correction for the wind-induced undercatch.

  * •

TX, daily maximum temperature (Lussana, 2020d). DOI is 10.5281/zenodo.3923700.
In addition to TX, the archive also provides TXa (TX alternative), which is
the field of daily maximum temperature without the check for consistency
against TG and TN.

  * •

TN, daily minimum temperature (Lussana, 2020c). DOI is 10.5281/zenodo.3923697.
In addition to TN, the archive also provides TNa (TN alternative), which is
the field of daily minimum temperature without the check for consistency
against TG and TX.

##  Supplementary Material

Figure 11: same as Fig. 2 but for the typical January. Figure 12: same as Fig. 2 but for the typical February. Figure 13: same as Fig. 2 but for the typical March. Figure 14: same as Fig. 2 but for the typical April. Figure 15: same as Fig. 2 but for the typical May. Figure 16: same as Fig. 2 but for the typical June. Figure 17: same as Fig. 2 but for the typical July. Figure 18: same as Fig. 2 but for the typical August. Figure 19: same as Fig. 2 but for the typical September. Figure 20: same as Fig. 2 but for the typical October. Figure 21: same as Fig. 2 but for the typical November. Figure 22: same as Fig. 2 but for the typical December. Figure 23: Same as Fig. 7 but for spring months: March, in the left panel; April, in the middle; May, in the right panel. Figure 24: Same as Fig. 7 but for summer months: June, in the left panel; July, in the middle; August, in the right panel. Figure 25: Same as Fig. 7 but for autumn months: September, in the left panel; October, in the middle; November, in the right panel. Figure 26: Same as Fig. 8 but for spring months: March, in the left panel; April, in the middle; May, in the right panel. Figure 27: Same as Fig. 8 but for summer months: June, in the left panel; July, in the middle; August, in the right panel. Figure 28: Same as Fig. 8 but for autumn months: September, in the left panel; October, in the middle; November, in the right panel. Figure 29: Same as Fig. 9 but for spring months: March, in the left panel; April, in the middle; May, in the right panel. Figure 30: Same as Fig. 9 but for summer months: June, in the left panel; July, in the middle; August, in the right panel. Figure 31: Same as Fig. 9 but for autumn months: September, in the left panel; October, in the middle; November, in the right panel. Figure 32: Same as Fig. 10 but for spring months: March, in the left panel; April, in the middle; May, in the right panel. Figure 33: Same as Fig. 10 but for summer months: June, in the left panel; July, in the middle; August, in the right panel. Figure 34: Same as Fig. 10 but for autumn months: September, in the left panel; October, in the middle; November, in the right panel. Table 1: Overview of variables and mathematical notation. All the vectors are column vectors if not otherwise specified. If $\mathbf{X}$ is a matrix, $\mathbf{X}_{i}$ is its $i$th column (column vector) and $\mathbf{X}_{i,:}$ is its $i$th row (row vector). If $\mathbf{x}$ is a column vector, $\mathbf{x}_{j}$ is its $j$th element. symbol | description | dimension
---|---|---
$m$ | number of grid points | -
$p$ | number of observations | -
$l$ | index of OI configuration | $l=1,\ldots,4$
$\boldsymbol{\lambda}$ | OI parameters | not specified
$\mathbf{x}^{g,a}(\boldsymbol{\lambda})$ | TG analysis over the grid, obtained with OI parameters $\boldsymbol{\lambda}$ | m
$\mathbf{x}^{x,a}(\boldsymbol{\lambda})$ | TX analysis over the grid, obtained with OI parameters $\boldsymbol{\lambda}$ | m
$\mathbf{x}^{n,a}(\boldsymbol{\lambda})$ | TN analysis over the grid, obtained with OI parameters $\boldsymbol{\lambda}$ | m
$\mathbf{x}^{g,\mathrm{IDI}}(\boldsymbol{\lambda})$ | TG IDI over the grid, obtained with OI parameters $\boldsymbol{\lambda}$ | m
$\mathbf{x}^{x,\mathrm{IDI}}(\boldsymbol{\lambda})$ | TX IDI over the grid, obtained with OI parameters $\boldsymbol{\lambda}$ | m
$\mathbf{x}^{n,\mathrm{IDI}}(\boldsymbol{\lambda})$ | TN IDI over the grid, obtained with OI parameters $\boldsymbol{\lambda}$ | m
$\mathbf{w}^{l}$ | weight for OI with $\boldsymbol{\lambda}_{l}$, used in the merging | m
$\boldsymbol{\alpha}$ | normalization factor for the merging | m
Algorithm 1 seNorge_2018 procedure for the production of daily gridded
datasets for an arbitrary day.

0: daily observations of temperature and precipitation collected from the data
sources

0: daily averaged wind speed gridded field from numerical model output

0: Gridded fields of long-term monthly averages of precipitation from
numerical model output

Production of 4 provisional versions of TG fields, OI with
$\boldsymbol{\lambda}_{l}$, $l=1,\ldots,4$

Merging of the TG versions into the definitive TG field

Adjust RR observations for wind-induced undercatch based on TG and the wind
fields

Production of RR gridded fields

Production of 4 provisional versions of TX fields, OI with
$\boldsymbol{\lambda}_{l}$, $l=1,\ldots,4$

Merging of the TX versions into the definitive TX field

Production of 4 provisional versions of TN fields, OI with
$\boldsymbol{\lambda}_{l}$, $l=1,\ldots,4$

Merging of the TN versions into the definitive TN field

Check consistency between TX and TG. Modify TX where the consistency is
violated

Check consistency between TN and TG. Modify TN where the consistency is
violated

## References

  * Førland and Tveito (1997) Førland, E., and O. Tveito (1997), Temperatur og snødata for flomberegning, DNMI Report, 28(97), 51.
  * Frogner et al. (2019) Frogner, I.-L., A. T. Singleton, M. Ø. Køltzow, and U. Andrae (2019), Convection-permitting ensembles: challenges related to their design and use, Quarterly Journal of the Royal Meteorological Society, 0(0), doi:10.1002/qj.3525.
  * Gandin and Hardin (1965) Gandin, L. S., and R. Hardin (1965), Objective analysis of meteorological fields, vol. 242, Israel program for scientific translations Jerusalem.
  * Klein Tank et al. (2002) Klein Tank, A., J. Wijngaard, G. Können, R. Böhm, G. Demarée, A. Gocheva, M. Mileta, S. Pashiardis, L. Hejkrlik, C. Kern-Hansen, et al. (2002), Daily dataset of 20th-century surface air temperature and precipitation series for the European climate assessment, International journal of climatology, 22(12), 1441–1453.
  * Lind et al. (2020) Lind, P., D. Belušić, O. B. Christensen, A. Dobler, E. Kjellström, O. Landgren, D. Lindstedt, D. Matte, R. A. Pedersen, E. Toivonen, and F. Wang (2020), Benefits and added value of convection-permitting climate modeling over fenno-scandinavia, Climate Dynamics, pp. 1–20, doi:10.1007/s00382-020-05359-3.
  * Lussana (2020a) Lussana, C. (2020a), senorge_2018 daily total precipitation amount 1957-2019, doi:10.5281/zenodo.3923703, observational gridded dataset over the Norwegian mainland.
  * Lussana (2020b) Lussana, C. (2020b), senorge_2018 daily mean temperature 1957-2019, doi:10.5281/zenodo.3923706, observational gridded dataset over the Norwegian mainland.
  * Lussana (2020c) Lussana, C. (2020c), senorge_2018 daily minimum temperature 1957-2019, doi:10.5281/zenodo.3923697, observational gridded dataset over the Norwegian mainland.
  * Lussana (2020d) Lussana, C. (2020d), senorge_2018 daily maximum temperature 1957-2019, doi:10.5281/zenodo.3923700, observational gridded dataset over the Norwegian mainland.
  * Lussana et al. (2010) Lussana, C., F. Uboldi, and M. R. Salvati (2010), A spatial consistency test for surface observations from mesoscale meteorological networks, Quarterly Journal of the Royal Meteorological Society, 136(649), 1075–1088.
  * Lussana et al. (2018a) Lussana, C., T. Saloranta, T. Skaugen, J. Magnusson, O. E. Tveito, and J. Andersen (2018a), seNorge2 daily precipitation, an observational gridded dataset over Norway from 1957 to the present day, Earth System Science Data, 10(1), 235–249, doi:10.5194/essd-10-235-2018.
  * Lussana et al. (2018b) Lussana, C., O. E. Tveito, and F. Uboldi (2018b), Three-dimensional spatial interpolation of 2m temperature over Norway, Quarterly Journal of the Royal Meteorological Society, 144(711), 344–364, doi:10.1002/qj.3208.
  * Lussana et al. (2019) Lussana, C., O. E. Tveito, A. Dobler, and K. Tunheim (2019), senorge_2018, daily precipitation, and temperature datasets over norway, Earth System Science Data, 11(4), 1531–1551, doi:10.5194/essd-11-1531-2019.
  * Mohr (2008) Mohr, M. (2008), New routines for gridding of temperature and precipitation observations for “senorge. no”, Met. no Report, 8, 2008\.
  * Müller et al. (2017) Müller, M., M. Homleid, K.-I. Ivarsson, M. A. Køltzow, M. Lindskog, K. H. Midtbø, U. Andrae, T. Aspelien, L. Berggren, D. Bjørge, et al. (2017), Arome-metcoop: a nordic convective-scale operational weather prediction model, Weather and Forecasting, 32(2), 609–627.
  * Reistad et al. (2011) Reistad, M., Ø. Breivik, H. Haakenstad, O. J. Aarnes, B. R. Furevik, and J.-R. Bidlot (2011), A high-resolution hindcast of wind and waves for the north sea, the norwegian sea, and the barents sea, Journal of Geophysical Research: Oceans, 116(C5).
  * Tarantola (2005) Tarantola, A. (2005), Inverse Problem Theory and methods for model parameter estimation, SIAM, Philadelphia.
  * Tveito et al. (2000) Tveito, O., E. Førland, R. Heino, I. Hanssen-Bauer, H. Alexandersson, B. Dahlström, A. Drebs, C. Kern-Hansen, T. Jónsson, E. Vaarby Laursen, et al. (2000), Nordic temperature maps, DNMI report, 9(00).
  * Tveito and Førland (1999) Tveito, O. E., and E. J. Førland (1999), Mapping temperatures in Norway applying terrain information, geostatistics and GIS, Norsk Geografisk Tidsskrift-Norwegian Journal of Geography, 53(4), 202–212.
  * Tveito et al. (2002) Tveito, O. E., H.-C. Udnæs, Z. Mengistu, R. Engeset, and E. J. Førland (2002), New snow maps for Norway, in Proceedings XXII Nordic Hydrological Conference 2002, pp. 527–532.
  * Tveito et al. (2005) Tveito, O. E., I. Bjørdal, A. O. Skjelvåg, and B. Aune (2005), A gis-based agro-ecological decision system based on gridded climatology, Meteorological Applications, 12(1), 57–68.
  * Uboldi et al. (2008) Uboldi, F., C. Lussana, and M. Salvati (2008), Three-dimensional spatial interpolation of surface meteorological observations from high-resolution local networks, Meteorological Applications, 15(3), 331–345.
  * Wolff et al. (2015) Wolff, M. A., K. Isaksen, A. Petersen-Øverleir, K. Ødemark, T. Reitan, and R. Brækkan (2015), Derivation of a new continuous adjustment function for correcting wind-induced loss of solid precipitation: results of a norwegian field study, Hydrology and Earth System Sciences, 19(2), 951–967, doi:10.5194/hess-19-951-2015.