1

Embedded Algorithmic Noise-Tolerance for SignalProcessing and Machine Learning Systems via Data

Path DecompositionSai Zhang Student Member, IEEE and Naresh R. Shanbhag, Fellow, IEEE

Abstract—Low overhead error-resiliency techniques such asalgorithmic noise-tolerance (ANT) have been shown to be particu-larly effective for signal processing and machine learning kernels.However, the overhead of conventional ANT can be as large as30% due to the use of explicit estimator block. To overcome thisoverhead, embedded ANT (E-ANT) is proposed [1] where theestimator is embedded in the main computation block via datapath decomposition (DPD). E-ANT reduces the logic overheadto be below 8% as compared with the 20% − 30% associatedwith conventional reduced precision replica (RPR) ANT systemwhile maintaining the same error compensation functionality.DPD was first proposed in our original paper [1] where itsbenefits were studied in the case of a simple multiply-accumulator(MAC) kernel. This paper builds upon [1] by: 1) providingconditions for the existence of DPD, 2) demonstrating DPD for avariety of commonly employed kernels in signal processing andmachine learning applications, and 3) evaluating the robustnessimprovement and energy savings of DPD at the system level foran SVM-based EEG seizure classification application. Simulationresults in a commercial 45 nm CMOS process show that E-ANTcan compensate for error rates up to 0.38 for errors in FEonly, and 0.17 for errors in FE and CE, while maintain a truepositive rate ptp > 0.9 and a false positive rate pfp ≤ 0.01. Thisrepresents a greater than 3-orders-of-magnitude improvement inerror tolerance over the conventional system. This error toleranceis employed to reduce energy via the use of voltage overscaling(VOS). E-ANT is able to achieve 51% energy savings when errorsare in FE only, and up to 43% savings when errors are in bothFE and CE.

I. INTRODUCTION

Error resiliency techniques have been proposed [2] to en-hance energy efficiency by reducing design margins and com-pensating for the resultant errors. Large design margins arisefrom the need to provide robustness in the presence of process,voltage, and temperature variations [3], and represent anenergy overhead as high as 3×-to-4× [4]. The key to the useof error resiliency for energy reduction is that such techniquesneed to be low overhead and yet effective in compensatingfor high error rates. Classical fault-tolerance techniques suchas N-modular redundancy (NMR) rely on replication of themain computation block and as a result are ineffective for

Copyright (c) 2015 IEEE. Personal use of this material is permitted.However, permission to use this material for any other purposes must beobtained from the IEEE by sending a request to pubs-permissions@ieee.org.

Sai Zhang is with the Department of Electrical and Computer Engi-neering, University of Illinois at Urbana-Champaign, Urbana, USA, e-mail:szhang12@illinois.edu.

Naresh R. Shanbhag is with the Department of Electrical and ComputerEngineering, University of Illinois at Urbana-Champaign, Urbana, USA, e-mail: shanbhag@illinois.edu.

the purposes of energy reduction. Hence, low overhead errorresiliency techniques such as RAZOR [5], [6], error-detectionsequential (EDS) [7], confidence driven computing (CDC) [8]and statistical error compensation (SEC) techniques [2] havebeen proposed to enhance energy efficiency.

Error resiliency techniques have two sources of overhead:error detection and error correction. Techniques such as RA-ZOR [5], [6], EDS [7], and CDC [8] achieve lower errordetection overhead compared with NMR. RAZOR employsa shadow latch for error detection with an error detectionoverhead of 1.2% (with 28.5% overhead in RAZOR flip flop).EDS [7] employs a dynamic transition detector with a time-borrowing data path latch (TDTB) or a double sampling staticdesign with time-borrowing data path latch (DSTB) to achieveerror detection and the error detection overhead is similar toRAZOR. CDC [8] employs fine-grained temporal redundancyand confidence checking with an error detection overhead of8%−46%. However, these techniques employ a roll back basederror correction scheme, thus incur error correction overheadof greater than 2×, and can compensate for error rates < 0.01.

Unlike the roll back based techniques, SEC is a classof system level error compensation techniques that utilizessignal and error statistics and hence is particularly well-suitedfor signal processing and machine learning systems. Thesetechniques include algorithmic noise tolerance (ANT), softNMR, stochastic sensor network on a chip (SSNOC) [2], andhave been shown to compensate for error rates ranging from0.21 to 0.89, with an combined error detection and correctionoverhead ranging from 5% to 30% resulting in energy savingsranging from 35% to 72%.

ANT [2], is a specific SEC technique that has been shownto be effective in compensating for high error rates in signalprocessing and machine learning kernels. For example, thereduced precision replica (RPR) ANT technique and predictionbased ANT was employed to compensate for error rates of0.27 ∼ 0.58 in an ECG processor [9], [10] while delivering therequired application-level performance. The overhead in ANTranges from 5% to 30% [9] due to the use of explicit estimatorblocks in error compensation. This overhead, though smallcompared to other techniques, limits the achievable systemlevel energy efficiency between 28% ∼ 41%.

In this paper, we propose embedded statistical error com-pensation where the main block is reformulated at the archi-tectural level to embed an estimator for low overhead errordetection and correction. In particular, embedded ANT (E-ANT) is proposed by embedding a RPR estimator into the

2

main block via a technique called data path decomposition(DPD). Such decomposition is achieved by decomposing themain block into the most significant bit (MSB) and the leastsignificant bit (LSB) components and employing the MSBcomponent as the estimator output. As a result, DPD basedE-ANT achieves the same error compensation functionalitywith lower complexity and improved energy savings as theconventional ANT technique. DPD was first proposed in ouroriginal paper [1] where its benefits were studied in thecase of a simple multiply-accumulator (MAC) kernel. Thispaper builds upon [1] by: 1) providing conditions for theexistence of DPD, 2) demonstrating DPD for a variety ofcommonly employed kernels in signal processing and machinelearning applications (filter, butterfly unit, exponential kerneland others), and 3) evaluating the robustness improvement andenergy savings of DPD at the system level for an SVM-basedEEG seizure classification application.

E-ANT is applied to the design of an EEG seizure classifica-tion system [11] consisting of a frequency selective filter bankas the feature extractor (FE) and a support vector machine(SVM) as the classification engine (CE). Simulation resultsin a commercial 45 nm CMOS process show that E-ANTcan compensate for error rates of up to 0.38 (errors in FEonly), up to 0.17 (errors in FE and CE), and maintain the truepositive rate ptp > 0.9 and false positive rate pfp ≤ 0.01. Thisrepresents a greater than 3-orders-of-magnitude improvementin error tolerance over the conventional architecture. This errortolerance is employed to reduce energy via the use of voltageoverscaling (VOS). E-ANT is able to achieve 51% energysavings when errors are in FE only, and up to 43% savingswhen errors are in both FE and CE.

The rest of the paper is organized as follows. Section IIdescribes the background of the ANT technique and the SVMEEG classification system architecture. Section III presentsE-ANT and derives E-ANT architectures for a wide classof common signal processing and machine learning kernels.Section IV presents the design optimization and application ofE-ANT to the SVM EEG classification system. Conclusionsare presented in Section V.

II. BACKGROUND

A. Conventional ANT

Conventional ANT incorporates a main block (M) and anestimator (E) as shown Fig. 1(a). The M-block implementsthe algorithm of interest and is conventionally error-free. InANT, the M-block is permitted to make errors, which are thencompensated for by the rest of the blocks in Fig. 1(a) includingthe E-block. In RPR ANT, the E-block is obtained by reducingthe precision of the M-block. The M-block is subject to largemagnitude errors η (e.g., timing errors due to critical pathviolations which typically occur in the MSBs) while the E-block is subject to small magnitude errors e (see Fig. 1(b),e.g., due to quantization noise in the LSBs), i.e.:

ya = yo + η (1)ye = yo + e (2)

where yo, ya, and ye are the error-free, the M, and E-blockoutputs, respectively. ANT exploits the difference in the errorstatistics of η and e to detect and compensate for errors toobtain the final corrected output y as follows:

y =

{ya if |ya − ye| ≤ Thye otherwise

(3)

where Th is an application dependent threshold parameterchosen to maximize the performance of ANT. In this paper,Th is chosen to equal max(|yo − ye|) as this ensures that theM-block output ya will always be selected [9] when the outputis error free. The error rate pη is defined as:

pη = 1− Pη(0) = Pr{η 6= 0} (4)

where Pη(·) is the error probability mass function (PMF)of η. The errors η are most conveniently obtained by applyingvoltage overscaling (VOS) where the supply voltage Vdd isscaled as follows:

Vdd = KvosVdd−crit (5)

where Kvos is the voltage overscaling factor, and Vdd−critis the minimum voltage needed for error free operation in theM-block. Note that for the ANT system to work properly, theE-block is not permitted to make large magnitude errors suchas those arising from timing violations. This helps maintainthe difference in the error statistics at the output of the M andE-block as shown in Fig. 1(b).

(a) (b)

Figure 1. Algorithmic noise-tolerance (ANT): a) conventional architecture,and b) the error statistics in the main (M) and estimator (E) blocks.

The performance improvement achieved by ANT can beevaluated by employing a system level metric such as thesignal-to-noise ratio (SNR). Assume that the error-free outputin Fig. 1(a) is expressed as:

yo = s+ ns (6)

where s and ns represent the signal and noise components inthe error-free output yo, respectively. At the application level,one is interested in the ratio of the signal power σ2

s to the noisepowers at the outputs of the M, E-block and ANT system.Thus, the following application level SNRs can be defined:

3

SNRM,a = 10log10(σ2s

σ2ns + σ2

η

) (7)

SNRE,a = 10log10(σ2s

σ2ns + σ2

e

) (8)

SNRANT,a = 10log10(σ2s

σ2ns + σ2

nr

) (9)

where σ2s , σ2

ns , σ2η , σ2

e , σ2yoand σ2

nr are the variances ofthe signal s, noise ns, M-block hardware error η, E-blockestimation error e, error-free output yo, and residual errornr = yo − y, respectively. It is also of interest to evaluatehow ’noisy’ the circuit fabric is with respect to an error-free(conventional) architecture. By definition, the output of suchan architecture is yo. Thus, we define the circuit level SNRsas follows:

SNRM,c = 10log10(σ2yo

σ2η

) (10)

SNRE,c = 10log10(σ2yo

σ2e

) (11)

SNRANT,c = 10log10(σ2yo

σ2nr

) (12)

If error detection is ideal, then nr ∈ {0, e}, and itsprobability mass function (PMF) Pηr (nr) is given by:

Pηr (nr) =

{1− pη if nr = 0

pη if nr = e

and

σ2nr = pησ

2e

where pη is the error rate of the M-block defined in (4).Therefore, (9) and (12) can be expressed as:

SNRANT,a = 10log10(σ2s

σ2n + pησ2

e

) (13)

SNRANT,c = 10log10(σ2yo

pησ2e

) (14)

Since e is the small magnitude LSB error and η is thelarge magnitude MSB error, pησ2

e � σ2e � σ2

η . This furtherimplies that SNRANT,a � SNRE,a � SNRM,a, andSNRANT,c � SNRE,c � SNRM,c. Thus, the output SNRof the ANT system is significantly greater than the SNR at theoutput of either the M or E-block. This phenomenon occursin spite of the fact that the ANT system output y ∈ {ya, ye}(see Fig. 1(a)), i.e., y equals the output of either the M or E-block. The reason for this unique feature of ANT is becauseit exploits the difference in the error statistics (see Fig. 1(b))at the output of the M and E-block.

B. EEG classification system using SVM

Portable health monitoring is an important class of appli-cations that can benefit from the design of energy efficientmachine learning kernels. It has been shown [12] that epilepticseizures can be efficiently detected by analyzing the EEGsignal using an SVM kernel. The EEG seizure classificationsystem [12] shown in Fig. 2(a) consists of a frequency selectivefilter bank to extract signal energy in the 0− 20 Hz range anda SVM classifier. The filter bank has passband of 3 Hz witha transition band of 1.5 Hz. Eight channels are employed tocover the entire frequency range [12].

SVM [13] is a popular supervised learning method forclassification and regression. An SVM operates by first train-ing the model (the training phase) followed by classification(the classification phase). During the training phase, featurevectors with labels are used to train the model. During theclassification phase, the SVM produces a predictive label whenprovided with a new feature vector. The SVM training can beformulated as the following optimization problem to determinethe maximum margin classifier [13] (see Fig. 2(b)):

min 12‖w‖

2+ C

∑i

ξi

s.t.yi(w

Txi−b) ≥ 1−ξiξi ≥ 0

(15)

where C is the cost factor, ξi is the soft margin, xi is thefeature vector, yi is the label corresponding to the featurevector xi, w is the weight vector, and b is the bias. The trainedmodel is represented by:

y = wTo x− b (16)

where wo are the optimized weights. It can be shown thatthe optimum weights are represented as a linear combinationof the feature vectors that lie on the margins (see Fig. 2(b)),i.e., support vectors:

wo =

Ns∑i=1

αiyixs,i (17)

where Ns and xs,i are the number of support vectors andith support vector, respectively. The linear model can thus berepresented as:

y =

Ns∑i=1

αiyixTs,ix− b (18)

The linear SVM in (18) can be easily extended into non-linear SVM by employing the kernel trick [13], resulting in:

y =

Ns∑i=1

αiyiK(xs,i,x)− b (19)

where K(xs,i,x) is a kernel function. Popular kernel func-tions include polynomial, radial basis function (RBF), andothers [14].

4

(a)

(b)

Figure 2. EEG seizure classifier with SVM: a) system architecture, and b)principle of SVM.

III. PROPOSED E-ANT TECHNIQUE

The key idea in E-ANT is to reuse part of the M-blockto generate an estimate of its error-free output yo. This is incontrast to conventional ANT, where an explicit E-block isrequired. We will show that such embedding of the E-blockcan be performed at the architectural level, where DPD isproposed to transform an existing architecture into an errorresilient one.

A. DPD based E-ANT

In RPR ANT, the M and E-block process the same data butwith different precisions. This redundancy can be exploited toembed the E-block into the M-block via DPD. In particular,DPD decomposes the M-block into MSB and LSB compo-nents, and employs the output of the MSB component as anestimate of the error-free M-block output yo. By ensuring thatthe critical path of the MSB block is always shorter than thatof the M-block, the requirements on the error statistics (seeFig. 1(b)) on the M and E-block are satisfied.

Let ya = f(x) denote the M-block functionality, where xand ya are the input and output of the M-block, respectively.A Bx-bit input x = x0x1...xBx−1 can be written in 2’scomplement form [15], as follows:

x = −x0 +

Bx−1∑i=1

xi2−i

= xM + xL2−(Bmsb−1) (20)

where xM is the value of Bmsb MSB bits, and xL is thevalue of Bx −Bmsb LSB bits, as shown below:

xM = −x0 +

Bmsb−1∑i=1

xi2−i (21)

xL =

Bx−1∑i=Bmsb

xi2−(i−Bmsb+1) (22)

Therefore, the M-block output is expressed as:

ya = f(x) = f(xM + xL2−(Bmsb−1))

In E-ANT, we decompose f(x) as follows:

ya = f(x)

= f(xM + xL2−(Bmsb−1))

= g(fM (xM ), fL(xM , xL)) (23)

where fM (xM ) and fL(xM , xL) are functions that arecombined by the operator g(·) to generate the final output ya.Since this decomposition utilizes the finite precision nature ofarithmetic units, it is referred to as DPD. We show that DPDexists if f(x) is n-times differentiable or can be piecewiseapproximated. An E-ANT system can be obtained via DPD byensuring that: 1) the critical path of fM (xM ) is shorter thanthat of g(fM (xM ), fL(xM , xL)), and 2) fM (xM ) generatesan estimate ye of the error-free output yo. The operation ofDPD based E-ANT is described as follows:

ya = g(fM (xM ), fL(xM , xL))

ye = fM (xM )

y =

{ya if |ya − ye| ≤ Thye otherwise

where Th is the error detection threshold as in (3). Next,we describe several methods to achieve DPD.

B. DPD via Taylor expansion

Taylor expansion can be employed to achieve DPD. If f(x)is n-times differentiable in the input range x ∈ [xl, xu]. Inthe appendix A, we show that DPD for f(x) using Taylorexpansion is given by:

f(x) ≈ fM (xM ) + fL(xM , xL) (24)

where

fM (xM ) = f(x0)

+

n∑k=1

k∑i=0

[f (k)(x0)

k!(ki

)(−x0)k−i]xi,M

fL(xM , xL) =

n∑k=1

k∑i=0

[f (k)(x0)

k!(ki

)(−x0)k−i]xi,L

xi,M = xiM

xi,L =

i−1∑j=0

(ij

)xjM (xL2−(Bmsb−1))i−j

5

where xM and xL are defined in (21) and (22), respectively.As a special case, when a first order Taylor expansion is

employed at x0 = 12 (xl + xu), i.e., at center of the input

dynamic range, (24) simplifies into:

f(x) ≈ f(x0) + f ′(x0)(x− x0) (25)

where f ′(x0) is the first order derivative of f(x) at x0.Substituting (20) into (25), we obtain the DPD of f(x) asfollows:

f(x) ≈ f(x0) + f ′(x0)(xM + xL2−(Bmsb−1)

−x0,M − x0,L2−(Bmsb−1))

= fM (xM ) + fL(xL)2−(Bmsb−1) (26)

where

fM (xM ) = f(x0) + f ′(x0)(xM − x0,M )

fL(xL) = f ′(x0)(xL − x0,L)

and fM (xM ) can be used as the E-block. Note that inthe decomposition in (26), only x is decomposed into MSBand LSB components and the factor f ′(x0) remains in fullprecision. If a simpler E-block is required, f ′(x0) can alsobe decomposed into MSB and LSB part, as shown in SectionIII-D. The pivot point x0 should be chosen such that the errormetric, e.g., the mean square error, between the original andthe E-ANT kernel is minimized.

C. DPD via piecewise linear (PWL) approximation

The PWL approximation can be employed when f(x) (x ∈[xl, xu]) is non-differentiable or the input dynamic range islarge.

The PWL approximation employs N+1 points (xk, f(xk))where xk = xl + k

N (xu − xl) and k = 0, 1, ..., N toapproximate f(x) as:

f(x) ≈N∑k=1

pk(x)

pk(x) =

{akx+ bk xk ≤ x < xk+1

0 otherwise(27)

where x0 = xl, xN = xu, ak = f(xk+1)−f(xk)xk+1−xk , and bk =

xk+1f(xk)−xkf(xk+1)xk+1−xk . Each segment pk(x) can be decomposed

by noting that for a linear function p(x), substituting for xfrom (20), we have

p(x) = p(xM + xL2−(Bmsb−1))

= p(xM ) + p(xL)2−(Bmsb−1) (28)

Therefore, substituting (20) into (27), we obtain:

pk(x) =

{pk,M (xM ) + pk,L(xL)2−(Bmsb−1) xk ≤ x < xk+1

0 otherwise(29)

where

pk,M (xM ) = akxM + bk

pk,L(xL) = akxL

and pk,M (xM ) can be employed as the E-block.Note that other piecewise approximation methods such as

spline interpolation [16] where each segment is approximatedwith a low order polynomial can also be employed for DPD.Each low order polynomial can be decomposed in a mannersimilar to (24).

Next, we apply DPD to obtain E-ANT architectures forarithmetic units and compute kernels commonly used in signalprocessing and machine learning.

D. E-ANT Arithmetic Unit Architectures

1) E-ANT Adder: The output of a two-operand adder isgiven by:

ya = x1 + x2

where x1 and x2 are the input operands. We first decomposethe operands into MSB and LSB components according to(20):

x1 = x1M + x1L2−(Bmsb−1)

x2 = x2M + x2L2−(Bmsb−1)

where xiM and xiL are defined in (21) and (22), respec-tively.

Since addition is a linear function, DPD can be easilyobtained from (28) as follows:

ya = x1M + x2M + x1L2−(Bmsb−1) + x2L2−(Bmsb−1)

= fM + fL2−(Bmsb−1) (30)

where fM = x1M + x2M and fL = x1L + x2L. The dataflow graph (DFG) and the symbol of the E-ANT adder areshown in Fig. 3(a) and Fig. 3(b), respectively.

2) E-ANT Multiplier: Employing the DPD in (21)-(23), theE-ANT multiplier can be derived as follows:

ya = x1x2

= (x1M + x1L2−(Bmsb−1))(x2M + x2L2−(Bmsb−1))

= fM + fL2−(Bmsb−1) (31)

where fM = x1Mx2M and fL = x1Lx2M + x1x2L. Fig.4(a) and Fig. 4(b) show the DFG and symbol of the E-ANTmultiplier.

6

(a)

(b)

Figure 3. E-ANT Adder: a) DFG, and b) symbol.

(a)

(b)

Figure 4. E-ANT Multiplier: a) DFG, and b) symbol.

3) E-ANT multiply-accumulator (MAC): MAC operation isdescribed as:

ya[n] = x[n]w[n] + ya[n− 1] (32)

We first decompose x[n], w[n] and y[n − 1] according to(20):

x[n] = xM [n] + xL[n]2−(Bmsb−1) (33)

w[n] = wM [n] + wL[n]2−(Bmsb−1) (34)

ya[n− 1] = ya,M [n− 1] + ya,L[n− 1]2−2(Bmsb−1) (35)

The E-ANT MAC can be obtained by substituting (33)-(35)in to (32), and employing (30)-(31) to decompose ya[n] asfollows:

ya[n] = xM [n]wM [n] + ya,M [n− 1]

+ (xL[n]wM [n] + (xM [n] + xL[n]2−(Bmsb−1))wL[n]

+ ya,L[n− 1]2−(Bmsb−1))2−(Bmsb−1)

= xM [n]wM [n] + ya,M [n− 1]

+ (xL[n]wM [n] + x[n]wL[n]

+ ya,L[n− 1]2−(Bmsb−1))2−(Bmsb−1)

= fM + fL2−(Bmsb−1)

+ ya,L[n− 1]2−(Bmsb−1))2−(Bmsb−1) (36)

where fM = xM [n]wM [n] + ya,M [n − 1] and fL =xL[n]wM [n] + x[n]wL[n] + ya,L[n− 1]2−(Bmsb−1). Fig. 5(a)and Fig. 5(b) show the DFG and the symbol of the E-ANTMAC.

(a)

(b)

Figure 5. E-ANT MAC unit: a) DFG, and b) symbol.

E. E-ANT Signal Processing and Machine Learning Kernels

Complex E-ANT kernels can be derived by employing theE-ANT arithmetic units derived in section III-D.

1) E-ANT FIR Filter: One of the most important kernels ininformation processing is filtering/convolution. We can derivean E-ANT FIR filter by employing (30)-(31) as follows:

ya[n] =

N−1∑i=0

w[i]x[n− i]

=

N−1∑i=0

fiM +∑i

fiL2−(Bmsb−1)

7

where fiM = xM [n− i]wM [i] and fiL = xL[n− i]wM [i] +x[n − i]wL[i] for i = 0...N − 1. Fig. 6 shows the DFGs ofthe direct form and transposed form E-ANT FIR filter wherewe make use of the symbols in Fig. 3(b), 4(b), and 5(b) tosimplify the DFGs.

(a)

(b)

Figure 6. E-ANT FIR filter: a) the DFG of direct form FIR filter, and b) theDFG of transposed form FIR filter.

2) E-ANT Fast Fourier Transform (FFT) Butterfly Unit(BU): BU is the main data processing unit in FFT processors.A general BU implements the following function:

y1r,a = x1r + x2ry1i,a = x1i + x2i

d = x1 − x2y2r,a = drWr − diWi

y2i,a = drWi + diWr

where x1 and x2 are the inputs, y1 and y2 are the outputs,and W is the twiddle factor. The real and imaginary parts aredenoted by r and i subscripts, respectively. E-ANT FFT BUcan be derived as shown in Table I.

Table IDPD FOR FFT BU

y1r,a = x1r + x2r = y1r,M + y1r,L2−(Bmsb−1),where

y1r,M = x1r,M + x2r,M and y1r,L = x1r,L + x2r,L.y1i,a = x1i + x2i = y1i,M + y1i,L2

−(Bmsb−1), wherey1i,M = x1i,M + x2i,M and y1i,L = x1i,L + x2i,L.

y2r,a = drWr − diWi = y2r,M + y2r,L2−(Bmsb−1), where

y2r,M = dr,MWr,M − di,MWi,M andy2r,L = dr,LWr,M + drWr,L − (di,LWi,M + diWi,L).

y2i,a = drWi + diWr = y2i,M + y2i,L2−(Bmsb−1), where

y2i,M = dr,MWi,M + di,MWr,M andy2i,L = dr,LWi,M + drWi,L + di,LWr,M + diWr,L

The DFG of the E-ANT FFT BU is shown in Fig. 7.3) E-ANT Exponential Kernel: Exponential kernel (e−x)

is a critical component in many machine learning algorithmssuch as kernel SVM [12], [17], Gaussian Mixture Model [18],and others [19], [20]. Taylor expansion in Section III-B and

Figure 7. DFG of the E-ANT FFT butterfly unit.

PWL approximation in Section III-C can be employed toobtain E-ANT exponential kernels.

Assuming that the input dynamic range is scaled to [0,1], a2nd order Taylor expansion leads to:

ya = e−x ≈ e−x0 − e−x0 × (x− x0) +e−x0(x− x0)2

2(37)

When x0 = 0.5, (37) simplifies to

ya ≈ ax2 + bx+ c (38)

where a = 0.3033, b = −0.9098, and c = 0.9856. TheTaylor expansion based E-ANT exponential block can thus bederived from (24) as follows:

ya ≈ fM + fL2−(Bmsb−1)

where

fM = ax2M + bxM + c

fL = a(2xMxL + x2L2−(Bmsb−1)) + bxL

The DFG is shown in Fig. 8(a).Alternatively, PWL approximation can be employed to

obtain an E-ANT exponential kernel. Assume that two linearfunctions on [0, 0.5] and [0.5, 1] are used to approximate theexponential function on the interval [0, 1], then according to(27):

ya = e−x ≈{

a1x+ b1 0 ≤ x < 0.5a2x+ b2 0.5 ≤ x < 1

where a1 = −0.7869, b1 = 1, a2 = −0.4773 and b2 =0.8452. We first decompose ai, bi (i = 1, 2) according to(20):

ai = ai,M + ai,L2−(Bmsb−1)

bi = bi,M + bi,L2−2(Bmsb−1)

Since each segment is linear, they can be decomposed byusing (28):

8

ya,i = aix+ bi= fi,M + fi,L2−(Bmsb−1)

where

fi,M = ai,MxM + bi,M

fi,L = ai,MxL + ai,Lx+ bi,L2−(Bmsb−1)

The resultant E-ANT exponential kernel has a reconfig-urable architecture where different approximations are chosenaccording to the input values. The DFG of the PWL basedE-ANT exponential kernel is shown in Fig. 8(b).

(a)

(b)

Figure 8. DFG of the E-ANT exponential kernel: a) Taylor expansion based,and b) PWL approximation based

IV. SIMULATION RESULTS

This section presents the design optimization of the pro-posed E-ANT technique, and shows the simulation results ina 45 nm CMOS process when E-ANT is applied to an SVMEEG classification system. This system is chosen due to itsdata path dominant architecture, which makes it an attractiveplatform to apply DPD. It has been shown that other machinelearning methods such as extreme learning machines (ELMs)[21], [22], [23] might outperform SVM in terms of learningspeed or accuracy in EEG seizure classification problems.DPD is also applicable to these algorithms. However, in thispaper, we focus on SVM as it has been implemented inintegrated circuits [24] and hence offers a point of comparison.

A. Evaluation Methodology

Figure 9(a) shows the evaluation methodology employedto quantify system-level performance metrics and to estimatesystem-level energy consumption that integrates circuit, archi-tecture, and system level design variables. The methodologyconsists of two parts: 1) system-level error injection, and 2)system-level energy estimation. Comparison of the proposed E-ANT with conventional approach (no error compensation) andretraining based approach in [11] is done using a commercial45 nm CMOS process. The results are shown in Section IV-Ein Fig. 14.

System-level error injection is done as follows:1) Characterize delay vs. Vdd of basic gates such as AND

and XOR using HSPICE for 0.2 V ≤ Vdd ≤ 1.2 V.2) Develop structural Verilog HDL models of key kernels

needed in the EEG classification system using the basicgates characterized in Step 1. These kernels are a 12 binput, 8 b coefficient, and 16 b output, 44-tap FIR filter(used in the FE) and a 8 b input, 8 b coefficient, and 19 boutput vector-matrix multiplication kernel (used in theSVM CE). In designing the FE and CE, floating pointmodels were first developed to verify functionality. Toconvert floating point design to fixed point data path, weinstrumented the dynamic range at the output of the FIRfilter and the vector-matrix multiplication kernel usingapplication level data. The filter output was truncatedto 16 b with 1 b sign bit, 2 b integer length, and 13 bfraction length. Full precision was maintained in theabsolute-sum block as shown in Fig. 12, and the outputwas truncated and normalized to 8 b with 1 b sign bit,and 7 b fraction as required by the SVM algorithm.Truncation was performed at the output of the first andsecond MAC in the CE (see Fig. 12) to manage bit-growth in the similar manner.

3) HDL simulations of these kernels were conducted atdifferent voltages by including the appropriate voltage-specific delay numbers obtained in Step 1 into the HDLmodel. The error PMFs of these kernels and error ratespη are obtained for different supply voltages (and thusvoltage overscaling factor Kvos).

4) System performance evaluation and design optimizationare done by injecting errors into a fixed point MATLAB-model of the EEG classification system. The errors areobtained by sampling the error PMFs obtained in Step3.

Figure 9(c) shows the error PMF Pη(η) of the 44-tap low passFIR filter used in the FE at Vdd = 0.9 V (fclk = 76 MHz)which corresponds to a Kvos = 0.75 and an error rate pη =0.05, and Fig. 9(d) shows the error rate pη increases from10−5 at Vdd = 1.15 V to 0.99 at Vdd = 0.5 V as the voltagescales down.

System-level energy estimation is done as follows:1) Obtain a full adder (FA) count NFA of the kernel being

analyzed.2) Conduct a one-time characterization of the energy con-

sumption of a FA incorporating both dynamic and leak-age energies as follows:

9

(a) (b)

(c) (d)

Figure 9. Evaluation methodology: a) simulation setup, and b) comparisonof the energy model and HSPICE simulations in a 45 nm CMOS process, c)error PMF at Vdd = 0.9V, and d) error rate pη vs. Vdd for the 44-tap lowpass filter employed in the FE, the CHB-MIT EEG data set [11] is employedas input.

EFA = CFAV2dd + VddIleak(Vdd)

1

fclk(39)

with

Ileak(Vdd) = µCoxW

L(m−1)V 2

T e−VtmVT e

−ηdVddmVT (1−e

−VddVT )

(40)where CFA is the effective load capacitance of theFA and is extracted from HSPICE, Vdd is the supplyvoltage, Vt, VT , µ, Cox, and ηd are the thresholdvoltage, the thermal voltage, the carrier mobility, the gatecapacitance per unit W/L, and the drain induced barrierlowering (DIBL) coefficient, respectively, obtained fromthe process files, and m is a constant related to the sub-threshold slope factor and is a fitting parameter.

3) The energy estimate of the kernel is obtained as Eop =NFAEFA.

Figure 9(b) shows the modeling results of the FA and ripplecarry adder (RCA) for various bit width demonstrating theaccuracy and scalability of the energy model. The energymodel is within 5% (for 0.2 V ≤ Vdd ≤ 1.2 V) of circuitsimulation results.

B. E-ANT Design Optimization

The methodology in Fig. 9(a) is employed to performoptimization for the E-ANT kernels proposed in Sect. III,and the E-ANT MAC kernel in Fig. 5(a) is used as anexample. Since we adopt VOS to obtain different error rates,the parameters to be optimized are the voltage overscalingfactor Kvos and the E-block bit width Bmsb, where we assumethat Bx,msb = Bw,msb = Bmsb. The optimization framework

is general enough to include the case when Bx,msb 6= Bw,msb.A grid search algorithm is employed to systematically de-termine the optimum setting K∗vos and B∗msb satisfying theperformance metric, as shown in Algorithm 1 below. We adoptmean squared error (MSE) with respect to the floating pointkernel as the performance metric:

MSE = E(y − yfl)2 (41)

where y and yfl indicate the E-ANT and floating pointoutput, respectively. The maximum E-block length Bmaxunder which the E-block does not make errors is determinedby Kvos. The optimization routine gives the optimum E-ANTconfiguration, including K∗vos, B

∗msb and minimum energy

E∗op, at the output.

Algorithm 1 Energy optimization algorithm for E-ANT1. Initialize K∗vos = 1, B∗msb = 0, E∗op = energy of conventionalMAC, MSEreq = specified MSE requirement.2. Kvos = Kvos − ∆, Bmsb = 0. Obtain maximum E-blockprecision Bmax to ensure error-free E-block operation.3. Bmsb = Bmsb+1. If Bmsb > Bmax, then exit, else computeMSE according to (23).4. If MSE < MSEreq, then calculate energy E(Kvos)according to (21), else go to step 35. If E∗op > E(Kvos), then E∗op = E(Kvos), and B∗msb =Bmsb6. Go to step 2

Algorithm 1 is employed to optimize E-ANT MAC for 8band 16b precision with MSE requirements of 10−2 ∼ 10−5.The iso-MSE plots in the Bmsb and pη plane (see Fig. 10(a)and Fig. 10(c)) indicate that the optimum Bmsb increases asthe error rate pη increases because a higher precision E-blockis needed to compensate for the M-block errors. Figure 10(b)shows that the 8 bit E-ANT MAC achieves energy savings of16% ∼ 69%, while as the 8 bit ANT MAC fails to achieveenergy savings at the tight MSE requirement of 10−5 dueto E-block overheads. Figure 10(d) shows that the 16 bit E-ANT MAC achieves 59% energy savings compared with theconventional MAC. The overhead of the E-ANT architectureis below 8% compared with the 36.4% and 13.9% overheadsfor the 8 bit and 16 bit ANT architecture, respectively.

Figure 11 shows that the energy savings increase as theMSE requirement increases for a fixed Bx. This is becausea larger MSE requirement allows the MAC to operate at ahigher pη and can thus reduce the E-block overheads. Thisis also confirmed in Fig. 10(b) and Fig. 10(d). Additionally,the energy savings increase as Bx increases for a fixed MSErequirement because a large Bx tends to tolerate more LSBerrors, thus enabling the MAC to operate at a higher pη .

C. EEG Classification System Set-up

To evaluate the performance of E-ANT, the filter kernelin the FE and the vector-matrix multiplication kernel in theSVM CE shown in Fig. 12 are implemented employing theE-ANT MAC as shown in Fig. 5, and are characterized viathe procedure described in section IV-B. For the filter bank in

10

(a) (b)

(c) (d)

Figure 10. Optimization of E-ANT MAC, the dashed line illustrates thatthe maximum E-block precision Bmax decreases as pη increases, indicatingthat to ensure error-free E-block operation, the E-block bit width is upperbounded. The solid lines show the optimum Bmsb configuration for eachpη at different MSE requirements, with the circle marker indicating the(B∗msb, p∗η) pair achieving the MSE requirements with minimum Eop: a)

optimization results of an 8×8 E-ANT MAC for different MSE requirements,b) normalized energy of an 8×8 conventional MAC, ANT MAC and E-ANTMAC, c) optimization results of a 16× 16 E-ANT MAC for different MSErequirements, and d) normalized energy of a 16 × 16 conventional MAC,ANT MAC, and E-ANT MAC.

Figure 11. Energy savings vs. input precision and MSE.

the FE, we use an input of 12 bit, with the MSB 8 bit takenas the E-block. For the CE, the input precisions of the twoMACs are chosen to be 8 bit, and E-block precisions are 4bit.

We employ the CHB-MIT EEG data set [11] to train theSVM and use leave-one-out cross validations to evaluate thesystem performance. The system performance metric em-ployed is the true positive (TP) rate ptp and false positive/alarm(FP) rate pfp, defined as:

ptp =TP

TP + FN

pfp =FP

FP + TN

where TP , FN , FP , and TN are the number of truepositives, false negatives, false positives, and true negatives,respectively. A good classifier achieves high values of ptp (> 0.9) at a small constant false alarm rate pfp (< 0.01).

Three implementations are considered, i.e., the uncompen-sated system (denoted as CONV), the system which performsretraining with erroneous features, similar to the one proposedin [11] (denoted as RETRAIN), and the system with E-ANT (denoted as E-ANT). In the retraining method [11], theclassifier is trained with features extracted in the presence ofVOS errors. Unlike in the retraining method [11] where theCE needs to be error free, E-ANT can tolerate errors in boththe FE and CE. Therefore, two setups are considered in ourexperiment: 1) errors in FE only, and 2) errors in both FEand CE. The maximum value of the error rate pη for whichptp > 0.9 and pfp < 0.01 is referred to as the error tolerancepη−max of the architecture. In the first setup, pη−max is theerror rate in the FE, and in the second setup, pη−max is themaximum of the error rate in the FE and CE.

Figure 12. Second order polynomial kernel SVM EEG classification systemarchitecture.

D. SNR Performance

Figure 13(a) shows that the improvement in the applicationlevel SNR (see (7)-(9)) achieved by E-ANT at the FE outputis significant. In particular, the SNR of the M-block (also theSNR of the conventional system with errors), SNRM,a, dropscatastrophically from 42 dB to 10 dB for values of pη as lowas 8 × 10−4. The SNR of the E-block, SNRE,a, is constantat 23 dB for pη ≤ 0.42. This is because the E-block makessmall magnitude estimation errors e. For pη > 0.42, SNRE,adrops catastrophically as the E-block also starts to make largemagnitude timing errors. In contrast, the ANT system SNR,SNRANT,a, is at least 10 dB higher than either SNRE,a orSNRM,a for values of pη as high as 0.1, and approaches theE-block SNR as pη increases.

The circuit level SNRs (see (10) - (12)) also exhibit a similartrend in Fig. 13(b). Furthermore, the SNR analysis in sectionII-A is validated by plotting (13) and (14) in Fig. 13(a) andFig. 13(b), respectively.

11

(a) (b)

Figure 13. SNR at the output of the FE: a) application level SNR, and b)circuit level SNR.

E. Classification Performance and Energy Savings

As shown in Fig. 14(a), ptp drops sharply as circuit errorrate increases in CONV system where no SEC is applied.The RETRAIN system does slightly better than the CONVsystem because the classifier is retrained to adapt to the erroraffected features. However, pη−max is still only around 10−3.This is due to the fact that the errors under investigation aretiming errors. Unlike the stuck-at-faults errors in [11], timingerrors are dynamic and depend on the state of circuit, so theerror pattern observed during training might not be the sameas during the test. In contrast, when E-ANT is applied, ptpdegrades gracefully as pη increases. As a result, the E-ANTsystem can achieve pη−max as high as 0.38. Figure 14(a) alsoshows that the ptp is always lower than 0.9 when only E-block is employed, i.e., the E-block on its own is unable tomeet the performance specifications. Similarly, when errorspresent in both FE and CE (Fig. 14(b)), both the CONVsystem and RETRAIN system achieve pη−max < 10−3, whileE-ANT achieves a pη−max of 0.17. These are of 2-orders-of-magnitude (errors in FE only) and 3-orders-of-magnitude(errors in both FE and CE) greater than the existing systems.The receiver operating characteristic (ROC) curve at pη−maxis shown in Fig. 14(c) when errors are in FE only, and in Fig.14(d) when errors are in both FE and CE. In both experiments,the ROC of the CONV as well as the RETRAIN systemapproaches the ROC of a random classifier which outputs ±1with equal probability, while the ROC of the E-ANT system(w/ or w/o retraining) remains close to the ROC of an idealclassifier.

Principle component analysis (PCA) is performed on thefeature vectors to understand the reason why CONV systemfails but E-ANT system is able to maintain good performance.Fig. 14(e) shows that when no SEC is applied, circuit errorshave two effects on the feature vectors: 1) errors make it harderto separate the positive and negative samples, and 2) the entirefeature space is shifted due to the accumulation block in theFE. The SVM fails to correctly perform classification withoutknowledge of the error statistics. Figure 14(f) shows that thelarge magnitude errors are compensated and converted to smallresidual errors when E-ANT is applied. This will cause a verysmall shift in the feature space. As a result, the SVM classifiercan still perform correct classification. One way to improveE-ANT further is to incorporate retraining. In this method,

the classifier is trained employing features that are subject toresidual errors after the correction via E-ANT. However, asshown in Fig. 14(a), the improvement is minor due to the factthat the residual errors are typically small.

(a) (b)

(c) (d)

(e) (f)

Figure 14. Simulation results: a) ptp of CONV, RETRAIN and E-ANT withpfp = 0.01 when errors are in FE only, b) ptp of CONV, RETRAIN andE-ANT with pfp = 0.01 when errors are in both FE and CE, c) ROC curveof CONV, RETRAIN and E-ANT at pη−max when errors are in FE only,d) ROC curve of CONV, RETRAIN and E-ANT at pη−max when errors arein both FE and CE, e) PCA results of error-free and erroneous features forCONV, and f) PCA results of error-free and erroneous features for E-ANT.

Table II compares the pη−max, FE energy/feature (EF ), andCE energy/decision (EC) of the three systems. The E-ANTsystem can achieve pη−max of 0.38 when errors are in FEonly, and 0.17 when errors are in both FE and CE. WhenVOS is applied for energy saving, the E-ANT system is ableto achieve 51% energy savings when errors are in FE onlycompared with the CONV system. When both FE and CE arein error, the E-ANT system is able to achieve 43% and 29%energy savings in the FE and CE, respectively.

V. CONCLUSIONS

In this paper, an embedded statistical error compensationtechnique named E-ANT is proposed and employed as a lowpower technique in designing machine learning and signal

12

Table IIPERFORMANCE AND ENERGY COMPARISON

Errors in FE only Errors in both FE and CEpη−max Energy savings in FE pη−max Energy savings in FE Energy savings in CE

Conventional 8× 10−4 NA 2× 10−4 NA NAError resilient retraining 3× 10−3 7% 8× 10−4 13% 12%

E-ANT 0.38 51% 0.17 43% 29%

processing kernels. E-ANT employs DPD to embed the RPRE-block into the M-block. The DPD is general and is derivedfor a wide class of compute kernels. The effectiveness of theproposed E-ANT technique has been demonstrated throughthe design of an SVM EEG seizure classification system wheresimulation results in a commercial 45 nm CMOS process showthat E-ANT achieves up to 0.38 error tolerance and up to 51%energy savings compared with an uncompensated system.

This works shows that by exploring architectural transforms,it is possible to design architectures that are inherently errorresilient without an explicit E-block. It opens up a few researchdirections to extend or generalize existing SEC techniques.In particular, other low complexity error compensation tech-niques can be investigated that improve upon existing SECtechniques such as SSNOC and soft-NMR [2]. Moreover, withthe adoption near/sub-threshold voltage design and continuousscaling of CMOS process, PVT induced errors and defectsinduced errors are becoming a growing concern for the designof low power embedded platforms. The effectiveness of errorresiliency techniques to enhance system robustness in presenceof these new error models can also be explored.

ACKNOWLEDGMENT

This work was supported in part by Systems on NanoscaleInformation fabriCs (SONIC), one of the six SRC STARnetCenters, sponsored by MARCO and DARPA.

APPENDIX - A

In this appendix, we derive (24). Consider the nth orderTaylor expansion of f(x) as shown below:

f(x) = f(x0) +

n∑k=1

f (k)(x0)

k!(x− x0)k (42)

Substituting the binomial expansion of (x − x0)k in (42),we obtain

f(x) = f(x0)

+

n∑k=1

f (k)(x0)

k!

k∑i=0

(ki

)xi(−x0)k−i

= f(x0)

+

n∑k=1

k∑i=0

[f (k)(x0)

k!(ki

)(−x0)k−i]xi (43)

Next, employing (20) and the binomial expansion, we writexi as:

xi = (xM + xL2−(Bmsb−1))i

=

i∑j=0

(ij

)xjM (xL2−(Bmsb−1))i−j

= xiM +

i−1∑j=0

(ij

)xjM (xL2−(Bmsb−1))i−j

= xi,M + xi,L (44)

where

xi,M = xiM

xi,L =

i−1∑j=0

(ij

)xjM (xL2−(Bmsb−1))i−j

Substituting (44) into (43), we obtain:

f(x) ≈ f(x0)

+

n∑k=1

k∑i=0

[f (k)(x0)

k!(ki

)(−x0)k−i]xi

= f(x0)

+

n∑k=1

k∑i=0

[f (k)(x0)

k!(ki

)(−x0)k−i](xi,M + xi,L)

= fM (xM ) + fL(xM , xL) (45)

where

fM (xM ) = f(x0)

+

n∑k=1

k∑i=0

[f (k)(x0)

k!(ki

)(−x0)k−i]xi,M

fL(xM , xL) =

n∑k=1

k∑i=0

[f (k)(x0)

k!(ki

)(−x0)k−i]xi,L

This completes the derivation of (24).

REFERENCES

[1] S. Zhang and N. Shanbhag, “Embedded error compensation for energyefficient dsp systems,” in Signal and Information Processing (Global-SIP), 2014 IEEE Global Conference on, Dec 2014, pp. 30–34.

[2] N. Shanbhag, R. Abdallah, R. Kumar, and D. Jones, “Stochastic compu-tation,” in Design Automation Conference (DAC), 2010 47th ACM/IEEE,June 2010, pp. 859–864.

[3] S. Borkar, T. Karnik, S. Narendra, J. Tschanz, A. Keshavarzi, and V. De,“Parameter variations and impact on circuits and microarchitecture,” inDesign Automation Conference, 2003. Proceedings, June 2003, pp. 338–342.

13

[4] K. Nepal, R. Bahar, J. Mundy, W. Patterson, and A. Zaslavsky, “Tech-niques for designing noise-tolerant multi-level combinational circuits,” inDesign, Automation Test in Europe Conference Exhibition, 2007. DATE’07, April 2007, pp. 1–6.

[5] D. Ernst, S. Das, S. Lee, D. Blaauw, T. Austin, T. Mudge, N. S. Kim,and K. Flautner, “RAZOR: circuit-level correction of timing errors forlow-power operation,” Micro, IEEE, vol. 24, no. 6, pp. 10–20, Nov 2004.

[6] D. Blaauw, S. Kalaiselvan, K. Lai, W.-H. Ma, S. Pant, C. Tokunaga,S. Das, and D. Bull, “RAZOR II: In situ error detection and correctionfor PVT and SER tolerance,” in Solid-State Circuits Conference, 2008.ISSCC 2008. Digest of Technical Papers. IEEE International, Feb 2008,pp. 400–622.

[7] J. Tschanz, K. Bowman, C. Wilkerson, S.-L. Lu, and T. Karnik, “Re-silient circuits; enabling energy-efficient performance and reliability,”in Computer-Aided Design - Digest of Technical Papers, 2009. ICCAD2009. IEEE/ACM International Conference on, Nov 2009, pp. 71–73.

[8] C.-H. Chen, D. Blaauw, D. Sylvester, and Z. Zhang, “Design andevaluation of confidence-driven error-resilient systems,” Very LargeScale Integration (VLSI) Systems, IEEE Transactions on, vol. 22, no. 8,pp. 1727–1737, Aug 2014.

[9] R. Abdallah and N. Shanbhag, “An Energy-Efficient ECG Processorin 45-nm CMOS Using Statistical Error Compensation,” Solid-StateCircuits, IEEE Journal of, vol. 48, no. 11, pp. 2882–2893, Nov 2013.

[10] M. Chen, J. Han, Y. Zhang, Y. Zou, Y. Li, and X. Zeng, “An error-resilient wavelet-based ecg processor under voltage overscaling,” inBiomedical Circuits and Systems Conference (BioCAS), 2014 IEEE, Oct2014, pp. 628–631.

[11] N. Verma, K. H. Lee, K. J. Jang, and A. Shoeb, “Enabling system-levelplatform resilience through embedded data-driven inference capabilitiesin electronic devices,” in Acoustics, Speech and Signal Processing(ICASSP), 2012 IEEE International Conference on, March 2012, pp.5285–5288.

[12] J. Yoo, L. Yan, D. El-Damak, M. Altaf, A. Shoeb, and A. Chandrakasan,“An 8-Channel Scalable EEG Acquisition SoC With Patient-SpecificSeizure Classification and Recording Processor,” Solid-State Circuits,IEEE Journal of, vol. 48, no. 1, pp. 214–228, Jan 2013.

[13] V. Vapnik, “An overview of statistical learning theory,” Neural Networks,IEEE Transactions on, vol. 10, no. 5, pp. 988–999, Sep 1999.

[14] C. J. C. Burges, “A tutorial on support vector machines for patternrecognition,” Data Mining and Knowledge Discovery, vol. 2, pp. 121–167, 1998.

[15] K. K. Parhi, VLSI digital signal processing systems : designand implementation. New York: Wiley, 1999, a Wiley-Intersciencepublication. [Online]. Available: http://opac.inria.fr/record=b1097682

[16] M. T. Heath, Scientific Computing: An Introductory Survey, 2nd ed.,E. M. Munson, Ed. McGraw-Hill Higher Education, 1996.

[17] A. Ganapathiraju, J. Hamaker, and J. Picone, “Applications of SupportVector Machines to speech recognition,” Signal Processing, IEEE Trans-actions on, vol. 52, no. 8, pp. 2348–2355, Aug 2004.

[18] H. Hu, M.-X. Xu, and W. Wu, “GMM Supervector Based SVMwith Spectral Features for Speech Emotion Recognition,” in Acoustics,Speech and Signal Processing, 2007. ICASSP 2007. IEEE InternationalConference on, vol. 4, April 2007, pp. IV–413–IV–416.

[19] S. Lawrence, C. Giles, A. C. Tsoi, and A. Back, “Face recognition: aconvolutional neural-network approach,” Neural Networks, IEEE Trans-actions on, vol. 8, no. 1, pp. 98–113, Jan 1997.

[20] D. Wedge, D. Ingram, D. McLean, and Z. Bandar, “On global-localartificial neural networks for function approximation,” Neural Networks,IEEE Transactions on, vol. 17, no. 4, pp. 942–952, July 2006.

[21] Y. Song, J. Crowcroft, and J. Zhang, “Automatic epileptic seizuredetection in {EEGs} based on optimized sample entropy andextreme learning machine,” Journal of Neuroscience Methods, vol.210, no. 2, pp. 132 – 146, 2012. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0165027012002671

[22] Q. Yuan, W. Zhou, S. Li, and D. Cai, “Epileptic {EEG} classificationbased on extreme learning machine and nonlinear features,” EpilepsyResearch, vol. 96, no. 1âAS2, pp. 29 – 38, 2011. [Online]. Available:http://www.sciencedirect.com/science/article/pii/S0920121111001185

[23] G.-B. Huang, H. Zhou, X. Ding, and R. Zhang, “Extreme learningmachine for regression and multiclass classification,” Systems, Man, andCybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 42, no. 2,pp. 513–529, April 2012.

[24] K. Lee, S.-Y. Kung, and N. Verma, “Low-energy formulations of supportvector machine kernel functions for biomedical sensor applications,”Journal of Signal Processing Systems, vol. 69, no. 3, pp. 339–349, 2012.

Sai Zhang received the B.S. in Electronic Scienceand Technology (with distinction) from Universityof Science and Technology of China. Since 2010,he has joined the University of Illinois at Urbana-Champaign and received the M.S. degree in Electri-cal and Computer Engineering in 2012. Currently, heis a PhD student in Electrical and Computer Engi-neering, University of Illinois at Urbana-Champaign.Since 2010, he has been a Research Assistant in Co-ordinated Science Laboratory. During the summersof 2013 and 2014, he was with Texas Instruments,

Inc., Dallas, in the Embedded Processing Systems R&D Labs, where he wasinvolved with designing reconfigurable error correction codec for memory andresearching written once memory coding schemes for NVMs. His researchinterests are in the design of low power error resilient integrated circuits andsystems for digital signal processing and machine learning. He is a studentmember of the IEEE.

Naresh R. Shanbhag is the Jack Kilby Professor ofElectrical and Computer Engineering at the Univer-sity of Illinois at Urbana-Champaign. He receivedhis Ph.D. degree from the University of Minnesota(1993) in Electrical Engineering. From 1993 to1995, he worked at AT&T Bell Laboratories atMurray Hill where he led the design of high-speedtransceiver chip-sets for very high-speed digital sub-scriber line (VDSL), before joining the Universityof Illinois at Urbana-Champaign in August 1995.He has held visiting faculty appointments at the

National Taiwan University (Aug.-Dec. 2007) and Stanford University (Aug.-Dec. 2014). His research interests are in the design of energy-efficientintegrated circuits and systems for communications, signal processing andmachine learning. He has more than 200 publications in this area and holdsthirteen US patents. He is also a co-author of the research monographPipelined Adaptive Digital Filters published by Kluwer Academic Publishersin 1994. Dr. Shanbhag received the 2010 Richard Newton GSRC IndustrialImpact Award, became an IEEE Fellow in 2006, received the 2006 IEEEJournal of Solid-State Circuits Best Paper Award, the 2001 IEEE Transactionson VLSI Best Paper Award, the 1999 IEEE Leon K. Kirchmayer Best PaperAward, the 1999 Xerox Faculty Award, the IEEE Circuits and SystemsSociety Distinguished Lecturership in 1997, the National Science FoundationCAREER Award in 1996, and the 1994 Darlington Best Paper Award from theIEEE Circuits and Systems Society. In 2000, Dr. Shanbhag co-founded andserved as the Chief Technology Officer of Intersymbol Communications, Inc.,(acquired in 2007 by Finisar Corp (NASDAQ:FNSR)) a semiconductor start-up that provided DSP-enhanced mixed-signal ICs for electronic dispersioncompensation of OC-192 optical links. Since January 2013, he is the foundingDirector of the Systems On Nanoscale Information fabriCs (SONIC) Center, a5-year multi-university center funded by DARPA and SRC under the STARnetprogram.