Analog Integrated Circuits and Signal Processing, 41, 89–92, 2004c© 2004 Kluwer Academic Publishers. Manufactured in The Netherlands.

Back-Gate Effect to Generate Derivative of Neuron Activation Function

AMIT K. GUPTA1 AND NAVAKANTA BHAT2

1Semiconductor Products Sector, MOTOROLA, Gurgaon, India2Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore, India

E-mail: K.Amit@motorola.com; navakant@ece.iisc.ernet.in

Received December 2, 2003; Revised January 19, 2004; Accepted January 19, 2004

Abstract. A simple neuron circuit is presented which can generate the sigmoid neuron activiation function (NAF)and its derivative (DNAF) by introducing asymmetric body biasing in the cross-coupled differential pair. The modeof operation is controlled externally. This results in the efficient utilization of the hardware by realizing NAF andDNAF using the same building blocks. The circuit is prototyped and characterized as a proof of concept on 1.5 µmAMI technology.

Key Words: derivative of neuron activation function, cross-coupled differential pair configuration, subthresholdoperation, back-gate effect

Introduction

A neural network can be trained for a particular prob-lem during the training phase. In this phase, the networkis provided with a set of desired input-output vectors.The weights of the network are updated such that thedesired output is approximately achieved for an appliedinput. Most of the training algorithms, such as errorback propagation, require the computation of DNAFfor the adaptation of weight vector. This is a major con-cern in the analog implementation of such algorithms.

Derivative Computation

The sigmoid functions are common choice for NAF.Some circuits in the literature [1] realize DNAF bycomputing actual derivative. This involves extra hard-ware, such as adders and multipliers. The other methodinvolves computation of derivative using difference ap-proximation formulas [2–4]. Derivative of the neuronactivation function can simply be evaluated by us-ing either the forward difference or central differenceapproximation.

f ′(Vi ) ≈ f (Vi + �V ) − f (Vi )

�V(1)

f ′(Vi ) ≈ f(Vi + �V

2

) − f(Vi − �V

2

)�V

(2)

In order to reduce the error in the derivative, �V shouldbe in the range of a few tens of millivolts. DNAF is typ-ically computed by applying either external or on chipgenerated small offset voltage, �V , at one of the inputof differential pair [2, 3]. This results in significant in-terconnect routing complexity to access inputs of all theneurons in a large neural network. The generation ofon chip reference voltage in millivolts range increasesthe circuit complexity.

Use of asymmetry in device dimensions of thecross-coupled pair to generate DNAF has beenreported in [4]. In this work, we use back-gate effect inthe cross-coupled differential pair to generate DNAF.Using the body terminal of one of the driver transistorof the differential pair does not require any additionalhardware with each neuron, since it eliminates theneed for accessibility of inputs. The body bias voltagecan be generated either on-chip or off-chip. The circuitis implemented in subthreshold region for low powerapplication.

Circuit Description

The drain current of a MOS transistor operating in sub-threshold is given by

ID = Ioeq[κVGS+(1−κ)VBS]/kT (1 − e−qVDS/kT ) (3)

90 Gupta and Bhat

Fig. 1. Schematic diagram of the neuron circuit.

where κ = (1+CD/COX)−1. CD and COX are the deple-tion and gate oxide capacitances respectively. Typicallyκ varies between 0.5 and 0.9. Io = kx W/L where kx

depends on process parameters. VGS, VBS and VDS arethe gate-to-source, body-to-source and drain-to-sourcevoltages respectively.

The currents in the two branches of a matched dif-ferential pair operating in subthreshold region havesigmoid relationship to the input differential volt-age. The input referred offset voltage of a simpledifferential amplifier with body terminal of the in-put transistors tied at different potentials is given by[5]

VOS = 1 − κ

κ�Vbody, (4)

where �Vbody is the difference in the body-sourcepotentials of the two transistors. Figure 1 showsthe circuit diagram of NAF and DNAF generator.This circuit has two modes of operation. DNAFmode (�Vbody = Vbody �= 0) would enable the circuitto function in learning phase whereas the NAF mode(�Vbody = Vbody = 0) would be operational in recallphase. In DNAF mode, Vbody is adjusted to getrequired asymmetry and hence input referred off-set voltage. MP11 − MP12 and MP21 − MP22 arethe driver transistors of the two required differen-tial matched pairs. The dimensions of these tran-sistors are the same. The body voltages of MP12and MP21 are modulated for asymmetry. Since the

circuit is implemented in n-well technology, thedriver transistors are realized using PMOS. Transis-tors MN11, MN12, MN21 and MN22 have the samedimensions.

In NAF mode, INAF is given as

INAF = I12 − I11 = ISS tanhqκVdiff

2kT(5)

In DNAF mode, IDNAF is given by

IDNAF = I21 − I11

= ISS

[1

1 + eqκ(Vdiff−VOS)− 1

1 + eqκ(Vdiff+VOS)

]

(6)

IDNAF ≈ VOSdINAF

dVdiff, (7)

where Vdiff is the differential input voltage and VOS isgiven by (4).

Test Results

The proposed circuit has been experimentally proto-typed and characterized as a proof of concept on 1.5 µmAMI technology with the reduced power supply volt-age, VDD = 2.5 V and ISS = 20 nA. To minimize therandom offset voltage, devices of size 100 µm/5 µmare used as the driver transistors of the cross-coupledconfiguration. Figure 2 shows the variation of input re-ferred offset voltage with the asymmetry in the bodybias of differential pairs in the neuron circuit. The sim-plistic model of (4) with constant κ matches the ex-perimental result fairly well. Forward biasing of thesource-body junction should be very small to maintainlow level of substrate leakage current as compared tothe current levels in the circuits. We have used Vbody

= 150 mV, which results in the offset voltage of 30mV and always maintains source-body junction in re-verse biasing condition. The offset voltage due to ran-dom process variation is 2.5 mV. Thus the input re-ferred offset voltage due to introduced asymmetry dom-inates over the random offset voltage. Experimentallyobtained NAF, DNAF and derivative computed fromNAF are shown in Fig. 3. Good match is observedbetween DNAF and the derivative computed fromNAF.

Back-Gate Effect to Generate Derivative of Neuron Activation Function 91

Fig. 2. Variation of VOS with the asymmetry in the body bias. The dash-dotted and circled curves depict theoretical and experimental resultsrespectively.

Fig. 3. Experimentally obtained neuron activation function and its derivative. The dashed and solid line represent the NAF and DNAFrespectively. The derivative computed from the NAF is shown by dash-dotted curve.

Conclusion

In this paper we have presented application of back-gate effect to generate neuron activation function and

its approximate scaled derivative. The technique uti-lizes asymmetry in the body biasing of the input tran-sistors of cross-coupled configuration which resultsin the offset voltage at the inputs. The circuit has

92 Gupta and Bhat

been successfully prototyped on AMI 1.5 µm CMOStechnology.

Acknowledgment

The authors would like to thank the IISc-Space Tech-nology Cell and the Department of Science and Tech-nology, Government of India for their support.

References

1. T. Shima, T. Kimura, Y. Kamatani, T. Itakura, Y. Fujita, and T.Iida, “Neuro chips with on-chip back-propagation and /or hebbian

learning.” IEEE J. Solid State Circuits, vol. 27, no. 12, pp. 168–1875, 1992.

2. C. Lu, B. Shi, and L. Chen, “Push-pull output neuron circuit.”Electron. Lett., vol. 37, pp. 1531–1533, 2001.

3. G. Bogason, “Generation of a neuron transfer function and itsderivative.” Electron. Lett., vol. 29, no. 21, pp. 1867–1869,1993.

4. A.K. Gupta and N. Bhat, “Asymmetric cross-coupled differentialpair to realize neuron activation function and its derivative,” IEEETrans. on Circuits and Systems-II, (Accepted).

5. R.R. Harrison, “A wide-linear-range subthreshold CMOStransconductor employing the back-gate effect,” in Proc. IEEEIntl. Symp. on Circuits and Systems, 2002, vol. 3, pp. 727–730.