Computer Architecture: A Constructive Approach Branch Direction Prediction – Six Stage Pipeline

Computer Architecture: A Constructive Approach

Branch Direction Prediction –Six Stage Pipeline

Joel EmerComputer Science & Artificial Intelligence Lab.Massachusetts Institute of Technology

April 23, 2012 L19-1http://csg.csail.mit.edu/6.S078

NA pred with decode feedback


F

Fetch

fr D

Decode

dr R

RegRead

rr X

Execute

xr M

Memory

mr W

Write-backxf

NextAddress

Prediction

df

Decode detected mispredicts Non-branch

When nextPC != PC+4 => use PC+4

Unconditional target known at decode When nextPC != known target => use known target

Conditional branch When nextPC != PC+4 or decoded target => use PC+4


Can we do better than PC+4?

Dynamic Branch Prediction


Branch direction prediction:

Learn and predict the direction a branch will go

Standard prediction principles:

Temporal correlationThe way a branch resolves may be a good predictor of the way it will resolve at the next execution

Spatial correlation Several branches may resolve in a highly correlated manner (a preferred path of execution)

One-bit predictor


0 0Fetch PC

Branch? Target PC

+

I-Cache

Opcode offsetInstruction

kBHT Index

2k-entryBHT,1 bits/entry

Taken/¬Taken?

Fetch

Decode

Predict branch will go same direction it went last time

One-bit predictor// Interface

interface DirectionPred; method ActionValue#(Tuple2#(Bool, DirInfo)) predict(Addr addr);

method Action train(DirInfo dirInfo, Bool taken);endinterface

// Feedback information

typedef 64 BPRows;typedef Bit#(TLog#(BPRows)) DirLineIndex;

typedef DirLineIndex DirInfo;


One-bit predictor (continued)module mkDirectionPredictor(DirectionPred); RegFile#(DirLineIndex, Bool) dirArray <- mkRegFileFull();

method ActionValue#(Tuple2#(Bool, DirInfo)) predict(Addr addr);

DirLineIndex index = truncate(addr >> 2); return tuple2(dirArray.sub(index), index); endmethod

method Action train(DirInfo dirInfo, Bool taken); DirLineIndex index = dirInfo; dirArray.upd(index, taken); endmethodendmodule


Array of prediction bits

Return prediction saved in array

Update array with last actual

behaviorWhen should we train?

Two-bit PredictorSmith, 1981


• Assume 2 direction prediction bits per instruction

On ¬taken

On taken

1 1 Strongly taken

1 0 Weakly taken

0 1 Weakly ¬taken

0 0 Strongly ¬taken

How well does one-bit predictor do on short trip count loops?

Implement using saturating counter

Saturating Countertypedef Bit#(2) Counter;

function Counter updateCounter(Bool dir, Counter counter); return dir?saturatingInc(counter) :saturatingDec(counter);endfunction

function Counter saturatingInc(Counter counter); let plusOne = counter + 1; return (plusOne == 0)?counter:plusOne;endfunction

function Counter saturatingDec(Counter counter); return (counter == 0)?0:counter-1;endfunction


How do we determine prediction from counter?

Two-bit predictor


0 0Fetch PC

kBHT Index

2k-entryBHT,1 bits/entry

Taken/¬Taken?

Two-bit predictortypedef 64 BPRows;typedef Bit#(TLog#(BPRows)) DirLineIndex;

// DirInfo datatypedef struct { DirLineIndex index; Counter counter;} DirInfo deriving(Bits, Eq); module mkDirectionPredictor(DirectionPred);

// Direction predictor stateRegFile#(DirLineIndex,Counter) cntArray <- mkRegFileFull();


Feedback state for training

Two-bit predictor (continued)method ActionValue#(Tuple2#(Bool, DirInfo)) predict(Addr addr); DirInfo info = ? info.index = truncate(addr >> 2); info.counter = cntArray.sub(index);

Bool taken = (truncate(counter >> 1) == 1);

return tuple2(taken, info); endmethod

method Action train(DirInfo info, Bool taken); cntArray.upd(info.index, updateCounter(taken, info.counter)); endmethodendmodule


Training information is

index and counter

Prediction is high bit of counter

Train by updating counter

Exploiting Spatial CorrelationYeh and Patt, 1992


Implemented with a history register, ‘hist’, that records the direction of the last N branches executed by the processor.

if (x[i] < 7) theny += 1;

if (x[i] < 5) thenc -= 4;

If first condition false, second condition also false

Also works well for short trip count loops.

Ghist predictortypedef 64 BPRows;typedef Bit#(TLog#(BPRows)) DirLineIndex;

typedef Bit#(2) Counter;

// DirInfo datatypedef struct { DirLineIndex hist; Counter counter;} DirInfo deriving(Bits, Eq); module mkDirectionPredictor(DirectionPred);

// Direction predictor stateReg#(DirLineIndex) hist <- mkReg(0);RegFile#(DirLineIndex,Counter) cntArray <- mkRegFileFull();


Global history predictor method ActionValue#(Tuple2#(Bool, DirInfo)) predict(Addr addr); DirInfo info = ?; info.hist = hist; info.counter = cntArray.sub(hist);

Bit#(1) pred = truncate(info.counter >> 1);

hist <= truncate(hist << 1 | zeroExtend(pred)); return tuple2((pred == 1), info); endmethod


How good are predictions while waiting for training?

Shift new prediction into history register

Calculate feedback information

Global history predictormethod Action train(DirInfo info, Bool taken);

counterArray.upd(info.hist, updateCounter(taken, info.counter));endmethod

method Action repair(DirInfo info, Bool taken);

hist <= truncate((info.hist << 1) | zeroExtend(pack(taken)));

endmethod endmodule


What is the state of ‘hist’ afterredirects from decode and execute?

Restore history to state it would be in after the

desired prediction

NA pred with decode feedback


F

Fetch

fr D

Decode

dr R

RegRead

rr X

Execute

xr M

Memory

mr W

Write-backxf

NextAddress

Prediction

df

DirectionPrediction

Direction prediction recipeExecute Send redirects on mispredicts (unchanged) Send direction prediction training

Decode Check if next address matches direction pred Send redirect if different

Fetch Generate prediction Learn from feedback Accept redirects from later stages


Add direction feedbacktypedef struct { Bool correct; NaInfo naPredInfo; Addr nextAddr; DirInfo dirPredInfo; Bool taken;} Feedback deriving (Bits, Eq);

FIFOF#(Tuple3#(Epoch,Epoch,Feedback)) decFeedback<-mkFIFOF;FIFOF#(Tuple2#(Epoch,Feedback)) execFeedback <- mkFIFOF;


Feedback needs information for training

direction predictor

Execute (branch analysis)// after executing instruction...

let nextEeEpoch = eeEpoch;let cond = execData.execInst.cond; let nextPc = cond?execData.execInst.addr : execData.pc+4;if (nextPC != execData.nextAddrPred) nextEeEpoch += 1;

eeEpoch <= newEeEpoch;execFeedback.enq(tuple2(nextEeEpoch, Feedback{correct: (nextPC == execData.nextAddrPred), taken: cond, dirPredInfo: execData.dirPredInfo, naPredInfo: execData.naPredInfo, nextAddr: nextPc}));

// enqueue instruction to next stage


Recall: may have been set in decode

Always send feedback

Decode with mispredict detectrule doDecode; let decData = newDecData(fr.first); let correctPath = (decData.execEpoch != deEpoch) ||(decData.decEpoch == ddEpoch);

let instResp = decData.fInst.instResp; let pcPlus4 = decData.pc+4;

if (correctPath) begin decData.decInst = decode(instResp, pcPlus4); let target = knownTargetAddr(decData.decInst); let brClass = getBrClass(decData.decInst); let predTarget = decData.nextAddrPred; let predDir = decData.takenPred;


Determine if epoch of incoming instruction is on

good path

New exec epoch

Same dec epoch

Decode with mispredict detect let decodedTarget = case (brClass) NonBranch: pcPlus4; UncondKnown: target; CondBranch: (predDir?target:pcPlus4); default: decData.nextAddrPred; endcase; if (decodedTarget != predTarget) begin decData.decEpoch = decData.decEpoch + 1; decData.nextAddrPred = decodedTarget; decFeedback.enq( tuple3(decData.execEpoch, decData.decEpoch, Feedback{correct: False, naPredInfo: decData.naPredInfo, nextAddr: decodedTarget, dirPredInfo: decData.dirPredInfo, taken: decData.takenPred})); enddr.enq(decData); end // of correct path April 23, 2012 L19-22http://csg.csail.mit.edu/6.S078

Wrong next addr?

Tell exec addr of next instruction!

Send feedback

New dec epoch

Enqueue to next stage on correct path

Calculate target as best as decode can

Decode with mispredict detect else begin // incorrect path decData.decEpoch = ddEpoch; decData.execEpoch = deEpoch; end ddEpoch <= decData.decEpoch; deEpoch <= decData.execEpoch; fr.deq;

endrule


Preserve current epoch if instruction on incorrect path

decData.*Epoch have been set properly so we always save them.

Handling redirect from executeif (execFeedback.notEmpty) begin match {.execEpoch, .fb} = execFeedback.first; execFeedback.deq; if(!fb.correct) begin dirPred.repair(fb.dirPredInfo, fb.taken); dirPred.train(fb.dirPredInfo, fb.taken); naPred.repair(fb.naPredInfo, fb.nextAddr); naPred.train(fb.naPredInfo, fb.nextAddr); feEpoch <= execEpoch; fetchPc <= feedback.nextAddr; end else begin dirPred.train(fb.dirPredInfo, fb.taken); naPred.train(fb.naPredInfo, fb.nextAddr); enqInst; endend


Train and repair on redirect

Just train on correct prediction

Handling redirect from decodeelse if (decFeedback.notEmpty) begin decFeedback.deq; match {.execEpoch, .decEpoch, .fb} = decFeedback.first; if (execEpoch == feEpoch) begin if (!fb.correct) begin // epoch unchanged fdEpoch <= decEpoch; dirPred.repair(fb.dirPredInfo, fb.taken); naPred.repair(fb.naPredInfo, fb.nextAddr); fetchPc <= feedback.nextAddr; end else // dec feedback on correct prediction enqInst; end else // dec feedback, but in fetch is in new exec epoch enqInst;else // no feedback enqInst;


Just repair never train on feedback

from decode

Computer Architecture: A Constructive Approach Branch Direction Prediction – Six Stage Pipeline

Documents

Transcript of Computer Architecture: A Constructive Approach Branch Direction Prediction – Six Stage Pipeline