Applying Machine Learning to Circuit Design
description
Transcript of Applying Machine Learning to Circuit Design
![Page 1: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/1.jpg)
Applying Machine Learning
to Circuit Design
David HettlingerAmy Kerr
Todd Neller
![Page 2: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/2.jpg)
Channel Routing Problems (CRPs)
Chip DesignEach silicon wafer contains hundreds of chips.Chip components, specifically transistors, are etched into the silicon.Often groups of components need to be wired together.
![Page 3: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/3.jpg)
A Simple CRP Instance3 4 1 0 2 4
4 2 0 3 1 2
Net number 4
Silicon Channel
![Page 4: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/4.jpg)
One Possible Solution
0
2
3 4
5
1
3 4 1 0 2 4
4 2 0 3 1 2Goal: Decrease the number of horizontal tracks.
![Page 5: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/5.jpg)
CRPs: Constraints
Horizontal: Horizontal segments cannot overlap.Vertical : If subnet x has a pin at the top of column a, and subnet y has a pin at the bottom of column a, then subnet x must be placed above subnet y.
![Page 6: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/6.jpg)
Simulated Annealing (SA)Background
“Annealing” came from a process blacksmiths use.Metal is heated then cooled slowly to make it as malleable as possible.Statistical physicists developed SA.
![Page 7: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/7.jpg)
SA Problem ComponentsDefinable states: The current configuration (state) of a problem instance must be describable.New state generation: A way to generate new states.Energy function: A formula for the relative desirability of a given state.Temperature and annealing schedule: A temperature value and a function for changing it over time.
![Page 8: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/8.jpg)
A Visual Example
Local Minimum Global
Minimum
![Page 9: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/9.jpg)
Applying Simulated Annealing to CRPs
Definable states: The partitioning of subnets into groups.
New states generation: Change the grouping of the subnets.
Energy function: The number of horizontal tracks needed to implement a given partition.
Temperature and annealing schedule: Start the temperature just high enough to accept any new configuration. As for the annealing schedule, reinforcement learning can help find that.
![Page 10: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/10.jpg)
A Simple CRP Example
Start State of a CRP Instance Partition Graph of this State
![Page 11: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/11.jpg)
A Simple CRP ExampleStates 1 and 2
Partition Graphs of theses States
![Page 12: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/12.jpg)
A Simple CRP ExampleStates 1, 2 and 3 Partition Graphs of these States
![Page 13: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/13.jpg)
A Simple CRP ExampleStarting through Ending States Partition Graphs of these States
![Page 14: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/14.jpg)
A Generated CRP InstanceStart State A Solution
15 Horizontal Tracks
12 Horizontal Tracks
![Page 15: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/15.jpg)
SA Problem ComponentsDefinable states: The current configuration (state) of a problem instance must be describable.New state generation: A way to generate new states.Energy function: A formula for the relative desirability of a given state.Temperature and annealing schedule: A temperature value and a function for changing it over time.
![Page 16: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/16.jpg)
The Drunken Topographer
Imagine an extremely hilly landscape with many hills and valleys high and lowGoal: find lowest spotMeans: airlift a drunk!Starts at random spotStaggers randomlyMore tired rejects more uphill steps
![Page 17: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/17.jpg)
Super-Drunks, Dead-Drunks, and Those In-
BetweenThe Super-Drunk never tires
Never rejects uphill stepsHow well will the Super-Drunk search?
The Dead-Drunk is absolutely tired
Always rejects uphill stepsHow well will the Dead-Drunk search?
Now imagine a drunk that starts in fine condition and very gradually tires.
![Page 18: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/18.jpg)
Traveling Salesman Problem
Have to travel a circuit around n cities (n = 400)Different costs to travel between different cities (assume cost = distance)State: ordering of cities (> 810865 orderings for 400 cities)Energy: cost of all travelStep: select a portion of the circuit and reverse the ordering
![Page 19: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/19.jpg)
Determining the Annealing Schedule
The schedule of the “cooling” is critical
Determining this schedule by hand takes days
Takes a computer mere hours to compute!
![Page 20: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/20.jpg)
Reinforcement Learning Example
Goal: Ace a class
Trial & Error: study for various amts. of timeShort term reward: exam grades, amt
free time Long term rewards:
Grade affects future opportunities: i.e. whether we can slack off
laterOur semester grades (goal is to max. this!)
Need to learn how long we need to study to get an A
![Page 21: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/21.jpg)
Reinforcement Learning (RL)
Learns completely by trial & error
Receives rewards for each action
Goal: maximize long-term numerical reward
1. Immediate reward (numerical)
2. Delayed reward: actions affect future situations & opportunities for future
rewards
No preprogrammed knowledge
No human supervision/mentorship
![Page 22: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/22.jpg)
RL: The Details
Agent = the learner (i.e. the student)
Environment = everything the agent cannot completely control.
Includes reward functions (i.e. grade scale)Descript. of current state (i.e. current average)
Call this description a “Sensation”
Agent
Environment
Sensation Reward Action
![Page 23: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/23.jpg)
RL: Value FunctionsUse immediate & delayed rewards to evaluate desirability of actions/learn taskValue function of a state-action pair, Q(s,a) The expected reward for taking action a
from state s
Strategy: Most of the time, choose the action that corresponds to the maximal Q(s,a) value for the state.
Remember, must explore sometimes!
Includes immediate & delayed reward
![Page 24: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/24.jpg)
RL: Q-Functions
Agent tries various actions.
We must learn Q(s,a)
To start, set Q(s,a) = 0 for all s, a.
Each time experiences action a from state s, updates estimate of Q(s,a) towards the actual reward experienced
If usually pick the action a’ that has the maximal Q-value for that state
max. total reward optimal performance
![Page 25: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/25.jpg)
Example: Grid WorldCan always move up, down, right, left
Board wraps around
Goal
Start
Get to goal in as few steps as
possible.
Reward = ?
Meaning of Q?
What are the optimal paths?
![Page 26: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/26.jpg)
Applying RL to CRP
Can learn an approx. optimal annealing schedule using RL
Reward function:
Penalized for the amount of time used to find this better configuration Computer learns to find an approx
optimal annealing schedule in a time-efficient manner.
Program self-terminates!
Rewarded for reaching better configuration
![Page 27: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/27.jpg)
ConclusionsCRP is an interesting but complex problemSimulated Annealing helps us solve CRPsSimulated Annealing requires annealing schedule (how to change temperature over time)Reinforcement learning – which is just learning through trial & error – lets a computer learn an annealing schedule in hours instead of days.
![Page 28: Applying Machine Learning to Circuit Design](https://reader036.fdocuments.us/reader036/viewer/2022062723/56813fef550346895daafd09/html5/thumbnails/28.jpg)
Any Questions?