Obstacle Avoidance in Crowd Simulations: A Machine Learning Method for Intelligent Steering

Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering

Obstacle Avoidance in Crowd Simulations: A Machine Learning Method for Intelligent Steering

Steve Agland (98075207) Machine Learning (31734)

[email protected]

[email protected]

Lecturer: Paul Kennedy

Abstract This paper presents a machine learning system developed for the purpose of refining an existing distributed cognitive model that was designed to simulate crowds of creatures in computer-generated animated films (Agland, 2002). The original model was specifically tailored for a particular animation project involving a crowd of Slinkies, stretchy spring-like toys and the unlikely recipients of the gift of life. The cognitive model simulated the simple behaviour of this crowd responding to various stimuli, such as forces of attraction and repulsion (members of the crowd moving towards an target or away from a threat). It also provided a rudimentary collision-avoidance system; but in this respect if failed to produced what should appear to be intelligent behaviour. This paper outlines a system, based on a neural network (NN), designed to solve the problem of trying to find an algorithm for an agent to use when navigating its way through a field of obstacles towards (or away from) a target. The problem involves a number of considerations that were revealed during the development of the first project, such balancing the need to head for “safe”, or sparsely populated, areas with the need to reach the target within a reasonable amount of time. Agents using the initial method described in (Agland, 2002) were seen to either charge suicidally into dense groups of other agents or to run wildly off course in order to avoid any risk of confrontation. Such behaviour did not necessarily always result in “failure,” but it did appear unnatural when played as an animation.

Figure 1 – A crowd of 16 Slinky agents, initialised in a randomly offset grid and colour-coded for identification.

The method described in this paper attempts to use human-generated sample behaviour to train a neural network to make navigation decisions that address the twin needs of safety and efficiency, as well as appearing sensible and natural.

Background More information on the motivations and broader scope of this project can be found in (Agland, 2002) but a description of the existing model will be provided here in order to justify the design decisions made when developing this extension.

Steve Agland (98075207) 1 of 21 25 Oct 2002

Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering The existing model represents a crowd of agents similar to the popular (Renolds, 1987) boids flock, but with some modifications such as 2-dimentionality (the agents cannot fly) and more flexible influences. Each agent has its own physical attributes, such as size, starting position, maximum velocity, maximum acceleration etc. The model also includes other entities called “distractions.” The behaviour of distractions is entirely predetermined and serves only to influence the agents (and therefore, provide the animator with some control over the crowd) rather than exhibit any simulated behaviour of their own. Both agents and distractions project “influences” of varying strengths and types, which spread over a certain radius around them (reducing in strength the further they spread). The simulation loops over a time unit: 25 frames every second. Note that the simulation is not run in real-time, but outputs 3D data for import into a 3D animation application. Each frame, every agent in the crowd analyses the influences that are affecting it – taking into account the direction, the strength and type (attract, repel, “look at” or avoid) – and adjusting the movement of its “head” and its “foot” accordingly. Some aspects of this model were important to consider when designing the machine learning system that was to enhance the collision-avoidance behaviour of these agents:

• Behavioural decisions are made by the agent 25 times every second. This means that the decisions must be made quickly in order for the simulation to run in a reasonable amount of time. However, one can also conclude that an agent, when faced with a field of obstacles, need only decide the best direction in which to begin (rather than plot a comprehensive and winding course) since a revised decision can be made very soon afterwards – before the position of the agent, or of the potentially mobile obstacles, has changed significantly.

• The model of the crowd (at least in its current manifestation) operates primarily on a 2D plane, and vertical data can effectively be ignored.

• If agents are not moving, then using a collision-avoidance decision maker is unnecessary; if they are moving then they will always have a “target” – a position in space that they are heading towards. This may sometimes be a virtual point of on the opposite side of the agent from a repulsive influence.

• Obstacles surrounding the agent are of less importance the further they are oriented from the “target line,” that is, the direction the agent would ideally head if there were no other obstacles. In fact, it could be assumed that any obstacles that are not within 90 degrees on either side of the target line can be safely ignored (if one of these obstacles is another agent moving quickly towards the first, we can assume its own obstacle-avoidance decisions will prevent a collision.

• Obstacles further away from an agent are of less relevance than those close-by. They may still have some influence on decision making, however, since a “clear run” is preferable to a direction that will eventually require more course adjustments.

Since this problem is not one of classification or prediction, there was no need to seek a source of “real world” examples to use as training data. The goal was to generate natural intelligent-seeming behaviour in the agents. There are, in a simple sense, two general ways this could be done: using supervised or unsupervised learning. Unsupervised learning would involve developing a system that could accurately determine the “success” of a chosen path after experimentation. While technically possible, such an approach would be as difficult, if not more so, than trying to code the decision-making algorithm by hand. It is for this reason that the simpler method of supervised learning (or teaching by example) was chosen.



Generating the Training Data The most accessible source of intelligent obstacle navigation decisions is, of course, a human. An interactive system was devised to allow a user to respond to randomly generated scenarios with suggested routes. The aim was to create a system that allowed the rapid generation of pairs of vectors: one representing the scenario, and used as input into a neural network; and one representing the user’s suggested route(s), to be compared to the output of the neural network for the purposes of training and testing it. It was decided that the best way to represent a scenario in a vector was to split the agent’s “field of vision” into 18 10-degree arcs that cover the 90 degrees on either side of the target line. The values of the vector elements would signify the proximity of any obstacles in that arc. These values would be reals that vary between –1 and 1. Values of minus 1 would mean that there were no obstacles in that arc closer than the target point; a value of 0 meaning there was an obstacle half the distance to the target away; and a value of 1 meaning there was an obstacle touching the agent (hopefully this would never be the case).

Figure 2. A portion of the Training Data Generator user interface. The user may browse through an infinite set of randomly generated scenarios, which vary greatly in terms of obstacle density and the distance to the target. The green spot in the centre represents the agent, the red spot at the top represents the target, and the blue spots scattered throughout represent randomly placed obstacles. The obstacles in front of the target are summarized in an 18-element vector (each representing an arc of 10 degrees), presented in the upper graph on the left. Taller columns (greater values) mean closer obstacles in that arc. The user suggests navigational directions by clicking on areas the agent should head for – the further from the agent the more strongly recommended the route. This allows the user to specify ranked alternate routes. These are shown on the map as green spots connected by lines to the agent. The neural network’s current “guesses” as to which was to go in this scenario are shown by the dull red “spikes” on the map. Both the user’s and the neural net’s solutions are summarized in the lower graph on the left. See: http://agi.bpa.nu/~agi/helix_crowd/?scenarioid=104


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering It was decided that this was preferable to a range of 0 to 1, because many scenarios would involve values of zero, which may not work well in a neural network where synaptic weights need a non-zero value to be effective. The user’s provided solutions would be stored in a similar vector. The higher the value in each arc, the more strongly the user recommends taking that direction for this particular scenario. The training interface was written in PHP, which includes some image-generating functionality, and runs as a website. When a user visits the site, they are presented with the next unsolved scenario. These scenarios are randomly generated as previous ones are completed. See Figure 2 for more information on the functionality of the user interface. As the user provides suggested solutions to obstacle scenarios, the scenario map, the scenario summary vector, the user’s suggested solutions and the solution summary vector are all stored in a database for later retrieval. Once the neural network (NN) was built, trained and running (see below), the web interface was extended to also display the output of the NN for the given scenario. This provided visual representation of the NN’s performance in comparison to the suggested solutions, and assisted in assessment and refinement of its construction and training methods. The user can also decide to retrain the network – the database is exported, divided into training and testing data, and the training procedure run using the established successful parameters (saving the weights when the test data error hit a minimum). The user could then view the NN’s new decisions based on the updated training data.

Building the Neural Network A neural network seemed a suitable machine learning method to apply to this problem, as they are well known as successful function approximators, and this is precisely what was needed in this case: a set of numeric information is know about a situation, and a set of numeric values is needed as a solution. Neural networks are, after all, models of biological brains, and it is a brain, or at least a small portion of it, that is being simulated here. Genetic algorithms were also considered as a method, but the nature of the function that needed to be approximated meant that genetic algorithms would probably end up being used to evolve the weights of a NN anyway. This was an unnecessary complexity in an application where NN back-propagation was a sufficiently effective training procedure. The choice to use an 18-element vector as input and output was an arbitrary decision that seemed a reasonable balance between accuracy and efficiency – and, of course, is easier for a decimal-based human to conceptualise as a 180° arc. Once a sufficient amount of training data had been generated (about 300 scenarios for training, and a further 50 for testing), experimentation with neural network parameters begun in order find the the most effective configuration. D. R. Tveter’s AI Backprop software was used to create, train and test an artificial neural network. The PHP-based user interface exported the training data, stored in the database, in a suitable format for use with this software. Instead of the standard sigmoid function, a “TanH” function was used to squash the output values so that they fitted in a range between –1 and 1. It was found that the network reached its minimum error quite quickly, after about 90-120 iterations of the training data, before it started to rise again.


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering A lot of experimentation was required to find the optimal parameters. A learning rate of 0.001 appeared to be the most effective, even though the minimum error was still being reached very quickly: an average of about 0.18 per unit for the test data. The addition of hidden layers of any size always yielded worse results for the test data, although over-fitting lead to more accurate results on the training data – reaching below an error rate of 0.1 with a 36-neuron hidden layer. Eventually the use of hidden layers was abandoned, although a bias node with a constant value of 1.0 is used on the output layer. Meanwhile – even though a 0.18 average error seemed an acceptable error rate for an application that did not require pinpoint accuracy (human-generated data is always guaranteed to be inconsistent anyway) – the choices made by the network as displayed in the in the user interface (see Figure 2) did not always seem entirely reasonable. In some scenarios, often when there were many obstacles nearby in all directions, the NN would output an apparently “wild” decision to head strongly off to the side, around 70-90 degrees off the course of the target. While this might have resulted in some clever “circumnavigating” behaviour by agents in tricky situations, it certainly was not a suggested policy in any of the training data. Additionally the incidence of this unexpected phenomenon was too high to favourable, so some more tweaking was required for the NN. The creation of more training data, to the point of having some 550 scenarios solved by humans, certainly improved the performance of the NN. This was not so much apparent by any change in the minimum error, but by a noticeable correlation in the user interface between the user’s choices and the NN’s. Some more consideration of the internal workings of the NN lead to an idea that a change in the “sharpness” or “gain” of the output could improve accuracy, and in fact, changing the value of this variable from 1.0 to 0.6 brought the minimum error rate down to about 0.15. Any further reductions saw a rise in the minimum error, so the gain was left at 0.6.

Figure 3 – The flow of data during the training and testing phase.



Implementation In order to verify the practicality of the NN’s performance, it needed to be integrated with the existing simulation software. This software was written in LISP/CLOS, and while it may have been possible to use system calls to link the NN to the simulation software, it was decided that it would be simpler and more efficient to recreate the neural network in LISP. Therefore, a simple neural network class was created – capable of only the functionality required of it for this application: that is, reproducing exactly the behaviour of Tveter’s NN software when supplied with the same weights. The LISP NN was not, however, capable of learning, only processing. Soon, comparisons of the output of Tveter’s software to that of the LISP-based emulator proved to be identical. The beginning of the simulation a NN object is created and initialised with the trained weights produced by Tveter’s Backprop software. The simulation engine was then modified – removing the previous steering algorithm and replacing it with one that converts the list of “avoid me” influences being applied to an agent to the 18-element vector required by the NN. This is then processed by the NN and the output is analysed to determine the optimal direction in which the agent should head.

Test Run

Frame 1 – Four agents on the left begin their existence instantiated on the four corners of an irregular square. To the right is the target, which moves slowly north during the simulation. Between is a field of five obstacles, placed deliberately to get in the way of the agents.

Frame 25 – The agents have begun approaching the target on what appear to be direct courses.

Frame 50 – The larger agent at the back is moving slowly for cinematic reasons, due to it size. The lower-most agent has certainly deviated from its course to squeeze through the gap between the first two obstacles.

Frame 75 – The first two agents have moved through the gap without a collision. The leftmost agent is doing well too. The large agent at the back appears to be hesitating – there are four obstacles between it and the target.



Frame 125 – The leading agents have past what should be their last obstacle – one of them coming close, but not too close, to colliding with it.

Frame 150 – The first two agents have reached the target. The agent at the rear is taking the long way around, but skimming the edge for efficiency’s sake. Although its difficult to see in a still frame, the middle agent is heading directly for the right-most obstacle asit steers around the other.

Frame 175 – Two agents are now hovering inside the target ball. When viewing the animation, the middle agent makes a smooth but significant course correction after passing its last obstacle and heading straight for the target.

Frame 100 – The first two agents have evidently decided to tackle the approaching obstacle in different ways – one going on one side, and one on the other. The large agent at the back seems ready to go around the outside of the obstacles field.

Frame 250 – Several seconds later, the last agent (with a slower maximum velocity than the others) finally reaches the target by cautiously around the outside.

Frames 1-250 – In this image we can see the paths taken by the four agents through the obstacle field. The most striking feature is that although the agents avoided colliding with any obstacles, they also managed to take extremely direct and efficient routes – thus realising the goal of the experiment.

Future Development A number of enhancements could be made to the current system that could produce a remarkable improvement in accuracy and performance. One major shortcoming of the current system is its poor ability to deal with unusually large or irregularly shaped obstacles. The scenario-generating training software uses only circular obstacles, and these are essentially treated as an arbitrary number of points around the edge of the circle – the larger the circle, the more points are used.


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering This was a fairly effective way of ensuring that if an obstacle was large enough to appear in more than one 10-degree arc, then it would registered in each one. The LISP simulation software, however, is not quite so sophisticated, mainly for speed reasons: dealing with 5 points of influence instead of one can severely affect speed. To compensate for this shortcoming a “spread” factor is used on very strong (close) influences to spread them over more than one arc. This is not, however, a very accurate method. A method similar to the 3D rendering technique of ray-tracing could be used on arbitrarily sized or shaped obstacles – that is by firing a ray (or several) down each of the 18 arcs to detect obstacles (Watt, 1993, pp. 267). In fact, a technique similar to this is used in the Massive software, developed in order to produce the immense battle scenes in the Lord of the Rings films. The agents are essentially able to “see” around them, and act accordingly (Duncan, 2002). Another improvement that would broaden the potential application scope of the system would be to extend it into three dimensions. This would allow for the simulation of birds or fish, planes or helicopters. It could also be used for land-based agents moving over unstable terrain. This would add significantly to the complexity of the network, as the scenario would need to be summarized in a 2D matrix rather than a 1D vector. Anticipation is a factor not address in the current system, and it could improve its performance remarkably in scenarios when many of the obstacles are in motion. Each frame the agent makes a decision based on the current position of each obstacle. However, if the obstacle is moving, then it will be in a different position by the time the agent reaches it. This may not be so much of a problem considering the agent update their direction 25 times ever frame. Nevertheless, more intelligent decision would undoubtedly be made if the obstacle’s velocity was also taken into account along with its position. Integrating velocity into the current model would be difficult. Positions can be mapped to a vector, but velocities cannot. An obstacle may be moving towards or away from the agent, or moving between arcs. One way to consider velocity without altering the established system would be to alter the positions of the obstacles before converting them to a summary vector. This would be based on the obstacle velocity and the estimated time it would take for the agent to reach the obstacle. The obstacle field then used would then be a sort of time warp, with predictions spreading further into the future the further from the agent you go. An improvement like this would increase the apparent intelligence of the decisions being made be the agents, such as heading for gaps in the obstacle field that have not yet even appeared.

Conclusion This paper presented a method for training a neural network to make intelligent navigation decisions when faced with a series of obstacles. The goal was to produce behaviour that addressed the twin needs of safety (collision avoidance) and efficiency (reaching the target in as little time as possible). A model was developed as an extension to an existing crowd simulation system. The experiment demonstrated that the system design was effective – at least as far as its current application goes. Wider tests and enhancements need to be carried out to investigate its feasibility in larger crowd systems with many moving agents and obstacles, and conflicting agent goals. The success of the initial system, however, is encouraging and indicative that the method is an effective approach to tackling the problem.



Glossary 2D – Two-dimensional. In this paper, this generally refers to the horizontal plane, excluding the vertical “third” dimension. 3D – Three-dimensional. In this paper, this usually refers to model, object or point that has attributes in three dimensions (front/back, left/right, up/down). Agent – An artificial entity, capable of responding to stimuli and exhibiting behaviour. In the case of this project, an agent is a crowd member. Crowd – A group of agents, moving around and influencing one another. Houdini – A high-end 3D animation package produced by Side Effects Software and used in this project to convert the 3D data output by the software into 3D animation for analysis and review. Influence – An effect, emanating from a point in space, that can cause behavioural changes in nearby agents. Agents themselves generate influences of their own. Obstacle – Effectively an “avoid me” influence (see above). Obstacles aren’t so much repulsive as best avoided. A “field” of obstacles is analysed by the system described in this project in order to decide the best direction in which an agent should head. Slinky – The common name for a spring-like toy (actually a brand name, although not all “magic springs” are Slinky-brand). They are famously known for the ability to “walk” down stairs, but, as yet, are not so well known for their ability to congregate, avoid obstacles or have animated adventures.

References

1. Kennedy, P., Cognitive Modelling Lecture Notes: Neural Networks, 2002 2. Osborne, T., Machine Learning Lecture Slides: Neural Nets, 2002 3. Prechelt, L., comp.ai.neural-nets FAQ,

http://www-2.cs.cmu.edu/Groups/AI/html/faqs/ai/neural/faq.html 4. Agland, S., Crowd Simulation: A Distributed Cognitive Model, UTS, 2002

http://www.slinkies.net/crowd_simulation.pdf 5. Reynolds, C. W., Flocks, Herds, and Schools: A Distributed Behavioral Model, Computer

Graphics, 21(4), July 1987, pp, 25-34 6. Watt, A., 3D Computer Graphics, Addison-Wesley, 1993 7. Duncan, J., Ring Masters, Cinefex, No: 98, 2002, pp: 84 – 90 8. Rist, R., Cognitive Modelling Lecture Notes: Production Systems, 2002 9. Tveter, D.R., Professional Version Basis of AI Backprop, Hypertext Documentation, 1998 10. Steele, G. L., Common Lisp the Language, 2nd Edition, Digital Press, 1990 11. Pitman, K., et. al., [Xanalys] Common Lisp Hyperspec,

http://www.xanalys.com/software_tools/reference/HyperSpec/ 12. Bakken, S., et. al. PHP Manual, PHP Documentation Group, http://www.php.net/, 2002 13. Side Effects Software, Houdini Reference Manual, Side Effects Software, 1999



Appendix A – Training Engine and User Interface (PHP Code) At the time of writing, this interface can be accessed and used at: http://agi.bpa.nu/~agi/helix_crowd/ <? ################################################################# ## HELIX CROWD SIMULATOR ------- (C) Copyright Steve Agland 2002 ## ##-----------------------------------------------------------------## ## $Source: /home/cvsroot/helix_crowd/web/index.php,v $ ## $Revision: 1.3 $ ## $Author: agi $ ## $Date: 2002/10/23 06:08:43 $ ##-----------------------------------------------------------------## ## [email protected] ## ################################################################# # # The purpose of this script is generate random situations, which # are 2D confiurations with the agent at the center and a target at # one of the edges. Dispersed throughout out randomly placed and sized # circles define('SQUIZLIB_PATH','/home/agi/mysource/squizlib'); define('BP_PATH','/home/agi/probp'); include_once(SQUIZLIB_PATH.'/object/object.inc'); include_once(SQUIZLIB_PATH.'/db/db.inc'); ################################################## # A Function that draws a filled circle of a certain # radius on an image function imageCircle($im,$x,$y,$r,$c) { $vert_count = (int) $r * 4; $points = array(); for($i = 0; $i < $vert_count; $i++) { $rad = deg2rad(($i / $vert_count) * 360); $points[] = sin($rad) * $r + $x; $points[] = cos($rad) * $r + $y; } imagePolygon($im,$points,$vert_count,$c); } function imageFilledCircle($im,$x,$y,$r,$c) { $vert_count = max(3,(int) $r * 4); $points = array(); for($i = 0; $i < $vert_count; $i++) { $rad = deg2rad(($i / $vert_count) * 360); $points[] = sin($rad) * $r + $x; $points[] = cos($rad) * $r + $y; } imageFilledPolygon($im,$points,$vert_count,$c); } ####################################### # Returns a random number - different for # each scenarioid, and within elemnts of # a scenario, but should be consistant # within a scenarioid if repeated. function get_rand() { if($GLOBALS['rand_seed']) { $GLOBALS['rand_seed'] += 789; } else { $GLOBALS['rand_seed'] = $GLOBALS['scenarioid'] * 837291; } mt_srand($GLOBALS['rand_seed']); return mt_rand() / mt_getrandmax(); } ######################################################## # These functions normalise values in 1 (length) or 2 (area) # dimentions taking into accounr $GLOBALS['distance'], the radius # of the world. function d_norm($v) { return $v / $GLOBALS['distance']; } function d_norm2($v) { return $v / ($GLOBALS['distance'] * $GLOBALS['distance']); }


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering ############################################ # Converts a MySQL datetime to a timestamp function mysql_to_timestamp($datetime) { if (!list($d,$t) = explode(" ",$datetime)) $d = $datetime; list($y,$m,$d) = explode("-",$d); list($h,$i,$s) = explode(":",$t); return mktime($h,$i,$s,$m,$d,$y); } ######################################################## # Converts an array of points into a summary vector function point_array_to_summary_vector($array,$const_radius=1,$invert) { $summary_vector = array(); for($i = 0; $i < 18; $i++) $summary_vector[$i] = 0; for(reset($array); $o = current($array); next($array)) { $r = ($o['r']?$o['r']:$const_radius); # For each obstacles analyse the position of a few points around # the circumference and see which arc they lie in. The number of points # to analyse depends on how relatively large the obstacle is $point_count = round(3 + d_norm($r) * 60); $start_at = get_rand() * 360; for($i = 0; $i < $point_count; $i++) { $deg = $start_at + ($i * 360) / $point_count; $x = d_norm(sin(deg2rad($deg)) * $r + $o['x']); $y = d_norm(cos(deg2rad($deg)) * $r + $o['y']); if($y < 0) continue; # Ignore points behind the agent $d = sqrt(pow($x,2)+pow($y,2)); # Distance from agent $closeness = min(1,max(0, $invert ? $d : 1 - $d)); $segment = floor((rad2deg(atan($x/$y)) + 90) / 10); if($closeness > $summary_vector[$segment]) { $summary_vector[$segment] = $closeness; } } } return $summary_vector; } ################################################################## # Runs a vector through the existing NN and returns the resulting # guessed solution. This is the exciting stuff! function run_nn($vector) { $f = fopen('tmp_vector.dat','w'); for($i = 0; $i < 18; $i++) { $vector[$i] = sprintf('%.4f',$vector[$i]); } fwrite($f,implode(' ',$vector)."\nq\n"); fclose($f); $o = shell_exec(BP_PATH.'/bp ../bp/helix_go.bp < tmp_vector.dat'); $r = array_splice(split("[ \n]+",$o),1,-1); # Scale to fit in with the 0..1 thing we have going in here (as opposed to -1..1). for($i = 0; $i < 18; $i++) { $r[$i] = $r[$i]/2+0.5; } return $r; } ################################################################## # Runs a vector through the existing NN and returns the resulting # guessed solution. This is the exciting stuff! function retrain_nn() { return shell_exec(BP_PATH.'/bp < ../bp/helix_retrain.bp'); } #################################################### # Define some constants, arbitrarily chosen with the # application in mind. define('MIN_DISTANCE', 15); define('MAX_DISTANCE', 200); define('MIN_DENSITY', 1); # In 100x100 units (10,000 square units). define('MAX_DENSITY', 100); define('MIN_RADIUS', 0.7); # Of the circular agents define('MAX_RADIUS', 4);


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering $db = new Db('helix_crowd'); ############################### # Design a randomised scenarios $scenarioid = (int) $_REQUEST['scenarioid']; if($data = $db->single_row("SELECT * FROM scenario WHERE scenarioid=$scenarioid")) { $map_filename = sprintf("scenarios/%06d_map.gif",$scenarioid); $solution_filename = sprintf("scenarios/%06d_solution.gif",$scenarioid); $summary_filename = sprintf("scenarios/%06d_summary.gif",$scenarioid); # Scenario already defined. $distance = $data['distance']; $area = $distance * $distance; $o_count = $data['o_count']; $a_radius = $data['a_radius']; $create_time = mysql_to_timestamp($data['create_time']); $record_time = mysql_to_timestamp($data['record_time']); $record_ip = $data['record_ip']; $obstacles = unserialize($data['obstacles']); $summary_vector = unserialize($data['summary_vector']); $solutions = unserialize($data['solutions']); $solution_vector = unserialize($data['solution_vector']); ################################################## # Process an solutions that may have bene provided if(($x = $_REQUEST['x']) && ($y = $_REQUEST['y'])) { $x = ($x - 250) / 250 * $distance; $y = ($y - 250) / 250 * -$distance; if($d = sqrt(pow($x,2)+pow($y,2)) > $distance) { $solution_msg = 'Outside of target zone.'; $failed = 1; } # Now, if they've clicked on a spot that's inside another # solution, we'll assume they're trying to REMOVE that solution. if(is_array($solutions) && !$failed) { for($s = reset($solutions); $s; $s = next($solutions)) { if(sqrt(pow($s['x']-$x,2)+pow($s['y']-$y,2)) < $a_radius * 2 + 1) { $updated = 1; $solution_msg = 'Solution removed.'; $removeid = key($solutions); break; } } if(isset($removeid)) unset($solutions[$removeid]); } else { $solutions = array(); } if(!$updated && !$failed) { # Check for collisions with obstacles, other solutions or or the origin if(sqrt(pow($x,2)+pow($y,2)) < $a_radius * 3) { $solution_msg = 'Too close to centre.'; } else { $c_o_count = count($obstacles); for($j = 0; $j < $c_o_count; $j++) { $o_r = $obstacles[$j]['r']; $o_x = $obstacles[$j]['x']; $o_y = $obstacles[$j]['y']; if(sqrt(pow($o_x-$x,2)+pow($o_y-$y,2)) < $a_radius + $o_r + 1) { $failed = 1; $solution_msg = 'Too close to an obstacle.'; } } if(!$failed) { $solutions[] = array('x'=>$x,'y'=>$y); $updated = 1; $solution_msg = 'Solution added.'; } } } if($updated) { # Update solution vector $solution_vector = point_array_to_summary_vector($solutions,$a_radius,1); # Update database


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering $db->update("UPDATE scenario SET solutions='".addslashes(serialize($solutions))."', solution_vector='".addslashes(serialize($solution_vector))."', record_time=now(), record_ip='$GLOBALS[REMOTE_ADDR]' WHERE scenarioid=$scenarioid"); # Delete all the images if(file_exists($map_filename)) unlink($map_filename); if(file_exists($summary_filename)) unlink($summary_filename); if(file_exists($solution_filename)) unlink($solution_filename); } } } else { $scenarioid = (int) $db->insert("INSERT INTO scenario (create_time) VALUES (now())"); $map_filename = sprintf("scenarios/%06d_map.gif",$scenarioid); $solution_filename = sprintf("scenarios/%06d_solution.gif",$scenarioid); $summary_filename = sprintf("scenarios/%06d_summary.gif",$scenarioid); # Create a new scenario. # Now, the physical size of the scenario depends on how far # the agent is from the target. Lets make this a random number # between the min and maximum distances. $distance = get_rand() * (MAX_DISTANCE - MIN_DISTANCE) + MIN_DISTANCE; # $distance is the basis for many things, such as the width and height # of the world, and its acts as a scaling factor when normalising # the whole scenario. # Lets work out the density of obstacles in the scenario. # This depends on the size of the world, the more space, the more # can fit. I.e. it is directly proportional to the square of $distance, # with a random factor thrown in for good measure. $area = $distance * $distance; $o_count = round( get_rand() * $area / 10000 * (MAX_DENSITY - MIN_DENSITY) + MIN_DENSITY ); # Radius of the agent. $a_radius = get_rand() * (MAX_RADIUS - MIN_RADIUS) + MIN_RADIUS; # Now generate all the obstacles. $obstacles = array(); for($i = 0; $i < $o_count; $i++) { $r = get_rand() * (MAX_RADIUS - MIN_RADIUS) + MIN_RADIUS; # Try to find a spot to put it that isnt on top of another obstacle for($try = 0; $try < 20; $try++) { $x = get_rand() * 2 * $distance - $distance; $y = get_rand() * 2 * $distance - $distance; # Too close to the agent? if(sqrt(pow($x,2)+pow($y,2)) < $r + $a_radius + 1) continue; $c_o_count = count($obstacles); for($j = 0; $j < $c_o_count; $j++) { $o_r = $obstacles[$j]['r']; $o_x = $obstacles[$j]['x']; $o_y = $obstacles[$j]['y']; if(sqrt(pow($o_x-$x,2)+pow($o_y-$y,2)) < $r + $o_r + 1) continue 2; } break; } if($try >= 20) { $o_count = $i; break; } $obstacles[$i]['r'] = $r; $obstacles[$i]['x'] = $x; $obstacles[$i]['y'] = $y; } # Work out the summary vector based on the obstacle positions. # The summary vector is an array of 18 elements (0-17), each # one representing an 10-degree arc of the horizon. It covers # the 180 degrees that hasthe target in the centre, on the line in # between arcs 8 and 9. The value of each element represents how # close an obstacles are in that arc - only counting obstacles # that are within the radius defined by the distance to the target. # Arcs with no obstacles in them, or obstacles on the border will


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering # have values of 0. Arcs with an obstacle halfway to the border # will have value of 0.5, while arcs containing very close obstacles # will have values approachng 1. $summary_vector = point_array_to_summary_vector($obstacles,1,0); # Save the scenario to the database $db->update("UPDATE scenario SET distance=$distance, o_count=$o_count, a_radius=$a_radius, obstacles='".addslashes(serialize($obstacles))."', summary_vector='".addslashes(serialize($summary_vector))."' WHERE scenarioid=$scenarioid"); } # Run the neural network to get the computer-guessed solution to this scenario $computed_vector = run_nn($summary_vector); ###################### # Create the map image if(!file_exists($map_filename)) { ########################### # Create the image resource $im = imageCreate(500,550); # Colour allocations $fbg_colour = imageColorAllocate($im, 200, 200, 230); $bbg_colour = imageColorAllocate($im, 150, 150, 170); $agent_colour = imageColorAllocate($im, 60, 200, 60); $target_colour = imageColorAllocate($im, 200, 60, 60); $obstacle_colour = imageColorAllocate($im, 60, 60, 200); $solution_colour = imageColorAllocate($im, 90, 150, 90); $computed_colour = imageColorAllocate($im, 150, 90, 90); $black = imageColorAllocate($im, 0, 0, 0); $darkgrey = imageColorAllocate($im, 96, 96, 96); $white = imageColorAllocate($im, 255, 255, 255); # Draw the background imageFilledRectangle($im,0,0,499,249,$fbg_color); imageFilledRectangle($im,0,250,499,499,$bbg_colour); imageRectangle($im,0,0,499,549,$black); # Overlay this with the NN's chosen solution space. $verts = array(250,250); foreach($computed_vector as $segmentid => $value) { $verts[] = round(-sin(deg2rad($segmentid*10+90)) * $value * 250 + 250); $verts[] = round(cos(deg2rad($segmentid*10+90)) * $value * 250 + 250); $verts[] = round(-sin(deg2rad($segmentid*10+95)) * $value * 2 * 250 + 250); $verts[] = round(cos(deg2rad($segmentid*10+95)) * $value * 2 * 250 + 250); $verts[] = round(-sin(deg2rad($segmentid*10+100)) * $value * 250 + 250); $verts[] = round(cos(deg2rad($segmentid*10+100)) * $value * 250 + 250); } imageFilledPolygon($im,$verts,(int)count($verts)/2,$computed_colour); # Division lines on the 2D Map for($angle = 0; $angle < 360; $angle += 10) { $x = round(sin(deg2rad($angle)) * 250 + 250); $y = round(cos(deg2rad($angle)) * 250 + 250); imageLine($im,250,250,$x,$y,$black); } for($i = 0.1; $i <= 1; $i += 0.1) { imageCircle($im,250,250,$i * 250,$darkgrey); } # Draw the target imageFilledCircle($im,250,0,d_norm(5)*250,$target_colour); # Draw all the obstacles foreach($obstacles as $o) { imageFilledCircle($im, 250+d_norm($o['x'])*250, 250-d_norm($o['y'])*250, d_norm($o['r'])*250, $obstacle_colour); } # Draw all the solutions if(is_array($solutions)) { foreach($solutions as $s) { imageFilledCircle($im,


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering 250+d_norm($s['x'])*250, 250-d_norm($s['y'])*250, d_norm($a_radius)*250, $solution_colour); imageLine($im,250,250,250+d_norm($s['x'])*250, 250-d_norm($s['y'])*250,$solution_colour); imageLine($im,249,250,250+d_norm($s['x'])*250-1, 250-d_norm($s['y'])*250,$solution_colour); imageLine($im,251,250,250+d_norm($s['x'])*250+1, 250-d_norm($s['y'])*250,$solution_colour); } } # The agent. imageFilledCircle($im,250,250,d_norm($a_radius)*250,$agent_colour); # Division lines on the 1D Map imageFilledRectangle($im,0,500,499,549,$black); imageLine($im,0,499,499,499,$white); for($angle = 10; $angle < 180; $angle += 10) { $x = round(($angle/180)*499); imageLine($im,$x,500,$x,548,$white); } # Colour the 1D summary map showing target and obstacles for($i = 0; $i < 18; $i++) { $r = pow(1 - abs($i - 8.5) / 8.5,2) * 255; $g = $solution_vector[$i] * 255;; $b = $summary_vector[$i] * 255; $segment_colors[$i] = imageColorAllocate($im,$r,$g,$b); imageFilledRectangle($im, round($i/18*499+1), 500, round(($i+1)/18*499-1), 548, $segment_colors[$i] ); } # Write the image out to a file imageGIF($im, $map_filename); } ################################ # Create the summary vector image if(!file_exists($summary_filename)) { $im = imageCreate(180,100); # Colour allocations $bg_colour = imageColorAllocate($im, 0xee,0xf8,0xff); $border_colour = imageColorAllocate($im, 0xdd,0xe8,0xee); $black = imageColorAllocate($im, 0, 0, 0); ImageFilledRectangle($im,0,0,179,99,$bg_colour); ImageRectangle($im,0,0,179,99,$border_colour); for($i = 0; $i < 18; $i++) { $v = $summary_vector[$i]; $c = imageColorAllocate($im,64,64,64+191*$v); ImageFilledRectangle($im,$i*10,99-$v*99,$i*10+8,99,$c); ImageRectangle($im,$i*10,99-$v*99,$i*10+8,99,$black); } # Write the image out to a file imageGIF($im, $summary_filename); } ################################ # Create the solution vector image if(!file_exists($solution_filename)) { $im = imageCreate(180,100); # Colour allocations $bg_colour = imageColorAllocate($im, 0xee,0xf8,0xff); $border_colour = imageColorAllocate($im, 0xdd,0xe8,0xee); $black = imageColorAllocate($im, 0, 0, 0); ImageFilledRectangle($im,0,0,179,99,$bg_colour); ImageRectangle($im,0,0,179,99,$border_colour);


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering for($i = 0; $i < 18; $i++) { $v = $solution_vector[$i]; $cv = $computed_vector[$i]; $c = imageColorAllocate($im,64,64+191*$v,64); $cc = imageColorAllocate($im,128+127*$v,64,64); # Computed ImageFilledRectangle($im,$i*10,99-$cv*99,$i*10+5,99,$cc); ImageRectangle($im,$i*10,99-$cv*99,$i*10+5,99,$black); # Real ImageFilledRectangle($im,$i*10+3,99-$v*99,$i*10+8,99,$c); ImageRectangle($im,$i*10+3,99-$v*99,$i*10+8,99,$black); } # Write the image out to a file imageGIF($im, $solution_filename); } # Print the page ?> <html> <head> <title>Helix Crowd - Agent Nevigation Trainer - Scenario No. <?=$scenarioid?></title> </head> <style> body,p,td,ul,li { font-family: verdana,arial,helvetica; font-size: 10pt; } </style> <body marginwidth=0 marginheight=0 leftmargin=0 topmargin=0 bgcolor=#ffffff> <table cellpadding=5 cellspacing=0 border=0 width=100% height=100%><td><td align=center> <form name=solution type=get> <input type=hidden name=scenarioid value=<?=$scenarioid?>> <input type=hidden name=time value=<?=time()?>> <table cellpadding=1 cellspacing=0 border=0 bgcolor=#000000 width=750><tr><td><table cellpadding=5 cellspacing=1 border=0> <tr bgcolor=#99aaee> <td align=center colspan=2><h2>Helix Crowd - Agent Navigation Trainer</td> </tr> <tr bgcolor=#ccddff> <td colspan=2 align=center> <table cellpadding=0 cellspacing=5 border=0> <tr> <td bgcolor=#000000 valign=top> <table cellspacing=1 cellpadding=2 border=0 height=100%> <tr bgcolor=#ffffff> <td colspan=2 align=center><b>SCENARIO INFO</td> </tr> <tr bgcolor=#eef8ff> <td><b>Scenario ID</b></td> <td><?=$scenarioid?></td> </tr> <tr bgcolor=#eef8ff> <td><b>Target Distance</b></td> <td><?=sprintf('%.2f',$distance)?></td> </tr> <tr bgcolor=#eef8ff> <td><b>Agent Radius</b></td> <td><?=sprintf('%.2f',$a_radius)?></td> </tr> <tr bgcolor=#eef8ff> <td><b>No. of Obstacles</b></td> <td><?=$o_count?></td> </tr> <tr bgcolor=#eef8ff> <td colspan=2 align=center> <img src="<?=$summary_filename?>?<?=time()?>" border=0 width=180 height=100><br> <?if($scenarioid>1){?> <a href="./?scenarioid=<?=$scenarioid-1?>&<?=time()?>"><b><<Prev.</b></a> <?}else{?> <<Prev. <?}?> |


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering <a href="./?scenarioid=<?=$scenarioid+1?>&<?=time()?>"><b>Next>></b></a> </td> </tr> <tr></tr> <tr bgcolor=#ffffff> <td colspan=2 align=center><b>SOLUTIONS</td> </tr> <tr bgcolor=#eef8ff> <td><b>Solutions</b></td> <td><?=$solutions?count($solutions):0?></td> </tr> <tr bgcolor=#eef8ff> <td colspan=2 align=center> <img src="<?=$solution_filename?>?<?=time()?>" border=0 width=180 height=100><br> <span style="color:#008800"><?=$solution_msg?></span> </td> </tr> <tr></tr> <tr bgcolor=#ffffff> <td colspan=2 align=center><b>GENERAL RULES</td> </tr> <tr bgcolor=#eef8ff> <td><b>Min. Distance</b></td> <td><?=MIN_DISTANCE?></td> </tr> <tr bgcolor=#eef8ff> <td><b>Max. Distance</b></td> <td><?=MAX_DISTANCE?></td> </tr> <tr bgcolor=#eef8ff> <td><b>Min. Density*</b></td> <td><?=MIN_DENSITY?></td> </tr> <tr bgcolor=#eef8ff> <td><b>Max. Density*</b></td> <td><?=MAX_DENSITY?></td> </tr> <tr bgcolor=#eef8ff> <td><b>Min. Radius</b></td> <td><?=MIN_RADIUS?></td> </tr> <tr bgcolor=#eef8ff> <td><b>Max. Radius</b></td> <td><?=MAX_RADIUS?></td> </tr> <tr bgcolor=#eef8ff> <td><b>Arc Segments</b></td> <td>18</td> </tr> </table> </td> <td valign=top><input type=image src="<?=$map_filename?>?<?=time()?>" border=0 width=500 height=550 alt="<?="Click the position that you recommend the green agent should head for, considering that:\n\n * It wants to reach the red target.\n * It must avoid hitting blue obstacles.\n * It will be given the chance to reevaluate its direction very soon after it begins moving.\n * The quicker it gets to the target (the more direct its route) the better.\n * The further from the green spot you click, the more highly recommended your recommendation is compared to other clicks for this scenario.\n * Try to avoid clicking in the lower (dark) half unless in an impossible situation."?>"></td> </tr> <tr> <td colspan=2>* Density of obstacles per 10,000 square units (100x100).</td> </tr> </table> </td> </tr> <tr bgcolor=#eef8ff> <td colspan=2> <a href="./?export=1&scenarioid=<?=$scenarioid?>">Retrain</a> <? ############ # Exporting? if($export) {


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering $data = $db->associative_array("SELECT scenarioid, summary_vector, solution_vector FROM scenario WHERE solution_vector != ''"); $tr = fopen('../bp/helix_training.dat','w'); $te = fopen('../bp/helix_test.dat','w'); $count = count($data); $i = 0; foreach($data as $scenarioid => $datum) { if($i++ < $count * 0.75) { $out = &$tr; } else { $out = &$te; } foreach(unserialize($datum['summary_vector']) as $v) { fwrite($out,sprintf("%.5f ",$v*2-1)); } fwrite($out,"\n"); foreach(unserialize($datum['solution_vector']) as $v) { fwrite($out,sprintf("%.5f ",$v*2-1)); } fwrite($out,"\n"); } fclose($tr); fclose($te); echo " : Exported $count records (".ceil($count*0.75)." training, ".floor($count*0.25)." test)."; # Now do the retraining. $output = retrain_nn(); echo "<br><b>Retraining output:</b><pre>$output</pre>"; # Clear the images. $d = opendir('scenarios'); while($f = readdir($d)) { if(substr($f,-4) == '.gif') unlink("scenarios/$f"); } closedir($d); } else { echo "(Click to export the current data and retrain the neural network.)"; } ?> </td> </tr> <tr bgcolor=#eef8ff> <td colspan=2> <h3>Instructions</h3> <p>This interface is designed to help you to quickly build a large set of training data for self-steering agent (robot, A.I. life form etc). You are presented with an infinite set of randomly generated scenarios. In each one, the agent (represented by a <span style="color:#44dd44">green</span> spot in the centre) is trying to reach the target (represented by a <span style="color:#dd4444">red</span> spot at the top) while avoiding hitting any obstacles (represented by <span style="color:#4444dd">blue</span> spots scattered over the map). Scenarios vary greatly in terms of size (distance to target) and density of obstacles. <p>Click the position that you recommend the green agent should head for, considering that: <ul> <li>It wants to reach the red target. <li>It must avoid hitting blue obstacles. <li>It will be given the chance to reevaluate its direction very soon after it begins moving. <li>The quicker it gets to the target (the more direct its route) the better. <li>The further from the green spot you click, the more highly recommended your recommendation is compared to other clicks for this scenario. <li>You should try to avoid clicking in the lower (dark) half unless in an impossible situation. </ul> <p>You may make more than one solution recommendation, and these are represented on the scenario map by <span style="color:#338833">dull green</span> spots connected to the agent by lines. You can cancel/remove a recommendation by clicking on it. <p>Use the <b>Prev</b> and <b>Next</b> buttons to move between scenarios. New scenarios are automatically generated as you move along. <p>Above the <b>Prev</b> and <b>Next</b> buttons is a summary graph. This represents the information stored about the scenario in the training data. It has 18 parts, each representing a 10-degree arc on the east-north-west horizon. The higher the column, the <b>closer</b> an obstacle is in that segment.


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering <p>A similar graph is created in the <b>Solutions</b> section as you place your solution recommendations; the columns of this graph are higher the <b>further</b> your recommendation is from the centre. Generally you should have tall columns in your green solution graph where you have short columns on the blue scenario graph. You should have most of your high green columns in the centre - that is, pointing towards the target. <p>These two summary graphs represent the 18-part vectors that will be fed to a neural network (NN) over and over as training/example data: as if to say <i>if you're in a situation like <span style="color:#333388">this</span>, then <span style="color:#338833">this</span> is the best direction to head.</i> With enough sample data, the NN should learn to make "intelligent" navigation decisions when avoiding obstacles while heading for a target. </td> </tr> <tr bgcolor=#eef8ff> <td colspan=2>© Copyright Steve Agland 2002 - <a href="mailto:[email protected]">[email protected]</a></td> </tr> </table></td></tr></table> </form> </td></tr></table> </body> </html>



Appendix B - Training Engine and User Interface (MySQL Database Schema) CREATE TABLE scenario ( scenarioid MEDIUMINT UNSIGNED NOT NULL AUTO_INCREMENT, distance REAL UNSIGNED NOT NULL, # From agent to target o_count MEDIUMINT UNSIGNED NOT NULL, # Obstacles in the scenario a_radius REAL UNSIGNED NOT NULL, # Radius of the agent create_time DATETIME NOT NULL, # Time the scenario was created. record_time DATETIME NOT NULL, # Time this scenario was solved by a human record_ip VARCHAR(15) NOT NULL, # IP of user who sovled the scenario. obstacles LONGTEXT NOT NULL, # Serialized array of obstacle position info summary_vector TEXT NOT NULL, # Array representing summary for NN input solutions LONGTEXT NOT NULL, # Array of coords of points to head for solution_vector TEXT NOT NULL, # Like summary, but strong segments are best to head PRIMARY KEY(scenarioid) );

Appendix C – Ten Sample Vector Tuples of Training Data -0.10102 0.07043 0.35955 -0.41855 0.35582 0.76050 0.76192 0.19870 -0.10842 0.35088 0.78266 0.74919 -0.22640 0.68260 0.77625 0.80597 0.90041 0.94088 -1.00000 -1.00000 -0.66826 -0.66117 -1.00000 -1.00000 -1.00000 -0.11906 0.29003 0.31148 -1.00000 -1.00000 -0.38939 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -0.97290 -1.00000 -1.00000 0.24721 -1.00000 -1.00000 -0.95447 -1.00000 -1.00000 -0.77949 -0.82358 -0.98511 -1.00000 -0.34183 -1.00000 -1.00000 -0.84280 -0.58814 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 0.89645 0.88599 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 0.25450 0.20311 -1.00000 -1.00000 0.47858 0.50725 0.42064 -0.44392 -1.00000 -0.03900 0.01354 -1.00000 -1.00000 -1.00000 0.35631 0.41164 -0.02656 0.10529 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 1.00000 -1.00000 -1.00000 -0.23741 -0.13691 -0.20655 -1.00000 -1.00000 -1.00000 -1.00000 -0.65086 -1.00000 -0.84579 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -0.81330 -1.00000 -0.50999 -0.53336 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 0.18103 0.94624 0.92294 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 0.70888 0.27301 0.49170 0.48327 -0.07411 -0.07344 0.03920 0.06900 -0.39235 0.90378 0.14226 -1.00000 0.82434 0.92271 0.35854 0.52457 0.52730 0.18993 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -0.24616 -0.27177 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 0.00776 -0.25274 0.27699 0.88143 0.80229 -0.26535 -0.25723 0.78515 0.92478 0.84032 -0.69651 -0.85104 0.64882 0.68863 0.61790 0.22801 0.32607 0.25349 -1.00000 -0.51108 -0.53149 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 0.56932 0.59970 -1.00000 -1.00000 -1.00000 -1.00000 -0.41022 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -0.35143 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 1.00000 1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -0.40965 -0.40344 -0.94883 -0.02494 -1.00000 -1.00000 -1.00000 -1.00000 0.10810 -1.00000 -1.00000 -0.58839 0.07060 -0.77533 -0.82007 -0.92923 -0.53525 0.10300 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 0.06169 -1.00000 -1.00000 0.97675 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 0.46467 0.45653 0.36348 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -0.83941 -0.95823 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 0.12686 0.12945 0.05810 1.00000 1.00000 0.60485 0.56957 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -0.23723 -0.16996 -0.19784 -1.00000 -1.00000 -1.00000 -0.32250 -0.24594 -0.37369 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 -1.00000 0.84385 1.00000 0.99831 -1.00000 -1.00000 -1.00000 0.01152 0.17798 0.15713 0.06751 -1.00000 -1.00000 -1.00000



Appendix D – Script for Tveter’s Backprop Software – Retraining. ***************************************************************** ** HELIX CROWD SIMULATOR ------- (C) Copyright Steve Agland 2002 ** **-----------------------------------------------------------------** ** $Source: /home/cvsroot/helix_crowd/bp/helix_retrain.bp,v $ ** $Revision: 1.1 $ ** $Author: agi $ ** $Date: 2002/10/23 06:07:53 $ **-----------------------------------------------------------------** ** [email protected] ** ***************************************************************** * * This is an initialisation file for Donald R. Tveter's AI Backprop * software. It is run by the training interface to update the weights. * This is a general problem type, as opposed to a classification problem f pg * 5000 lines to a page, to stop it "More?"ing :( f P 5000 * Create a 2-layer neural network, with 18 inputs and 18 outputs. This was found to be the most successful configuration. A hiddenlayer provided better results on the training dataset, but worse on the test dataset. m 18 18 * Seed the random number function s 28378 * Clear and initialize the network with random weights ci * Read in the training data rt /home/agi/helix_crowd/bp/helix_training.dat * Read in the test data tf /home/agi/helix_crowd/bp/helix_test.dat * Set the eta, or learning rate. This was found to be the most successful value - converging on a minimum within only about 120 iterations. e 0.001 * set eta, the learning rate to 0.5 (and eta2 to 0.5) * Use Tan activation, which output values between -1 and 1 a aht * hidden layers a aot * output layer * Tolerance per output unit to 0.1 * Overall tolerance t 0.15 * Saves the weights when a minimum is reached swem+ * File to save the weights to f w r sw /home/agi/helix_crowd/bp/helix_weights.txt * Change the gain/scaling thingy a Do 0.6 * Run the training routine 500 times. r 500 10 * Quit q



Appendix E – Sample Tveter Backprop Retraining Output Note that the following listing contains some output of the PHP script. The minimum was reached extremely early in the run, somewhere in the 20’s. Since this is not a classification problem, and due to the nature of the data, is was unlikely that the NN would ever get precise matches with the test data. Close enough is good enough. Retrain : Exported 573 records (430 training, 143 test). Retraining output: no data file, stdin assumed >>>>>>>>>>>>>>>>>>>>>>>>>>>>seed = 28378, range = - 1.00 to + 1.00 >>430 training patterns read >>143 test patterns read >>>>>>>running . . . 10 0.47 % 0.17816 0.70 % 0.16238 20 0.47 % 0.16787 0.00 % 0.15603 30 0.00 % 0.17764 0.00 % 0.16437 40 0.00 % 0.18317 0.00 % 0.17104 50 0.00 % 0.17961 0.00 % 0.16922 60 0.00 % 0.17957 0.00 % 0.17096 70 0.00 % 0.18063 0.00 % 0.17293 80 0.00 % 0.17888 0.00 % 0.17072 90 0.00 % 0.17920 0.00 % 0.17110 100 0.00 % 0.17962 0.00 % 0.17210 110 0.00 % 0.17954 0.00 % 0.17184 120 0.00 % 0.17959 0.00 % 0.17201 130 0.00 % 0.17975 0.00 % 0.17225 140 0.00 % 0.17982 0.00 % 0.17227 150 0.00 % 0.17991 0.00 % 0.17231 160 0.00 % 0.17998 0.00 % 0.17233 170 0.00 % 0.18005 0.00 % 0.17236 180 0.00 % 0.18011 0.00 % 0.17240 190 0.00 % 0.18016 0.00 % 0.17244 200 0.00 % 0.18021 0.00 % 0.17249 210 0.00 % 0.18024 0.00 % 0.17255 220 0.00 % 0.18027 0.00 % 0.17261 230 0.00 % 0.18030 0.00 % 0.17266 240 0.00 % 0.18033 0.00 % 0.17271 250 0.00 % 0.18035 0.00 % 0.17275 260 0.00 % 0.18037 0.00 % 0.17278 270 0.00 % 0.18039 0.00 % 0.17281 280 0.00 % 0.18041 0.00 % 0.17284 290 0.00 % 0.18043 0.00 % 0.17286 300 0.00 % 0.18045 0.00 % 0.17288 310 0.00 % 0.18046 0.00 % 0.17289 320 0.00 % 0.18048 0.00 % 0.17290 330 0.00 % 0.18049 0.00 % 0.17291 340 0.00 % 0.18050 0.00 % 0.17291 350 0.00 % 0.18051 0.00 % 0.17291 360 0.00 % 0.18052 0.00 % 0.17291 370 0.00 % 0.18053 0.00 % 0.17290 380 0.00 % 0.18053 0.00 % 0.17289 390 0.00 % 0.18054 0.00 % 0.17288 400 0.00 % 0.18054 0.00 % 0.17287 410 0.00 % 0.18054 0.00 % 0.17286 420 0.00 % 0.18055 0.00 % 0.17285 430 0.00 % 0.18055 0.00 % 0.17283 440 0.00 % 0.18055 0.00 % 0.17282 450 0.00 % 0.18055 0.00 % 0.17281 460 0.00 % 0.18055 0.00 % 0.17280 470 0.00 % 0.18056 0.00 % 0.17278 480 0.00 % 0.18056 0.00 % 0.17277 490 0.00 % 0.18056 0.00 % 0.17276 500 0.00 % 0.18056 0.00 % 0.17275 >>



Appendix F – Selected files from the LISP/CLOS Simulation Software Only those relevant files are included here – that is, those modified or added since the publication of the previous paper on this project. This paper, including code appendices, can be found at: http://www.slinkies.net/crowd_simulation.pdf ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; HELIX CROWD SIMULATOR ------- (C) Copyright Steve Agland 2002 ;; ;; ;;-----------------------------------------------------------------;; ;; $Source: /home/cvsroot/helix_crowd/main.lisp,v $ ;; $Revision: 1.9 $ ;; $Author: agi $ ;; $Date: 2002/10/23 06:08:10 $ ;;-----------------------------------------------------------------;; ;; [email protected] ;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; ;; ; NOTES ; One unit in measurement = one centimeter ;;;;;;;;;;;;;;;;;;;;;; ; Define some globals ;(defconstant *source_root* "/home/agi/lisp/helix_crowd") (defconstant *source_root* ".") (setf *WARN-ON-FLOATING-POINT-CONTAGION* NIL) ;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Special function defintion (defun include (file) (load (concatenate 'string *source_root* "/" file))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; This function reparses the source code, presumably after modification (defun reload_helix () (include "main.lisp")) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Include function definition files (include "general.lisp") ;;;;;;;;;;;;;;;;;;;;;;; ; Third-party libraries (include "lib/atof.lisp") (include "lib/matrix.lisp") ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Include class definition files (include "datareader.lisp") (include "datawriter.lisp") (include "world.lisp") (include "neuralnetwork.lisp") ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; And then someone created the *world* (defun helix () (defvar *world* (make-instance 'World)) (populate *world*) ; Create a brain for all the agents to share (write-line "") (write-string "Creating brain (steering neural network): ") (defvar *brain* (make-instance 'NeuralNetwork)) (build *brain* (list 18 18)) (load_weights *brain* (concatenate 'string *source_root* "/bp/helix_weights.txt")) (write-string "....") (run_simulation *world* 300)) (helix)


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; HELIX CROWD SIMULATOR ------- (C) Copyright Steve Agland 2002 ;; ;; ;;-----------------------------------------------------------------;; ;; $Source: /home/cvsroot/helix_crowd/neuralnetwork.lisp,v $ ;; $Revision: 1.1 $ ;; $Author: agi $ ;; $Date: 2002/10/23 06:05:33 $ ;;-----------------------------------------------------------------;; ;; [email protected] ;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; This class defines a simply neural network system - capable ; of performing, but not learning. It reads in its weights ; from an external file, generated by Donald R. Tveter's AI ; Backprop program. This is where NN development should be ; done. It only works with two-layer networks. (defclass NeuralNetwork () ( (layers :accessor layers :documentation "A list of layers, which are integers representing the number of neurons in them." :initform () :initarg :layers :type 'list) (weights :accessor weights :documentation "An array of layers, which are arrays of neurons, which are arrays weights for each of the neurons in the preceeding layer." :initform () :initarg :weights :type 'array) (bias_weights :accessor bias_weights :documentation "An array of layers, which are arrays of neurons, which are weights to that layer's bias node, which is assumed ot have a constant value of 1." :initform () :initarg :bias_weights :type 'array) (input :accessor input :documentation "A vector representing the current input values." :initform #() :initarg :input :type 'array) )) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Establishes the number of neurons in each layer (defmethod build ((nn NeuralNetwork) (l list)) (setf max_neurons (eval (cons 'max l))) (setf (weights nn) (make-array (list (length l) max_neurons max_neurons) :initial-element 0)) (setf (bias_weights nn) (make-array (list (length l) max_neurons) :initial-element 0)) (setf (layers nn) l)) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Reads in weights from an provided weights file (defmethod load_weights ((nn NeuralNetwork) (f string)) (setf dr (make-instance 'DataReader :filename f)) (read_line dr) ; Clear the first header line (loop when (setf line (ws_read_line dr)) do ;; We decrement all the indexes because the weight files start at 1 instead of 0 (setf layer (1- (parse-integer (nth 5 line)))) (setf neuron (1- (parse-integer (nth 6 line)))) (if (equal "b" (setf input (nth 3 line))) (setf input -1) (setf input (1- (parse-integer input)))) (setf value (tolerant-parse-float (first line))) (if (< input 0) ; Bias weight (setf (aref (bias_weights nn) layer neuron) value) (setf (aref (weights nn) layer neuron input) value)) while line) (close_stream dr))


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Given a vector, with an input for each of the input neurons ; will return a result vector for each of the output neurons (defmethod process ((nn NeuralNetwork) (input vector)) (setf (input nn) input) (get_layer_output nn (1- (length (layers nn))))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; A recursive function that returns the outputs for a ; layer based on the input, which is the output of the ; previous layer. (defmethod get_layer_output ((nn NeuralNetwork) (layer_no integer)) (if (> layer_no 0) (progn (setf input (get_layer_output nn (1- layer_no))) (setf output (make-array (list (nth layer_no (layers nn))) :initial-element 0.0)) (loop for i from 0 below (length output) do (setf (aref output i) (aref (bias_weights nn) layer_no i)) (loop for j from 0 below (length input) do (setf (aref output i) (+ (aref output i) (* (aref input j) (aref (weights nn) layer_no i j)))))) (map 'vector #'output_squash output)) ; Squash those numberz (input nn))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; The squashing function applied to output values. Currently ; this only uses the tanh method (to produce values between -1 ; and 1, which is what we want in this application) ; Hard-coded D (scale/gain) modifier ; fill fix this if I ever use this for another application (defun output_squash (v) (tanh (* 0.6 v)))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; HELIX CROWD SIMULATOR ------- (C) Copyright Steve Agland 2002 ;; ;;-----------------------------------------------------------------;; ;; $Source: /home/cvsroot/helix_crowd/agent.lisp,v $ ;; $Revision: 1.7 $ ;; $Author: agi $ ;; $Date: 2002/10/21 12:37:43 $ ;;-----------------------------------------------------------------;; ;; [email protected] ;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; An agent is an "intelligent" object that changes its state ; according to environmental factors ; Now for the purposes of this application, the agents are ; slinkies, or essentailly double-headed worms. They have two ; ends each of which has a position and a rotation. ; Each end/position/rotation combo also has a velocity, hend ; the first 8 slots below (defclass Agent (Entity) ( ; These first 8 slots are for 3D information, storing ; the position and rotation of the two ends. The other ; four are "velocity" values - that is, how fast each ; of the previous four attributes is changing, and in ; which direction. ; ; End 1 (pos1 :accessor pos1 :documentation "The position one end of the agent in 3D space" :initform #(0 0 0) ; Origin :initarg :pos1 :type 'vector) (rot1 :accessor rot1 :documentation "The rotation one end of the agent in 3D space" :initform #(0 0 0) ; Origin :initarg :rot1 :type 'vector) ; End 2 (pos2 :accessor pos2 :documentation "The position the other end of the agent in 3D space" :initform #(0 0 0) ; Origin


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering :initarg :pos2 :type 'vector) (rot2 :accessor rot2 :documentation "The rotation the other end of the agent in 3D space" :initform #(0 0 0) ; Origin :initarg :rot2 :type 'vector) ; Velocities ; End 1 (vpos1 :accessor vpos1 :documentation "The position velocity one end of the agent in 3D space" :initform #(0 0 0) ; Origin :initarg :vpos1 :type 'vector) (vrot1 :accessor vrot1 :documentation "The rotation velocity one end of the agent in 3D space" :initform #(0 0 0) ; Origin :initarg :vrot1 :type 'vector) ; End 2 (vpos2 :accessor vpos2 :documentation "The position velocity the other end of the agent in 3D space" :initform #(0 0 0) ; Origin :initarg :pos2 :type 'vector) (vrot2 :accessor vrot2 :documentation "The rotation velocity the other end of the agent in 3D space" :initform #(0 0 0) ; Origin :initarg :rot2 :type 'vector) ; Other slinky-generating parameters (radius :accessor radius :documentation "The radius of the coils." :initform 5 :initarg :radius :type 'float) (coil_count :accessor coil_count :documentation "The number of coils." :initform 40 :initarg :documentation :type 'float) (strand_height :accessor strand_height :documentation "The height of the slinky strand." :initform 5 :initarg :strand_height :type 'float) (stepp :accessor stepp :documentation "A value between 0 and 1 indicating whether the slinky sits on one foot or the other, or somewhere slinkin' in between." :initform 0.4 :initarg :stepp ; Corresponds to "step" in houdini :/ :type 'float) (step_speed :accessor step_speed :documentation "To do with how stretchy the slinky is as it steps." :initform 0.3 :initarg :step_speed :type 'float) (step_height :accessor step_height :documentation "Controls the extent of the arc made by the coils as they step." :initform 1 :initarg :step_height :type 'float) ; Personality data. These are weightings which determine how the ; agent behaves. (max_vel1 :accessor max_vel1 :documentation "The maximum velocity any pos1 may move. cm/frame" :initform 0.25 :initarg :max_vel1 :type 'float) (max_vel2 :accessor max_vel2 :documentation "The maximum velocity any pos2 may move. cm/frame" :initform 0.25 :initarg :max_vel2 :type 'float) (max_rot_vel :accessor max_rot_vel :documentation "The maximum velocity any of its points may move. cm/frame" :initform 20 :initarg :max_rot_vel :type 'float) (accel1 :accessor accel1


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering :documentation "The rate at which pos1 velocities may increase. cm/frame/frame" :initform 0.2 :initarg :accel1 :type 'float) (acce2 :accessor accel2 :documentation "The rate at which pos2 velocities may increase. cm/frame/frame" :initform 0.1 :initarg :accel2 :type 'float) (agility :accessor agility :documentation "The rate at which rotational velocities may increase. deg/frame/frame" :initform 4 :initarg :agility :type 'float) ; A Datawriter for writing out geometric information for Houdini's use (out :accessor out :documentation "A Datawriter for writing out geometric information for Houdini's use" :initform () :initarg out :type 'datawriter) )) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Makes sure rotations are modulated ;(defmethod (setf rot1) ((a Agent) (v vector)) ; (print "[rot1]") (setf (slot-value a 'rot1) (mod_balanced_lookat_vector v))) ;(defmethod (setf rot2) ((a Agent) (v vector)) ; (print "[rot2]") (setf (slot-value a 'rot2) (mod_balanced_lookat_vector v))) ;(defmethod (setf vrot1) ((a Agent) (v vector)) ; (print "[vrot1]") (setf (slot-value a 'vrot1) (vabstrim (mod_balanced_lookat_vector v) (max_rot_vel a)))) ;(defmethod (setf vrot2) ((a Agent) (v vector)) ; (print "[vrot2]") (setf (slot-value a 'vrot2) (vabstrim (mod_balanced_lookat_vector v) (max_rot_vel a)))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Initializes the physical attributes of the ; agent based on a list of values sent through ; in an expected order. (defmethod initialise ((a Agent) (data list) (id integer)) ; ID (setf (id a) id) ; Geometirc data ; First three data = pos1 vector (setf (pos1 a) (make-array 3 :initial-contents (subseq data 0 3))) ; Second three data = pos2 vector (setf (pos2 a) (make-array 3 :initial-contents (subseq data 3 6))) ; First end rotation from face-down to point at the third vector from pos1 (setf (rot1 a) (lookat_vector3 (pos1 a) (make-array 3 :initial-contents (subseq data 6 9)))) ;(setf (rot1 a) (make-array 3 :initial-contents (subseq data 6 9))) ; Second end always init flat (on ground) (setf (rot2 a) #(0 0 0)) ; Other slinky attributes (setf (radius a) (nth 9 data)) (setf (coil_count a) (nth 10 data)) (setf (strand_height a) (nth 11 data)) (setf (stepp a) (nth 12 data)) (setf (step_speed a) (nth 13 data)) (setf (step_height a) (nth 14 data)) ; Aglility less the bigger you are (setf (accel1 a) (/ 0.5 (radius a))) (setf (accel2 a) (/ 0.3 (radius a))) (setf (agility a) (/ 5 (radius a))) ; Speed less the bigger you are (setf (max_vel1 a) (/ 4.0 (radius a))) (setf (max_vel2 a) (/ 3.0 (radius a))) (setf (max_rot_vel a) (/ 20 (radius a))) ; Influences: (update_inate_influences a)


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering ; The output file (setf (out a) (make-instance 'datawriter :filename (concatenate 'string *source_root* "/out/agent" (write-to-string (id a)) ".chan"))) ;(print_data a) a) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Updates the agents attributes based on the environ ; ment in the current frame (defmethod update_frame_data ((a Agent)) ; The point the agent most wants to look at, and how much (setf look_at_pos '(0 . #(0 0 0))) ; The point the agent most wants to move to, and how much (setf move_to_pos (cons 0 (xz (pos2 a)))) ; The points the agent most wants to escape, and how much (setf move_from_poss ()) ; The points the agent most wants to steer away from, and how much (setf steer_from_poss ()) (loop for effect in (get_influence_summary *world* (v/ (v+ (pos1 a) (pos2 a)) 2) ; Influence effect from the midpoint of the two ends 0.1 ; Ignore any influence with a distance-factored strength less than this (list (id a))) ; Ignore my id, I don't need influences from myself do (cond ; Look-at effects ((= (aref effect 0) *influence_look_at*) (when (> (aref effect 2) (first look_at_pos)) (setf look_at_pos (cons (aref effect 2) (aref effect 1))))) ; Attracting/move-to effects ; Note that attacting, repelling and avoiding are all done on ; the XZ plane only... y has no effect ; Note all the positions are converted to RELATIVE positions ((= (aref effect 0) *influence_attract*) (when (> (aref effect 2) (first move_to_pos)) (setf move_to_pos (cons (aref effect 2) (v- (xz (aref effect 1)) (xz (pos2 a))))))) ; Avoiding/repelling ((= (aref effect 0) *influence_repel*) (setf move_from_poss (cons (cons (aref effect 2) (v- (xz (aref effect 1)) (xz (pos2 a)))) move_from_poss))) ((= (aref effect 0) *influence_avoid*) (setf rel_pos (v- (xz (aref effect 1)) (xz (pos2 a)))) (if (and ; Big and close? Run away. Small/far? Steer clear. (< (vdistance rel_pos)) (> (aref effect 2) 5)) ; Repelling (setf move_from_poss (cons (cons (/ (aref effect 2) 5) rel_pos) move_from_poss)) ; Avoiding (setf steer_from_poss (cons (cons (aref effect 2) rel_pos) steer_from_poss)))) )) ; React to lookat effects (setf diff (v- (lookat_vector3 (pos1 a) (rest look_at_pos)) (rot1 a))) (setf (aref diff 1) (mod_balanced_angle (y diff))) ; agility & max_rot_vel (setf (vrot1 a) (vabstrim (v+ (vrot1 a) (vabstrim (v- diff (vrot1 a)) (agility a))) (max_rot_vel a))) ; Work out the position of the pos1 so that its horizontally between ; the pos2 and the lookat point. This section automatically keeps ; pos1 following pos 2 around (setf optimal_pos1xz (v- (v* (vnormalise


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering (v- (xz (rest look_at_pos)) (xz (pos2 a)))) (* 3 (radius a))) (v- (xz (pos1 a)) (xz (pos2 a))))) ; Now, the further things are away the more slinkies "crane" to see them ; so... they stretch up to the sqrt of the distance to the lookat point ; their radius adds to this however. (setf optimal_pos1 (vector (x optimal_pos1xz) (- (* 2 (+ (radius a) (sqrt (vdistance (pos2 a) (rest look_at_pos))))) (y (pos1 a))) (y optimal_pos1xz))) ; accel & max_vel (setf (vpos1 a) (vabstrim (v+ (vpos1 a) (vabstrim (v- optimal_pos1 (vpos1 a)) (accel1 a))) (max_vel1 a))) ; Now we do the velocity of pos2, which is essentially the movement ; of the slinky as a whole. ; A slinky ca only have one destination, but it can run away from ; more than one thing. Running away from something is essentially ; the same as running towards something in the opposite direction ; Lets just combine all the repulsions and one (if any) attraction ; into one great big direction. (if (> (first move_to_pos) -1) (setf direction (v* (vnormalise (rest move_to_pos)) (first move_to_pos))) (setf direction #(0 0))) ; Now adjust this, weightedly, so that the direction is AWAY from repulsive points (loop for move_from_pos in move_from_poss do (setf direction (v- direction (v* (vnormalise (rest move_from_pos)) (first move_from_pos))))) ; Okay, either there's somewhere we want to go or otherwise we want to stop/stay (if (and (= 0 (x direction)) (= 0 (y direction))) ; If we're moving/Unless we're stopped, slow down! (unless (and (= 0 (x (vpos2 a))) (= 0 (z (vpos2 a)))) (setf (vpos2 a) (v+ (vpos2 a) (vabstrim (v* (vpos2 a) -1) (* 1 (accel2 a)))))) ; Better at stopping than starting ; Otherwise, accelerate towards our goal (progn ; Now factor in the steering (if (= 0 (length steer_from_poss)) ; Nothing to steer from, go as fast as we can (setf hesitation 1.0) ; Let the brain work out which way to go (progn (setf direction (best_angle_from_nn_output direction (process *brain* (obstacle_field_to_nn_vector direction steer_from_poss)))) ; Size of direction from the best_angle.. function relation to sureity (setf hesitation (vdistance direction)))) (setf (vpos2 a) (vabstrim (v+ (vpos2 a) (vabstrim (vector (x direction) 0 (y direction)) (* hesitation (accel2 a)))) (max_vel2 a))))) ; Update positions based on velocities (setf (pos1 a) (v+ (pos1 a) (vpos1 a))) ;(setf (rot1 a) (v+ (rot1 a) (vrot1 a))) (setf (rot1 a) (mod_balanced_lookat_vector (v+ (rot1 a) (vrot1 a)))) (setf (pos2 a) (v+ (pos2 a) (vpos2 a))) ;(setf (rot2 a) (v+ (rot2 a) (vrot2 a))) (setf (rot2 a) (mod_balanced_lookat_vector (v+ (rot2 a) (vrot2 a)))) ; Update the influence positions (update_inate_influences a) ) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; saves the current frame's data to the otput file (defmethod save_frame_data ((a Agent)) (write_line_floats (out a)


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering (append (coerce (pos1 a) 'list) (coerce (rot1 a) 'list) (coerce (pos2 a) 'list) (coerce (rot2 a) 'list) (list (radius a) (coil_count a) (strand_height a) (stepp a) (step_speed a) (step_height a))))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; This sets up the influences so that they're ; up to date etc (defmethod update_inate_influences ((a Agent)) (clear_influences a) (add_influence a (pos1 a) *influence_look_at* ; Lookat the upper end (+ 0.5 (* (radius a) 0.1)) ; Most slinkies are as attractive to look at as eachother, although bigger ones often win (* (radius a) 10)) ; Fairly wide range of lookat appeal (add_influence a (pos1 a) ; top end *influence_avoid* ; Avoid me! (+ 5 (* (y (pos1 a)) 0.02)) ; Higher slinkies are more avoidable (* (radius a) 10)) (add_influence a (pos2 a) ; bottom end *influence_avoid* ; Avoid me! 5 (* (radius a) 10)) ; Higher slinkies are more avoidable ) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; For the NEW steering method. This takes a field of ; avoidance obstacles and converts it to an 18-element ; vector for use in the neural network (defun obstacle_field_to_nn_vector (direction obstacles) ; We'll ignore the stengths (setf max_strength (eval (cons 'max (setf strengths (mapcar #'first obstacles))))) (setf min_strength (eval (cons 'min strengths))) (setf r (make-array '(18) :initial-element -1)) (setf dir_rot (circular_angle (x direction) (y direction))) (setf dir_dis (eval (cons 'max (mapcar #'vdistance (mapcar #'rest obstacles))))) (loop for obstacle in obstacles do (when (setf v (point_to_nn_value dir_rot dir_dis (rest obstacle))) ; Really strong things spread fear through the arcs (setf spread (/ (- (first obstacle) min_strength) (- max_strength min_strength))) (setf (aref r (first v)) (max (aref r (first v)) (rest v))) (when (> spread 0.5) (when (< (first v) 17) (setf (aref r (1+ (first v))) (max (aref r (1+ (first v))) (*(rest v)spread)))) (when (> (first v) 0) (setf (aref r (1- (first v))) (max (aref r (1- (first v))) (*(rest v)spread))))) (when (> spread 0.75) (when (< (first v) 16) (setf (aref r (+ (first v) 2)) (max (aref r (+ (first v) 2)) (*(rest v)(* spread 0.75))))) (when (> (first v) 1) (setf (aref r (- (first v) 2)) (max (aref r (- (first v) 2)) (*(rest v)(* spread 0.75)))))))) r) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Given the direction (angle) of a target, its distance ; and another point, returns the a tuple with the first ; part the index of one of the 18 elements it belongs in ; and its value. -1 if it belongs in no element (defun point_to_nn_value (dir_rot dir_dis p) (when ; The point it within the target zone and is has an index between 0 and 17 inclusive (and (< (setf d (vdistance p)) dir_dis) (< (setf index (floor (/ (mod (- (circular_angle_v p) (- dir_rot 90)) 360) 10))) 18)) (cons index (1+(*(/ d dir_dis) -2))))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Takes a direction point and the ouput from an the nn brain ; and returns a new direction based on the recommended angle


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering (defun best_angle_from_nn_output (direction v) (setf a 90.0) (setf best_i 9) (setf max_value -1) (loop for i from 0 below 18 do (when (> (aref v i) max_value) (setf max_value (aref v (setf best_i i))))) (if (> max_value -1) (progn (setf a (+ (* best_i 10.0) 5.0)) (when (and (> best_i 0) (< best_i 17)) (if (> (aref v (1- best_i)) (aref v (1+ best_i))) (setf a (- a (* (/ (aref v (1- best_i)) max_value) 5))) (setf a (+ a (* (/ (aref v (1+ best_i)) max_value) 5))))) (setf a (mod (+ a (- (circular_angle_v direction) 90)) 360)) (v* (circular_vector a) (/ (1+ max_value) 2))) direction)) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Used in steering, this function converts ; a list of obstructions into a list of "gaps" ; Obstructions are supplied in a form of a vector: ; #(weight width_in_degrees direction_in_degrees) orderd by direction (defun obstructions_to_gaps (obstructions) (obstructions_to_gaps_rec ; The (0.1 . 90) is a virtual avoidance point to allow making a gap on the very left (cons '(0.2 . -90) (sort obstructions #'< :key #'rest)))) (defun obstructions_to_gaps_rec (obstructions) (if (> (length obstructions) 1) (cons (two_obstructions_to_gap (subseq obstructions 0 2)) (obstructions_to_gaps_rec (rest obstructions))) (list (two_obstructions_to_gap (list (first obstructions) '(0.2 . 90)))))) (defun two_obstructions_to_gap (obstructions) (setf a (weighted_avg (subseq obstructions 0 2))) (vector (first a) (- (rest (second obstructions)) (rest (first obstructions))) (rest a))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Evaluates the worthiness of a gap, returning a score ; Takes into account gap width, weight of avoidance and ; the closeness to the preferred direction (0) (defun evaluate_gap (gap) (* (/ 1 (aref gap 0)) (expt (/ (aref gap 1) 180) 1.5) ; Expt so it has more power (/ 1 (+ 5 (abs (aref gap 2)))))) ; 10 to give this more importance ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Finishes up after the simulation (defmethod cleanup ((a Agent)) (close_stream (out a))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Prints out the 3D information about this agent ; for debuggin purposes (defmethod print_data ((a Agent)) (print "AGENT ID:") (princ (id a)) (print "End 1 Position:") (princ (pos1 a)) (print "End 1 Rotation:") (princ (rot1 a)) (print "End 1 Position:") (princ (pos2 a)) (print "End 2 Rotation:") (princ (rot2 a)) (print "End 2 Position Velocity:") (princ (vpos1 a)) (print "End 1 Rotation Velocity:") (princ (vrot1 a)) (print "End 2 Position Velocity:") (princ (vpos2 a)) (print "End 2 Rotation Velocity:") (princ (vrot2 a)) a)


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; HELIX CROWD SIMULATOR ------- (C) Copyright Steve Agland 2002 ;; ;;-----------------------------------------------------------------;; ;; $Source: /home/cvsroot/helix_crowd/datareader.lisp,v $ ;; $Revision: 1.3 $ ;; $Author: agi $ ;; $Date: 2002/10/23 06:04:01 $ ;;-----------------------------------------------------------------;; ;; [email protected] ;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; This handy little class just reads tables of numeric ; data in from a tab-delimted .chan file generated by ; Houdini (other formats too complicated ot bother with) (defclass DataReader () ((filename :accessor filename :documentation "The path to the file being read in" :initform "" :initarg :filename :type 'string) (input_stream :accessor input_stream :documentation "The filestream, which'll be opened when we need it" :initform ()))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Opens the filestream, ready to go ; If its already open, leave it open ; return the stream in either case (defmethod open_stream ((dr DataReader)) (unless (AND (input_stream dr) (open-stream-p (input_stream dr))) (setf (input_stream dr) (open (filename dr)))) (input_stream dr)) ;;;;;;;;;;;;;;;;;;;;;;; ; Closes the filestream (defmethod close_stream ((dr DataReader)) (close (input_stream dr))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Returns a list, one element for each tab-separated ; value in the current line of the file.. advances one line (defmethod read_line ((dr DataReader)) (split (read-line (open_stream dr) ()) #\Tab)) (defmethod ws_read_line ((dr DataReader)) ; Any whitespace (ws_split (read-line (open_stream dr) ()))) (defmethod read_line_floats ((dr DataReader)) (mapcar #'tolerant-parse-float (read_line dr))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Closes the steam, same thing essentially (defmethod reset ((dr DataReader)) (close_stream dr))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; HELIX CROWD SIMULATOR ------- (C) Copyright Steve Agland 2002 ;; ;; ;;-----------------------------------------------------------------;; ;; $Source: /home/cvsroot/helix_crowd/general.lisp,v $ ;; $Revision: 1.6 $ ;; $Author: agi $ ;; $Date: 2002/10/23 06:04:42 $ ;;-----------------------------------------------------------------;; ;; [email protected] ;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;; ; General handy functions ;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Splits a string by a certain charater (defun split (str chr) (if (setf n (position chr str)) (cons (subseq str 0 n) (split (subseq str (1+ n)) chr))


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering (list str))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Splits a string by chunks of whitespace (defun ws_split (str) (if (and (setf n (min (if (setf pos (position #\Tab str)) pos (length str)) (if (setf pos (position #\Space str)) pos (length str)))) (< n (length str))) (if (> n 0) (cons (subseq str 0 n) (ws_split (subseq str (1+ n)))) (ws_split (subseq str (1+ n)))) (if (> (length str) 0) (list str) ()))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Joins a list of strings into a single string (defun join (list chr) (if (> (length list) 0) (if (> (length list) 1) (concatenate 'string (first list) (if (stringp chr) chr "") (join (rest list) chr)) (first list)) "")) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Takes a dotted list ((weight . value)*) ; returns (total_weight . weighted_average_value) (defun weighted_avg (list) (setf n (first list)) (if (> (length list) 1) (progn (setf r (weighted_avg (rest list))) (setf x (+ (first r) (first (first list)))) (cons x (/ (+ (* (first r) (rest r)) (* (first (first list)) (rest (first list)))) x))) (first list))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Lazy float parser, return 0 if the ; string is blank or full of junk (defun tolerant-parse-float (str) (unless (setf r (parse-float str :junk-allowed t)) (setf r 0)) r) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Simply prints a log message out ; Might want to override this to print ; to something else.. its just for log messages (defun prlog (str) (princ str)) ;;;;;;;;;;;;;;;;;;;;; ; Handy ref functions (defmethod x ((v vector)) (aref v 0)) (defmethod y ((v vector)) (aref v 1)) (defmethod z ((v vector)) (aref v 2)) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Handy vector trimming functions (defmethod xy ((v vector)) (vector (aref v 0) (aref v 1))) (defmethod xz ((v vector)) (vector (aref v 0) (aref v 2))) (defmethod yz ((v vector)) (vector (aref v 1) (aref v 2))) ;;;;;;;;;;;;;;;;;;;;;;;;; ; Vector arithmatic ; Self-explanatory, really? (defmethod v+ ((v1 vector) (v2 vector)) (map 'vector '+ v1 v2)) (defmethod v- ((v1 vector) (v2 vector)) (map 'vector '- v1 v2))


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering (defmethod v* ((v1 vector) (v2 vector)) (map 'vector '* v1 v2)) (defmethod v* ((v1 vector) (v2 real)) (v* v1 (make-array (length v1) :initial-element v2))) (defmethod v/ ((v1 vector) (v2 vector)) (map 'vector '/ v1 v2)) (defmethod v/ ((v1 vector) (v2 real)) (v/ v1 (make-array (length v1) :initial-element v2))) (defmethod vabs ((v1 vector)) (map 'vector 'abs v1)) (defmethod vdistance ((v1 vector) &optional (v2 () vector)) (when (> (length v2) 0) (setf v1 (v- v1 v2))) (if (> (length v1) 1) (sqrt (+ (expt (aref v1 0) 2) (expt (vdistance (subseq v1 1)) 2))) (aref v1 0))) (defmethod vnormalise ((v1 vector)) (v/ v1 (vdistance v1))) ; Makes sure none of a vectors elements exceeds a certain max abs value (defun abstrim (val max) (max (* -1 (abs max)) (min (abs max) val))) (defmethod vabstrim ((v1 vector) (v2 vector)) (map 'vector 'abstrim v1 v2)) (defmethod vabstrim ((v1 vector) (max real)) ; Uses the length of the vector to trim (if (> (vdistance v1) max) (v* (vnormalise v1) max) v1)) ;;;;;;;;;;;;;;;;;;;;; ; Radians and degrees (defun rad2deg (x) (/ x (/ pi 180))) (defun deg2rad (x) (* x (/ pi 180))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Exponential decay or attentuation - how the influence dies off (defun attenuate (s d a) (* s (expt 2.7182817 (* -0.6931472 (/ d a))))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Takes two points in 3D space. Returns the lookat ; vector to look from the first point to the second. ; A lookat vector is defined as a 3D vector of which ; only the first two components are significant. ; This function will always return 0 for the Z component. ; Assuming Y is the vertial and that a lookat vector ; of (0 0 0) is looking straight down, then ; the X component is the angle in DEGREES to rotate ; the vector around the X axis until it it pointing at ; the correct ELEVATION (which may be above or below ; the horizon). ; The X component should always be >= 0 and <= 180 ; The Y component is the angle in degrees from the ; positive X axis to get the correct lookat AZIMUTH. ; The Y component should always be >= -180 and <= 180 ; E.g.: ; (180 0 0) -- Looking straight up ; (90 0 0) -- Looking along the positive X axis ; (135 -90 0) -- Looking along the negative Z axis, halfways upwards ; The Z component will eventually be interpreted as being ; the "twist", however if it not used in this simulation. (defmethod lookat_vector3 ((p1 vector) (p2 vector)) (setf d (v- p2 p1)) (setf hdist (vdistance (vector (x d) (z d)))) (if (not (eq hdist 0)) (vector (+ (rad2deg (atan (/ (y d) hdist))) 90) (mod_balanced_angle (circular_angle (* -1 (z d)) (* -1 (x d)))) ; Ensure 0) (cond (( > 0 (y d)) #(0 90 0)) (( < 0 (y d)) #(0 90 0)) (#(0 0 0)))))


Obstacle Avoidance in Crowd Simulations: Machine Learning (31734) A Machine Learning Method for Intelligent Steering ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Ensures an angle in degrees falls between -180 and +180 (defun mod_balanced_angle (a) (- (mod (+ a 180) 360) 180)) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Ensures an angle in degrees falls between -180 and +180 (defun mod_balanced_lookat_vector (v) (vector (aref v 0) (mod_balanced_angle (aref v 1)) (aref v 2))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Given a pair of coords, calculates the angle ; around the circle on the plane (in degrees) ; Starts at positive x axis (defun circular_angle_v (v) (circular_angle (x v) (y v))) (defun circular_angle (x y) (setf a (rad2deg (if (not (= x 0)) (abs (atan (/ y x))) (/ pi 2)))) (cond ((and (> y 0) (<= x 0)) (- 180 a)) ; Upper left quad ((and (<= y 0) (< x 0)) (+ 180 a)) ; Lower left quad ((and (< y 0) (>= x 0)) (- 360 a)) ; Lower right quad (superfluous) (a))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ; Given an angle, works out a normalized 2D vector ; Opposite of circular_angle (defun circular_vector (a) (vector (cos (deg2rad a)) (sin (deg2rad a)))) ; Okay so I didn't need to think about all this ; (setf a (mod_balanced_angle a)) ; (cond ; ((and (> 0 a) (< 90 a)) ; Upper right ; (vector (cos (deg2rad a)) (sin (deg2rad a)))) ; ((and (> 90 a) (< 180 a)) ; Upper left ; (vector (cos (deg2rad a)) (sin (deg2rad a)))) ; ((and (< 0 a) (> -90 a)) ; Lower right ; (vector (cos (deg2rad a)) (sin (deg2rad a)))) ; ((and (< -90 a) (> -180 a)); Lower left ; (vector (cos (deg2rad a)) (sin (deg2rad a)))) ; ((eq 0 a) #(1 0)) ; ((eq 90 a) #(0 1)) ; ((eq -90 a) #(0 -1)) ; ((eq 180 (abs a)) #(-1 0))))


Obstacle Avoidance in Crowd Simulations: A Machine Learning Method for Intelligent Steering

Documents

Transcript of Obstacle Avoidance in Crowd Simulations: A Machine Learning Method for Intelligent Steering