Presentation: Tool Support for Essential Use Cases to Better Capture Software Requirements
Tool use-presentation
-
Upload
del-bao -
Category
Technology
-
view
17 -
download
0
Transcript of Tool use-presentation
![Page 1: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/1.jpg)
Depei Bao
![Page 2: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/2.jpg)
What is the problem?●A robot with intense interaction with environmental
resources, such as another object, substance, surface or medium.
●The goal of the robot is to make some change to other object and/or itself mediated by body parts and/or other entities, i.e., tools.
![Page 3: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/3.jpg)
What is the problem? (cont.)1.A general knowledge representation and reasoning
mechanism for rational decision making in any situation (independent of the specific knowledge structure)
2.Planning and reasoning with the aid of object ontological structure
3.Cognitive process to mediate sensory input and motor control in real interaction
4.Sensor signal recognition and control signal generation
![Page 4: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/4.jpg)
What is the problem(cont.)
●The upper level constrains the lower level
●The lower level realizes the upper level
●Focus on 2 and 3
![Page 5: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/5.jpg)
Basic Ideas●General decision making: means-end reasoning,
utility theory, preference handling, etc.●Domain: physical object manipulation• affordance-
based knowledge representation and schema●Cognitive bridge: embodied verification and planning●Sensor to knowledge and knowledge to controller
![Page 6: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/6.jpg)
Pieces of Philosophy, Ontology and Psychology1.Reasoning and Verification
2.Affordance and Interactive properties
3.Embodied Knowledge and Reasoning
![Page 7: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/7.jpg)
![Page 8: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/8.jpg)
What is Reasoning?●Ways to retrieve relevant information from a
knowledge base represented and organized in a certain structure
●Typical structure: proposition or predicate logic
![Page 9: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/9.jpg)
Relevant=Rational●Rational criteria
1.Consistent logic inference
2.Conform to mental states: desire, belief and intention
3.With constraints of physical world and social codes (beliefs)
4.Context-sensitive
![Page 10: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/10.jpg)
Symbol structure manipulation?●The curse of AI: common sense and context
reasoning, such as frame problem●Solution 1: Nonmonotonic logic for reasoning●Solution 2: Capture the semantics of symbols?
symbol grounding problem
![Page 11: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/11.jpg)
Verification●Determine the truth value of a symbol●Only the system can understand the semantics of
the symbol, i.e., the meaning a symbol represents●How?
1.Symbol network (such as semantic networks)
2.Knowledge is embodied
3.Grounded on sensor data
4.Symbol to controller mapping
![Page 12: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/12.jpg)
![Page 13: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/13.jpg)
Affordance
●A concept coined by ecological psychologist James Gibson
●Denote a resource or support that the environment offers an animal for action.
●A goal-biased way to perceive an object identified by its function
![Page 14: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/14.jpg)
Why affordance?●Intuitively natural approach of our perception
and cognition●As a manner of knowledge representation
suitable to context-based physical behavior●Context: the environment elements combined
with the agent’s ability provide the opportunity (as the precondition) for the proper action/effect
![Page 15: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/15.jpg)
The Semantics of Affordance●Sth O can afford an action A or an effect E (by some
implicit action).●=some properties of O satisfy the requirement of A or E. ●Four types of affordances
1.First-order or binding affordance: affords an action, e.g., can climb, or affords a status of the agent or other objects, e.g., can support.
2.Second-order or output affordance: affords changes to itself, e.g., afford to be struck into the wall or affords changes to other objects, e.g., hammer.
![Page 16: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/16.jpg)
Interactive properties●How to represent properties requirement?
1.Properties requirement is relative
2.The meanings of properties are grounded on object properties
3.The mapping (or meaning) is different for different actions even though they share the same name
•interactive properties
![Page 17: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/17.jpg)
Interactive properties (Cont.)
●In essence, interactive properties are constraints on object properties, e.g., hardness constrains on size, shape and material of an object
●Constraints also among interactive properties, e.g., sharpness and hardness
●Constraints also among affordances for a task.●Three levels of constraints satisfaction problem
(CSP) on object properties (for verification)
![Page 18: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/18.jpg)
Object properties●Properties of an object independent of specific task
and can directly recognized from sensor data.●What the purpose they serve?
1.Object recognition: necessary properties to identify an entity as an instance of an object class.
2.Parameters to distinguish among different object classes and among instances of the same object class to meet specific functional requirement.
![Page 19: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/19.jpg)
Question: object classes are contingent?●Two sorts of object classes possess different
ontological natures1.An artifact with a designed purpose2.A natural occurrence object identified only by its
substance●Share similar representational structure3.Common primitive properties: shape, material,
color, etc. (attribute-value)4.Compositional parts5.Intrinsic and extrinsic states
![Page 20: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/20.jpg)
Identity of an object●An object class constraints the properties of its
compositional parts and their compositional relationships.
●For each primitive part, the constraints are imposed on the values of relevant attributes.
●An object may exhibit different properties in different internal state.
●Is-a relation and part-of relation
![Page 21: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/21.jpg)
Identity of an object (Cont.)●A natural occurrence object, constraints are
determined only by natural properties●An artifact is identified by function, not by natural
properties●Designed function for an artifact class=generalized
affordance properties, ●a prototype, a looser constraint on properties than
its instances●Introduce attribute not common to all objects
![Page 22: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/22.jpg)
Perception●Recognition of object properties is a process to
match sensor pattern●Refer to the determinable/determinates, e.g., color
and genus/species, e.g., shape
![Page 23: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/23.jpg)
Situation awareness and tracking●A tracking process for the environmental situation,
purpose-independent perception●Obtained information can be used for verification
without the need to search from the beginning●Object identification, position, velocities and
orientations of objects and agents, spatial relation of object, subject-object relations, events, etc.
●States are tracked at every perception loop.
![Page 24: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/24.jpg)
![Page 25: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/25.jpg)
What is the meaning of embodiment●Knowledge is obtained, represented and verified
in terms of subject experience.●Body-related reasoning and verification. ●Body information is implied in affordance
representation●Empathy
![Page 26: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/26.jpg)
Knowledge level embodiment●Every interactive property of an affordance implies a
body configuration, body dynamics and body properties, but the properties of a body part is not explicitly considered and/or set in verification.
●Past experience gives metrics to every such mapping from interactive properties to properties of an object for an action.
●Similarity comparison between an instance and properties prototype by the metrics for the verification of an affordance.
![Page 27: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/27.jpg)
Controller level embodiment●Only at the control level, unconsciously, we consider the
body configuration (after verification of a symbol, this is a mapping from the semantics of the affordance to a controller).
●Control level embodiment: during an actual operation, we recognize the specific sensory feedback pattern to verify the interaction and dynamically adapt body configuration, i.e., the record of the real experience.
![Page 28: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/28.jpg)
Schema●“a mental codification of experience that includes a
particular organized way of perceiving cognitively and responding to a complex situation or set of stimuli”
--Merriam-Webster's Collegiate Dictionary●A knowledge organization and retrieval method●Grouping relevant knowledge (precondition and
solution) of a task together
![Page 29: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/29.jpg)
Symbol to control signal●A controller?
1.A low level dynamic feedback system
2.Generating spatial and kinematic trajectories of the agent’s body and apply a certain style of force onto the target
![Page 30: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/30.jpg)
Symbol to control signal (Cont.)●Order=Mapping from embodied symbol knowledge●Order from cognition or not●Mapping Levels?●Feedback through cognitive channel or not
![Page 31: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/31.jpg)
Body image●Under the assumption of embodied knowledge
representation, centralized body knowledge is rarely used for reasoning at an interaction
●The high-level self image only exists as a social element.
●At the low level, however, there is a body model for controller derivation.
![Page 32: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/32.jpg)
Body-tool extension●Two levels: the cognitive level and the controller
level●The cognitive level: changed semantics of
affordance symbol●The controller level: body model extension (different
mapping from cognition)●The ‘feeling’ is still recognized from sensory pattern●Triggering? Learned Pattern for a specific action?
![Page 33: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/33.jpg)
![Page 34: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/34.jpg)
The framework●From ideas to realization●Glue various components●Process for reasoning from the goal to the plan
beyond symbol manipulation
![Page 35: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/35.jpg)
Three steps●A single cycle of means-end reasoning
1.Target effect analysis (end)
2.Effectual action search (means)
3.Plan construction (refine the means in terms of the end and situation)
●Example: tries to hammer a nail into wall
![Page 36: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/36.jpg)
Target effect analysis●Effect?
1.In multi-attribute decision theory terminology, effect=outcome of an action=an attribute vector (propositional or numeric) of states
2.Thus, effect=a subject/object status description capturing the static/dynamic meaning (states or states change) of the status
●Description Extension●Verb-centered representation
![Page 37: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/37.jpg)
Verb-based representation●A verb has two semantic dimensions: the effect/state
and the manner of the action●Direct grounding of these two meanings: ‘remove’ a
state and ‘strike’ on a controller pattern●Otherwise, specify more meanings by extending
along verb links (relations) with context check
●Effect: the index to effectual action and plan schema
![Page 38: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/38.jpg)
Verb relations●WordNet
1.Troponymy: the super-sub (is-a) relation, e.g., scribble and write
2.Causation: e.g. , give/have, reach/hold
3.(Temporal) inclusion: the part-of relation
4.Backward presupposition: prerequisite (not the cause), e.g., win/play
![Page 39: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/39.jpg)
Example●the target state is (the depth would be specified,
which is dependent on the previous end).●The action is a direct description of the final state.
![Page 40: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/40.jpg)
Effectual action search●An effectual action can be retrieved from the target
state category by some analysis on the situation●Why?
1.follow human problem solving pattern: schema
2.Two second-order affordances (passive and positive) are complementary and mutual dependent, fix the target.
![Page 41: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/41.jpg)
Example●In the embedding category, measure the properties
of x’s tips (i.e., the shape and material of a nail) and y’s surface (wall), and then derive striking.
![Page 42: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/42.jpg)
Plan Construction●Decision= action sequence schema + verification
schema + adjustment●Action sequence schema: an effectual action has a
prototype of action sequence specification, e.g., facilitating actions
●Verification schema:
1.an effectual action and its facilitating actions has corresponding affordance schema (precondition)
2.These affordances have relations, e.g., causal
![Page 43: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/43.jpg)
Plan Construction (Cont.)●If necessary, a more general action is decomposed
into further primitive actions along the hierarchy until an action can be directly mapped to a controller
●Finally, a plan is represented as a finite automaton: state: action primitives, transitions: expected observed outcome, which can be mapped to a lower level controller (state) and sensor data (transition)
![Page 44: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/44.jpg)
Example●By measuring the target(the nail, wall and depth), striking
(the action mode is impulsive) requires an object able to provide enough impulse on each blow.
●Two functional parts and the functional relation
![Page 45: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/45.jpg)
●Interactive properties also include spatial requirement such as interface matching
![Page 46: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/46.jpg)
●Available space verification, also embodied●A facilitating action: the nail affords to be stabilized●in the preparation phase
![Page 47: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/47.jpg)
Decision making●Goal≠Plan Performance
1.Uncertainty/reliability of the plan: calculated as the result of verification
2.The cost of plan
3.The cost of verification●More sophisticated agent modeling
![Page 48: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/48.jpg)
Verb extension and application●‘is-a’ relation: the basic verb organization
1.A sub-verb is the sub-type of a super verb with more semantic constraints.
2.e.g., ‘take’ is a kind of ‘shift’ focusing on the source whereas ‘give’ focuses on the recipient.
3.The constraint information is useful for grounding and planning, e.g., from ‘move’ to ‘step on’ (step 2 and 3)
![Page 49: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/49.jpg)
●Causation: effect extension (step 1) and effectual action selection (step 2)
●(Temporal) inclusion: the part-of relation1.In plan schema selection (step 3)2.From a general action category (e.g., an effectual
action) to primitive actions●Backward presupposition: prerequisite (not the
cause), e.g., win/play (step 2 and 3, different conditions)
![Page 50: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/50.jpg)
![Page 51: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/51.jpg)
Functional primitives●Properties requirement: Mapped to object properties ●Nonmonotonic logic
1.There is no hope to check all possible preconditions and side-effects for an action
2.Defeat the conclusion (an affordance) if we can get extra information from other sources, e.g., object properties
![Page 52: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/52.jpg)
Functional relations●behavior relations (also a kind of affordance) of two
or more functional parts, e.g., ‘swing together’●mapped to spatial relations and physical properties
(for the relation) of functional parts, e.g., for ‘swing together’, the effecter and the handle should be rigid and be connected in a way that affords swinging in the same direction.
![Page 53: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/53.jpg)
![Page 54: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/54.jpg)
Verification procedures●May also involve search for an object●The search is finally guided by object class
knowledge, i.e., properties constraints●Uncertainty in affordance verification (later)●CSP, heuristics for the sequence of verification
![Page 55: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/55.jpg)
Failure●The effect is not achieved as expected●Cues to indicate the failure: e.g., sensor feedback●Diagnosis: attribute reasons to the failure
1.Sensor inaccurate
2.Uncertain verification
3.Unconsidered environmental conditions
4.Situation change
5.Disregarded properties
6.Side-effects to the target
![Page 56: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/56.jpg)
Feedback mechanism●Sensor: vision, tactile, proprioceptional, balance(?)●Proprioceptional:
1.the sense of the relative position of neighboring parts of the body
2.Feedback traced by controller, return (map) to symbols in the cognitive part if such information (such as the current body posture) is needed
![Page 57: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/57.jpg)
Verification of affordances●Mapping knowledge from the measurement of the
target properties to properties of the affording object bridged by interactive properties.
●Only for the target as an object, a subject is implicit●For four kinds of affordances
1.Pure action: target: action itself
2.Static status: subject/object parameters
3.Passive affordance: self-mapping
![Page 58: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/58.jpg)
Example1.Positive affordance: the required effect/change to
another object. The effectual action determines the target properties to measure, which constrains the object’s properties in terms of its interactive properties.
●Example, measure the properties of the nail, the wall and the embedded depth.
![Page 59: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/59.jpg)
Uncertainty of verification●Constraints often specify two thresholds for a
parameter t1 (guaranteed) and t2 (excluded)●Effectual process●Out of ranges: reasons against the effect●Reasons are uncertain●A number: uncertain properties for the extent to
which (or the probability of) the reason contributes to the failure of the process.
●Piercing: sharpness: the angle of the tip, from contacting the wall to the specified depth
![Page 60: Tool use-presentation](https://reader038.fdocuments.us/reader038/viewer/2022103105/5873c7581a28abbc788b7dd7/html5/thumbnails/60.jpg)
Conclusion●An implementation?●A knowledge intensive approach to the problem
because of its generality●Domain specific information is represented as
knowledge rather than embedded in structure.●Cognitive framework, leave sensor pattern
recognition and controller construction