Post on 20-Dec-2015
Action Rules DiscoveryAction Rules Discoverywithout pre-existing without pre-existing classification rulesclassification rules
BByy
Zbigniew W. RasZbigniew W. Ras1,21,2 and Agnieszka Dardzinska and Agnieszka Dardzinska1,31,3
1)1) University of North Carolina, Charlotte, NC, USAUniversity of North Carolina, Charlotte, NC, USA2)2) Polish Academy of Science, Warsaw, PolandPolish Academy of Science, Warsaw, Poland3)3) Bialystok Technical University, Bialystok, PolandBialystok Technical University, Bialystok, Poland
1. Introduction : 1. Introduction : Action RuleAction Rule
An action rule is a rule extracted from a datatable that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute
A B D
a1 b2 d1
a2 b2
a2 b2 d2
Information System
1. Introduction : 1. Introduction : Action RuleAction Rule
An action rule is a rule extracted from a database that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute
A B D
a1 b2 d1
a2 b2
a2 b2 d2
Information System
Assumption:
- attributes are partitioned into stable and flexible
1. Introduction : 1. Introduction : Action RuleAction Rule
An action rule is a rule extracted from a database that describes a possible transition of objects from one state to another with respect to a distinguished attribute called a decision attribute
A B D
a1 b2 d1
a2 b2
a2 b2 d2
Information System
Assumption:
- attributes are partitioned into stable and flexible
Values of attributes can be changed
1. Introduction : 1. Introduction : Action RuleAction Rule
Action rule is defined as a term
A B D
a1 b2 d1
a2 b2
a2 b2 d2
Information System
conjunction of fixed condition features shared by both groups
proposed changes in values of flexible features
desired effect of the action
[(ω) ∧ (α → β)] →(ϕ→ψ)
2. Background and Objectives
XX a a bb cc dd
xx11 aa11 bb11 cc11 dd11
xx22 aa22 bb11 cc22 dd11
xx33 aa22 bb22 cc22 dd11
xx44 aa22 bb11 cc11 dd11
xx55 aa22 bb33 cc22 dd11
xx66 aa11 bb11 cc22 dd22
xx77 aa11 bb22 cc2 2 dd11
xx88 aa11 bb22 cc11 dd33
Information System Information System
S = (X,A,V)S = (X,A,V)XX = { = {xx11, , xx22, , xx33, , xx44, , xx55, , xx66, , xx77, , xx88}}AA = {a, = {a, b, c, db, c, d}}
X is a set of objectsX is a set of objectsA is a set of attributes (partial A is a set of attributes (partial functions: X functions: X V) V) VVa a is a set of values of attribute a,is a set of values of attribute a,where a where a A, and V = A, and V = {V{Vaa : a : a A}, A},
Information System SInformation System S
Decision Attribute : {d}Stable Attributes : Ast= {a}Flexible Attributes : Afl = {b,c}
Constraints: Minimum Support 1
Minimum Confidence 2
3. ARED Algorithm (Stable and Flexible Att)
XX a a bb cc dd
xx11 aa11 bb11 cc11 dd11
xx22 aa22 bb11 cc22 dd11
xx33 aa22 cc22 dd11
xx44 aa22 bb11 cc11 dd11
xx55 aa22 bb33 cc22 dd11
xx66 aa11 bb11 dd22
xx77 aa11 bb22 cc2 2 dd11
xx88 aa11 bb22 cc11 dd33
Decision System
2. Background and Objectives2. Background and Objectives
Action rule generation in earlier work used existing Action rule generation in earlier work used existing classification rules (generated, for instance, by rule classification rules (generated, for instance, by rule discovery algorithms such as LERS or ERID)discovery algorithms such as LERS or ERID)
Proposed AlgorithmProposed Algorithm
1. Extract action rules directly from an information 1. Extract action rules directly from an information system without using pre-existing classification rules.system without using pre-existing classification rules.
2. Extract action rules that have minimal attribute 2. Extract action rules that have minimal attribute involvement.involvement.
Set of rules R with supporting objects
Objects a b c d
x1, x2, x3, x4 0 L
x1, x3 0 L
x2, x4 2 L
x2, x4 1 L
x5, x6 3 L
x7, x8 2 1 H
x7, x8 1 2 H
Objects a b c
x1, x2, x3, x4 0
x1, x3 0
x2, x4 2
x2, x4 1
x5, x6 3
Objects b c
x1, x3 0
x2, x4 2
x2, x4 1
x5, x6 3
Objects b
x2, x4 2
x5, x6 3
c = ?c = 0
Objects b c
x1, x2, x3, x4
Objects b
x1, x3
a = 0
Objects b
x2, x4
a = ?
Objects a b c
x7, x8 2 1
x7, x8 1 2
Objects b c
x7, x8 1
a = 2
Objects b c
x7, x8 1 2
a = ?
Stable Attributes: {a, c}
Flexible Attribute: b
Decision Attribute: d
T1 T2
T3 T4
(T3, T1) : (a = 2) (b, 21) ( d, L H)
(a = 2) (b, 31) ( d, L H)
Objects b
x7, x8 1
c = ? c = 2
Objects b
x7, x8 1
c = ?
Objects b
x1, x2, x3, x4
T5
T6
c = 1
Action Rules Discovery System DEAR
3. Action Rules
Atomic Action Term: (a, a1 → a2 )
attribute a1, a2 ∈ Va
If a1 = a2 then a – stable on a1
3. Action Rules
Atomic Action Term: (a, a1 → a2 )
attribute a1, a2 ∈ Va
If a1 = a2 then a – stable on a1Set of Action Terms:
1. If t – atomic action term, then t – an action term2. If t1, t2 – action terms, then t1*t2 – an action term3. If t – action term containing sub-terms (a, a1 → a2 ), (b, b1 →
b2), then a ≠ bDomain of Action Term t: Dom(t) – set of all attribute names listed in t
3. Action Rules
Atomic Action Term: (a, a1 → a2 )
attribute a1, a2 ∈ Va
If a1 = a2 then a – stable on a1Action Rule: r = [t1 → t2]
Dom(t2)={d} and Dom(t1) in A
action term
atomic action term
3. Example
XX a a bb cc dd
xx11 aa11 bb11 cc11 dd11
xx22 aa22 bb11 cc22 dd11
xx33 aa22 bb22 cc22 dd11
xx44 aa22 bb11 cc11 dd11
xx55 aa22 bb33 cc22 dd11
xx66 aa11 bb11 cc22 dd22
xx77 aa11 bb22 cc2 2 dd11
xx88 aa11 bb22 cc11 dd33
(a, a2 → a2) (b, b1 → b3)(c, c2 → c2)(d, d1 → d2)
Decision System SDecision System S
atomic action terms
r=[[(a, a2→a2)*(b, b1→b3)] → (d, d1→d2)
action rule
Dom (r) ={a, b, d}
4. Interpretation NS of action terms
1. If (a, a1 → a2) – atomic action term then:NS((a, a1 → a2))=[{x X:a(x)=a1∈ }, {x∈X:a(x)=a2}]
2. If t1=(a, a1 → a2) * t and NS(t)=[Y1, Y2], then:
NS(t1)=[ Y1 ∩ {x∈X:a(x)=a1}, Y2 ∩ {x∈X:a(x)=a2} ]
Information System S=(X, A, V)Information System S=(X, A, V)
Assume that [Y1, Y2]∩[Z1, Z2] = [Y1∩Z1, Y2∩Z2], and
NS(t1)=[Y1, Y2],NS(t2)=[Z1, Z2], then:NS(t1*t2)=NS(t1)∩NS(t2)
4. Interpretation NS of action terms
Information System S=(X, A, V)Information System S=(X, A, V)
NS(t1)=[Y1, Y2]
NS(t2)=[Z1, Z2]
Action Rule: r = [t1 → t2]
Support:
Confidence:
)(
)(
)(
)()(
2
22
1
11
Ycard
ZYcard
Ycard
ZYcardrconf
)()sup( 11 ZYcardr
XX a a bb cc dd
xx11 aa11 bb11 cc11 dd11
xx22 aa22 bb11 cc22 dd11
xx33 aa22 bb22 cc22 dd11
xx44 aa22 bb11 cc11 dd11
xx55 aa22 bb33 cc22 dd11
xx66 aa11 bb11 cc22 dd22
xx77 aa11 bb22 cc2 2 dd11
xx88 aa11 bb22 cc11 dd33
Information System SInformation System S
r = [[(a, a2→a2)*(b, b1→b2)] → (d, d1→d2)
NS((a, a2→a2))=[{x2, x3, x4, x5}, {x2, x3, x4, x5}] NS((b, b1→b2))=[{x1, x2, x4, x6}, {x3, x7, x8}] NS((d, d1→d2))=[{x1, x2, x3, x4, x5, x7}, {x6}]
XX a a bb cc dd
xx11 aa11 bb11 cc11 dd11
xx22 aa22 bb11 cc22 dd11
xx33 aa22 bb22 cc22 dd11
xx44 aa22 bb11 cc11 dd11
xx55 aa22 bb33 cc22 dd11
xx66 aa11 bb11 cc22 dd22
xx77 aa11 bb22 cc2 2 dd11
xx88 aa11 bb22 cc11 dd33
Information System SInformation System S
r = [[(a, a2→a2)*(b, b1→b2)] → (d, d1→d2)
NS((a, a2→a2))=[{x2, x3, x4, x5}, {x2, x3, x4, x5}] NS((b, b1→b2))=[{x1, x2, x4, x6}, {x3, x7, x8}] NS((d, d1→d2))=[{x1, x2, x3, x4, x5, x7}, {x6}]
NS((a, a2→a2)*(b, b1→b2))=[{x2, x4}, {x3}]
sup (r) = 2, conf (r) = 0
5. Algorithm ARED
-Similar to ERID and LERS-Only positive marks yield action rules-Action terms of length k are built from unmarked action terms of length k – 1 and unmarked atomic action terms of length 1
S=(X, A ∪ {d}, V) – decision systemλ1, λ2 – minimum support and confidenceCS(a)={NS(ta): ta – an atomic action term built from elements in Va}
a ∈ A
5. ARED Algorithm
For decision attribute in S:
NS(d,d1 d2)=[{x1,x2, x3, x4, x5, x7}, {x6}]
XX a a bb cc dd
xx11 aa11 bb11 cc11 dd11
xx22 aa22 bb11 cc22 dd11
xx33 aa22 bb22 cc22 dd11
xx44 aa22 bb11 cc11 dd11
xx55 aa22 bb33 cc22 dd11
xx66 aa11 bb11 cc22 dd22
xx77 aa11 bb22 cc2 2 dd11
xx88 aa11 bb22 cc11 dd33
stable attribute
flexible attributes
Object reclassification from class d1 to d2 λ1=2, λ2=1/4
Notation: t1=(b,b1b1), t2=(b,b2b2), t3=(b,b3b3), t4=(a,a1a2), t5=(a,a1a1), t6=(a,a2a2), t7=(a,a2a1), t8=(c,c1c2), t9=(c,c2c1), t10=(c,c1c1),t11=(c,c2c2),
t12 = (d,d1 d2),
5. ARED Algorithm
For decision attribute in S:
NS(d,d1 d2)=[{x1,x2, x3, x4, x5, x7}, {x6}]
Object reclassification from class d1 to d2 λ1=2, λ2=1/4
For classification attribute in S:
NS(t1)=[{x1,x2, x4, x6}, {x1,x2, x4, x6}]
Not marked λ1=3
NS(t2)=[{x3,x7, x8}, {x3,x7, x8}] Mark “-” λ1=1
NS(t3)=[{x5}, {x5}] Mark “-” λ1=1
NS(t4)=[{x1,x6, x7, x8}, {x2,x3, x4, x5}]
Mark “-” λ2=0
NS(t5)=[{x1,x6, x7, x8}, {x1,x6, x7, x8}]
Not marked λ1=4
NS(t6)=[{x2,x3, x4, x5}, {x2,x3, x4, x5}]
Mark “-” λ2=0
Mark “+” λ1=4,λ2=1/4 NS(t7)=[{x2,x3, x4, x5}, {x1,x6, x7, x8}]
5. ARED Algorithm
For decision attribute in S:
NS(t12)=[{x1,x2, x3, x4, x5, x7}, {x6}]
Object reclassification from class d1 to d2 λ1=2, λ2=1/4
For classification attribute in S:
NS(t1)=[{x1,x2, x4, x6}, {x1,x2, x4, x6}]
Not marked λ1=3
NS(t2)=[{x3,x7, x8}, {x3,x7, x8}] Marked “-” λ2=0
NS(t3)=[{x5}, {x5}] Marked “-” λ1=1
NS(t4)=[{x1,x6, x7, x8}, {x2,x3, x4, x5}]
Marked “-” λ2=0
NS(t5)=[{x1,x6, x7, x8}, {x1,x6, x7, x8}]
Not marked λ2=2
NS(t6)=[{x2,x3, x4, x5}, {x2,x3, x4, x5}]
Marked “-” λ2=0
Mark “+” λ1=4, λ2=1/4NS(t7)=[{x2,x3, x4, x5}, {x1,x6, x7, x8}] r = [t7 t1]
5. ARED Algorithm
For decision attribute in S:
NS(t12)=[{x1,x2, x3, x4, x5, x7}, {x6}]
Object reclassification from class d1 to d2 λ1=2, λ2=1/4
For classification attribute in S:
NS(t8)=[{x1,x4, x8}, {x2, x3, x5, x6, x7}]
Not marked
Marked “-”
NS(t10)=[{x1, x4, x8}, {x1, x4, x8}] Marked “-”
NS(t11)=[{x2, x3, x5, x6, x7}, {x2, x3, x5, x6, x7}]
Not marked
Rule r = [t8→t12], conf = 2/3 *1/5 <λ2, sup=2 ≥ λ1
NS(t9)=[{x2, x3, x5, x6, x7}, {x1, x4, x8}]
5. ARED Algorithm
For decision attribute in S:
NS(t12)=[{x1,x2, x3, x4, x5, x7}, {x6}]
Object reclassification from class d1 to d2 λ1=2, λ2=1/4
For classification attribute in S:
Marked “+”
NS(t1*t11)=[{x2, x6}, {x2, x6}] Not marked
NS(t5*t8)=[{x1, x8}, {x6, x7}] Marked “-”
Rule r = [t1*t8→t12], conf = 1/2 ≥ λ2, sup=2 ≥ λ1
Now action terms of length 2 from unmarked action terms of length 1
NS(t1*t5)=[{x1, x6}, {x1, x6}] Not marked
NS(t1*t8)=[{x1, x4}, {x2, x6}]
Rule r = [t5*t8→t12], conf = 1/2 ≥ λ2, sup= 1 < λ1
NS(t5*t11)=[{x6, x7}, {x6, x7}] Not marked
NS(t8*t11)=[Ø, {x2, x3, x5, x6, x7}] Marked “-”
5. ARED Algorithm
For decision attribute in S:
NS(t12)=[{x1,x2, x3, x4, x5, x7}, {x6}]
Object reclassification from class d1 to d2 λ1=2, λ2=1/4
For classification attribute in S:
Now action terms of length 3 from unmarked action terms of length 2
(t1*t5*t8) –extension of (t5*t8)
Marked “-”
Algorithm STOPS!
5. ARED Algorithm
For decision attribute in S:
NS(t12)=[{x1,x2, x3, x4, x5, x7}, {x6}]
Object reclassification from class d1 to d2 λ1=2, λ2=1/4
For classification attribute in S:
Action rules:
[[(b,b1→b1)*(c,c1→c2)] → (d, d1→d2)]
[[(a,a2→a1] → (d, d1→d2)]
XX a a bb cc dd
xx11 aa11 bb11 cc11 dd11
xx22 aa22 bb11 cc22 dd11
xx33 aa22 bb22 cc22 dd11
xx44 aa22 bb11 cc11 dd11
xx55 aa22 bb33 cc22 dd11
xx66 aa11 bb11 cc22 dd22
xx77 aa11 bb22 cc2 2 dd11
xx88 aa11 bb22 cc11 dd33
6. Application
Database HEPAR (Medical Center of Postgraduate Education, Warsaw, Poland) prepared by Dr. med. Hanna Wasyluk
758 758 patientspatients described by 106 attributes (including 31 laboratory tests described by 106 attributes (including 31 laboratory tests with values discretized to: “below normal”, “normal”, “above normal”). with values discretized to: “below normal”, “normal”, “above normal”). 14 attributes are stable. Two tests are invasive tests: 14 attributes are stable. Two tests are invasive tests: HBsAg [in tissue], HBcAg [in tissue]. There are 7 decision values:
I. I. Acute hepatitisAcute hepatitisIIa. IIa. Subacute hepatitis (types BC)IIb. IIb. Subacute hepatitis (alcohol-abuse)IIIa.IIIa. Chronic hepatitis (curable)IIIb.IIIb. Chronic hepatitis (non-curable)IV.IV. Cirrhosis hepatitisV.V. Liver cancer
We can ask for re-classifications: IIb We can ask for re-classifications: IIb I, IIIa I, IIIa I I
6. Application
Database HEPAR (heavily incomplete)
Handling data incompletenessHandling data incompleteness
Attributes with more than Attributes with more than 90% 90% of null values have been of null values have been removedremoved
Subjective attributes have been removedSubjective attributes have been removed [example: [example: History of alcohol abuse]
Removal of absolute medical testsRemoval of absolute medical tests
RSES RSES null value imputation module was used to make the null value imputation module was used to make the resulting dataset complete.resulting dataset complete.
6. Application
ChosenChosen redu reducct t has no invasive tests has no invasive tests, 11% , 11% incomplete dataincomplete data{m, n, q, u, y, aa, ah, ai, am, an, aw, bb, bg, by, cj, cm}
m Bleedingn Subjaundice symptomsq Eructationu Obstructiony Weight lossaa Smoking ah History of viral hepatitis (stable)ai Surgeries in the past (stable)am History of hospitalization (stable)an Jaundice in pregnancy aw Erythematous dermatitisbb Cysts bg Sharp liver edge (stable)bm Blood cell plaqueby Alkaline phosphatasecj Prothrombin index cm Total cholesteroldd Decision attribute
6. ApplicationAction Rules discovery in Database HEPAR
No history of viral hepatitis butbut withwith history of shistory of surgery and hospitalization, , ssharp liver edge normal, , no sno subjaundice symptoms, , ttotal cholesterol normal, , eerythematous dermatitis normal, wweight normal, no cysts , patient does not smoke
[(ah=1)[(ah=1) (ai=2) (ai=2) (am=2) (am=2)(bg=1)](bg=1)](cm=1)(cm=1) (aw=1) (aw=1)(u, (u, 1) 1)(bb=1)(bb=1) (aa=1) (aa=1) (q, (q, 1) 1) (m, (m, 1) 1) (n=1) (n=1) (bm, (bm, down) down) (y=1) (y=1) (by, norm (by, norm up) up) (dd, IIIa (dd, IIIa I ) I )
[[(ah=1)(ah=1) (ai=2)(ai=2) (am=2) (am=2) (bg=1)] (bg=1)](cm=1)(cm=1)(aw=1)(aw=1)(u, (u, 1) 1) (bb=1) (bb=1) (aa=1) (aa=1) (q, (q, 1) 1)(m, (m, 1) 1)(n=1)(n=1)(bm, (bm, down) down)(y=1)(y=1) (by, norm (by, norm down) down)
(dd, IIIa (dd, IIIa I ) I ) Two quite similar action rules have been constructedTwo quite similar action rules have been constructed::
By getting rid of obstruction, eructation, bleeding, by decreasing the bBy getting rid of obstruction, eructation, bleeding, by decreasing the blood cell plaque and by changing the level of a and by changing the level of alkaline phosphatase we should be able we should be able reclassify the patient from IIIa group to Ireclassify the patient from IIIa group to I. . Medical doctor should decide for a given patient if Medical doctor should decide for a given patient if the alkaline phosphatase level level needs to be decreased or increasedneeds to be decreased or increased. .
Attribute values of Attribute values of ttotal cholesterol, wweight, and smoking have to stay unchangedhave to stay unchanged. .
• Presented algorithm discovers action rules directly from a decision table.
• The proposed algorithm generates the shortest action rules without using pre-existing classification rules
• Future work includes- Action rule generation from incomplete information system
- Action rule generation when attributes are hierarchical
- Considering different flexibilities of attributes (e.g. the social condition is most likely less flexible than
the health condition )
7. Conclusion and Future Work