SLIDE 1
Decision Trees and Random Forests Modeling by using P Systems
jsempere@dsic.upv.es http://personales.upv.es/jsempere José M. Sempere Department of Information Systems and Computation Universitat Politècnica de València
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 2 Goals of this work: (1) Provide algorithms to define P systems from decision trees and random forests (2) Introduce machine learning techniques to obtain decision trees and random forests from given data through membrane and object rules based on P systems working in entropic/functional manner Previous Works
- Decision Tree Models Induced by Membrane Systems (2015) J. Wang, J. Hu, M.J. Pérez-Jiménez, A.Riscos-Núñez
ROMJIST Vol.18 No.3 pp 228-239
- Self-constructing Recognizer P Systems. D. Díaz-Pernil, F. Peña-Cantillana, M.A. Gutiérrez-Naranjo. In Proceedings of
the Thirteenth Brainstorming Week on Membrane Computing , pp 137-154. Fenix Editora. 2014. Díaz-Pernil et al. Wang et al. Our approach Trees defined by the membrane structure Tree-like objects Trees defined by the membrane structure Non-deterministic search of structures External Induction Algorithm Algorithm runs by P rules within an entropic manner 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 3
A Decision Tree is a representation for a discrete-valued function !: #$×#&× ⋯×#( → * #$
+,-./(#$. 2, 2$) +,-./(#$. 2, 25)
#& #6
+,-./(#6. 2, 2$) +,-./(#6. 2, 27)
*. 2$ *. 26
+,-./ #6. 2, 28 ∈ {#6. 2 = 28, #6. 2 ≤ 28, #6. 2 ≥ 28, #6. 2 ≠ 28, …} 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 4
An example: protein-protein interactions prediction
(From “What are decision trees ?”. Carl Kingsford & Steven L. Salzberg. Nature Biotechnology 26 No. 9, 1011 – 1013 (2008) )
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 5
Implementing decision trees by membrane structures (I) Objects alphabet 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany Membrane labels alphabet
SLIDE 6
Implementing decision trees by membrane structures (I) 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany An algorithm to translate decision trees to P systems
SLIDE 7
Implementing decision trees by membrane structures (I) 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany An algorithm to translate decision trees to P systems
SLIDE 8
Implementing decision trees by membrane structures (I) 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany An algorithm to translate decision trees to P systems
SLIDE 9
Implementing decision trees by membrane structures (I) 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany An algorithm to translate decision trees to P systems
SLIDE 10
Implementing decision trees by membrane structures (II) ECYes ECNo SFYes SFNo SCLNo SCLYes GDL5Yes GDL5No 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 11
Implementing decision trees by membrane structures (III) ECYes ECNo SFYes SFNo SCLNo SCLYes GDL5Yes GDL5No
∀ ", $, % ∈ '(), *+ ,-'().-/".0$12/3% → [.0$],-7() ∀ ", $, % ∈ '(), *+ ,-*+.-/".0$12/3% → [.-/"12/3%],-8+
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 12 Implementing decision trees by membrane structures (IV) ECYes ECNo SFYes SFNo SCLNo SCLYes GDL5Yes GDL5No
∀ ", $, % ∈ '(), *+ ,-'().-/".0$12/3% → [.0$],-7() ∀ ", $, % ∈ '(), *+ ,-*+.-/".0$12/3% → [.-/"12/3%],-8+
12/3'()→ [7,.]12/37() 12/3*+→ [89]12/38+
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 13 Implementing decision trees by membrane structures (V) ECYes ECNo SFYes SFNo SCLNo SCLYes GDL5Yes GDL5No
∀ ", $, % ∈ '(), *+ ,-'().-/".0$12/3% → [.0$],-7() ∀ ", $, % ∈ '(), *+ ,-*+.-/".0$12/3% → [.-/"12/3%],-8+
12/3'()→ [7,.]12/7() 12/3*+→ [89]12/8+
[YES] → [ ] YES [NO] → [ ] NO [NO] → [ ] NO
[NO] → [ ] NO [YES] → [ ] YES
[YES] → [ ] YES [NO] → [ ] NO
[NO] → [ ] NO [YES] → [ ] YES
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 14
A parsing example ECYes ECNo SFYes SFNo SCLNo SCLYes
GDL5Yes GDL5No
ECno SFyes SCLyes GDL5no 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 15
A parsing example ECYes ECNo SFYes SFNo SCLNo SCLYes
GDL5Yes GDL5No
SCLyes GDL5no 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 16
A parsing example ECYes ECNo SFYes SFNo SCLNo SCLYes
GDL5Yes GDL5No
GDL5no 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 17
A parsing example ECYes ECNo SFYes SFNo SCLNo SCLYes
GDL5Yes GDL5No
NO 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 18
A parsing example ECYes ECNo SFYes SFNo SCLNo SCLYes
GDL5Yes GDL5No
NO 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 19
A parsing example ECYes ECNo SFYes SFNo SCLNo SCLYes
GDL5Yes GDL5No
NO 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 20
A parsing example ECYes ECNo SFYes SFNo SCLNo SCLYes
GDL5Yes GDL5No
NO 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 21
A parsing example ECYes ECNo SFYes SFNo SCLNo SCLYes
GDL5Yes GDL5No
NO 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 22
Applying a machine learning technique inside a P system
Sample ! " # Decision 1 High High High Yes 2 High High High Yes 3 Low High Low Yes 4 Medium High High Not
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 23 Applying a machine learning technique inside a P system
Sample ! " # Decision 1 High High High Yes 2 High High High Yes 3 Low High Low Yes 4 Medium High High Not !$%&'
(
"$%&'
(
#$%&'
(
)*+%,%-."/0
(
!$%&'
1
"$%&'
1
#$%&'
1
)*+%,%-."/0
1
!2-3
4
"$%&'
4
#2-3
4
)*+%,%-."/0
4
!5*6%78
9
"$%&'
9
#2-3
9
)*+%,%-.:;
9
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 24 Applying a machine learning technique inside a P system
Sample ! " # Decision 1 High High High Yes 2 High High High Yes 3 Low High Low Yes 4 Medium High High Not !$%&'
(
"$%&'
(
#$%&'
(
)*+%,%-."/0
(
!$%&'
1
"$%&'
1
#$%&'
1
)*+%,%-."/0
1
!2-3
4
"$%&'
4
#2-3
4
)*+%,%-."/0
4
!5*6%78
9
"$%&'
9
#2-3
9
)*+%,%-.:;
9
A generic algorithm to build decision trees from data
Input: A finite set of supervised tuples E Output: A decision tree T Method: 1) Create an arbitrary root 2) If all the tuples belong to class <= then return(root, <=) else 1. Select an attribute ! with values >(, >1, … , >5 2. Make a partition of E according to the attribute value /(, /1, … , /5 ∶ / = ⋃%D(
5 /%
3. Build decision trees for every subset: E(, E1, … , E5 endMethod
FGGH
I = JK I = JL I = JM N
K
NL NM
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 25 Applying a machine learning technique inside a P system
Sample ! " # Decision 1 High High High Yes 2 High High High Yes 3 Low High Low Yes 4 Medium High High Not !$%&'
(
"$%&'
(
#$%&'
(
)*+%,%-."/0
(
!$%&'
1
"$%&'
1
#$%&'
1
)*+%,%-."/0
1
!2-3
4
"$%&'
4
#2-3
4
)*+%,%-."/0
4
!5*6%78
9
"$%&'
9
#2-3
9
)*+%,%-.:;
9
A generic algorithm to build decision trees from data
Input: A finite set of supervised tuples E Output: A decision tree T Method: 1) Create an arbitrary root 2) If all the tuples belong to class <= then return(root, <=) else 1. Select an attribute ! with values >(, >1, … , >5 2. Make a partition of E according to the attribute value /(, /1, … , /5 ∶ / = ⋃%D(
5 /%
3. Build decision trees for every subset: E(, E1, … , E5 endMethod
Suppose that the attribute for the node root is X, then we apply the following three rules (membrane creation)
!$%&'
(
"$%&'
(
#$%&'
(
)*+%,%-."/0
(
!$%&'
1
"$%&'
1
#$%&'
1
)*+%,%-."/0
1
→ [ "$%&'
(
#$%&'
(
)*+%,%-."/0
(
"$%&'
1
#$%&'
1
)*+%,%-."/0
1
]!$%&' !2-3
4
"$%&'
4
#2-3
4
)*+%,%-."/0
4
→ [ "$%&'
4
#2-3
4
)*+%,%-."/0
4
]!2-3 !5*6%78
9
"$%&'
9
#2-3
9
)*+%,%-.:;
9
→ [ "$%&'
9
#2-3
9
)*+%,%-.:;
9 ]!5*6%78
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 26 Applying a machine learning technique inside a P system How can the appropriate rules be selected ?
- External algorithm selects the rule and we apply a translation scheme from decision trees to P systems
- Apply a P system working within an entropic manner (CMC17)
- Apply rules according to a functional criterium: (FUNCTIONAL WORKING MANNER !!!)
!: # $ → & 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany '(, … , '+ : ' → , A function defines the selections of the rules (instead of selection by priority) Every rule has some parameters (elements from V) and the function ! is calculated over them. The rule to be applied has the highest value for the function !.
SLIDE 27
Random forests
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
FINAL DECISION input data decision trees ensemble decision criteria
SLIDE 28
From decision trees to random forests
Random Forest algorithm
mtry: number of features for node splitting ntree: number of trees in the forest
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 29
From cell-like P systems to tissue-like P systems
Initially we have only one cell with cell-creation rules and all the input data
!" #$ … &' → !" #$ … &'(!" #$)
!" #$ … &'
!" #$ … &' → !" #$ … &'(#$ &')
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 30
From cell-like P systems to tissue-like P systems
Initially we have only one cell with cell-creation rules and all the input data
!" #$ … &' → !" #$ … &'(!" #$)
!" #$ … &' !" #$ … &' !" #$
!" #$ … &' → !" #$ … &'(#$ &')
#$ &' 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 31
From cell-like P systems to tissue-like P systems
Initially we have only one cell with cell-creation rules and all the input data
!" #$ … &' → !" #$ … &'(!" #$)
!" #$ … &' !" #$ … &' !" #$
!" #$ … &' → !" #$ … &'(#$ &')
#$ &' Apply rules to create a decision tree 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 32
YES YES YES NO YES NO
YES → (#$%)' NO → (())'
Rules inside every P system at skin region (communication rules) * 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 33
YES YES YES NO YES NO ! 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 34
YES YES YES NO YES NO ! Majority rules ##$% > #'( → #$%*+, #'( > ##$% → '(*+, 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 35
YES ! Majority rules ##$% > #'( → #$%*+, #'( > ##$% → '(*+, 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
SLIDE 36 19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
Conclusions (... work in progress)
- A definition of decision trees by P systems
- Introducing machine learning approach by P systems working in an entropic/functional manner
- A definition of random forests by (tissue) P systems
- Introducing ensemble decision criteria by (majority) rules
SLIDE 37
19th International Conference on Membrane Computing (CMC19) 4-7 September 2018 , Dresden, Germany
