defragTrees

Python code for tree ensemble interpretation proposed in the following paper.

• S. Hara, K. Hayashi, Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach. In Proceedings of the 21th International Conference on Artificial Intelligence and Statistics (AISTATS'18), pages 77--85, 2018.

Requirements

To use defragTrees:

• Python3.x
• Numpy
• Pandas

To run example codes in example directory:

• Python: XGBoost, Scikit-learn
• R: randomForest

To replicate paper results in paper directory:

• Python: Scikit-learn, Matplotlib, pylab
• R: randomForest, inTrees, nodeHarvest

Usage

Prepare data:

• Input X: feature matrix, numpy array of size (num, dim).
• Output y: output array, numpy array of size (num,).
  • For regression, y is real value.
  • For classification, y is class index (i.e., 0, 1, 2, ..., C-1, for C classes).
• Splitter splitter: thresholds of tree ensembles, numpy array of size (# of split rules, 2).
  • Each row of splitter is (feature index, threshold). Suppose the split rule is second feature < 0.5, the row of splitter is then (1, 0.5).

Import the class:

from defragTrees import DefragModel

Fit the simplified model:

Kmax = 10 # uppder-bound number of rules to be fitted
mdl = DefragModel(modeltype='regression') # change to 'classification' if necessary.
mdl.fit(X, y, splitter, Kmax)
#mdl.fit(X, y, splitter, Kmax, fittype='EM') # use this when one wants exactly Kmax rules to be fitted

Check the learned rules:

print(mdl)

For further deitals, see defragTrees.py. In IPython, one can check:

import defragTrees
defragTrees?

Examples

Simple Examples

See example directory.

Replicating Paper Results

See paper directory.