Current Slide

Efficiency and Interpretability

• Efficiency of backpropagation: Each epoch (one iteration through the training set) takes O(|D| * w), with |D| tuples and w weights, but # of epochs can be exponential to n, the number of inputs, in worst case
• For easier comprehension: Rule extraction by network pruning
• Simplify the network structure by removing weighted links that have the least effect on the trained network
• Then perform link, unit, or activation value clustering
• The set of input and activation values are studied to derive rules describing the relationship between the input and hidden unit layers
• Sensitivity analysis: assess the impact that a given input variable has on a network output. The knowledge gained from this analysis can be represented in rules