ADALINE with 2 features decision function. Chapter 2, Page 40

[manas007]

Hello,
While plotting the decision boundary with just 2 features for ADALINE, the decision class function is simply
w1 ∗ x1 + w2 ∗ x2 + b = 0

and therefore in simple slope intercept form, x2 can be simplified to
x2 = (−w1/w2) ∗ x1 − (b/w2)

when I plot this, the line does not separate the two classes. I posted the question on cross-validated exchange as well and the answer was to label y as {-1,1} instead of {0,1} and indeed the plot works correctly. I am wondering why is this?

Link to question

[manas007]

@rasbt is there an explanation why the labels should be {-1,1} instead of {0,1} ? The above plot works when labels are {-1,1} but not when labels are {0,1}. This is not explained in the book and therefore is confusing when drawing the boundary using the simple line equation instead of the meshgrid.

[rasbt]

Hi there, that's a good question. I had the class labels as -1 & 1 in earlier editions of the book (https://github.com/rasbt/python-machine-learning-book/blob/master/code/ch02/ch02.ipynb) but it's not necessary, and I changed it to 0 and 1 to make it more consistent with scikit-learn.

However, note that for this, I set the threshold to 0.5 so that it's between 0 and 1:

    def predict(self, X):
        """Return class label after unit step"""
        return np.where(self.activation(self.net_input(X)) >= 0.5, 1, 0)

And hence you have to also add that in your equation when plotting the boundaries.

[rasbt]

@manas007 Let me know if this helps

[manas007]

@rasbt thank you for your response. Shouldn't the equation of the boundary remains the same irrespective of the threshold ?

[rasbt]

Because you have class labels 0 and 1, the prediction threshold needs to be between the two, in this case 0.5. If you have classes -1 and 1, the prediction threshold is 0.

If you have the threshold 0.5, you have to incorporate it into the equation, otherwise it will offset by 0.5.