## Python Machine Learning

Sebastian Raschka

Mentioned 2

Mentioned in questions and answers.

I'm beginner in machine learning. I begin my study with the book python machine learning and some videos online.

I'm confused by the implementation of Perceptron in "python machine learning". This is the formula:

And this is the python implementation for the formula:

But by the formula, it's W * X, not X * W in the implementation. They're not same for matrix.(For numpy dot, if X is 2-D array, it's matrix multiplication.). Why does the implementation not follow the formula?

The actual values in above python codes from the Iris-example in the book are these:

``````w[1:]:  [-0.68  1.82]

X: [[ 3.3   0.  ]
[ 3.32  0.  ]
[ 3.34  0.  ]
...,
[ 7.94  6.08]
[ 7.96  6.08]
[ 7.98  6.08]]
``````

Thanks.

A good question here. The answer should be two-fold.

## W dot X =?= X dot W

You are perfectly right on that exchanging X and W yeilds different result for matrix multiplication.

However, in this situation, w and x are actually vectors, or m*1 matrix. The dot product results in a scalar. So in this situation, 'x dot w' and 'w dot x' are the same.

As you can see that x is interpreted as [x0, x1, x2, ..., xm], which indicates its vector nature.

What this function does is combining the inputs of a neuron with weight being `w`. These inputs are outputs from neurons from the previous layer. We know a neuron's output is a scalar.

## About the bias term W_[0]

Actually the implementation is different in that a w_[0] is appended to the end of the polynomial. That is called a bias term, which modifies the result of linear combination of the inputs, or the `x`. It is a common practice in the implementations to use this bias term. But in math, we usually omit it since it doesn't change the linear nature of the combination. Of course in some situations this bias term is explicitly listed in the math representation.

## ====== Update =======

As the question has been updated, a further explain on the actual situation is added.

In this case, the first dimension is 'broadcast' to the result.

Consider a simpler situation:

``````w[1:]:  [-0.68  1.82]

X: [ 3.3   0.  ]
``````

in which as @Peater and I explained, X and w are both vectors.

In the actual setup, X is not a matrix, but rather a vactor variable that changes on sampling, or time. Say X is of dimension [n * 2], that means x is a two-dimensional vector with n samples.

Here sample means different evaluations of a variable in different time frames, or different pixels. This dimension of X is broadcast to the result.

i am newbie to scikit-learn and machine learning in general. i want to write linear classifier that can separate male from female and find weights of separator lines in case i used height only and used both features weight and height.

i have text file that contain three columns where represent weight , height and gender.

``````|---------------------|------------------|------|
|      124.5          |     5.2          |   1  |
|---------------------|------------------|------|
|      187            |     5.7          |   0  |
|---------------------|------------------|------|
``````

note: o represent male and 1 represent female

i have managed to use perceptron algorithm from sklearn.linear_model but i dont know how to get weights of separator line

``````import numpy as np
from sklearn import svm
import matplotlib.pyplot as plt
from sklearn.linear_model import perceptron
from pandas import *
import fileinput

f = fileinput.input('data.txt')
_male = []
_female = []
_target = []
X_1 = []
X_2 = []
Y = []

for line in f:
temp = line.split(",")
if str(temp[2]) == '0\n' :
X_1.append(round(float(temp[0]),2))
X_2.append(round(float(temp[1]),2))
Y.append(0)
else:
X_1.append(round(float(temp[0]), 2))
X_2.append(round(float(temp[1]), 2))
Y.append(1)
inputs = DataFrame({
'Height' : X_1,
'Weight' : X_2,
'Targets' : Y
})
colormap = np.array(['r', 'k'])
plt.scatter(inputs.Height,inputs.Weight, c=colormap[inputs.Targets], s=40)
net = perceptron.Perceptron(n_iter=1000, verbose=0, random_state=None,   fit_intercept=True, eta0=0.002)

# Train the perceptron object (net)
net.fit(inputs[['Height', 'Weight' ]],inputs['Targets'])
# Output the values
print "Coefficient 0 " + str(net.coef_[0, 0])
print "Coefficient 1 " + str(net.coef_[0, 1])
print "Bias " + str(net.intercept_)

plt.scatter(inputs.Height,inputs.Weight, c=colormap[inputs.Targets])

# Calc the hyperplane (decision boundary)
ymin, ymax = plt.ylim()
w = net.coef_[0]
a = -w[0] / w[1]
xx = np.linspace(ymin, ymax)
yy = a * xx - (net.intercept_[0]) / w[1]

# Plot the hyperplane
plt.plot(xx, yy, 'k-')
plt.show()
``````

Your question is a bit broad, and you can start by reading the scikit-learn documents online. I also highly recommend Python Machine Learning which is a great book.

That being said, here are a few pointers to get you started:

1. First, you need to make sure you have installed the following python libraries:

• numpy
• pandas
• scikit-learn
• matplotlib (optional)
• ipython and jupyter (optional)
2. You want to get a better understanding of your data. Any visualization could be helpful in this case. I suggest using a scatter plot. You can use heights and weights for your x and y values, and two different colors for males and females. You will then be able to visually confirm whether these two groups are separable or not. If they are, then you can hope to train a model to classify these groups given their heights and weights.

3. scikit-learn does not like missing values. So, if you have rows with missing values, you either need to remove them from your data, or fill the missing values with a reasonable value (e.g. the mean value). Take a look at the Imputer class for this.

4. Assuming you have a lot of data, you want to split it into two sets, one for training your model, and the other for testing the quality of your model. Take a look at train_test_split in sklearn to achieve this. You probably want about %10 to %20 of your data for testing.

5. Once you have a dataset for training your model, you need to choose an estimator, and then train it by calling the .fit() method, and specifying your features (inputs), and target (output). There are several estimators available in scikit-learn. You may choose logistic regression for now.

6. The next step is to check how good your model is by testing it against the test dataset. Take a look at the .score() method for your estimator to see how to do it.

7. The first trained model is usually not that good and you need to find ways to improve it by finding better input parameters for your estimator, choosing another estimator, or even trying a different technique. This means a lot of iterations between steps 5 and 6 above, and this is an important part of any Machine Learning project. In order to tune the parameters of your estimator, you may want to use GridSearchCV.

For small projects like yours, you may want to take a look at sler which simplifies working with scikit-learn quite a lot. You should be able to do all the work above with a few lines of code (there are a few examples available to learn from).

Realated tags