Python Machine Learning

Sebastian Raschka

Mentioned 2

Unlock deeper insights into Machine Leaning with this vital guide to cutting-edge predictive analyticsAbout This Book- Leverage Python's most powerful open-source libraries for deep learning, data wrangling, and data visualization- Learn effective strategies and best practices to improve and optimize machine learning systems and algorithms- Ask - and answer - tough questions of your data with robust statistical models, built for a range of datasetsWho This Book Is ForIf you want to find out how to use Python to start answering critical questions of your data, pick up Python Machine Learning - whether you want to get started from scratch or want to extend your data science knowledge, this is an essential and unmissable resource.What You Will Learn- Explore how to use different machine learning models to ask different questions of your data- Learn how to build neural networks using Keras and Theano- Find out how to write clean and elegant Python code that will optimize the strength of your algorithms- Discover how to embed your machine learning model in a web application for increased accessibility- Predict continuous target outcomes using regression analysis- Uncover hidden patterns and structures in data with clustering- Organize data using effective pre-processing techniques- Get to grips with sentiment analysis to delve deeper into textual and social media dataIn DetailMachine learning and predictive analytics are transforming the way businesses and other organizations operate. Being able to understand trends and patterns in complex data is critical to success, becoming one of the key strategies for unlocking growth in a challenging contemporary marketplace. Python can help you deliver key insights into your data - its unique capabilities as a language let you build sophisticated algorithms and statistical models that can reveal new perspectives and answer key questions that are vital for success.Python Machine Learning gives you access to the world of predictive analytics and demonstrates why Python is one of the world's leading data science languages. If you want to ask better questions of data, or need to improve and extend the capabilities of your machine learning systems, this practical data science book is invaluable. Covering a wide range of powerful Python libraries, including scikit-learn, Theano, and Keras, and featuring guidance and tips on everything from sentiment analysis to neural networks, you'll soon be able to answer some of the most important questions facing you and your organization.Style and approachPython Machine Learning connects the fundamental theoretical principles behind machine learning to their practical application in a way that focuses you on asking and answering the right questions. It walks you through the key elements of Python and its powerful machine learning libraries, while demonstrating how to get to grips with a range of statistical models.

More on

Mentioned in questions and answers.

I'm beginner in machine learning. I begin my study with the book python machine learning and some videos online.

I'm confused by the implementation of Perceptron in "python machine learning". This is the formula:

enter image description here

And this is the python implementation for the formula: enter image description here

But by the formula, it's W * X, not X * W in the implementation. They're not same for matrix.(For numpy dot, if X is 2-D array, it's matrix multiplication.). Why does the implementation not follow the formula?

The actual values in above python codes from the Iris-example in the book are these:

w[1:]:  [-0.68  1.82]

X: [[ 3.3   0.  ]
    [ 3.32  0.  ]
    [ 3.34  0.  ]
    [ 7.94  6.08]
    [ 7.96  6.08]
    [ 7.98  6.08]]


A good question here. The answer should be two-fold.

W dot X =?= X dot W

You are perfectly right on that exchanging X and W yeilds different result for matrix multiplication.

However, in this situation, w and x are actually vectors, or m*1 matrix. The dot product results in a scalar. So in this situation, 'x dot w' and 'w dot x' are the same.

As you can see that x is interpreted as [x0, x1, x2, ..., xm], which indicates its vector nature.

What this function does is combining the inputs of a neuron with weight being w. These inputs are outputs from neurons from the previous layer. We know a neuron's output is a scalar.

About the bias term W_[0]

Actually the implementation is different in that a w_[0] is appended to the end of the polynomial. That is called a bias term, which modifies the result of linear combination of the inputs, or the x. It is a common practice in the implementations to use this bias term. But in math, we usually omit it since it doesn't change the linear nature of the combination. Of course in some situations this bias term is explicitly listed in the math representation.

====== Update =======

As the question has been updated, a further explain on the actual situation is added.

In this case, the first dimension is 'broadcast' to the result.

Consider a simpler situation:

w[1:]:  [-0.68  1.82]

X: [ 3.3   0.  ]

in which as @Peater and I explained, X and w are both vectors.

In the actual setup, X is not a matrix, but rather a vactor variable that changes on sampling, or time. Say X is of dimension [n * 2], that means x is a two-dimensional vector with n samples.

Here sample means different evaluations of a variable in different time frames, or different pixels. This dimension of X is broadcast to the result.

i am newbie to scikit-learn and machine learning in general. i want to write linear classifier that can separate male from female and find weights of separator lines in case i used height only and used both features weight and height.

i have text file that contain three columns where represent weight , height and gender.

|      124.5          |     5.2          |   1  |
|      187            |     5.7          |   0  |

note: o represent male and 1 represent female

i have managed to use perceptron algorithm from sklearn.linear_model but i dont know how to get weights of separator line

import numpy as np
from sklearn import svm
import matplotlib.pyplot as plt
from sklearn.linear_model import perceptron
from pandas import *
import fileinput

f = fileinput.input('data.txt')
_male = []
_female = []
_target = []
X_1 = []
X_2 = []
Y = []

for line in f:
    temp = line.split(",")
    if str(temp[2]) == '0\n' :
        X_1.append(round(float(temp[0]), 2))
        X_2.append(round(float(temp[1]), 2))
inputs = DataFrame({
'Height' : X_1,
'Weight' : X_2,
'Targets' : Y
colormap = np.array(['r', 'k'])
plt.scatter(inputs.Height,inputs.Weight, c=colormap[inputs.Targets], s=40)
net = perceptron.Perceptron(n_iter=1000, verbose=0, random_state=None,   fit_intercept=True, eta0=0.002)

# Train the perceptron object (net)[['Height', 'Weight' ]],inputs['Targets'])
# Output the values
print "Coefficient 0 " + str(net.coef_[0, 0])
print "Coefficient 1 " + str(net.coef_[0, 1])
print "Bias " + str(net.intercept_)

plt.scatter(inputs.Height,inputs.Weight, c=colormap[inputs.Targets])

# Calc the hyperplane (decision boundary)
ymin, ymax = plt.ylim()
w = net.coef_[0]
a = -w[0] / w[1]
xx = np.linspace(ymin, ymax)
yy = a * xx - (net.intercept_[0]) / w[1]

  # Plot the hyperplane
   plt.plot(xx, yy, 'k-')

Your question is a bit broad, and you can start by reading the scikit-learn documents online. I also highly recommend Python Machine Learning which is a great book.

That being said, here are a few pointers to get you started:

  1. First, you need to make sure you have installed the following python libraries:

    • numpy
    • pandas
    • scikit-learn
    • matplotlib (optional)
    • ipython and jupyter (optional)
  2. You want to get a better understanding of your data. Any visualization could be helpful in this case. I suggest using a scatter plot. You can use heights and weights for your x and y values, and two different colors for males and females. You will then be able to visually confirm whether these two groups are separable or not. If they are, then you can hope to train a model to classify these groups given their heights and weights.

  3. scikit-learn does not like missing values. So, if you have rows with missing values, you either need to remove them from your data, or fill the missing values with a reasonable value (e.g. the mean value). Take a look at the Imputer class for this.

  4. Assuming you have a lot of data, you want to split it into two sets, one for training your model, and the other for testing the quality of your model. Take a look at train_test_split in sklearn to achieve this. You probably want about %10 to %20 of your data for testing.

  5. Once you have a dataset for training your model, you need to choose an estimator, and then train it by calling the .fit() method, and specifying your features (inputs), and target (output). There are several estimators available in scikit-learn. You may choose logistic regression for now.

  6. The next step is to check how good your model is by testing it against the test dataset. Take a look at the .score() method for your estimator to see how to do it.

  7. The first trained model is usually not that good and you need to find ways to improve it by finding better input parameters for your estimator, choosing another estimator, or even trying a different technique. This means a lot of iterations between steps 5 and 6 above, and this is an important part of any Machine Learning project. In order to tune the parameters of your estimator, you may want to use GridSearchCV.

For small projects like yours, you may want to take a look at sler which simplifies working with scikit-learn quite a lot. You should be able to do all the work above with a few lines of code (there are a few examples available to learn from).