Evaluating a Classification Model

Url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/pima-indians-diabetes/pima-indians-diabetes.data'

You can also calculate various category metrics useful for model selection.

Since the under report (spam to the inbox) has a wider tolerance than false detection (spam filter captures non-spam)

Since the false detection (the ordinary transaction is marked as possibly invalid) has a wider tolerance than false detection (undetected illegal transaction)

/Users/ritchieng/anaconda3/envs/py3k/lib/python3.5/site-packages/sklearn/utils/ validation.py: 386: Dreprecation Warning: Since the data is 0.17 and it is going to be deprecated to 0.19, pass 1d array Please increase ValueError. If the data has a single feature, use X.reshape (-1, 1) to transform the data, if it contains a single sample use X.reshape (1, -1) Transform the data.

By default, the threshold is converted to class prediction using a threshold of 0.5 (for binary problems).

If random selection of positive and negative observations is chosen, AUC indicates the possibility that the classifier will specify a higher probability of prediction for positive observations.

As a simple example, we often use that accuracy when evaluating classification models. After all, the probability that the model will output the correct answer. More complicated is that you can ask questions such as PAC learning, "Accuracy of the algorithm will be X if you give N examples to train". Actually, I do not think so, but it still applies to our model. Just because the probability is continuous and not deterministic, it is important to return cat-like things rather than deterministic. It is deterministic and I think that the picture above is a dog, so it is really difficult to know how to change parameters to behave in a good example and output correct answers without sacrificing performance. As opposed to this, there is a direct way to increase the likelihood of giving an image to a cat, as we saw in this chapter.

When evaluating a classification model via an expected value frame, you can calculate the probability of each cell in the confusion matrix, multiply the advantage of that cell and summarize the result. For example, if the actual positive price of a customer held with the addition of an activity is 0.9 and the customer's profit is held at 100 dollars, the expected profit for that cell is 90 dollars. 2. Data science is closer to software engineering than research and development. The data content of many data scientists needs to be familiar with Agile and other software engineering frameworks. This is reasonable, as many data scientists are hired by technology companies, but I believe that data scientists can be managed in the same way.