Empirical or Statistical Approaches to Cross-Validating

There are two ways to cross-validate, but it is empirical or statistical (Levy, 2013). To use the empirical method you can get predictors and criteria from the previous sample and compare them with other samples to see how they match the original samples collected . Next, enter this data as the predicted score of the new sample to obtain the cross validation score (Levy, 2013). By comparing the predicted score with the actual score, you can see the validity of the sample.

Cross-validation is a method of measuring predictive performance of a statistical model mainly in an independent data set (obtained from http://robjhyndman.com). One way to measure the predictive power of a model is to test the model with a series of data that is not used in training data. Data minor calls this a "test set", and the data used for estimation is a "training set". The main purpose of verification is to avoid over fitting. Over fitting occurs when machine learning algorithms (eg, classifiers) not only identify the signals in the data set, but also identify noise. The noise here means that the model is too sensitive to the characteristics of the dataset and makes no sense. The ultimate consequence of over fitting is that the classifier that seems to work well for training data may not work well for new data from the same problem. This is very bad possibility.

In this blog post, we explain the pitfalls when using conventional cross validation and time series data. Specifically, 1) divide the time series without causing data leakage, 2) use nested cross validation to obtain unbiased estimation error in independent test set, 3) data including multiple time series Due to differences in terms in verification literature that crossovers using sets we have clearly defined our resume process. First, the data set is divided into subsets called training sets and the other subset is called test set. If you need to adjust the parameters, divide the training set into a training subset and a validation set. Learn the model with the training subset and select the parameter that minimizes the error of the validation set. Finally, train the model with the complete training set using the selected parameters and record the error in the test set

X_prediction = predict_data.drop ('t_id', axis = 1) It is necessary to perform cross validation on training data. Cross-validation is a method by which the model is generalized to an unknown independent data set, and the method used to estimate the actual accuracy. The goal here is to define a dataset to test the model in the training phase to limit problems such as over fitting. We used the k-split cross validation. The original sample is randomly divided into k identical sub-samples. Of the k sub-samples, a single sub-sample is held as the verification data of the test model and the remaining k - 1 sub-samples are used as the training data. Next, the cross-validation process is repeated k times and each k sub-samples are used only once as verification data. You can then generate a single estimate by averaging the results of folded k.