Here is an example of what I meant, it is simply the Fixed Split Sample with ID's and breakpoints inserted, so you can see the numbers of the examples that are being used for both training and test. With what you have you can easily be training on examples that occur after the examples you are testing! And see if there is a significant difference. I think it does, depending on the learning algorithm.Īnd I guess you could test this quite easily.Įvaluate the performance of your learning algorithm 3 times, using cross validation, random order percentage split, and fixed order percentage split. Now does it matter in what order you feed your training examples to your learning algorithm? Well that’s a hard question actually. (t-0 isn’t because t is fully deterministic)īut since t-0 and t-1 are correlated 100% we might as well also throw it away, without loosing any information. Secondly we can’t use training examples of the form: So I think its best to just throw it away. I don’t think adding a training example with ? adds any useful information. Let’s say we take window size of 2, so we are adding the information of the barometer readings from the pervious hour u-1. Now when we make the problem a bit harder, by adding barometer information, I think this still holds. So train on: (randomly selected 6,4,8,1,2) Intuitively I would do a “keep order” 66% split to test the performance.īut I guess training on a “random order” 66% split would work just as well. So we could start simple, throw away barometer for now, and see how well we can forecast v+1 using just t. Unlike the value of u+1, since we cannot look ahead into the future and see what our barometer looks like. We already know the value of t+1, since time or 'hour of the day' is a fully determinative attribute. Let's say we wish to predict the wind speed at the next hour, so the value of v+1. Let's say we have a simple dataset, made by hand. Change W-REPTree to any regression learning algorithm that suits you.īut are we really training on future examples, and testing on past examples? If not chance all occurrences of close to your own label.Ĥ. My dataset also uses the class attribute label: 'close' so this should be the same. click on 'CSVExampleSource', and change it so loads your own dataset.ģ. save the text below as an xml file and open in in RMĢ.

Rapidminer studio prediction