When you develop a Machine Learning Technique, you need to know how better is your solution compared with other solutions.
There are a lot of methods, but the most used are Fold-Cross Validation and Bootstrap. Both are commonly used in classifiers system. In my thesis "Rule Induction Using Ants", we find not knowing with of two techniques we would use.
- Bootstrap has low variance, but large bias in some problems.
- K -Fold Cross Validation with moderate values (10-20), reduce the variance but increase bias.
- Using Stratified strategy is better in terms of variance and bias, comparated with Regular Cross Validation.
That's all folks, I wish this Post could help someone.