Multi-Objective Programming in SVMs

Multi-Objective Programming in SVMs

Jinbo Bi

Department of Mathematical Sciences
Rensselaer Polytechnic Institute

Abstract. We propose a general framework for support vector machines (SVM) based on the principle of multi-objective optimization. The learning of SVMs is formulated as a multi-objective program (MOP) by setting two competing goals to minimize the empirical risk and the model capacity. Distinct approaches to solving the MOP introduce various SVM formulations. The proposed framework enables a more effective minimization of the VC bound on the generalization risk. We develop a feature selection approach based on the MOP framework and demonstrate its effectiveness on hand-written digit data.

  • This paper has been accepted by the 20th International Conference on Machine Learning, 2003.
  • The RSVM package written in C++:
  • The MatLab codes for the feature selection experiments:

    The MatLab codes basically use the RSVM package to optimize the first step of MOPFS Algorithm 1 (see paper), and use AMPL commands which call MINOS 5.5 (a commercial optimization software) to optimize the second step of MOPFS Algorithm 1.

Contact Jinbo Bi (jinbo@engr.uconn.edu) for information about this page.