Clustering techniques, such as hierarchical clustering, k-means algorithm and self-organizing maps, are widely used to analyze gene expression data. Results of these algorithms depend on several parameters, e.g., the number of clusters. However, there is no theoretical criterion to determine such parameters'. In order to overcome this problem, we propose a method using mixture of PCA models trained by a variational Bayes (VB) estimation. In our method, good clustering results are selected based on the free energy obtained within the VB estimation. Furthermore, by taking an ensemble of estimation results, a robust clustering is achieved without any biological knowledge. Our method is applied to a clustering problem for gene expression data during a sporulation of Bacillus subtilis and it is able to capture characteristics of the sigma cascade.