In maximum penalized likelihood estimation, approaches of cross-validation (CV) are often useful in selecting a smoothing parameter. The CV score based on squared-error criterion behaves more badly than the likelihood-based score. However, it is expensive to calculate the likelihood-based score. Hence we propose a method for simple calculation of this score. The simple calculation is derived as an analogue of the deletion lemma in ordinary or penalized least squares, and is shown to be related to the one-step approximation to the estimates of parameters for the Newton-Raphson method. Our method is applied to binary data from some case studies in the context of logistic regression. It is illustrated that the simple calculation method well behaves and gives a good approximation to the likelihood-based score calculated by the delete-one method,