We consider the outfit grading problem for outfit recommendation, where we assume that users have a closet of items and we aim at producing a score for an arbitrary combination of items in the closet. The challenge in outfit grading is that the input to the system is a bag of item pictures that are unordered and vary in size. We build a deep neural network-based system that can take variable-length items and predict a score. We collect a large number of outfits from a popular fashion sharing website, Polyvore, and evaluate the performance of our grading system. We compare our model with a random-choice baseline. The performance of our model achieves 84% in both accuracy and precision, showing our model can reliably grade the quality of an outfit. We also built an outfit recommender on top of our grader to demonstrate the practical application of our model for a personal closet assistant.