In the paper, we present a real-time lipreading LSI for recognizing spoken words from lip movement. The LSI recognizes up to 8 words based on the Hidden Markov Model (HMM). The LSI accepts the 256 x 256 8-bit gray-scale images from a camera, and outputs the 3-bit symbol code of words for 43 images (corresponding to 1.53 sec). We present a lipreading algorithms optimized for hardware implementation. We have designed the lipreading LSI and fabricated a 4.9 mm x 4.9 mm chip using 0.35 mum process via VDEC Rohm. The LSI performs real-time recognition at 40 MHz operation.