Panic disorder (PD) is characterized by recurrent and unexpected panic attacks, subsequent anticipatory anxiety, and phobic avoidance. Recent epidemiological and genetic studies have revealed that genetic factors contribute to the pathogenesis of PD. We performed whole-exome sequencing on one Japanese family, including multiple patients with panic disorder, which identified seven rare protein-altering variants. We then screened these genes in a Japanese PD case-control group (384 sporadic PD patients and 571 controls), resulting in the detection of three novel single nucleotide variants as potential candidates for PD (chr15: 42631993, T>C in GANC; chr15: 42342861, G>T in PLA2G4E; chr20: 3641457, G>C in GFRA4). Statistical analyses of these three genes showed that PLA2G4E yielded the lowest p value in gene-based rare variant association tests by Efficient and Parallelizable Association Container Toolbox algorithms; however, the p value did not reach the significance threshold in the Japanese. Likewise, in a German case-control study (96 sporadic PD patients and 96 controls), PLA2G4E showed the lowest p value but again did not reach the significance threshold. In conclusion, we failed to find any significant variants or genes responsible for the development of PD. Nonetheless, our results still leave open the possibility that rare protein-altering variants in PLA2G4E contribute to the risk of PD, considering the function of this gene.