The root-knot nematode Meloidogyne graminicola is an important parasitic nematode that causes huge economic losses to rice production in China. In the present study, 54 M. graminicola populations were collected from the major rice-growing areas of ten provinces in China. The cytochrome oxidase subunit I (COI) gene sequence of M. graminicola populations in the studied locations were PCR-amplified, sequenced, and evaluated for genetic diversity analysis. The total number of mutations, haplotypes (Hap) numbers, the average number of nucleotide differences (k), haplotype diversity (Hd), and nucleotide diversity (π) of mtCOI gene were 39, 15, 5.37, 0.646, and 0.00682, respectively. The significant differences in Fst value (0.593) and a low level of gene flow (0.333) were detected between the 54 M. graminicola populations. High genetic diversity was observed within each population and a small genetic distance was found between them. The phylogenetic analyses showed that 54 M. graminicola populations were divided into three large groups corresponding to the Central Region (CR), Southern China (SC) and the Yangtze Valley (YV). Hap8 was the most widely distributed and was considered to be an origin of haplotype, revealing a separate evolutionary origin in China. The high genetic differentiation of M. graminicola populations was result of variation within each of the defined geographical groups, according to an analysis of molecular variance. No significant relationship was observed between the genetic distance and geographical distance in M. graminicola populations. To the best of our knowledge, this is the first study conductedon the genetic diversity of M. graminicola infesting rice in China and provides a theoretical basis for the future management of the M. graminicola population and potential approach for increasing production of rice.