Infrared nonlinear (IR NLO) optical crystals have an essential position in military and civilian fields because of their ability to convert lasers from near infrared (NIR) to mid/far infrared (MIR/FIR). In this work, two alkali-metal chalcogenides, Rb₂MGe₃S₈ [M=Zn (1), Cd (2)], were successfully synthesized by high-temperature solid-state reactions. Both compounds feature a two-dimensional layered structure and have a large optical band-gap, the experimental band-gap of 1 and 2 are 3.24 eV and 3.16 eV, respectively. Compound 1 belongs to the centrosymmetric group P-1, while 2 belongs to the non-centrosymmetric space group P2(1)2(1)2(1) and exhibits obvious NLO effect, which is comparable to that of KH₂PO₄ (KDP) (@1064 nm) at the particle size of 50~75 μm. Particle-size dependent NLO response measurements indicated that 2 is non-phase-matchable. Compound 2 exhibits a high laser-induced damage threshold of 16.6×AGS at 1064 nm. Through the analysis of the crystal structures of these two compounds, the reason why their formulas have the same stoichiometric ratio but symmetries are different is the structure change of basic building unit [MGe₃S₈]^2– in 1 and 2. All M atoms in both compounds are coordinated by four S atoms to form MS₄ tetrahedra. In each [CdGe₃S₈]^2– unit of 2, three S atoms bonded to the Cd atom are also bonded to all Ge atoms in that unit, that is to say, each CdS₄ tetrahedron is connected to the other three GeS₄ tetrahedra by sharing S vertices. Unlike the coordination manner in the [CdGe₃S₈]^2– unit of 2, there are only two S atoms bonded to both Zn and Ge atoms in [ZnGe₃S₈]^2– unit of 1. This structure change of [MGe₃S₈]^2– unit eventually led to the non-centrosymmetric transformation. What’s more, to get insight into the origin of NLO effect of 2, theoretical calculations of electronic band structure and NLO susceptibility were performed based on density functional theory.