基于图形处理单元与密度拟合近似的单精度耦合簇CCSD和CCSD(T)程序

doi:10.6023/A22070313

摘要/Abstract

作者此前工作表明, 在耦合簇CCSD (Coupled-Cluster approaches within the singles and doubles approximation)与CCSD(T) (CCSD approaches augmented by a perturbative treatment of triple excitations)计算中结合单精度数与消费型图形处理单元(GPU), 可以显著提高计算速度. 然而由于CCSD(T)计算对内存的巨大需求以及消费型GPU的内存限制, 在利用消费型GPU进行加速时, 不考虑利用空间对称性的情况下, 此前开发的CCSD(T)程序仅能用于计算300~400个基函数的体系. 利用密度拟合(Density-Fitting, DF)处理双电子积分可以显著降低CCSD(T)计算过程中的内存需求, 本工作发展了基于密度拟合近似并结合单精度数进行运算的DF-CCSD(T)程序, 该程序可用于包含700个基函数的无对称性体系的单点能计算, 以及包含1700个基函数的有对称性体系. 本工作所使用的计算节点配置了型号为Intel I9-10900k的CPU和型号为RTX3090的GPU, 与用双精度数在CPU上的计算相比, 利用单精度数结合GPU进行运算可以将CCSD的计算速度提升16倍, (T)部分可提升40倍左右, 而使用单精度数引入的误差可忽略不计. 在程序开发过程中, 作者发展了一套可利用GPU或CPU结合单精度数或双精度数进行含空间对称性的矩阵操作代码库. 基于该套代码库, 可以显著降低开发含空间对称性的耦合簇代码的难度.

关键词: 耦合簇, 密度拟合, 图形处理单元(GPU), 单精度

It has been reported by our group that using single-precision data and consumer graphics processing units (GPUs) can significantly improve computation speed of CCSD (Coupled-Cluster approaches within the singles and doubles approximation) and CCSD(T) (CCSD approaches augmented by a perturbative treatment of triple excitations). However, CCSD(T) can only be employed for small molecules with about 300~400 basis functions when using consumer GPUs for acceleration without spatial symmetry due to the memory limitation of GPU. Using density-fitting approximation can significantly reduce the memory requirements in CCSD(T) calculations. In this paper, DF-CCSD(T) codes based on the density fitting approximation together with single precision data was developed. All the matrix contractions were performed employing GEMM in CUBLAS on GPU or in Intel MKL on CPU. The other operations such as matrix expansion and transpose were performed using OpenACC on GPU or OpenMP on CPU. Those codes can be applied to single point energies for systems with around 700 basis functions without spatial symmetry and to molecules with about 1700 basis functions with symmetry on a GPU with 24 Gb memory. The server employed in this work has an Intel I9-10900k CPU and a RTX3090 GPU. CCSD calculations with single-precision data on GPU are about 16 times faster and it is about 40 times faster for the (T) part compared with the calculations on CPU using double precision data on this server. Error introduced by single precision data is negligible. A code library that can employ GPU or CPU using either single precision or double precision data to perform matrix operations with spatial symmetry was also reported in this work. Direct product decomposition (DPD) method was employed to deal with spatial symmetry. Complexity of developing coupled cluster codes with spatial symmetry can be significantly reduced with this library. The computational accuracy of single precision DF-CCSD(T) was compared with CCSD(T)-F12a and DLPNO-CCSD(T). The results shown that DF-CCSD(T) would be more stable than the other two approaches in describing chemical properties.

Key words: coupled-cluster, density fitting, graphics processing units (GPU), single precision

引用此文

王治钒, 何冰, 路艳朝, 王繁. 基于图形处理单元与密度拟合近似的单精度耦合簇CCSD和CCSD(T)程序[J]. 化学学报, 2022, 80(10): 1401-1409.

Zhifan Wang, Bing He, Yanzhao Lu, Fan Wang. Single-precision CCSD and CCSD(T) Calculations with Density Fitting Approximations on Graphics Processing Units[J]. Acta Chimica Sinica, 2022, 80(10): 1401-1409.

导出引用 EndNote|Reference Manager|ProCite|BibTeX|RefWorks

分享此文

图/表 7

参考文献 26

[1]	Bartlett, R. J.; Musiał, M. Rev. Mod. Phys. 2007, 79, 291. doi: 10.1103/RevModPhys.79.291
[2]	Raghavachari, K.; Trucks, G. W.; Pople, J. A.; Head-Gordon, M. Chem. Phys. Lett. 1989, 157, 479. doi: 10.1016/S0009-2614(89)87395-6
[3]	Řezáč, J.; Hobza, P. J. Chem. Theor. Comput. 2013, 9, 2151. doi: 10.1021/ct400057w
[4]	(a) Vahtras, O.; Almlöf, J.; Feyereisen, M. W. Chem. Phys. Lett. 1993, 213, 514. doi: 10.1016/0009-2614(93)89151-7
	(b) Van Alsenoy, C. J. Comput. Chem. 1988, 9, 620. doi: 10.1002/jcc.540090607
[5]	(a) Kats, D.; Korona, T.; Schütz, M. J. Chem. Phys. 2007, 127, 064107. doi: 10.1063/1.2755778
	(b) Korona, T.; Jeziorski, B. J. Chem. Phys. 2008, 128, 144107. doi: 10.1063/1.2889006
	(c) Boström, J.; Pitoňák, M.; Aquilante, F.; Neogrády, P.; Pedersen, T. B.; Lindh, R. J. Chem. Theor. Comput. 2012, 8, 1921. doi: 10.1021/ct3003018
	(d) DePrince, A. E.; Sherrill, C. D. J. Chem. Theor. Comput. 2013, 9, 2687. doi: 10.1021/ct400250u
	(e) Epifanovsky, E.; Zuev, D.; Feng, X.; Khistyaev, K.; Shao, Y.; Krylov, A. I. J. Chem. Phys. 2013, 139, 134105. doi: 10.1063/1.4820484
	(f) DePrince, A. E.; Kennedy, M. R.; Sumpter, B. G.; Sherrill, C. D. Mol. Phys. 2014, 112, 844. doi: 10.1080/00268976.2013.874599
	(g) Lesiuk, M. J. Chem. Phys. 2020, 152, 044104. doi: 10.1063/1.5129883
	(h) Bozkaya, U.; Sherrill, C. D. J. Chem. Phys. 2017, 147, 044104. doi: 10.1063/1.4994918
	(i) Shen, T.; Zhu, Z.; Zhang, I. Y.; Scheffler, M. J. Chem. Theor. Comput. 2019, 15, 4721. doi: 10.1021/acs.jctc.8b01294
[6]	(a) Harvey, M. J.; De Fabritiis, G. WIREs: Comput. Mol. Sci. 2012, 2, 734. doi: 10.1002/wcms.1101
	(b) Walker, R. C.; Götz, A. W. Electronic Structure Calculations on Graphics Processing Units. (Wiley Online Library, 2016).
	(c) Bao, J. Z.; Feng, X. T.; Yu, J. G. Acta Phys. Chim. Sin. 2011, 27, 2019. (in Chinese) doi: 10.3866/PKU.WHXB20110830
	(鲍建樟, 丰鑫田, 于建国, 物理化学学报, 2011, 27, 2019.)
[7]	(a) Stopper, D.; Roth, R. J. Chem. Phys. 2017, 147, 064508. doi: 10.1063/1.4997636
	(b) Nitsche, M. A.; Ferreria, M.; Mocskos, E. E.; Gonzalez Lebrero, M. C. J. Chem. Theor. Comput. 2014, 10, 959. doi: 10.1021/ct400308n
	(c) Jia, W.; Fu, J.; Cao, Z.; Wang, L.; Chi, X.; Gao, W.; Wange, L. W. J. Comput. Phys. 2013, 251, 102.
	(d) Genovese, L.; Ospici, M.; Deutsch, T.; Mehaut, J. F.; Neelov, A.; Goedecker, S. J. Chem. Phys. 2009, 131, 034103. doi: 10.1063/1.3166140
[8]	(a) Tornai, G. J.; Ladjanszki, I.; Rak, A.; Kis, G.; Cserey, G. J. Chem. Theor. Comput. 2019, 15, 5319. doi: 10.1021/acs.jctc.9b00560
	(b) Rak, A.; Cserey, G. Chem. Phys. Lett. 2015, 622, 92. doi: 10.1016/j.cplett.2015.01.023
	(c) Asadchev, A.; Gordon, M. S. J. Chem. Theor. Comput. 2012, 8, 4166. doi: 10.1021/ct300526w
	(d) Wang, Y.; Tian, Y. Q.; Jin, Z.; Suo, B. B. Acta Chim. Sinica 2021, 79, 653. (in Chinese) doi: 10.6023/A21020044
	(王岩, 田英齐, 金钟, 索兵兵, 化学学报, 2021, 79, 653.) doi: 10.6023/A21020044
[9]	(a) Bykov, D.; Kjaergaard, T. J. Comput. Chem. 2017, 38, 228. doi: 10.1002/jcc.24678
	(b) Angel Martinez-Martinez, L.; Amador-Bedolla, C. J. Mex. Chem. Soc. 2017, 61, 60.
	(c) Song, C.; Martinez, T. J. J. Chem. Phys. 2016, 144, 174111. doi: 10.1063/1.4948438
	(d) Maurer, S. A.; Kussmann, J.; Ochsenfeld, C. J. Chem. Phys. 2014, 141, 051106. doi: 10.1063/1.4891797
[10]	Fales, B. S.; Levine, B. G. J. Chem. Theor. Comput. 2015, 11, 4708. doi: 10.1021/acs.jctc.5b00634
[11]	Fales, B. S.; Curtis, E. R.; Johnson, K. G.; Lahana, D.; Seritan, S.; Wang, Y.; Weir, H.; Martínez, T. J.; Hohenstein, E. G. J. Chem. Theor. Comput. 2020, 16, 2021. doi: 10.1021/acs.jctc.9b01257
[12]	Knizia, G.; Li, W.; Simon, S.; Werner, H. J. J. Chem. Theor. Comput. 2011, 7, 2387. doi: 10.1021/ct200239p
[13]	Pokhilko, P.; Epifanovsky, E.; Krylov, A. I. J. Chem. Theor. Comput. 2018, 14, 4088. doi: 10.1021/acs.jctc.8b00321
[14]	Wang, Z.; Guo, M.; Wang, F. Phys. Chem. Chem. Phys. 2020, 22, 25103. doi: 10.1039/D0CP03800H
[15]	Knizia, G.; Adler, T. B.; Werner, H.-J. J. Chem. Phys. 2009, 130, 054104. doi: 10.1063/1.3054300
[16]	(a) Guo, Y.; Riplinger, C.; Becker, U.; Liakos, D. G.; Minenkov, Y.; Cavallo, L.; Neese, F. J. Chem. Phys. 2018, 148, 011101. doi: 10.1063/1.5011798
	(b) Liakos, D. G.; Guo, Y.; Neese, F. J. Phys. Chem. A 2020, 124, 90. doi: 10.1021/acs.jpca.9b05734
[17]	Gauss, J.; Stanton, J. F.; Bartlett, R. J. J. Chem. Phys. 1991, 95, 2623.
[18]	(a) Sun, Q.; Berkelbach, T. C.; Blunt, N. S.; Booth, G. H.; Guo, S.; Li, Z.; Liu, J.; McClain, J. D.; Sayfutyarova, E. R.; Sharma, S.; Wouters, S.; Chan, G. K.-L. WIREs Comput. Mol. Sci. 2018, 8, 1340.
	(b) Sun, Q.; Zhang, X.; Banerjee, S.; Bao, P.; Barbry, M.; Blunt, N. S.; Bogdanov, N. A.; Booth, G. H.; Chen, J.; Cui, Z.-H.; Eriksen, J. J.; Gao, Y.; Guo, S.; Hermann, J.; Hermes, M. R.; Koh, K.; Koval, P.; Lehtola, S.; Li, Z.; Liu, J.; Mardirossian, N.; McClain, J. D.; Motta, M.; Mussard, B.; Pham, H. Q.; Pulkin, A.; Purwanto, W.; Robinson, P. J.; Ronca, E.; Sayfutyarova, E. R.; Scheurer, M.; Schurkus, H. F.; Smith, J. E. T.; Sun, C.; Sun, S.-N.; Upadhyay, S.; Wagner, L. K.; Wang, X.; White, A.; Whitfield, J. D.; Williamson, M. J.; Wouters, S.; Yang, J.; Yu, J. M.; Zhu, T.; Berkelbach, T. C.; Sharma, S.; Sokolov, A. Y.; Chan, G. K.-L. J. Chem. Phys. 2020, 153, 024109. doi: 10.1063/5.0006074
[19]	Dunning, T. H. J. Chem. Phys. 1989, 90, 1007. doi: 10.1063/1.456153
[20]	Weigend, F. Phys. Chem. Chem. Phys. 2002, 4, 4285. doi: 10.1039/b204199p
[21]	(a) Peterson, K. A.; Figgen, D.; Goll, E.; Stoll, H.; Dolg, M. J. Chem. Phys. 2003, 119, 11113. pmid: 17181347
	(b) Peterson, K. A.; Shepler, B. C.; Figgen, D.; Stoll, H. J. Phys. Chem. A 2006, 110, 13877. pmid: 17181347
[22]	Figgen, D.; Rauhut, G.; Dolg, M.; Stoll, H. Chem. Phys. 2005, 311, 227. doi: 10.1016/j.chemphys.2004.10.005
[23]	Stoychev, G. L.; Auer, A. A.; Neese, F. J. Chem. Theor. Comput. 2017, 13, 554. doi: 10.1021/acs.jctc.6b01041
[24]	Peverati, R.; Truhlar, D. G. Philos. T. R. Soc. A 2014, 372, 20120476.
[25]	Werner, H.-J.; Knowles, P. J.; Manby, F. R.; Black, J. A.; Doll, K.; Heßelmann, A.; Kats, D.; Köhn, A.; Korona, T.; Kreplin, D. A.; Ma, Q.; Miller, T. F.; Mitrushchenkov, A.; Peterson, K. A.; Polyak, I.; Rauhut, G.; Sibaev, M. J. Chem. Phys. 2020, 152, 144107. doi: 10.1063/5.0005081
[26]	Neese, F. WIREs Comput. Mol. Sci. 2012, 2, 73. doi: 10.1002/wcms.81

	CCSD			(T)
Molecular	SP	SP	DP	SP	SP	DP
	GPU	CPU	CPU	GPU	CPU	CPU
(H₂O)₇	–1.9878328	–1.9878329	–1.9878329	–0.0585549	–0.0585566	–0.0585566
C₆H₁₄	–1.1398594	–1.1398593	–1.1398594	–0.0440448	–0.0440449	–0.0440449
C₁₀H₁₆	–1.7801039	–1.7801038	–1.7801038	–0.0798821	–0.0798823	–0.0798823
AtF₅	–2.2048644	–2.2048637	–2.2048637	–0.0941050	–0.0940953	–0.0941050
I₃^-	–1.3032696	–1.3032697	–1.3032697	–0.0498108	–0.0498108	–0.0498108

	CCSD			(T)
Molecular	SP	SP	DP	SP	SP	DP
	GPU	CPU	CPU	GPU	CPU	CPU
(H₂O)₇	–1.9878328	–1.9878329	–1.9878329	–0.0585549	–0.0585566	–0.0585566
C₆H₁₄	–1.1398594	–1.1398593	–1.1398594	–0.0440448	–0.0440449	–0.0440449
C₁₀H₁₆	–1.7801039	–1.7801038	–1.7801038	–0.0798821	–0.0798823	–0.0798823
AtF₅	–2.2048644	–2.2048637	–2.2048637	–0.0941050	–0.0940953	–0.0941050
I₃^-	–1.3032696	–1.3032697	–1.3032697	–0.0498108	–0.0498108	–0.0498108

		Wall-time of a single RI-CCSD iteration				Wall-time of (T) correction calculation
		SP	SP	SP	DP	SP	SP	SP	DP
n	Number of basis	GPU Time	CPU Time	$\frac{\text{CPUTime}}{\text{GPUTime}}$	CPU Time	GPU Time	CPU Time	$\frac{\text{CPUTime}}{\text{GPUTime}}$	CPU Time
3	174	0.9 (0.4)	5.46 (2.24)	6.07 (5.6)	12.4 (5.2)	2.8 (1.8)	22.4 (13.9)	8 (7.7)	40 (33)
4	232	3.1 (1.4)	17.7 (9.3)	5.71 (6.64)	37.2 (19.0)	14 (6.7)	336 (105)	24 (15.7)	672 (244)
5	290	7.8 (3.7)	50.9 (24.6)	6.53 (6.65)	106 (52.1)	62 (23)	1514 (492)	24.4 (21.4)	2698 (1035)
6	348	18 (8)	129 (58)	7.17 (7.25)	243 (122)	189 (69)	4866 (1555)	25.7 (22.5)	8476 (3174)
7	406	35 (14)	283 (111)	8.09 (7.93)	523 (226)	542 (179)	12588 (4359)	23.2 (24.4)	22312 (8399)
8	464	66 (28)	508 (220)	7.7 (7.86)	1012 (436)	1514 (445)	31404 (11527)	20.7 (25.9)	53604 (19188)
9	522	131 (44)	913 (389)	6.97 (8.84)	1815 (774)	3601 (898)	65888 (22203)	18.3 (24.7)	113226 (40485)
10	580	529 (81)	1768 (656)	3.34 (8.1)	(1626)	7608 (2313)	119307 (46606)	15.7 (20.2)	(84115)

		Wall-time of a single RI-CCSD iteration				Wall-time of (T) correction calculation
		SP	SP	SP	DP	SP	SP	SP	DP
n	Number of basis	GPU Time	CPU Time	$\frac{\text{CPUTime}}{\text{GPUTime}}$	CPU Time	GPU Time	CPU Time	$\frac{\text{CPUTime}}{\text{GPUTime}}$	CPU Time
3	174	0.9 (0.4)	5.46 (2.24)	6.07 (5.6)	12.4 (5.2)	2.8 (1.8)	22.4 (13.9)	8 (7.7)	40 (33)
4	232	3.1 (1.4)	17.7 (9.3)	5.71 (6.64)	37.2 (19.0)	14 (6.7)	336 (105)	24 (15.7)	672 (244)
5	290	7.8 (3.7)	50.9 (24.6)	6.53 (6.65)	106 (52.1)	62 (23)	1514 (492)	24.4 (21.4)	2698 (1035)
6	348	18 (8)	129 (58)	7.17 (7.25)	243 (122)	189 (69)	4866 (1555)	25.7 (22.5)	8476 (3174)
7	406	35 (14)	283 (111)	8.09 (7.93)	523 (226)	542 (179)	12588 (4359)	23.2 (24.4)	22312 (8399)
8	464	66 (28)	508 (220)	7.7 (7.86)	1012 (436)	1514 (445)	31404 (11527)	20.7 (25.9)	53604 (19188)
9	522	131 (44)	913 (389)	6.97 (8.84)	1815 (774)	3601 (898)	65888 (22203)	18.3 (24.7)	113226 (40485)
10	580	529 (81)	1768 (656)	3.34 (8.1)	(1626)	7608 (2313)	119307 (46606)	15.7 (20.2)	(84115)

	(H₂O)₉		(H₂O)₁₀
	GPU	CPU	GPU	CPU
1) Initio	15.6	15.3	233.7	235
2) N⁵	60	53.6	84.5	87.1
3) Iabci	3.5	22	7	37.6
4) Iabcd	37.7	623	66	1050
5) Wmbej	10.79	169.9	320.2	319.2
6) left	2.5	18.9	3.5	30