How to calculate RAM for BSE

Deals with issues related to computation of optical spectra, in RPA (-o c) or by solving the Bethe-Salpeter equation (-o b). Includes local field effects, excitons, etc.

Moderators: Davide Sangalli, Daniele Varsano, andrea.ferretti, andrea marini, Conor Hogan, myrta gruning

haseebphysics1
Posts: 145
Joined: Sat Aug 17, 2019 2:48 pm

How to calculate RAM for BSE

Post by haseebphysics1 » Thu Jun 18, 2020 1:57 am

Dear Yamboers,

For the following parameters, what's the formula to calculate the ram/cpu for the BSE? I remember, this was answered somewhere in the forum but I could not find one. The dimension comes from Nk x NV x NC = dim, right?
# BSK|Identifier :5843
# |Dimension :155520
# |Bands :153 - 170
Now should I square dim, and then multiply by 16 and then further divide by 1024 thrice to get ans back in GBs? was that in this way?
If yes, then why should I square the "dim" matrix?

Regards,
Haseeb Ahmad
MS - Physics,
LUMS - Pakistan

User avatar
claudio
Posts: 284
Joined: Tue Mar 31, 2009 11:33 pm
Contact:

Re: How to calculate RAM for BSE

Post by claudio » Thu Jun 18, 2020 8:06 am

Dear Haseeb

if I'm correct the amount of memory you need should be:

Memory (Gb) = 155520 x 155520 x 4 byte / 1024^3 = 90 Gb

I considered that you compiled yambo in single precision, therefore
a single-precision floating number is large 4 byte (32 bits) and a Gigabiyte is 1024^3 bytes.

https://en.wikipedia.org/wiki/Single-pr ... int_format

https://en.wikipedia.org/wiki/Gigabyte

In general, from personal experience, you can diagonalize a BSE matrix of maximum 35.000 size,
while with iterative diagonalization you can arrive up to 130.000 or something more.

best
Claudio
Claudio Attaccalite
[CNRS/ Aix-Marseille Université/ CINaM laborarory / TSN department
Campus de Luminy – Case 913
13288 MARSEILLE Cedex 09
web site: http://www.attaccalite.com
Freely download scientific books from: http://www.freescience.info

User avatar
Daniele Varsano
Posts: 2605
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: How to calculate RAM for BSE

Post by Daniele Varsano » Thu Jun 18, 2020 8:29 am

Dear Haseeb,
then why should I square the "dim" matrix?
These are the number of elements of a squared matrix of dimension "dim".
a single-precision floating number is large 4 byte

This is a complex matrix so I think that there is a factor 2.

Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

haseebphysics1
Posts: 145
Joined: Sat Aug 17, 2019 2:48 pm

Re: How to calculate RAM for BSE

Post by haseebphysics1 » Sat Jun 20, 2020 12:05 am

Dear claudio and Daniele, thanks for the replies!
a single-precision floating number is large 4 byte (32 bits)
1: So, can you please suggest, how I can check my installation type? 32 or 64 bits? And for the same size of matrix 64-bit installation takes more memory?

2: In BSE, is it better to parallelize over k instead of eh or t to save memory, right?

3: And, in one of the calculations, when I changed the number of bands in BSE, the BSEkernel did not seem to be calculated again (I saw the log file), although in the output file dimension was updated! But I was worried that has it computed the right results or not? I removed all the ndb.BS files in the SAVE before starting the new calculation.



Regards,
You do not have the required permissions to view the files attached to this post.
Haseeb Ahmad
MS - Physics,
LUMS - Pakistan

User avatar
Daniele Varsano
Posts: 2605
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: How to calculate RAM for BSE

Post by Daniele Varsano » Sat Jun 20, 2020 5:38 am

Dear Haseeb,
1. the yambo compilation by default is in single-precision unless you do not include the "--enable-dp " option in the configure.
2. You have many k points so parallelizing over k should be the best strategy
3. Your calculation seems successful, the kernel has been written to the disk:

Code: Select all

[WR./BSEBnds_153-170_low_range_0-3//ndb.BS_Q1_CPU_0]
Actually I do not know why there is not the estimation time of the BSE kernel construction in the log files, anyway, the code spent more than 9h in the kernel building:

Code: Select all

X T_space Kernel :     9h23m
Note that you are using very low parameters in the kernel:

Code: Select all

  |RL vectors    [exchange]: 183
|RL vectors [correlation]:  75
In general, note you are dealing with a massive calculation (160 occupied bands and 976 k points), such calculation needs many resources. I do not know exactly what system you are dealing with (some complex bismuth tungsten oxide) , but maybe you can inspect if it is not possible to reduce the unit cell and if there are symmetries (in the present setup you have no symmetry) that for some reason are not taken into account by QE, this will help a lot.

Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

haseebphysics1
Posts: 145
Joined: Sat Aug 17, 2019 2:48 pm

Re: How to calculate RAM for BSE

Post by haseebphysics1 » Mon Jun 22, 2020 6:31 pm

Dear Daniele, thanks for this useful insight.
In general, note you are dealing with a massive calculation (160 occupied bands and 976 k points), such calculation needs many resources. I do not know exactly what system you are dealing with (some complex bismuth tungsten oxide) , but maybe you can inspect if it is not possible to reduce the unit cell and if there are symmetries (in the present setup you have no symmetry) that for some reason are not taken into account by QE, this will help a lot.
However, I want to ask that those 976 k-points are for the IBZ, while full BZ comprises 1944 (18 x 6 x 18 k-mesh). I calculated the dimension of the BSEmatrix with k-points of the full BZ k-points and which is also coming out to be consistent with the dimension mentioned in the report file!

Now, the only thing to reduce the size of the memory is to lower the bands or k-mesh size. Right? How looking for some symmetry will help me here?

Thanks,
Haseeb Ahmad
MS - Physics,
LUMS - Pakistan

User avatar
claudio
Posts: 284
Joined: Tue Mar 31, 2009 11:33 pm
Contact:

Re: How to calculate RAM for BSE

Post by claudio » Tue Jun 23, 2020 8:04 am

Dear Haseeb,

unfortunately, symmetries do not help with the size.

The size of the BSE matrix depends by the number of k-points in the BZ,
while symmetries help in the construction of the BSE matrix elements.

best
Claudio
Claudio Attaccalite
[CNRS/ Aix-Marseille Université/ CINaM laborarory / TSN department
Campus de Luminy – Case 913
13288 MARSEILLE Cedex 09
web site: http://www.attaccalite.com
Freely download scientific books from: http://www.freescience.info

User avatar
jasonhan0710
Posts: 2
Joined: Wed Dec 23, 2020 6:48 am
Location: China

Re: How to calculate RAM for BSE

Post by jasonhan0710 » Wed Dec 23, 2020 9:14 am

Hi Daniele,

Does the memory usage means the memory size allocated for each core? For example, if yambo reports that

Code: Select all

    BSK|Identifier              : 2358
       |Dimension               :  53248
       |Bands                   :  75   95
which means I need 21.125GB (53248^2*8/1024^3) memory. If my cluster has 192GB RAM and 64 cores, should I reduce the number of cores in BSE calculation?

Thank you!

Best,
Jason
Jason Han

Department of Physics
National University of Defense Technology
Hunan, China

User avatar
Daniele Varsano
Posts: 2605
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: How to calculate RAM for BSE

Post by Daniele Varsano » Wed Dec 23, 2020 1:57 pm

Dear Jason,

yes, you should use fewer cores per node.

Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

Fadil
Posts: 7
Joined: Sun May 24, 2020 1:56 pm

Re: How to calculate RAM for BSE

Post by Fadil » Thu Dec 24, 2020 11:50 am

Dear all,

My bse calculation terminates without any error in the 'diago solver' section.
According to the Ram calculation below,;
28224^2*8/1024^3 = 6GB
Approximately 6 GB of RAM is required per core.

There are 256 GB RAM on my system, I am using 16 cores.
You can find the report file in the attachment.
Do you have any idea what is the problem?
You do not have the required permissions to view the files attached to this post.
Dr. Fadıl İYİKANAT
ICFO-The Institute of Photonic Sciences
Barcelona-SPAIN

Post Reply