memory issue in RPA calculation

Run-time issues concerning Yambo that are not covered in the above forums.

Moderators: Daniele Varsano, andrea marini, Conor Hogan, myrta gruning

memory issue in RPA calculation

Postby eariel99 » Thu Apr 26, 2012 3:06 pm

Hi,
I have a problem running for the RPA dielectric function. Apparently the system stops for a memory issue, with the last messages

..........
[WF-Oscillators/G space loader] Wfs (re)loading |################### | [099%] 45s(E) 4 <49s> [WF-Oscillators/G space loader] Wfs (re)loading |####################| [100%] 45s(E) 45s(X)
<49s> [M 12.451 Gb] Free wf_disk ( 0.312)
<22m-16s> Dipole (T): |####################| [100%] 21m-11s(E) 21m-11s(X)
<22m-16s> [M 0.208 Gb] Free WF (12.242)
<22m-16s> [WARNING] Missing non-local pseudopotential contribution
[ERROR] STOP signal received while in :[04] Optics
[ERROR]Mem All. failed. Element WF require 12.24200 [Gb]

So I guess the WF demand 12.24 Gb, but the computer has 16 Gb.

the system, in brief, is a supercell of 16(CdTeO3), totaling 80 atoms and 576 electrons. the k-point grid is 4x4x4, which is computed non self consistently by Quantum ESPRESSO. In practice they are 36 kpoins after use of inversion symmetry. The RPA calculation with yambo is fine with a 2x2x2 k-point mesh.
I had previously computed the RPA dielectric function using the epsilon utility of Quantum ESPRESSO with a 6x6x6 mesh and no symmetry. Hence, in principle I should have enough RAM to perform the calculation, although I it depends on how each code allocates the memory.

Should I just run on a bigger computer or use some keyword to optimize the memory allocation.

Please, find attached the report files and the stdout, and rhe rpa input.

Thanks a lot
You do not have the required permissions to view the files attached to this post.
Eduardo Menendez Proupin
Departamento de Química Fisica Aplicada
Facultad de Ciencias
Universidad Autónoma de Madrid
28049 Madrid, Spain
On leave from: Departamento de Fisica, Facultad de Ciencias, Universidad de Chile URL: www.gnm.cl/emenendez
eariel99
 
Posts: 10
Joined: Tue Apr 24, 2012 10:35 pm

Re: memory issue in RPA calculation

Postby andrea marini » Thu Apr 26, 2012 3:23 pm

Dear Eduardo,

we will soon release e new version of Yambo with a more aggressive memory distribution. In the meantime try playing with the bands

Code: Select all
% BndsRnXd
    1 |  430 |               # [Xd] Polarization function bands
%


and with the G-vectors used to describe the wavefunctions

Code: Select all
FFTGvecs


FFTGvecs appears when the input file is created using the -V RL option.

Andrea
Andrea MARINI
Istituto di Struttura della Materia, CNR, (Italy)
User avatar
andrea marini
 
Posts: 447
Joined: Mon Mar 16, 2009 4:27 pm

Re: memory issue in RPA calculation

Postby eariel99 » Fri Apr 27, 2012 11:50 am

Thank you Andrea for your advise. I am playing and learning. I am pleased to see that with 430 bands and less k-points or less RL vectors, the dielectric function is equal to the one obtained with the epsilon program from QE.

Playing with the parameters a few more questions had arisen. I hope they are for Yes/Not answers. I number the questions below

Playing with the number of bands. first calculation with 430 bands, following with 330 bands I find this line in the report file

[RD./02-rpa//db.dipoles]------------------------------------
5 lines below
*ERR* X band range : 1 330
but later 18 line below
X band range : 1 430
1) Between both runs (with 330 and 430 bands) I did not initialize again (yambo -i; yambo). Should I initialize again?
2) Does the "ERR" line give a warning to be considered?
2b) or does it only indicate that 330 dipole matrix elements were read from the database en the next 100 dipoles are calculated?

3) I see that using mpirun -np 2 yambo uses the double of RAM than just yambo or mpirun -np 1 yambo.
Note a line in the report file: YAMBO@nunoa x 002 CPUs * 04/27/2012 11:48. It also takes almost the same time to obtain the RPA dielectric function as the serial run.
Is it normal or a problem with my compiled binary ?

4) Can I freely run succesive calculations changing freely the number of parallel processes ?

5) <---> [WARNING] Missing non-local pseudopotential contribution
Can I include it using p2y -c ?

Thanks
Eduardo Menendez Proupin
Departamento de Química Fisica Aplicada
Facultad de Ciencias
Universidad Autónoma de Madrid
28049 Madrid, Spain
On leave from: Departamento de Fisica, Facultad de Ciencias, Universidad de Chile URL: www.gnm.cl/emenendez
eariel99
 
Posts: 10
Joined: Tue Apr 24, 2012 10:35 pm

Re: memory issue in RPA calculation

Postby andrea marini » Fri Apr 27, 2012 1:13 pm

eariel99 wrote:1) Between both runs (with 330 and 430 bands) I did not initialize again (yambo -i; yambo). Should I initialize again?


No. The intialization is done once for all. You need to redo it only if you choosed a small set of G-vectors at the beginning and you want to increase that number. If you run yambo with no option you do not need to do it again.
eariel99 wrote:2) Does the "ERR" line give a warning to be considered?


That ERR means" ooops, there are no enough bands in the database. I recalculate it again"

eariel99 wrote:2b) or does it only indicate that 330 dipole matrix elements were read from the database en the next 100 dipoles are calculated?

No. All are recalculated. This is always the case except when you are restarting a run and that database you are writing accepts restarting points.

eariel99 wrote:3) I see that using mpirun -np 2 yambo uses the double of RAM than just yambo or mpirun -np 1 yambo.
Note a line in the report file: YAMBO@nunoa x 002 CPUs * 04/27/2012 11:48. It also takes almost the same time to obtain the RPA dielectric function as the serial run.
Is it normal or a problem with my compiled binary ?


For the memory you are right. The version you are using does not distribute the memory. The ner version we will release before the summer will have memory distributio and SCALAPCK support. It should LARGELY improve the situation.

Also for the CPU time. Indeed where is the code spending most of the time ? If it is in the builidng of Xo you should hasve the total time divided by two when you run with two cpu. If it is doring X inversion there you do not gain anything unless you do not link against SCALAPACK.

eariel99 wrote:4) Can I freely run succesive calculations changing freely the number of parallel processes ?


Absolutely yes!

eariel99 wrote:5) <---> [WARNING] Missing non-local pseudopotential contribution
Can I include it using p2y -c ?


Yes. This is a relatively new feature that has been tested. Please try it and let us know. As far as I remember I added the parallel support to p2y to calculate the commutator. So try running p2y with mpirun.

eariel99 wrote:Thanks


You're welcome ;)

Andrea
Andrea MARINI
Istituto di Struttura della Materia, CNR, (Italy)
User avatar
andrea marini
 
Posts: 447
Joined: Mon Mar 16, 2009 4:27 pm

Re: memory issue in RPA calculation

Postby eariel99 » Mon Apr 30, 2012 1:10 pm

Hi,
Is it expected to have mpirun -np 2 p2y -S -c running during 3 days (still running)? Just to remind: 576 electrons, 430 bands, and 32 k-points.
Also in serial (p2y -N -S -c), with 4 k-points, it has 1.2 days running.
For fcc Si, p2y runs in 3 seconds.
Best regards
Eduardo

eariel99 wrote:5) <---> [WARNING] Missing non-local pseudopotential contribution
Can I include it using p2y -c ?

Yes. This is a relatively new feature that has been tested. Please try it and let us know. As far as I remember I added the parallel support to p2y to calculate the commutator. So try running p2y with mpirun.
Eduardo Menendez Proupin
Departamento de Química Fisica Aplicada
Facultad de Ciencias
Universidad Autónoma de Madrid
28049 Madrid, Spain
On leave from: Departamento de Fisica, Facultad de Ciencias, Universidad de Chile URL: www.gnm.cl/emenendez
eariel99
 
Posts: 10
Joined: Tue Apr 24, 2012 10:35 pm

Re: memory issue in RPA calculation

Postby andrea marini » Fri May 04, 2012 11:46 am

eariel99 wrote:Hi,
Is it expected to have mpirun -np 2 p2y -S -c running during 3 days (still running)? Just to remind: 576 electrons, 430 bands, and 32 k-points.
Also in serial (p2y -N -S -c), with 4 k-points, it has 1.2 days running.
For fcc Si, p2y runs in 3 seconds.


Uff. It could be. That piece of code is not exactly "fast". Can you test the effect of the non-local part of the pseudo using a smller number of atoms ?

Keep in mind that the effect is strictly related to the kind of atoms you have. You could test on the respective bulks or using a smaller nanostructure.

In general for Si, diamond is small. For LiF and Cu is large. It depends on the non-locality of the pseudo.

Andrea
Andrea MARINI
Istituto di Struttura della Materia, CNR, (Italy)
User avatar
andrea marini
 
Posts: 447
Joined: Mon Mar 16, 2009 4:27 pm

Re: memory issue in RPA calculation

Postby eariel99 » Wed May 09, 2012 5:57 pm

Hi Andrea,
I was out of bussiness last week and past of this one. I have just made the calculations of the dielectric function at RPA level for zincblende CdTe, 16x16x16 k-points grid (not fully converged), and I find problems with the pseudopotential commutator (p2y -c ). The "flat" curve is with the nonlocal commutator parts (p2y -c) . Without the non local parts it looks better compared with expeiments and with WIEN2k reports, even when it is
My calculations
eps2_local_vs_nonlocal.jpeg
.

Experiment
eps2_PRB32p3811y1985.jpeg


WIEN2K calculation
eps2_ChinPhysB21p017101y2012.jpeg


Any attention to the pseudopotentials? Mine are of Troullier Martins type, PBE functional, Cd with configuration 4d105s2 and nonlinear core correction, and Te in 5s25p4.

best regards
Eduardo
You do not have the required permissions to view the files attached to this post.
eariel99
 
Posts: 10
Joined: Tue Apr 24, 2012 10:35 pm

Re: memory issue in RPA calculation

Postby andrea marini » Thu May 10, 2012 8:46 pm

eariel99 wrote:Any attention to the pseudopotentials? Mine are of Troullier Martins type, PBE functional, Cd with configuration 4d105s2 and nonlinear core correction, and Te in 5s25p4.


Dear Eduardo, actually I have no idea why the commutator is doing this nasty effect. Did you try the code on a simple solid, like LiF ? Simply to be sure that you did everything ok.

Then, if you did everything correct, keep on working without it as I do not see any simple manner I can help you. But, keep in mind that PBE is going to be supported in the next version of Yambo that we will release in a matter of week. This means that the yambo you're using is most probably switching to simpler LDA-based functionals. Check the report file or/and the log of the interfaces (a2y or p2y).

If you need further informations about this point and you are interested in testing the devel version of the GPL source contact me personally.

Cheers

Andrea
Andrea MARINI
Istituto di Struttura della Materia, CNR, (Italy)
User avatar
andrea marini
 
Posts: 447
Joined: Mon Mar 16, 2009 4:27 pm

Re: memory issue in RPA calculation

Postby eariel99 » Fri May 11, 2012 4:45 pm

I ran for Silicon and the dielectric function is quite reasonable with and without the nonlocal part.
I will keep on with CdTe without the the nonlocal part.
Thanks
Eduardo
Eduardo Menendez Proupin
Departamento de Química Fisica Aplicada
Facultad de Ciencias
Universidad Autónoma de Madrid
28049 Madrid, Spain
On leave from: Departamento de Fisica, Facultad de Ciencias, Universidad de Chile URL: www.gnm.cl/emenendez
eariel99
 
Posts: 10
Joined: Tue Apr 24, 2012 10:35 pm

Re: memory issue in RPA calculation

Postby andrea marini » Fri May 11, 2012 4:49 pm

Fine. But be careful with the PBE. Check when running p2y which kind of XC functional is using Yambo. Most probably it using some LDA-like.

Andrea
Andrea MARINI
Istituto di Struttura della Materia, CNR, (Italy)
User avatar
andrea marini
 
Posts: 447
Joined: Mon Mar 16, 2009 4:27 pm

Next

Return to Other issues

Who is online

Users browsing this forum: No registered users and 1 guest

cron