advice/help on scalability of yambo (Fermi-CINECA/PRACE)

Various technical topics such as parallelism and efficiency, netCDF problems, the Yambo code structure itself, are posted here.

Moderators: Daniele Varsano, andrea.ferretti, andrea marini, Conor Hogan, myrta gruning

advice/help on scalability of yambo (Fermi-CINECA/PRACE)

Postby perezlist2015 » Thu Feb 19, 2015 1:52 pm

hi

I am looking into the application(s) procedure(s) for CPU time via the PRACE programme, in order to submit
one to the "project access" option for GW and BSE calculations with yambo (starting from qe)

Since the "project access" description mentions the need to submit scalability data, I thought the team of developers
could be so kind as to provide the most recent scalability data; it would be greatly appreciated

I have seen somewhere a scalability test done on CINECA/Fermi that shows 1024-core as keeping linearity
(sorry but I did not keep track of the website). Will you please let me know whether I am right by assuming that
Fermi/CINECA is probably the best supercomputing center to run yambo in parallel, in light of the existing scalability information?
(the 11th PRACE project access call is open now, and they have restricted it to either CINECA/Fermi or barcelona supercomputing center/mare nostrum)
Probably, you could recommend any other technical reason that makes my assumption right or wrong?
From my point of view it would be as difficult to get the application through in either place (there does not exist any national supercomputing center in Luxembourg)

Another technical question: do you happen to know whether everyone is supposed to do his/her own scalability test via a "preparatory access" (2 month)
application, to test the code, or that is not necessary? I mean would they accept the information I could get from the developers?

thanks a lot for your understanding

regards

Manuel

Dr Manuel Pérez Jigato, Chargé de Recherche
Luxembourg Institute of Science and Technology (LIST)
Materials Research and Technology (MRT)
41 rue du Brill
L-4422 BELVAUX
Grand-Duché de Luxembourg
Tel (+352) 47 02 61 - 584
Fax (+352) 47 02 64
e-mail manuel.perez@list.lu

PS I would be very happy to discuss the possibility of a collaboration, including the ETSF via
perezlist2015
 
Posts: 11
Joined: Wed Feb 18, 2015 8:37 pm

Re: advice/help on scalability of yambo (Fermi-CINECA/PRACE)

Postby perezlist2015 » Thu Feb 19, 2015 2:09 pm

hi

I found the website I was mentioning in my previous email message, with the scalability work done at CINECA

http://www.hpc.cineca.it/news/yambo-ena ... 1024-cores

this other details is also important for me: does yambo run on 8192 cores at Fermi?

regards

Manuel

Dr Manuel Pérez Jigato, Chargé de Recherche
Luxembourg Institute of Science and Technology (LIST)
Materials Research and Technology (MRT)
41 rue du Brill
L-4422 BELVAUX
Grand-Duché de Luxembourg
Tel (+352) 47 02 61 - 584
Fax (+352) 47 02 64
e-mail manuel.perez@list.lu
perezlist2015
 
Posts: 11
Joined: Wed Feb 18, 2015 8:37 pm

Re: advice/help on scalability of yambo (Fermi-CINECA/PRACE)

Postby andrea.ferretti » Thu Feb 19, 2015 6:51 pm

Hi Manuel,

the pictures on the yambo website that you mention
http://www.hpc.cineca.it/news/yambo-ena ... 1024-cores
were obtained on a rather small system (a graphene nano ribbon with few tens of atoms) and were already showing some decent scalability up to one whole rack of Fermi BGQ (1024 nodes, 16cores/node).

More recently we have been checking the scalability further on larger systems and the situation seems even to improve, so I would say it safe to state that yambo can exploit 1024*16 cores.
This under the following conditions:
* the scalability above has been obtained by using the current development version of yambo (implementing MPI+OpenMP parallelism), which is going to be released at worst before April (in view of the yambo 2015 CECAM school)
* The OpenMP parallelism is actually very important to run on machines with little memory per core such as BGQ. At the moment we use
not more than 1 or 2 MPI tasks per node (against 16 cores/node on BGQ), plus 8-64 OpenMP threads per task (64 threads per node on BGQ is a reasonable maximum value)
* since yambo performs a lot of computation of independent qwuantities, the openmp scaling is rather good

regarding the prace application:

In general I think it is even recommended to try to use standard scaling data (in principle one should try to be somehow specific on the planned applications, but standard data for standard codes are usually well accepted)
Concerning whether to run on Fermi or MareNostrum, I think the main difference is the memory per core (which is also a very critical parameter for yambo)...

Fermi, as a BGQ, has 1 GB/core (16GB per node), while Marenostrum is based on a Intel architecture and has slim and fat nodes (2GB to 8GB/core)... I would then tentatively say that Fermi is more extreme for yambo since it surely requires the whole MPI+OpenMP support, while perhaps, if your memory needs were not too severe, MareNostrum would allow for a more classical parallelisation scheme...

hope it helps
Andrea
Andrea Ferretti, PhD
CNR-NANO-S3 and MaX Centre
via Campi 213/A, 41125, Modena, Italy
Tel: +39 059 2055322; Skype: andrea_ferretti
URL: http://www.nano.cnr.it
andrea.ferretti
 
Posts: 101
Joined: Fri Jan 31, 2014 11:13 am

Re: advice/help on scalability of yambo (Fermi-CINECA/PRACE)

Postby Daniele Varsano » Thu Feb 19, 2015 7:31 pm

Dear Manuel,
ETSF collaborations are still possible.
The call is open until 16th April, more information can be found here:
http://www.etsf.eu/news/spring_2015_cal ... e_16_april
http://www.etsf.eu/services/call_proposals

Please let us know if you intend to submit a proposal, by writing to some of us.
Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/
User avatar
Daniele Varsano
 
Posts: 2060
Joined: Tue Mar 17, 2009 2:23 pm

Re: advice/help on scalability of yambo (Fermi-CINECA/PRACE)

Postby perezlist2015 » Mon Feb 23, 2015 3:15 pm

hi Andrea,

thanks a lot for the reply, really helpful information!

will you please send to me concrete scalability data that I can use in my application, I cannot find anywhere what you call standard scalability data for yambo

in the main paper at computer physics communications "yambo: an ab-initio tool for excited state calculations", 180 (2009) 1392, by A Marini et al, there is not a single scalability plot

The web site I mentioned in my previous email (at cineca) only mentions the 1024-core information, but there are no plots

I found on another website a limited dataset/plot for scability on MareNostrum, but they only go up to 256 core, and that is not within the minimum accepted number of cores for none of the machines of the prace call
perezlist2015
 
Posts: 11
Joined: Wed Feb 18, 2015 8:37 pm

Re: advice/help on scalability of yambo (Fermi-CINECA/PRACE)

Postby perezlist2015 » Mon Feb 23, 2015 3:21 pm

hi Danielle

thanks for your kind reply

I certainly intend to submit an application to ETSF, although at the moment, I have to concentrate in preparing my "preparatory access" application to PRACE, with deadline in a week. I wil discuss the details of the project with you
(and other interested developers) as soon as I have more time to draft something

regards

Manuel
perezlist2015
 
Posts: 11
Joined: Wed Feb 18, 2015 8:37 pm

Re: advice/help on scalability of yambo (Fermi-CINECA/PRACE)

Postby Daniele Varsano » Mon Feb 23, 2015 3:34 pm

Dear Manule,
please fill your signature with your complete affiliation, it is a rule of the forum.
You can find a scalability plot in the news page of the yambo website, dated 11/12/2013. In the yambo paper there is not any plot as it is 6 year old and the scalability we are talking about is a recent feature, and the new parallelization scheme will come with the new release.
Andrea Ferretti can eventually give you more hint and information about the scalability tests.
Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/
User avatar
Daniele Varsano
 
Posts: 2060
Joined: Tue Mar 17, 2009 2:23 pm


Return to Technical Issues

Who is online

Users browsing this forum: No registered users and 1 guest

cron