rev 16: LFS support absent on CINECA sp6 !!

Various technical topics such as parallelism and efficiency, netCDF problems, the Yambo code structure itself, are posted here.

Moderators: Daniele Varsano, andrea.ferretti, andrea marini, Conor Hogan, myrta gruning

rev 16: LFS support absent on CINECA sp6 !!

Postby marco.govoni » Wed Nov 17, 2010 9:23 am

Dear all,
I changed topic because now we are working with rev 16, but the problem are the same posted in the previous topic and concerned rev 14 http://www.yambo-code.org/forum/viewtopic.php?f=10&t=262&start=0 .

Just to remind you:
I am trying to run a simulation where the production of a database (in my case ndb.kindx) is larger that 2Gb (in my case ~6Gb). This is called, according with the netcdf standard, a Large File. In order to store this file you have to carefully set Large File Support (LFS) in the netcdf calls.

I have found that:
running on CINECA sp6 YAMBO WORKS FINE ONLY WITHOUT NETCDF. Without linking them is the only way to let Yambo correctly store the Large Files and end the run without errors on CINECA sp6.

The problems now can only be:
1) Netcdf on sp6 CINECA are bad.
2) Yambo has a bug that stops the run when a large file has to be written using netcdf.

As a lot of people, besides the YAMBO community, has been using CINECA netcdf modules for their applications and, moreover, because I tried to linked Yambo with almost any flavours of hand-installed netcdf 4 and 3.6 without removing the error: I support the second point, but I would not bet money on it.

If you cannot reproduce the error on local machines, maybe it is because the data to be contained in the output large file has to be stored in ram before being printed and maybe you need Hpc facilities for this.
In order to focus the problem, of course you shall reproduce the error on architectures other than CINECA sp6. If you have access to other clusters able to handle almost 6 or 7 Gb of ram I can give you with no problems the inputs of my test (Andrea already has). They should run just on 1 cpu (just because the portion of the code connected to my test is serial) and take almost 4 h to be accomplished.

Let me know.

Marco
Marco Govoni
Physics Department, University of Modena and Reggio Emilia (Italy)
marco.govoni
 
Posts: 35
Joined: Thu May 21, 2009 3:46 pm

Re: rev 16: LFS support absent on CINECA sp6 !!

Postby faffinito » Wed Nov 17, 2010 2:34 pm

Hi Marco,
- I succesfully compiled yambo linking netcdf, so I would suppose your problem is runtime, isn't it?
- I suppose the problem is not in the NetCDF itself because, as you report, there many users that link NetCDF without reporting this kind of problems.
I just would like to add a consideration to your problem.
It would be extremely useful to have some instruction from the developers about what and how to link external libraries. For instance, when you compile NetCDF you need also HDF5. HDF5, in turn, need zlib and szlib. Each of these libraries can be compiled with a lot of option: single or double precision, static or not, with or without fortran support, in parallel or serial way. You can see that the number of combinations become huge and try to guess which is appropriate to Yambo is almost impossible without the support from the developers team. Maybe there is some instruction somewhere, but I didn't find it...
Let me know, please.

Fabio
Fabio Affinito, PhD
CINECA - SCAI group
InterUniversity Computer Center
Via Magnanelli, 6/3
Casalecchio di Reno (Bologna) ITALY
+39/051/6171794 (Phone)
faffinito
 
Posts: 9
Joined: Tue Aug 31, 2010 9:55 am

Re: rev 16: LFS support absent on CINECA sp6 !!

Postby feffeficus » Wed Nov 17, 2010 3:08 pm

On my side, I can suggest also the people with errors to attach their compilation option used on sp6 in terms of: modules loaded and configure options used. Like that we can try to have an historical of all the compilation flavours. what do you say?
Federico Iori - LSI - Ecole Polytechnique (Palaiseau) France - http://www.etsf.polytechnique.fr
feffeficus
 
Posts: 31
Joined: Tue Apr 07, 2009 4:39 pm

Re: rev 16: LFS support absent on CINECA sp6 !!

Postby myrta gruning » Wed Nov 17, 2010 3:16 pm

faffinito wrote:. For instance, when you compile NetCDF you need also HDF5.


Hallo Fabio,

HDF5 is not needed by Yambo. At present Yambo does not use any of the hdf5 featues.
Also in case Yambo will implement the use of NETCDF+HDF5 we will left the option of having NETCDF without HDF5.

Each of these libraries can be compiled with a lot of option: single or double precision, static or not, with or without fortran support, in parallel or serial way. ou can see that the number of combinations become huge and try to guess which is appropriate to Yambo is almost impossible without the support from the developers team. Maybe there is some instruction somewhere, but I didn't find it...


I do not think that the options for libraries linked to a library linked to NETCDF are influenced by yambo.
The standard options to compile those libs should be fine. If there is some special requirements may be because of netcdf, not because of yambo. Also I do not think that e.g. serial or parallel, static or dynamic compilation should in principle matter.
What we know is that is crucial to use the same compiler as for yambo. Having e.g. netcdf compiled with gcc/gfortran and yambo with icc/ifort will not work.
Unfortunately it is true that netcdf is becoming more and more complicated, and changing rapidily and linking more and more things. Note that all new features of netcdf4, that usually need to link to other libraries, are not needed by Yambo.
Please note we are not great experts in netcdf and related libraries, we are using the library and most of the time learning by experience as other users. Our experience is limited to some machine/compilers/version of netcdf. Through this forum one can in principle access to the experiences of other users that may have had and solved the same problem.

Regards,
Myrta
Dr Myrta GrĂ¼ning
Atomistic Simulation Centre
Queen's University Belfast - Northern Ireland

http://www.researcherid.com/rid/B-1515-2009
User avatar
myrta gruning
 
Posts: 303
Joined: Tue Mar 17, 2009 11:38 am

Re: rev 16: LFS support absent on CINECA sp6 !!

Postby faffinito » Wed Nov 17, 2010 5:38 pm

Hi Myrta,
following your suggestions, I decided to restart from scratch.

First I compiled NetCDF4.0.1 (there's a bug affecting 4.1.1, and I didn't want to waste time on that) with:

Code: Select all
    export CFLAGS="-g -O2 -q64 -qarch=pwr6 -qtune=pwr6 -qmaxmem=-1 -qsuppress=1501-245"
    export CXXFLAGS=$CFLAGS
    export FFLAGS=$CFLAGS
    export FCFLAGS=$CFLAGS
    ./configure --prefix=/gpfs/scratch/userinternal/cin0644a/netcdf4 \


Then I simply installed Yambo 3.2.3 r.696 with:

Code: Select all
        module load blacs
        module load scalapack
        export CPP=cpp
        export CC=xlc_r
        export F77=xlf_r
        export FC=xlf90_r
        export FCFLAGS='-O2 -q64 -qstrict -qarch=pwr6 -qtune=pwr6 -qmaxmem=-1 -qautodbl=dbl4 -qsuffix=f=f -qrealsize'
        ./configure --prefix=/sp6/userinternal/cin0644a/yambo \
                --with-fftw=/cineca/prod/libraries/fftw/3.2.2/xl--10.1/lib \
                --with-netcdf-lib=/gpfs/scratch/userinternal/cin0644a/netcdf4/lib \
                --with-netcdf-include=/gpfs/scratch/userinternal/cin0644a/netcdf4/lib/include \
                --with-p2y=4.0 \
                --with-iotk=/cineca/prod/build/applications/QuantumESPRESSO/4.1/xl--10.1/BA_WORK/QuantumESPRESSO-4.1/iotk  \
                --build=powerpc-ibm \
                --enable-netcdf-LFS


obtaining this error in compilation :

Code: Select all
"mod_itm.f", line 437.34: 1513-062 (S) Generic procedure reference can not be resolved due to incorrect actual argument attributes.
"mod_itm.f", line 471.34: 1513-062 (S) Generic procedure reference can not be resolved due to incorrect actual argument attributes.
"mod_itm.f", line 536.21: 1513-062 (S) Generic procedure reference can not be resolved due to incorrect actual argument attributes.
"mod_itm.f", line 551.21: 1513-062 (S) Generic procedure reference can not be resolved due to incorrect actual argument attributes.
"mod_itm.f", line 566.20: 1513-062 (S) Generic procedure reference can not be resolved due to incorrect actual argument attributes


Any suggestion about that?

Fabio
Fabio Affinito, PhD
CINECA - SCAI group
InterUniversity Computer Center
Via Magnanelli, 6/3
Casalecchio di Reno (Bologna) ITALY
+39/051/6171794 (Phone)
faffinito
 
Posts: 9
Joined: Tue Aug 31, 2010 9:55 am

Re: rev 16: LFS support absent on CINECA sp6 !!

Postby andrea marini » Wed Nov 17, 2010 5:43 pm

Fabio, anyway you can provide us access on SP ?It is really hard for me to understand where the error is without having access to the source itself.

Andrea
Andrea MARINI
Istituto di Struttura della Materia, CNR, (Italy)
User avatar
andrea marini
 
Posts: 447
Joined: Mon Mar 16, 2009 4:27 pm

Re: rev 16: LFS support absent on CINECA sp6 !!

Postby feffeficus » Fri Nov 26, 2010 5:33 pm

PROBLEM FIXED:

report on
Code: Select all
yambo_3.2.3_rev16  compiled linking Netcdf  and  --enable-netcdf-LFS=no (default)



the error previously reported

Code: Select all
[ERROR] STOP signal received while in :[09] Dynamic Dielectric Matrix (PPA)
[ERROR][NetCDF] NetCDF: One or more variable sizes violate format constraints


has been fixed for the moment.
The database ndb.pp that yambo was creating during the plasmon pole calculation was larger than 2Gb.

With Daniele support, I've launch yambo -S (with fragmentation option) for the plasmon pole calculation.
It works. In the end I obtain a ndb.pp database fragmented in n-part (less than 2Gb) as the number of q-point I have in my calculation.
So, I can say that this error I got was due to database larger than 2 Gb.

Code: Select all
SOLUTION: run with yambo -S

Problem fixed for the moment.

Thanks to all.
Federico Iori - LSI - Ecole Polytechnique (Palaiseau) France - http://www.etsf.polytechnique.fr
feffeficus
 
Posts: 31
Joined: Tue Apr 07, 2009 4:39 pm


Return to Technical Issues

Who is online

Users browsing this forum: No registered users and 1 guest

cron