Too many sub-directories

Various technical topics such as parallelism and efficiency, netCDF problems, the Yambo code structure itself, are posted here.

Moderators: Daniele Varsano, andrea.ferretti, andrea marini, Conor Hogan, myrta gruning

Too many sub-directories

Postby romz » Thu Jan 21, 2010 10:14 am

Hello,

I installed yambo version 3.2.1 revision 448 under linux with ext3 file system.
It works fine and yambo passed all tests.

However, the ext3 file system has a limit on the number of sub-directories.
The limit is about 32000 sub-directories. When I perform the BSE calculation for GaN with
hexagonal cell with many k-point, this limit can be exceeded.

My questions are following:
1. Is it possible to avoid creation of so many sub-directiories in SAVE directory?
2. What kind of file system do you use for yambo?
3. What do you suggest?


Best regards,
Zbigniew Romanowski

Institute of High Pressure Physics
Polish Academy of Sciences
Warsaw, Poland
romz
 
Posts: 1
Joined: Wed Jan 20, 2010 11:52 am

Re: Too many sub-directories

Postby andrea marini » Thu Jan 21, 2010 10:28 am

Hi,

the number of blocks the BS Hamiltonian is dived decides the number of subfolders. The number of blocks is given by the number of k-points you have in the whole BZ. How many k-points do you have ?

To avoid the fragmentation you can edit yourself the src/io/ioBS.F routine, and in particular, the lines

Code: Select all
 if (write_is_on(ID)) then
   db_is_large=BS_K_dim>6000
   if (BS_K_coupling) db_is_large=BS_K_dim>3000
   if (db_is_large) then
     dbs_fragment=.true.
     io_fragmented(ID)=.true.
   endif
 endif


If dbs_fragment=.TRUE. then the blocks are spread in subdirectories to speed up the I/O. You can force by hand the logical to be false. HOWEVER if your matrix is huge the I/O in the solver will be really slowed down.

Another solution is to replace subfolders with files. In this case you need to do a little bit more of coding. PLease first try the other option and let me know.

Andrea
Andrea MARINI
Istituto di Struttura della Materia, CNR, (Italy)
User avatar
andrea marini
 
Posts: 447
Joined: Mon Mar 16, 2009 4:27 pm

Re: Too many sub-directories

Postby andrea marini » Fri Jan 22, 2010 10:10 am

andrea marini wrote:Another solution is to replace subfolders with files. In this case you need to do a little bit more of coding. PLease first try the other option and let me know.


If you can solve your problem by using files instead of folders for the fragments of the BS Hamiltonian let me know as I already wrote a small patch to do this. I can post it if you need.
Andrea MARINI
Istituto di Struttura della Materia, CNR, (Italy)
User avatar
andrea marini
 
Posts: 447
Joined: Mon Mar 16, 2009 4:27 pm

Re: Too many sub-directories

Postby silvanabotti » Tue Jun 01, 2010 3:10 pm

Dear Andrea,
I think we have exactly the same problem (ext3 file system) with the maximum number of sub-directories.
In the last post you say you have a patch to replace directories with files. Can you post it????

ciao,
Silvana
silvanabotti
 
Posts: 3
Joined: Fri Jun 26, 2009 10:34 am

Re: Too many sub-directories

Postby andrea marini » Tue Jun 01, 2010 8:39 pm

Dear Silvana,

the patch has been included in one of the latest revisions in the SVN repository. Can you try to download it ? Instructions are given in the download page.

Ciao

Andrea
Andrea MARINI
Istituto di Struttura della Materia, CNR, (Italy)
User avatar
andrea marini
 
Posts: 447
Joined: Mon Mar 16, 2009 4:27 pm


Return to Technical Issues

Who is online

Users browsing this forum: No registered users and 1 guest