make in HPC environment fails with 'echo$ notfound'

Having trouble compiling the Yambo source? Using an unusual architecture? Problems with the "configure" script? This is the place to look.

Moderators: Daniele Varsano, andrea.ferretti, andrea marini, Conor Hogan, myrta gruning

Forum rules
If you have trouble compiling Yambo, please make sure to list:
(1) the compiler (vendor and release: e.g. intel 10.1)
(2) the architecture (e.g. 64-bit IBM SP5)
(3) if the problems occur compiling in serial/in parallel
(4) the version of Yambo (revision number/major release version)
(5) the relevant compiler error message

make in HPC environment fails with 'echo$ notfound'

Postby gshamov » Fri Nov 16, 2018 6:41 pm

Hi,

I am trying to build Yambo as part of QE 6.3. It works perfectly on an older HPC environment (CentOS 6.9, Intel compiler suite, OpenMPI 1.6). The make there is OS make 3.81.

However, on a newer HPC machine where we have some very recent userland from Nix (Make version 4.2.1) building Yambo fails for any compiler, be that Intel 16 or GCC 5.4.0.

Code: Select all
>>>[Linking yambo]<<<
make[3]: Entering directory '/home/gshamov/devel/qe/qe-6.3/yambo-stable/driver'
/bin/sh: echo$: command not found
make[3]: *** [Makefile:85: yambo] Error 127
make[3]: Leaving directory '/home/gshamov/devel/qe/qe-6.3/yambo-stable/driver'


I have seen already an answer on this forum, which appears to be related to the same problem : viewtopic.php?f=1&t=1396 . In fact, I believe this is one of our users, based on the paths there. Now the answer suggests forcing using Intel icc as the preprocessor. It doesn't explain why pure GCC build fails when there is no Intel in the paths.

The offending line seem to be the call to $(driver) function that has teh @echo at the end.

Code: Select all
$(target): $(dep_file) $(objs)
        $(driver)$
        $(link)$
        $(modmove)$
        $(dircheck)$
        @mv $@ $(exec_prefix)

Then,

Code: Select all
#
# Functions
#
define driver
 @if test -f $(prefix)/driver/yambo.h; then $(cpp) $(cppflags) $(dopts)                     $(prefix)/driver/yambo.h > $(prefix)/driver/yambo_cpp.h; fi
 @if test -f $(prefix)/driver/ypp.h; then $(cpp) $(cppflags) $(dopts)                    $(prefix)/driver/ypp.h > $(prefix)/driver/ypp_cpp.h; fi
 @( eval $(cc) $(cflags) -c $(prefix)/driver/getopt.c > /dev/null)
 @( eval $(cc) $(cflags) $(dopts) -D_$@ -c $(prefix)/driver/driver.c > /dev/null)
 @rm -f $(prefix)/driver/yambo_cpp.h $(prefix)/driver/ypp_cpp.h
 @echo
endef

For some reason, the new make results in following expansion (from make -n in the driver directory):

Code: Select all
[gshamov@ driver]$[i] make -n[/i]
if test -f /home/gshamov/devel/qe/qe-6.3/YAMBO/driver/yambo.h; then gcc -E -P  -D_MPI -D_FFTW -D_SCALAPACK -D_TIMING -D_linux                     /home/gshamov/devel/qe/qe-6.3/YAMBO/driver/yambo.h > /home/gshamov/devel/qe/qe-6.3/YAMBO/driver/yambo_cpp.h; fi
if test -f /home/gshamov/devel/qe/qe-6.3/YAMBO/driver/ypp.h; then gcc -E -P  -D_MPI -D_FFTW -D_SCALAPACK -D_TIMING -D_linux                    /home/gshamov/devel/qe/qe-6.3/YAMBO/driver/ypp.h > /home/gshamov/devel/qe/qe-6.3/YAMBO/driver/ypp_cpp.h; fi
( eval mpicc -g -O2 -D_C_US -D_FORTRAN_US  -c /home/gshamov/devel/qe/qe-6.3/YAMBO/driver/getopt.c > /dev/null)
( eval mpicc -g -O2 -D_C_US -D_FORTRAN_US  -D_MPI -D_FFTW -D_SCALAPACK -D_TIMING -D_linux -D_yambo -c /home/gshamov/devel/qe/qe-6.3/YAMBO/driver/driver.c > /dev/null)
rm -f /home/gshamov/devel/qe/qe-6.3/YAMBO/driver/yambo_cpp.h /home/gshamov/devel/qe/qe-6.3/YAMBO/driver/ypp_cpp.h
[b]echo$[/b]


See the DOLLAR? On CentOS6 it expands to the harmless and expected echo with no dollars.

I am at loss as to what to do about this issue. Could anyone please advise? Thank you very much in advance!

--
Grigory Shamov
WestGrid Site Lead / HPC Specialist
University of Manitoba
E2-588 EITC Building,
(204) 474-9625
gshamov
 
Posts: 2
Joined: Fri Nov 16, 2018 5:34 pm

Re: make in HPC environment fails with 'echo$ notfound'

Postby andrea.ferretti » Sat Nov 17, 2018 10:12 pm

Dear Grigory,

thanks for writing.
The problem seems related to some fragility of the make syntax used.
Have you tried any plain workaround like adding an empty line at the end of the definition of driver ?

Code: Select all
#
# Functions
#
define driver
 @if test -f $(prefix)/driver/yambo.h; then $(cpp) $(cppflags) $(dopts)                     $(prefix)/driver/yambo.h > $(prefix)/driver/yambo_cpp.h; fi
 @if test -f $(prefix)/driver/ypp.h; then $(cpp) $(cppflags) $(dopts)                    $(prefix)/driver/ypp.h > $(prefix)/driver/ypp_cpp.h; fi
 @( eval $(cc) $(cflags) -c $(prefix)/driver/getopt.c > /dev/null)
 @( eval $(cc) $(cflags) $(dopts) -D_$@ -c $(prefix)/driver/driver.c > /dev/null)
 @rm -f $(prefix)/driver/yambo_cpp.h $(prefix)/driver/ypp_cpp.h
 @echo

endef


In general, if you have to modify makefiles, you better edit ./sbin/make_makefiles.sh, which regenerates all makefiles (except the one in the yambo home dir).
This happens if makefiles are not there or if the __lock_xxx file is missing (I think).
In turn, ./sbin/make_makefiles.sh is generated by ./configure, starting from the ./sbin/make_makefiles.sh.in template.

take care
Andrea
Andrea Ferretti, PhD
CNR-NANO-S3 and MaX Centre
via Campi 213/A, 41125, Modena, Italy
Tel: +39 059 2055322; Skype: andrea_ferretti
URL: http://www.nano.cnr.it
andrea.ferretti
 
Posts: 102
Joined: Fri Jan 31, 2014 11:13 am

Re: make in HPC environment fails with 'echo$ notfound'

Postby gshamov » Mon Nov 19, 2018 11:50 pm

Hi Andrea,

Adding an empty line after @echo in the definition of driver was tried by me. It gives the standalone $ which fails just as well.

Code: Select all
[gshamov@ driver]$ make

$
/bin/sh: $: command not found
gshamov
 
Posts: 2
Joined: Fri Nov 16, 2018 5:34 pm

Re: make in HPC environment fails with 'echo$ notfound'

Postby andrea.ferretti » Wed Nov 21, 2018 2:51 pm

Dear Grigory,

with the help of a colleague I tried to sort out why dollars were added to the end of those lines in the Makefile, and didn't come to a clear reason.
Dropping them, as from the patch below, works in our case. Does it solve the problem in your case ?

take care
Andrea

Code: Select all
diff --git a/sbin/make_makefile.sh.in b/sbin/make_makefile.sh.in
index ef9c9ba..9207204 100755
--- a/sbin/make_makefile.sh.in
+++ b/sbin/make_makefile.sh.in
@@ -118,17 +118,17 @@ rm -f $cdir/$ofile.c
 if [ "$mode" = "x" ] ; then
 cat << EOF >> $cdir/Makefile
 \$(target): \$(dep_file) \$(objs)
-   \$(driver)\$
-   \$(link)\$
-   \$(modmove)\$
-   \$(dircheck)\$
+   \$(driver)
+   \$(link)
+   \$(modmove)
+   \$(dircheck)
    @mv \$@ \$(exec_prefix)
 EOF
 else
 rm -f ./lib/$target
 cat << EOF >> $cdir/Makefile
 \$(target): \$(dep_file) arcreate
-   \$(modmove)\$
+   \$(modmove)
 EOF
 fi
Andrea Ferretti, PhD
CNR-NANO-S3 and MaX Centre
via Campi 213/A, 41125, Modena, Italy
Tel: +39 059 2055322; Skype: andrea_ferretti
URL: http://www.nano.cnr.it
andrea.ferretti
 
Posts: 102
Joined: Fri Jan 31, 2014 11:13 am


Return to Compilation

Who is online

Users browsing this forum: No registered users and 0 guests

cron