How to record everythinig to a log file in Win7? [SOLVED]

[ljludwig]

Dear ALL:

When using Abinit 7.0.5 under 64 bit Windows 7, the way to run abinit is like :

..\..\bin\mpiexec -localonly 2 ..\..\bin\abinit < "t_kpt+spin.files"

(Actually all of them can be added to Environment Variable "PATH")

However, this makes logging on the screen.

Is there a way to log everything into a single log file, as it is in linux?

#abinit <ab.files>& log

I tried this: abinit < ab.files > log 2>&1

It works well when doing serial running, but failed to do mpiexec.

Any help, doc, and suggestion will be greatly appreciated. Thank you very much!!



Solution:

1st step, enter CMD in Administrator mode (right click CMD, "Run as Administrator"). Go to the abinit folder, abinit\bin, run smpd as: smpd -install
Then a new process called smpd will be in the "Processes" of Task Manager.

2nd step: mpiexec -localonly 4 abinit < ab.files > log 2>&1
is good enough to do the work. Here, mpiexec and abinit are added to the PATH environment Variable, so no need to write the full path.

In any case, it will work.

[Alain_Jacques]

I have very little experience with Windows specifics but I know that PowerShell captures the output of commands in string variables you can dump in files as for example

$log = mpirun.exe ...
$log | OutFile log.txt

I also had (mixed) success with the plain command prompt with cmd /c mpirun.exe ... `>log.txt 2`>`&1 (mixed because some programs use STDOUT and/or STDERR)

Last but not least, cygwin environment comes with a bash shell so the Unix way should work.

Kind regards,

Alain

[ljludwig]

I have very little experience with Windows specifics but I know that PowerShell captures the output of commands in string variables you can dump in files as for example

$log = mpirun.exe ...
$log | OutFile log.txt

I also had (mixed) success with the plain command prompt with cmd /c mpirun.exe ... `>log.txt 2`>`&1 (mixed because some programs use STDOUT and/or STDERR)

Last but not least, cygwin environment comes with a bash shell so the Unix way should work.

Kind regards,

Alain

Hi, Alain! Thank you so much for your answer. It helps a lot. The reason I turn to win7 64 bit version is that, the cygwin itself is 32 bit platform, which means on Cygwin, Abinit cannot be built 53 bit. Thus, when doing larger calculation, it always tells me "Fails to Allocate Memory" problem: even if I personally bought 32 Gb memory, it still cannot allocate 5Gb to one core (4 core in total).

I am not quite sure about the "real" reason, but
1) Input file is good, simply increasing ecut will lead to this memory problem.
2) Even a cluster, where I compiled abinit without using 64 bit compiling, it still has memory problem.

The windows 64bit version surprisingly surpasses all kind of problem, and can work eventually.

Any suggestions about 64 bit compiling? Shall I just add "--with-64-bit-flag"?

Many thanks!

[jbeuken]

Hi,

The windows 64bit version surprisingly surpasses all kind of problem, and can work eventually.
Any suggestions about 64 bit compiling? Shall I just add "--with-64-bit-flag"?

sorry, but I don't understand your question...

- have you use the win 64bits binary version available in the abinit website ?
- have you need help to compile a new version under windows 7 ?

my recipe is based on x86_64-w64-mingw32-gfortran.exe under cygwin ( but running under windows 2003 server ( aka XP 64bits) )

jmb

[Alain_Jacques]

Two different questions here:

You need a 64bit version of Abinit to use more than 4GB of memory. The --64-bit-flag configure option is the key to build a 64bit Abinit (on any kind of operating system). For Windows, it's a bit more challenging - you have to install a mingw C/Fortran compiler toolchain, combine it with MPICH2, etc ... - so it is advisable to use the ready to launch Windows binaries provided on Abinit website at http://ftp.abinit.org/abinit-7.0.5_i686_cygwin_gnu4.5.zip. OK you have to look at the source distribution page to find the Windows binaries

When you have a 64bit Abinit application, you can run it under cygwin bash shell (which is a 32bit environment) to benefit from the Unix stdin/stdout/stderr, pipes, ... There is no interaction between the two architectures apart here from reading/writing streams of data. What cannot be done is to use the cygwin 32bit compilers/libraries to build a 64bit Abinit executable (actually because cygwin doesn't provide 32bit/64bits fat libraries for cross-compiling)

Kind regards,

Alain

[ljludwig]

Hi,

The windows 64bit version surprisingly surpasses all kind of problem, and can work eventually.
Any suggestions about 64 bit compiling? Shall I just add "--with-64-bit-flag"?

sorry, but I don't understand your question...

- have you use the win 64bits binary version available in the abinit website ?
- have you need help to compile a new version under windows 7 ?

my recipe is based on x86_64-w64-mingw32-gfortran.exe under cygwin ( but running under windows 2003 server ( aka XP 64bits) )

jmb

Dear Prof. JMB:

Thanks a lot for your response. Actually, the problem is two-fold:

1) One is found to be the limitation caused by 32 bit compiling of Abinit, which limits the available memory to 4Gb.
2) The other limit is found to be the 32Gb physical memory limit, in both the cluster my personal computer.

Therefore, in order to do calculation with converged K-mesh and E-cut (even for 2-atom in one u.c. for dielectric function calculation), I have to use mkmem and mffmem.

This simple problem struggles me for days, hopefully it is now clear for future reference, what might cause the "Failed to Allocate Memory" Problem.

Cheers!

[ljludwig]

Two different questions here:

You need a 64bit version of Abinit to use more than 4GB of memory. The --64-bit-flag configure option is the key to build a 64bit Abinit (on any kind of operating system). For Windows, it's a bit more challenging - you have to install a mingw C/Fortran compiler toolchain, combine it with MPICH2, etc ... - so it is advisable to use the ready to launch Windows binaries provided on Abinit website at http://ftp.abinit.org/abinit-7.0.5_i686_cygwin_gnu4.5.zip. OK you have to look at the source distribution page to find the Windows binaries

When you have a 64bit Abinit application, you can run it under cygwin bash shell (which is a 32bit environment) to benefit from the Unix stdin/stdout/stderr, pipes, ... There is no interaction between the two architectures apart here from reading/writing streams of data. What cannot be done is to use the cygwin 32bit compilers/libraries to build a 64bit Abinit executable (actually because cygwin doesn't provide 32bit/64bits fat libraries for cross-compiling)

Kind regards,

Alain

Dear Alain:

Thank you so much for this clarification, it explains all the confusions I had in the previous few days. Please allow me restate and organize your answer
for the references of other people and future convenience.

1) Why meet "Failed to Allocate 2Gb memory" problem in Cygwin, even if I bought amount of memory of 16Gb.
This is because Cygwin is 32bit, and Abinit is compiled in 32 bit, without the -64-bit-flag compilation.
In this case, even if I first bought 16Gb memory, the total memory in use= 2Gb*4cores=8Gb > 32bit limit=4Gb.

2) Same input file and PsP, 64bit Win7 package can run it which cannot be run in Cygwin.
The win7 package run in cmd uses 64 but built, so does not have the above problem. As you mentioned, the limit can be 196Gb for win7 Professional, etc.

3) When K-mesh and Ecut continuously increase, finally in the Cluster and my PC, it fails to run again.
This is because the Cluster has an upper limit of 32Gb for a user like me, and my PC has 32Gb memory (one week ago I only have 4Gb...).
So physically it limits the maximum memory, and eventually leads to the "Failed to Allocate >8Gb memory" if 4 cores are used.

If want to deal with the problem, and the memory usage, my current method is add "mkmem" and "mffmem". Now no GW calculation yet. Hopefully there are more ways to reduce memory in sacrifice of speed.

Several things worthwhile mention using Win7 Abinit package:

1) In order to save time typing the full path of abinit, in win7 the folder \abinit\bin can be added to Environment Variable.
Right click Computer-->Properties --> Advanced System Settings-->Environment Variables--> Find "path", and add the full path of abinit\bin, separated by ";"
Then restart cmd, and type echo %path%, the new path should be seen.

2) Emacs has a windows version, too: http://ftp.gnu.org/pub/gnu/emacs/windows/

3) Install UnxUtils: http://sourceforge.net/projects/unxutils/ Then commands like grep, ls will be installed; add the installed folder to "path" Environment Variable, then
people can use cmd just as linux.

4) Run Abinit in Parallel in cmd:
enter cmd in Administrator mode, run smpd, which is located at \abinit\bin, run like : smpd -install, then the windows Task Manager will have additional process "smpd"
then enter: mpiexec -localonly 4 abinit < ab.files > log 2>&1


Thank you so much, Alain!! I obtained quite a lot help from you, so I hope organizing the solutions will help others.