Resource Management during Screening

[JEJohns]

I'm trying to do a Convergence on the energy cutoff for wavefunctions during a screening calculation. I've attached a successful input file below. My question is about the memory usage of Abinit during a screening calculation. I'm running these calculations on the Quest machine at Northwestern, which consists of groups of 72 nodes, each node having 8 processors and 48 GB of ram. The memory usage during the calculation, is highly (and it seems unnecessarily) front loaded. The initial memory usage increases linearly until it writes out the screening for the first several q-points. After it writes out the first several q-points, memory usage then drops drastically (in this case to about a 5th of total levels), stays at these levels throughout the calculation, and writes out to the screening file (*_o_SCR) after every q-point. I used a process monitor to capture the memory usage for 4 hours and plotted it below. If the memory usage during the initial ramp up in memory exceeds the 48 GB/node that I have access to, the program terminates. In order to get this job to process then, I needed to use multiple nodes with less than 8 processes per node (4 / node in this case) in order to give each process extra available memory. But I don't understand why this is necessary since after the first write out, the memory usage stays quite low. Results are similar using gwpara=1 or gwpara=2, though I am currently collecting the equivalent data with a resource monitor for gwpara=2 now. Is there a workaround for this?
--James Johns

[gmatteo]

I think that the peak in the memory you are observing is mainly due to
the allocation of an extra array needed for evaluating the optical limit of the head
and of the wings of the polarizability at q==0

See
http://www.abinit.org/documentation/hel ... ml#inclvkb

The default value for inclvkb is 1 although the corresponding algorithm is very slow and memory-demanding.
Could you rerun the same test with inclvkb 2 and then post the new graph? Thanks

Matteo

[JEJohns]

I'll do that asap. Unfortunately, all nodes are down & reserved for maintenance, so I'll do it as soon as they come back up and the queue clears (Probably thurs or friday).

[JEJohns]

Ok, so I ran the same input file, but with inclvkb=2 as you suggested. This used way less memory per node. Below is a graph of the memory usage. I read the documentation for inclvkb, but only 0 & 1 have any documentation, inclvkb=2 is mentioned, but I have no idea from the input variable help how it differs from 1. Am I missing important information in my screening?
--James

[gmatteo]

Dear James,

The two different algorithms corresponding to inclvkb 1 and 2 are equivalent, they just correspond to
a different formulation of the same problem.

When inclvkb==2 the commutator F(r1,r2) = [Vnl(r1,r2),r2] is rewritten in reciprocal space in a fully separable
form so that the storage of the huge two-dimensional matrix F(G1,G2) is not needed.
On the contrary Inclvkb==1 requires the entire F(G1,G2) matrix hence it is much more memory demanding and
much slower for large cutoff energies.

Calculations done with inclvkb 1 or 2 are exact in the sense that the head and the wings of the polarizability
in the optical limit are evaluated taking all the different contributions to the commutator [H,r] into account.

The heads and wings obtained with inclvkb=0 are not "exact" since the term [Vnl, r] is not evaluated.
Neglecting [Vnl, r] should not introduce significant errors in the GW results for bulk systems but might lead
to important errors in optical properties or in GW calculations done in isolated systems in which the weight of the
gamma point is important.

Matteo

[JEJohns]

Thanks so much Matteo. I don't think I would have ever figured that out without your help & clarification.