VASP 6.5.0 Data not present on GPU in rc_add

[thibautvery]

Hello,

I compiled VASP with the following environment nvidia-compilers/24.3 cuda/12.2.0 openmpi/4.1.5-cuda.
The options I included are:
- OpenACC
- OpenMP
- MKL
- python plugins

When I perform a test on a V100, I got a message that the variable c from std/mgrid.f90 rc_add line:1773 is not present on the GPU.

I attached the makefile.include and the file to reproduce the error.

Best regards

[ahampel]

Hi,

thank you for posting your question in the official VASP forum.

I tried to reproduce your problem without success. I used the VASP 6.5.0 released version and compiled your code with the nvidia compilers we have right now available: nvhpc/24.1, intel-oneapi-mkl/2023.2.0, and openmpi/4.1.6-cuda . I do not have a V100 card available, but I did run the job on two A30 cards with MPI, and OpenMP threading, without any problem:

Code: Select all

 running    2 mpi-ranks, with    8 threads/rank, on    1 nodes
 distrk:  each k-point on    2 cores,    1 groups
 distr:  one band on    1 cores,    2 groups
 Offloading initialized ...    2 GPUs detected
 vasp.6.5.0 16Dec24 (build Jan 16 2025 14:04:17) complex                         
 POSCAR found type information on POSCAR Ti
 POSCAR found :  2 types and     333 ions
 scaLAPACK will be used selectively (only on CPU)
 WARNING: type information on POSCAR and POTCAR are incompatible
 POTCAR overwrites the type information in POSCAR
 typ   2 type information:  O 
 LDA part: xc-table for (Slater+PW92), standard interpolation
 POSCAR, INCAR and KPOINTS ok, starting setup
 FFT: planning ... GRIDC
 FFT: planning ... GRID_SOFT
 FFT: planning ... GRID
 WAVECAR not read
 initial charge from wavefunction
 entering main loop
       N       E                     dE             d eps       ncg     rms          rms(c)
DAV:   1    -0.150681060769E+07   -0.15068E+07   -0.26885E+07  3326   0.836E+03
DAV:   2    -0.182111908472E+07   -0.31431E+06   -0.27999E+06  3880   0.514E+03
DAV:   3    -0.196552237558E+07   -0.14440E+06   -0.13482E+06  4008   0.408E+03
DAV:   4    -0.203824784578E+07   -0.72725E+05   -0.68104E+05  4264   0.252E+03
DAV:   5    -0.208701008822E+07   -0.48762E+05   -0.44843E+05  4200   0.191E+03    0.292E+03
RMM:   6    -0.140572163328E+07    0.68129E+06   -0.32376E+04  2178   0.913E+02    0.159E+03
RMM:   7    -0.523878390546E+06    0.88184E+06    0.53564E+04  3910   0.106E+03    0.148E+03
...

without making any adjustments to your input files. Note, that your POTCAR has more elements that the POSCAR, but this does not matter here...

The only thing I changed was to remove the new ML interface, which btw is really experimental right now, so use this with great care, the python plugins, and changed the compute capability to cc80 for the A30 cards. Can you once try to remove the MLFF and python plugin things from the build and see if this could make a difference. I attached my makefile.include just in case you spot a difference that I do not see. The gcc_toolchain things you can ignore.

Best regards,
Alex

[ahampel]

The problem is related to the python plugins. This is indeed a bug in the code currently. Please remove for the OpenACC nvidia port the -DPLUGINS option for now. I will create a known bug report on the wiki and we will fix this in an upcoming bugfix release.

Best regards,
Alex

[thibautvery]

I confirm that disabling the plugins solves the problem.

Thanks a lot.

[ahampel]

Perfect. I will leave this topic open until we have a patch in place. I will confirm he once more when the fix is ready so that you can use the plugins together with the openacc nvidia port.

Best,
Alex