Calculation of forces in QMC

General discussion of the Cambridge quantum Monte Carlo code CASINO; how to install and setup; how to use it; what it does; applications.
Vladimir_Konjkov
Posts: 170
Joined: Wed Apr 15, 2015 3:14 pm

Calculation of forces in QMC

Post by Vladimir_Konjkov »

I found an interesting article of Sandro Sorella and Luca Capriotti http://www.luca-capriotti.net/pdfs/Phys ... 234111.pdf on the
calculation of an arbitrarily large number of energy derivatives in QMC by using adjoint algorithmic differentiation.

Whether implemented this algorithm in CASINO? If not do you plan to implement this in future versions?

Best wishes, Vladimir.
In Soviet Russia Casino plays you.
Mike Towler
Posts: 239
Joined: Thu May 30, 2013 11:03 pm
Location: Florence
Contact:

Re: Calculation of forces in QMC

Post by Mike Towler »

Hi Vladimir,

As you've presumably gathered from the paper, algorithmic differentiation is a way of efficiently evaluating derivatives of a function by repeatedly applying the chain rule to the lines of computer code which define the function. This works in principle since any computed function can be decomposed into a sequential list of basic arithmetic operations, each of which is differentiable. In practice, the list is extremely long, and when the programmer realizes that a code like CASINO has hundreds of thousands of lines of code he will almost certainly throw up his hands in despair and go home and do something less boring instead.. The technique also presumably has the effect of making the source code practically unreadable, so this is probably a good idea..

Matthew Foulkes had a student (Tom Poole - who I understand has since left academia) that spent some time on this; he used his own simplified DMC code to play with rather than CASINO (and with good reason). I think they had some success with it, but I don't think anything was ever published (?) and for some reason I don't even have the slides for the couple of talks he gave us on the subject (one in the Electronic Structure Discussion Group I organize(d) in Cambridge, and one at the last of my annual QMC conferences in Italy) for you to look at. I will email Matthew to see if he has anything further to add.

I've never used Sandro's code so I don't know how general it is.

CASINO is able to compute forces using approximate analytic derivatives under certain restricted circumstances (Gaussian basis set with pseudopotentials only) but to my knowledge hardly anyone has used this facility since it was introduced. Search for the four papers by Badinski on http://vallico.net/casinoqmc/our-qmc-papers for more details.

CASINO can also do a restricted kind of 'molecular dynamics' by coupling with a DFT program (PWSCF) and using the DFT forces to move the atoms.

None of this is ideal, and we would like to do better for forces, but there have always been other priorities and not enough dedicated developers. The many 'developer' people listed on this page:

http://vallico.net/casinoqmc/things-to-do/

don't really mean it.. Feel free to volunteer! Didn't you describe yourself as a 'professional programmer' in your last post? We love those. ;)

Cheers,
Mike
Vladimir_Konjkov
Posts: 170
Joined: Wed Apr 15, 2015 3:14 pm

Re: Calculation of forces in QMC

Post by Vladimir_Konjkov »

Hello, Mike, I finally read the manual's "Atomic forces" p.35.
Tell me, how difficult to implement second derivatives of energy in CASINO?

This would be very helpful in the following situations:
1. Hessians calculation for TS (transition state) search.
2. Calculation of vibrational frequencies in harmonic approximation and also ZPVE (zero point vibrational energy).

As I understand, it should also work in the case of multireference wfn, that will give CASINO advantages since the implementation
of analytic second derivatives for the multireference coupled cluster methods are very few.

Unfortunately, since I only available obfuscated source code, I can not see even the implementation of the first derivatives calculation.

With great respect, Vladimir.
In Soviet Russia Casino plays you.
Mike Towler
Posts: 239
Joined: Thu May 30, 2013 11:03 pm
Location: Florence
Contact:

Re: Calculation of forces in QMC

Post by Mike Towler »

Hi Vladimir,
Unfortunately, since I only available obfuscated source code, I can not see even the implementation of the first derivatives calculation.
Sorry, I forgot to flick the switch to give developer access for your CASINO login ID - you should have told me before! Now done. Let me know if you have any problems accessing the developer version.

I'll answer your other question tonight. Busy right now..

Cheers,
Mike
Matthew_Foulkes
Posts: 3
Joined: Tue Jan 14, 2014 10:47 pm
Location: Imperial College, London
Contact:

Re: Calculation of forces in QMC

Post by Matthew_Foulkes »

Hi Vladimir,

Mike is right that Tom Poole, Peter Haynes, James Spencer and I tried algorithmically differentiating a small QMC code written by Tom himself. We did not attempt to differentiate the whole of CASINO for the reasons Mike explained; the work required would have been excessive and it would have made CASINO very hard to maintain.

Sandro Sorella's work is very impressive and there is a version of his code, TurboRVB, with AD built in. As far as I am aware, however, Sandro only uses AD to calculate VMC forces, not DMC forces, and only algorithmically differentiates that part of the code concerned with evaluating the local energy. Both of these simplifications reduce the effort required, but optimizing VMC wavefunctions well enough to make them good enough to yield reliable forces is not an easy job.

Tom's work was more ambitious, in that he developed a way of differentiating a whole DMC code (this was previously thought to be impossible; it is not even obvious how to define the derivative of a code that make decisions based on random numbers). He got pretty good results and was able to calculate the DMC forces on all of the atoms in hydrocarbons of moderate size (we went up to C_4 H_10) in a total time only an order of magnitude longer than required for a single DMC simulation. The statistical accuracy was pretty good, too --- comparable to the differences between forces calculated using different DFT functionals. Furthermore, because he used the reverse mode of AD, the time taken to calculate all of the forces divided by the time taken to calculate the total energy was roughly independent of the number of atoms (and hence the number of forces) required. Tom used his AD formulation of DMC to carry a quantum molecular dynamics simulation of the dimerisation of borane and the formation of the so-called banana bonds using DMC forces. This work has been written up and submitted but I would rather not circulate the paper until it has been accepted. Sadly, unless someone is able to pay a team of several postdocs for several years, it looks unlikely that Tom's work will make its way into CASINO.

Tom gave a talk about his work at the Towler Institute conference last year (or perhaps the year before) and may be willing to send you a copy of his slides (I don't have them and am surprised that Mike doesn't either). If you contact me off-list I'll send you Tom's current email address.

Best wishes, Matthew
Vladimir_Konjkov
Posts: 170
Joined: Wed Apr 15, 2015 3:14 pm

Re: Calculation of forces in QMC

Post by Vladimir_Konjkov »

Matthew_Foulkes wrote:Hi Vladimir,

Mike is right that Tom Poole, Peter Haynes, James Spencer and I tried algorithmically differentiating a small QMC code written by Tom himself. We did not attempt to differentiate the whole of CASINO for the reasons Mike explained; the work required would have been excessive and it would have made CASINO very hard to maintain.
Hi, Matthew.
Thanks for the detailed explanation. But as I understand, the algorithm used to calculate first derivatives of energy in CASINO described in http://journals.aps.org/pre/abstract/10 ... .76.036707 isn't algorithmically differentiating. Make clear if I'm wrong.
The article also states that a common problem to the evaluation of forces within both VMC and DMC methods is that the variance of the HFT force is infinite when the bare Coulomb potential is used for the electron-nucleus interaction, so that the choose of the pseudopotential is one way to solve the problem. Pseudopotential however, introduce an additional complexity, with its unlocality.
I would like to understand the following. As algorithmically differentiating itself can't solve the problem of infinite variance, so can it be regarded as a promising method of calculation energy derivatives? And which method is most promising?
Tom gave a talk about his work at the Towler Institute conference last year (or perhaps the year before) and may be willing to send you a copy of his slides (I don't have them and am surprised that Mike doesn't either). If you contact me off-list I'll send you Tom's current email address.

Best wishes, Matthew
It is not necessary, Google will do it for me: http://www.cmth.ph.ic.ac.uk/people/t.po ... earch.html (amazing video).
But there are other opinion about it in CASINO manual sec. 6.7 "How to run coupled DFT-DMC molecular dynamics calculations"

Best regards, Vladimir.
In Soviet Russia Casino plays you.
Matthew_Foulkes
Posts: 3
Joined: Tue Jan 14, 2014 10:47 pm
Location: Imperial College, London
Contact:

Re: Calculation of forces in QMC

Post by Matthew_Foulkes »

Dear Vladimir,
Vladimir_Konjkov wrote: Hi, Matthew.
Thanks for the detailed explanation. But as I understand, the algorithm used to calculate first derivatives of energy in CASINO described in http://journals.aps.org/pre/abstract/10 ... .76.036707 isn't algorithmically differentiating. Make clear if I'm wrong.
That's right. As I explained, Tom did his work using a separate standalone code that he wrote himself, because algorithmicially differentiating the whole of CASINO would have been too hard. AD forces are not available in CASINO.
The article also states that a common problem to the evaluation of forces within both VMC and DMC methods is that the variance of the HFT force is infinite when the bare Coulomb potential is used for the electron-nucleus interaction, so that the choose of the pseudopotential is one way to solve the problem. Pseudopotential however, introduce an additional complexity, with its unlocality.
I would like to understand the following. As algorithmically differentiating itself can't solve the problem of infinite variance, so can it be regarded as a promising method of calculation energy derivatives? And which method is most promising?
I agree that the variance of the naive Hellmann-Feynman estimator for forces is infinite and that this is a problem (although people like Caffarel and Sorella have made great progress towards solving it). AD does not solve this problem, so Tom used pseudopotentials. The use of pseudopotentials is an approximation (and not always a good one, especially for transition metals), but is in practice almost always necessary if you want to do DMC calculations involving atoms beyond the first row of the periodic table. The compute time required for full-core DMC simulations rises as a high power (4-5) of the nuclear charge, so DMC simulations without pseudopotentials are impractical for heavy atoms. This is true whether or not you are interested in forces: if you want to use DMC, you will soon find that you are using pseudopotentials too. Non-local pseudopotentials make Hellman-Feynman-style force estimatators much more complicated, but were easier to deal with in AD.

AD is in my view the most promising method for calculating DMC forces because it gives accurate answers for systems of (as far as we could tell) arbitrary size in a time that is only a small multiple (which is independent of system size) of the normal DMC run time. It is a pity that no major DMC code has AD built in or is likely to have AD built in in the near future.

Best wishes, Matthew
Matthew_Foulkes
Posts: 3
Joined: Tue Jan 14, 2014 10:47 pm
Location: Imperial College, London
Contact:

Re: Calculation of forces in QMC

Post by Matthew_Foulkes »

P.S. Section 6.7 of the CASINO manual is about the Grossman-Mitas approach, which does not use QMC forces. Instead, the DMC energies are evaluated along ionic trajectories found using some other method, most likely DFT molecular dynamics. If you want to find thermodynamic properties via thermodynamic integration from the DFT Hamiltonian this is a reasonable way to to do it, but it is not really DMC molecular dynamics.
Vladimir_Konjkov
Posts: 170
Joined: Wed Apr 15, 2015 3:14 pm

Re: Calculation of forces in QMC

Post by Vladimir_Konjkov »

Hi, Matthew.

So ok, I performed calculations of atomic forces in methane with VMC method and using PP, and got a file vmc.hist with 172 colums.
plot_hist tells me that:
O-----------O
| PLOT_HIST |
O-----------O

PLOT DATA FROM VMC.HIST
======================

Column label not recognised.
Label is: FORAX1
How do I get information about the forces acting on the atoms?
Maybe there is another utility that extracts atomic forces data from VMC.HIST?

Thanks in advance, Vladimir.
Attachments
CASINO_force.tgz
Result of atomic forces calculation in methane
(702.99 KiB) Downloaded 1422 times
In Soviet Russia Casino plays you.
Mike Towler
Posts: 239
Joined: Thu May 30, 2013 11:03 pm
Location: Florence
Contact:

Re: Calculation of forces in QMC

Post by Mike Towler »

Because - to a first approximation - no-one has ever used forces in CASINO, it seems that no-one bothered to update plot_hist to allow plotting the forces. It's not difficult, and I'll implement it tonight if I have time..

The reblock utility will calculate the forces for you from vmc.hist. I'll check it over to make sure it still works.

Mike
Post Reply