Optimized decorrelation period

[Katharina Doblhoff]

According to the manual, setting vmc_decorr_period to 0, the decorr period is automatically optimized. Indeed in the output I get something like "Optimized vmc_decorr_period: 5". Do I need to increase this number when performing varmin or emin calculations?

While we are at it: Casino seems to estimate the decorr period from 400 vmc steps. I thought the decorrelation length estimate converged very slowly with number of samples (which is why reblocking has certain advantages), or did I understand something fundamentally wrong here? So how can Casino estimate the docorr period from only 400 vmc steps?

[Mike Towler]

According to the manual, setting vmc_decorr_period to 0, the decorr period is automatically optimized. Indeed in the output I get something like "Optimized vmc_decorr_period: 5". Do I need to increase this number when performing varmin or emin calculations?

It is easy to think that the vmc_decorr_period keyword gives directly the frequency with which configs are written out, not least because this used to be true in early versions of the code. Prior to version 2.1 every vmc_decorr_period (or corper as it then was) moves the local energy would be evaluated and the configs written out. If vmc_nstep was bigger than vmc_nconfig_write then the final part of the calculation would be done without writing any configs.

However this behaviour was changed years ago in 2.1.13 such that the config writes were spaced as far apart as possible for the given number of VMC moves e.g. in a vmc_nstep=100000 calculation with vmc_nconfig_write=10000, configs would be written every (100000/10000)*vmc_decorr_period steps i.e. 10 times less frequently than you might think given the previous behaviour. In this case setting vmc_decorr_period to e.g. 15 instead of the default 3 in optimization/DMC runs serves only to make the config generation process significantly slower, as the config writes for the above examples are already 3*10 steps apart and not serially correlated. Increasing this to 15*10 steps therefore serves no useful purpose.

To stop people who misunderstood this point from accidentally making config generation runs unnecessarily slow I therefore later (2.13.354) changed the code so that vmc_decorr_period now implies the minimum number of moves separating config writes; if the length of the parent VMC run is much greater than the number of configs to be written then the length of the inner decorrelation loop will be reduced to as small a value as possible without writing configs more frequently than every 3 moves (or whatever the default vmc_decorr_period for pure VMC runs is). Here's some interesting timing data:

Code: Select all

  Example for Si222 : varmin with vmc_nstep=100000,vmc_nconfig_write=10000, and
                     default decorrelation periods (3 VMC, 15 VMC_OPT).

  VMC #1: E     = -62.172(5) ; var  = 2.27(1) (correlation.out.0)
  VMC #2: E     = -63.044(3) ; var  = 0.88(2) (correlation.out.1)
  VMC #3: E     = -63.031(3) ; var  = 0.856(7) (correlation.out.2)
  Total CASINO CPU time  ::: 1313.8301 seconds

  VMC #1: E     = -62.172(8) ; var  = 2.26(3) (correlation.out.0)
  VMC #2: E     = -63.051(4) ; var  = 0.858(9) (correlation.out.1)
  VMC #3: E     = -63.040(3) ; var  = 0.851(9) (correlation.out.2)
  Total CASINO CPU time  ::: 865.1600 seconds

  i.e. a 35% speedup with better results.

So the answer to your question is basically no. If you insist on automatically optimizing vmc_decorr_period by setting it to zero, and you wish according to the standard dogma to space out the config writes to several times the correlation period, then the only way to do this is to do more VMC energy-generating steps than VMC config generating steps (i.e. vmc_nstep > vmc_nconfig_write). As to whether spacing out the configs is actually worth it in optimizations, I don't think it really matters all that much in VMC but it obviously doesn't hurt. Feel free to test..

However, an important point is that usually one needs to have the number of VMC moves much greater than the number of configs anyway, in order to get the error bar on the mean energy small enough to see if your optimization is actually working properly after each config gen/optimization cycle, so in practice the question hardly ever arises..

While we are at it: Casino seems to estimate the decorr period from 400 vmc steps. I thought the decorrelation length estimate converged very slowly with number of samples (which is why reblocking has certain advantages), or did I understand something fundamentally wrong here? So how can Casino estimate the docorr period from only 400 vmc steps?

Where do you get 400 from? I didn't write that bit of the code (Pablo?) but as far as I recall the correlation period estimate is done using N 'extended equilibration steps' over and above the standard vmc_equil_nstep number of equilibration steps, and a suitable N is worked out on the fly.

The correlation period is very short in VMC (typically just a few moves) and very long in DMC because of the much smaller timestep we are required to use. You can see this in typical reblocking analyses.

Cheers,
Mike

[Vladimir_Konjkov]

Hi all.

I recently read the manual p.25.1 Variance minimization: the standard method
it says that:

It is clearly desirable for the VMC-generated configurations to be completely uncorrelated. This can be achieved by giving vmc_decorr_period a large value (e.g., 10). Reblocking VMC energies in a preliminary VMC run will allow the user to determine the correlation period for VMC energies, which in turn suggests a suitable value for vmc_decorr_period.

I usually use vmc_decorr_period=10 (not zero).

Isn't that right?

Vladimir

[Mike Towler]

Hi Vladimir,

It is right yes, but as explained above if vmc_nstep is greater than vmc_nconfig_write then internally CASINO will reduce vmc_decorr_period (in effect, the number of 'extra steps' per vmc_nstep) right back down again, but in such a way that the config writes are still at least 10 steps apart as requested. If it didn't do that the only practical result of increasing vmc_decorr_period would be to unnecessarily increase the time taken (see the timings above).

The point about setting it to zero is that the value of vmc_decorr_period is then automatically optimized. Of course this is of most use in pure VMC simulations, rather than config-generating VMC runs - but even then the number of situations where anyone would actually give a damn is clearly limited. For the most part, VMC is just a relatively quick tool that allows us to do DMC simulations.

Mike