Dear All, I was wondering if using the same SEED parameter with the dirloop_sse application in different runs should produce exactly the same results. It is defined as " The random number seed used in the next run.". But I failed to produce the same results with alps 1.35 or with alps 2.02. Regards, Marcelo Guimarães
Hi Marcello, Indeed if the same SEED is used you get the same results if the same code is taken. However, dirloop_sse was completely rewritten between 1.3 and 2.0. It is a new code. Thus the results should now only agree within error bars. Matthias On Jan 20, 2012, at 6:56 PM, Marcelo Guimarães wrote:
Dear All,
I was wondering if using the same SEED parameter with the dirloop_sse application in different runs should produce exactly the same results.
It is defined as " The random number seed used in the next run.". But I failed to produce the same results with alps 1.35 or with alps 2.02.
Regards,
Marcelo Guimarães
Dear Matthias, Thank you for your answer. But I think I did not made myself clear. I used the same input file , and ran it twice with alps 1.35 obtaining different results in each run. Using the same input file and running twice with alps 2.00.rc4 I got different results. All 4 results were different from each other. I could not obtain exactly the same results even using the same code and the same SEED in either versions. Am I doing something wrong? Or the result could never actually be the same? In case it helps, this is the input file used in all runs: FILE:parameter LATTICE="simple cubic lattice" MODEL = "spin" ALGORITHM="sse" MEASURE[Structure Factor]=false MEASURE[Correlations]=false MEASURE[Specific Heat]=true MEASURE[Stiffness]=true MEASURE[Magnetization]=true WHICH_LOOP_TYPE="locopt" EPSILON=0.5 local_S=1 J=-1 J0=-1 J1=-1 Jz=0 Jz'=0 {THERMALIZATION=10000;L=04;BETA=0.10;SWEEPS=1000;D=10.0;SEED=814} I converted it with parameter2xml Then I executed the dirloop_see v1.1 for alps 1.3.5: dirloop_sse --Tmin=10 parameter.in.xml with the result: ... <SCALAR_AVERAGE name="Energy"> <COUNT>8519</COUNT> <MEAN method="simple">250.472</MEAN> ... Then I repeated the command: dirloop_sse --Tmin=10 parameter.in.xml and obtained the result: ... <SCALAR_AVERAGE name="Energy"> <COUNT>3009</COUNT> <MEAN method="simple">250.553</MEAN> ... The same goes for the dirloop 4.1 based on alps 2.0.0rc4 code: same parameters same SEED: After extracting values with "convert2xml parameter.task1.out.run1" the 1st run produced: <SCALAR_AVERAGE name="Energy"> <COUNT>1473</COUNT> <MEAN method="simple">251.784</MEAN> And in the 2nd run: <SCALAR_AVERAGE name="Energy"> <COUNT>16076</COUNT> <MEAN method="simple">250.616</MEAN> Marcelo Guimarães 2012/1/21 Matthias Troyer <troyer@phys.ethz.ch>
Hi Marcello,
Indeed if the same SEED is used you get the same results if the same code is taken. However, dirloop_sse was completely rewritten between 1.3 and 2.0. It is a new code. Thus the results should now only agree within error bars.
Matthias
On Jan 20, 2012, at 6:56 PM, Marcelo Guimarães wrote:
Dear All,
I was wondering if using the same SEED parameter with the dirloop_sse application in different runs should produce exactly the same results.
It is defined as " The random number seed used in the next run.". But I failed to produce the same results with alps 1.35 or with alps 2.02.
Regards,
Marcelo Guimarães
Dear Marcelo, [while I'm not Matthias, I'd dare adding my 2pence nonetheless] Look at the COUNT field of your output files. The number of measurements taken is different in all your runs --- therefore the results are different. As far as I understand, the number of MC is not guaranteed in current versions of ALPS applications. (Maybe someone more knowledgeable can comment on this?) Especially for short runs, the actual number of MC steps is almost always larger than the number you request in the parameter files. To make a meaningful comparison of equivalent runs you need to take errorbars into account, just as Matthias said in his previous email. Cheers, Zhenya 2012/1/21 Marcelo Guimarães <marcelosg@gmail.com>
Dear Matthias,
Thank you for your answer. But I think I did not made myself clear.
I used the same input file , and ran it twice with alps 1.35 obtaining different results in each run. Using the same input file and running twice with alps 2.00.rc4 I got different results. All 4 results were different from each other.
I could not obtain exactly the same results even using the same code and the same SEED in either versions. Am I doing something wrong? Or the result could never actually be the same?
In case it helps, this is the input file used in all runs: FILE:parameter
LATTICE="simple cubic lattice" MODEL = "spin" ALGORITHM="sse" MEASURE[Structure Factor]=false MEASURE[Correlations]=false MEASURE[Specific Heat]=true MEASURE[Stiffness]=true MEASURE[Magnetization]=true WHICH_LOOP_TYPE="locopt" EPSILON=0.5 local_S=1 J=-1 J0=-1 J1=-1 Jz=0 Jz'=0 {THERMALIZATION=10000;L=04;BETA=0.10;SWEEPS=1000;D=10.0;SEED=814}
I converted it with parameter2xml
Then I executed the dirloop_see v1.1 for alps 1.3.5: dirloop_sse --Tmin=10 parameter.in.xml
with the result: ... <SCALAR_AVERAGE name="Energy"> <COUNT>8519</COUNT> <MEAN method="simple">250.472</MEAN> ... Then I repeated the command: dirloop_sse --Tmin=10 parameter.in.xml
and obtained the result: ... <SCALAR_AVERAGE name="Energy"> <COUNT>3009</COUNT> <MEAN method="simple">250.553</MEAN> ...
The same goes for the dirloop 4.1 based on alps 2.0.0rc4 code: same parameters same SEED:
After extracting values with "convert2xml parameter.task1.out.run1" the 1st run produced: <SCALAR_AVERAGE name="Energy"> <COUNT>1473</COUNT> <MEAN method="simple">251.784</MEAN>
And in the 2nd run: <SCALAR_AVERAGE name="Energy"> <COUNT>16076</COUNT> <MEAN method="simple">250.616</MEAN>
Marcelo Guimarães
2012/1/21 Matthias Troyer <troyer@phys.ethz.ch>
Hi Marcello,
Indeed if the same SEED is used you get the same results if the same code is taken. However, dirloop_sse was completely rewritten between 1.3 and 2.0. It is a new code. Thus the results should now only agree within error bars.
Matthias
On Jan 20, 2012, at 6:56 PM, Marcelo Guimarães wrote:
Dear All,
I was wondering if using the same SEED parameter with the dirloop_sse application in different runs should produce exactly the same results.
It is defined as " The random number seed used in the next run.". But I failed to produce the same results with alps 1.35 or with alps 2.02.
Regards,
Marcelo Guimarães
Dear Evgeni, Thank you for your answer. Now I think I undestand, even though the sequence generated is the same with the same seed, the program keeps running additional measurements until checked by the scheduler. Thanks, Marcelo Guimaraes 2012/1/21 Evgeni Burovski <evgeny.burovskiy@gmail.com>
Dear Marcelo,
[while I'm not Matthias, I'd dare adding my 2pence nonetheless]
Look at the COUNT field of your output files. The number of measurements taken is different in all your runs --- therefore the results are different.
As far as I understand, the number of MC is not guaranteed in current versions of ALPS applications. (Maybe someone more knowledgeable can comment on this?) Especially for short runs, the actual number of MC steps is almost always larger than the number you request in the parameter files.
To make a meaningful comparison of equivalent runs you need to take errorbars into account, just as Matthias said in his previous email.
Cheers,
Zhenya
2012/1/21 Marcelo Guimarães <marcelosg@gmail.com>
Dear Matthias,
Thank you for your answer. But I think I did not made myself clear.
I used the same input file , and ran it twice with alps 1.35 obtaining different results in each run. Using the same input file and running twice with alps 2.00.rc4 I got different results. All 4 results were different from each other.
I could not obtain exactly the same results even using the same code and the same SEED in either versions. Am I doing something wrong? Or the result could never actually be the same?
In case it helps, this is the input file used in all runs: FILE:parameter
LATTICE="simple cubic lattice" MODEL = "spin" ALGORITHM="sse" MEASURE[Structure Factor]=false MEASURE[Correlations]=false MEASURE[Specific Heat]=true MEASURE[Stiffness]=true MEASURE[Magnetization]=true WHICH_LOOP_TYPE="locopt" EPSILON=0.5 local_S=1 J=-1 J0=-1 J1=-1 Jz=0 Jz'=0 {THERMALIZATION=10000;L=04;BETA=0.10;SWEEPS=1000;D=10.0;SEED=814}
I converted it with parameter2xml
Then I executed the dirloop_see v1.1 for alps 1.3.5: dirloop_sse --Tmin=10 parameter.in.xml
with the result: ... <SCALAR_AVERAGE name="Energy"> <COUNT>8519</COUNT> <MEAN method="simple">250.472</MEAN> ... Then I repeated the command: dirloop_sse --Tmin=10 parameter.in.xml
and obtained the result: ... <SCALAR_AVERAGE name="Energy"> <COUNT>3009</COUNT> <MEAN method="simple">250.553</MEAN> ...
The same goes for the dirloop 4.1 based on alps 2.0.0rc4 code: same parameters same SEED:
After extracting values with "convert2xml parameter.task1.out.run1" the 1st run produced: <SCALAR_AVERAGE name="Energy"> <COUNT>1473</COUNT> <MEAN method="simple">251.784</MEAN>
And in the 2nd run: <SCALAR_AVERAGE name="Energy"> <COUNT>16076</COUNT> <MEAN method="simple">250.616</MEAN>
Marcelo Guimarães
2012/1/21 Matthias Troyer <troyer@phys.ethz.ch>
Hi Marcello,
Indeed if the same SEED is used you get the same results if the same code is taken. However, dirloop_sse was completely rewritten between 1.3 and 2.0. It is a new code. Thus the results should now only agree within error bars.
Matthias
On Jan 20, 2012, at 6:56 PM, Marcelo Guimarães wrote:
Dear All,
I was wondering if using the same SEED parameter with the dirloop_sse application in different runs should produce exactly the same results.
It is defined as " The random number seed used in the next run.". But I failed to produce the same results with alps 1.35 or with alps 2.02.
Regards,
Marcelo Guimarães
Indeed that is the case. If it is important for you we can add an easy check that stops after exactly the given number of sweeps as long as the code is not run in parallel. Matthias On Jan 21, 2012, at 4:39 PM, Marcelo Guimarães wrote:
Dear Evgeni,
Thank you for your answer. Now I think I undestand, even though the sequence generated is the same with the same seed, the program keeps running additional measurements until checked by the scheduler.
Thanks,
Marcelo Guimaraes
2012/1/21 Evgeni Burovski <evgeny.burovskiy@gmail.com> Dear Marcelo,
[while I'm not Matthias, I'd dare adding my 2pence nonetheless]
Look at the COUNT field of your output files. The number of measurements taken is different in all your runs --- therefore the results are different.
As far as I understand, the number of MC is not guaranteed in current versions of ALPS applications. (Maybe someone more knowledgeable can comment on this?) Especially for short runs, the actual number of MC steps is almost always larger than the number you request in the parameter files.
To make a meaningful comparison of equivalent runs you need to take errorbars into account, just as Matthias said in his previous email.
Cheers,
Zhenya
2012/1/21 Marcelo Guimarães <marcelosg@gmail.com> Dear Matthias,
Thank you for your answer. But I think I did not made myself clear.
I used the same input file , and ran it twice with alps 1.35 obtaining different results in each run. Using the same input file and running twice with alps 2.00.rc4 I got different results. All 4 results were different from each other.
I could not obtain exactly the same results even using the same code and the same SEED in either versions. Am I doing something wrong? Or the result could never actually be the same?
In case it helps, this is the input file used in all runs: FILE:parameter
LATTICE="simple cubic lattice" MODEL = "spin" ALGORITHM="sse" MEASURE[Structure Factor]=false MEASURE[Correlations]=false MEASURE[Specific Heat]=true MEASURE[Stiffness]=true MEASURE[Magnetization]=true WHICH_LOOP_TYPE="locopt" EPSILON=0.5 local_S=1 J=-1 J0=-1 J1=-1 Jz=0 Jz'=0 {THERMALIZATION=10000;L=04;BETA=0.10;SWEEPS=1000;D=10.0;SEED=814}
I converted it with parameter2xml
Then I executed the dirloop_see v1.1 for alps 1.3.5: dirloop_sse --Tmin=10 parameter.in.xml
with the result: ... <SCALAR_AVERAGE name="Energy"> <COUNT>8519</COUNT> <MEAN method="simple">250.472</MEAN> ... Then I repeated the command: dirloop_sse --Tmin=10 parameter.in.xml
and obtained the result: ... <SCALAR_AVERAGE name="Energy"> <COUNT>3009</COUNT> <MEAN method="simple">250.553</MEAN> ...
The same goes for the dirloop 4.1 based on alps 2.0.0rc4 code: same parameters same SEED:
After extracting values with "convert2xml parameter.task1.out.run1" the 1st run produced: <SCALAR_AVERAGE name="Energy"> <COUNT>1473</COUNT> <MEAN method="simple">251.784</MEAN>
And in the 2nd run: <SCALAR_AVERAGE name="Energy"> <COUNT>16076</COUNT> <MEAN method="simple">250.616</MEAN>
Marcelo Guimarães
2012/1/21 Matthias Troyer <troyer@phys.ethz.ch> Hi Marcello,
Indeed if the same SEED is used you get the same results if the same code is taken. However, dirloop_sse was completely rewritten between 1.3 and 2.0. It is a new code. Thus the results should now only agree within error bars.
Matthias
On Jan 20, 2012, at 6:56 PM, Marcelo Guimarães wrote:
Dear All,
I was wondering if using the same SEED parameter with the dirloop_sse application in different runs should produce exactly the same results.
It is defined as " The random number seed used in the next run.". But I failed to produce the same results with alps 1.35 or with alps 2.02.
Regards,
Marcelo Guimarães
Based on what you said I managed do the changes. Thank you both for the help. 2012/1/21 Matthias Troyer <troyer@phys.ethz.ch>
Indeed that is the case. If it is important for you we can add an easy check that stops after exactly the given number of sweeps as long as the code is not run in parallel.
Matthias
On Jan 21, 2012, at 4:39 PM, Marcelo Guimarães wrote:
Dear Evgeni,
Thank you for your answer. Now I think I undestand, even though the sequence generated is the same with the same seed, the program keeps running additional measurements until checked by the scheduler.
Thanks,
Marcelo Guimaraes
2012/1/21 Evgeni Burovski <evgeny.burovskiy@gmail.com>
Dear Marcelo,
[while I'm not Matthias, I'd dare adding my 2pence nonetheless]
Look at the COUNT field of your output files. The number of measurements taken is different in all your runs --- therefore the results are different.
As far as I understand, the number of MC is not guaranteed in current versions of ALPS applications. (Maybe someone more knowledgeable can comment on this?) Especially for short runs, the actual number of MC steps is almost always larger than the number you request in the parameter files.
To make a meaningful comparison of equivalent runs you need to take errorbars into account, just as Matthias said in his previous email.
Cheers,
Zhenya
2012/1/21 Marcelo Guimarães <marcelosg@gmail.com>
Dear Matthias,
Thank you for your answer. But I think I did not made myself clear.
I used the same input file , and ran it twice with alps 1.35 obtaining different results in each run. Using the same input file and running twice with alps 2.00.rc4 I got different results. All 4 results were different from each other.
I could not obtain exactly the same results even using the same code and the same SEED in either versions. Am I doing something wrong? Or the result could never actually be the same?
In case it helps, this is the input file used in all runs: FILE:parameter
LATTICE="simple cubic lattice" MODEL = "spin" ALGORITHM="sse" MEASURE[Structure Factor]=false MEASURE[Correlations]=false MEASURE[Specific Heat]=true MEASURE[Stiffness]=true MEASURE[Magnetization]=true WHICH_LOOP_TYPE="locopt" EPSILON=0.5 local_S=1 J=-1 J0=-1 J1=-1 Jz=0 Jz'=0 {THERMALIZATION=10000;L=04;BETA=0.10;SWEEPS=1000;D=10.0;SEED=814}
I converted it with parameter2xml
Then I executed the dirloop_see v1.1 for alps 1.3.5: dirloop_sse --Tmin=10 parameter.in.xml
with the result: ... <SCALAR_AVERAGE name="Energy"> <COUNT>8519</COUNT> <MEAN method="simple">250.472</MEAN> ... Then I repeated the command: dirloop_sse --Tmin=10 parameter.in.xml
and obtained the result: ... <SCALAR_AVERAGE name="Energy"> <COUNT>3009</COUNT> <MEAN method="simple">250.553</MEAN> ...
The same goes for the dirloop 4.1 based on alps 2.0.0rc4 code: same parameters same SEED:
After extracting values with "convert2xml parameter.task1.out.run1" the 1st run produced: <SCALAR_AVERAGE name="Energy"> <COUNT>1473</COUNT> <MEAN method="simple">251.784</MEAN>
And in the 2nd run: <SCALAR_AVERAGE name="Energy"> <COUNT>16076</COUNT> <MEAN method="simple">250.616</MEAN>
Marcelo Guimarães
2012/1/21 Matthias Troyer <troyer@phys.ethz.ch>
Hi Marcello,
Indeed if the same SEED is used you get the same results if the same code is taken. However, dirloop_sse was completely rewritten between 1.3 and 2.0. It is a new code. Thus the results should now only agree within error bars.
Matthias
On Jan 20, 2012, at 6:56 PM, Marcelo Guimarães wrote:
Dear All,
I was wondering if using the same SEED parameter with the dirloop_sse application in different runs should produce exactly the same results.
It is defined as " The random number seed used in the next run.". But I failed to produce the same results with alps 1.35 or with alps 2.02.
Regards,
Marcelo Guimarães
participants (3)
-
Evgeni Burovski
-
Marcelo Guimarães
-
Matthias Troyer