Re: [ALPS-users] DWA - which version / compiled is needed
Hey Lode, Really nice to hear from you! It has been ages! Answers inline.
On 26 May 2020, at 2:13 PM, Lode Pollet <Lode.Pollet@physik.uni-muenchen.de> wrote:
hi Tama,
how are you?
We are having a difficult year :) Business-wise, trading execution services are fine. - HK exchange is still functioning. Political-wise, we are facing more squeeze from Beijing :(
I've been thinking about a light version migrated to ALPSCore for some time but never started doing it although it would be a good time to clean up the code again. Based on my existing code this would not take too long. I have enough experience with ALPSCore for that. ALPSCore does not incorporate the applications so it is independent of a worm algorithm code. It would only be for Bose-Hubbard and (in-plane ferromagnetic) spin models on very simple lattices. Visualization should be split again from the core code, as this (for us) is an application from the past. There are too many parts of this code that are model-dependent (nowadays it is always a variant of those models that you need to simulate, but it requires some small changes to the code), and that makes me hesitant because I am not sure it will be that useful. So this is why I think that something light and flexible is better.
I need your help (and Matthias’s help) for the physics. I already lost touch with physics. Regarding integration into current middleware / virtualization, I would be happy to help. We just need to have a plan to spend this amount of man-days to implement this change. Maybe Matthias has a bigger plan for this?
I don't know about Kafka or elasticsearch. What do you have in mind on native ETL support for big data infrastucture?
Elasticsearch - for fast-search NoSQL DB (Really fast for non-aggregated search). Kafka - for real-time ETL streaming. If you need lower-latency, then you need gRPC + Protobuf. That’s another story. (We don’t want to invest in building such framework.)
Do you see any bottlenecks currently?
Not for me. Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat) We still have 4 years left to implement the migration :) There is still some time.
But anyone can send me their wishlist or suggestions and then I will see what I can do (but not before Corona times are over). You can also join if you wish, then I will set up a git or so.
lode
On 26. May 2020, at 04:22, Tama MA <tamama@yotcopi.com> wrote:
Containerize ALPS-DWA ?
- To maintain DWA further requires some effort. - One simple option would be to containerize DWA of course. With podman / runc in RHEL-8, this overhead is almost negligible. - Other virtualization option include conda.
I must admit that at the time DWA was written, we were at a different era. If given the time for another PhD with Matthias ( :) ), I would revamp this design to incorporate Kafka and Elasticsearch (++) - that supports built-in visualization and native-ETL support for modern big-data infrastructure.
The question now is really whether this code is worth our effort to improve further? (I wrote DWA, so I would be the best person to be assigned this task…)
Well, I leave this question to the community, and Matthias would have a bigger vote :)
If yes, how many man-days would we assign to this task?
Kind regards, Tama MA
On 26 May 2020, at 12:41 AM, Mateusz Łącki <mateusz.lacki@gmail.com> wrote:
Hi, I want to compile DWA from source on Linux. As far as I know "old" DWA verions from ALPS 2.2 work fine, but require ancient GCC (gcc at most 5.5). Newer DWA from ALPS 2.3 compiles ok with some of new compilers (at the very least with clang), but it quickly crashes with segmentation fault (Oskar Prosniak email from from 22/02/2018).
The problem is that it is not so easy to install gcc on a new machines these days, as gcc5 uses some deprecated header files which have been removed from glibc 2.28 and newer (ustat.h). Downgrading glibc is not really an option.
Is there any option these days to compole any version DWA from source, preferably 7650 (with the intent of make a few small changes to the source code) in such a way the updated DWA code can be deployed eg. on cluster (that is without using old linux distribution, without downgrading glibc, and preferably without gcc-5).
??
Best, Mateusz Łącki
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
Hi, As for downgrading gcc to a version able to compile old version of alps it turs out that it is still compilable, provided someone i willing to comment out lines that give compiler error (in general this is nasty approach, but ustat.h seems hardly used in gcc after all). It seems they are activated via some extra compiler options, and at the very least one can compile gcc 55 with c,c++,fortran support and this allows to compile alps.
Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat)
We still have 4 years left to implement the migration :) There is still some time.
To compile DWA and alps in general it seems that no work is necessary as it compiles great with e.g compilers available in repositories of current stable Debian (provided one uses a recent version of alps). I guess is possible that is is the new version of compiler that is responsible for the segmentation fault. Then I am wrong :) Best, Mateusz Łącki On Tue, 26 May 2020 at 09:40, Tama MA <tamama@yotcopi.com> wrote:
Hey Lode,
Really nice to hear from you! It has been ages!
Answers inline.
On 26 May 2020, at 2:13 PM, Lode Pollet <Lode.Pollet@physik.uni-muenchen.de> wrote:
hi Tama,
how are you?
We are having a difficult year :)
Business-wise, trading execution services are fine. - HK exchange is still functioning. Political-wise, we are facing more squeeze from Beijing :(
I've been thinking about a light version migrated to ALPSCore for some time but never started doing it although it would be a good time to clean up the code again. Based on my existing code this would not take too long. I have enough experience with ALPSCore for that. ALPSCore does not incorporate the applications so it is independent of a worm algorithm code. It would only be for Bose-Hubbard and (in-plane ferromagnetic) spin models on very simple lattices. Visualization should be split again from the core code, as this (for us) is an application from the past. There are too many parts of this code that are model-dependent (nowadays it is always a variant of those models that you need to simulate, but it requires some small changes to the code), and that makes me hesitant because I am not sure it will be that useful. So this is why I think that something light and flexible is better.
I need your help (and Matthias’s help) for the physics. I already lost touch with physics.
Regarding integration into current middleware / virtualization, I would be happy to help. We just need to have a plan to spend this amount of man-days to implement this change.
Maybe Matthias has a bigger plan for this?
I don't know about Kafka or elasticsearch. What do you have in mind on native ETL support for big data infrastucture?
Elasticsearch - for fast-search NoSQL DB (Really fast for non-aggregated search). Kafka - for real-time ETL streaming.
If you need lower-latency, then you need gRPC + Protobuf. That’s another story.
(We don’t want to invest in building such framework.)
Do you see any bottlenecks currently?
Not for me.
Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat)
We still have 4 years left to implement the migration :) There is still some time.
But anyone can send me their wishlist or suggestions and then I will see what I can do (but not before Corona times are over). You can also join if you wish, then I will set up a git or so.
lode
On 26. May 2020, at 04:22, Tama MA <tamama@yotcopi.com> wrote:
Containerize ALPS-DWA ?
- To maintain DWA further requires some effort. - One simple option would be to containerize DWA of course. With podman / runc in RHEL-8, this overhead is almost negligible. - Other virtualization option include conda.
I must admit that at the time DWA was written, we were at a different era. If given the time for another PhD with Matthias ( :) ), I would revamp this design to incorporate Kafka and Elasticsearch (++) - that supports built-in visualization and native-ETL support for modern big-data infrastructure.
The question now is really whether this code is worth our effort to improve further? (I wrote DWA, so I would be the best person to be assigned this task…)
Well, I leave this question to the community, and Matthias would have a bigger vote :)
If yes, how many man-days would we assign to this task?
Kind regards, Tama MA
On 26 May 2020, at 12:41 AM, Mateusz Łącki <mateusz.lacki@gmail.com> wrote:
Hi, I want to compile DWA from source on Linux. As far as I know "old" DWA verions from ALPS 2.2 work fine, but require ancient GCC (gcc at most 5.5). Newer DWA from ALPS 2.3 compiles ok with some of new compilers (at the very least with clang), but it quickly crashes with segmentation fault (Oskar Prosniak email from from 22/02/2018).
The problem is that it is not so easy to install gcc on a new machines these days, as gcc5 uses some deprecated header files which have been removed from glibc 2.28 and newer (ustat.h). Downgrading glibc is not really an option.
Is there any option these days to compole any version DWA from source, preferably 7650 (with the intent of make a few small changes to the source code) in such a way the updated DWA code can be deployed eg. on cluster (that is without using old linux distribution, without downgrading glibc, and preferably without gcc-5).
??
Best, Mateusz Łącki
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
Dear all, I have a binary of recent ALPS (2019-08-07) compile on Debian Jessie (GCC 4.9.2). Unfortunately, dwa-01-bosons tutorial crashes with segmentation fault just after the thermalization. Best, Synge
On May 27, 2020, at 14:12, Mateusz Łącki <mateusz.lacki@gmail.com> wrote:
Hi, As for downgrading gcc to a version able to compile old version of alps it turs out that it is still compilable, provided someone i willing to comment out lines that give compiler error (in general this is nasty approach, but ustat.h seems hardly used in gcc after all). It seems they are activated via some extra compiler options, and at the very least one can compile gcc 55 with c,c++,fortran support and this allows to compile alps.
Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat)
We still have 4 years left to implement the migration :) There is still some time.
To compile DWA and alps in general it seems that no work is necessary as it compiles great with e.g compilers available in repositories of current stable Debian (provided one uses a recent version of alps). I guess is possible that is is the new version of compiler that is responsible for the segmentation fault. Then I am wrong :)
Best, Mateusz Łącki
On Tue, 26 May 2020 at 09:40, Tama MA <tamama@yotcopi.com> wrote:
Hey Lode,
Really nice to hear from you! It has been ages!
Answers inline.
On 26 May 2020, at 2:13 PM, Lode Pollet <Lode.Pollet@physik.uni-muenchen.de> wrote:
hi Tama,
how are you?
We are having a difficult year :)
Business-wise, trading execution services are fine. - HK exchange is still functioning. Political-wise, we are facing more squeeze from Beijing :(
I've been thinking about a light version migrated to ALPSCore for some time but never started doing it although it would be a good time to clean up the code again. Based on my existing code this would not take too long. I have enough experience with ALPSCore for that. ALPSCore does not incorporate the applications so it is independent of a worm algorithm code. It would only be for Bose-Hubbard and (in-plane ferromagnetic) spin models on very simple lattices. Visualization should be split again from the core code, as this (for us) is an application from the past. There are too many parts of this code that are model-dependent (nowadays it is always a variant of those models that you need to simulate, but it requires some small changes to the code), and that makes me hesitant because I am not sure it will be that useful. So this is why I think that something light and flexible is better.
I need your help (and Matthias’s help) for the physics. I already lost touch with physics.
Regarding integration into current middleware / virtualization, I would be happy to help. We just need to have a plan to spend this amount of man-days to implement this change.
Maybe Matthias has a bigger plan for this?
I don't know about Kafka or elasticsearch. What do you have in mind on native ETL support for big data infrastucture?
Elasticsearch - for fast-search NoSQL DB (Really fast for non-aggregated search). Kafka - for real-time ETL streaming.
If you need lower-latency, then you need gRPC + Protobuf. That’s another story.
(We don’t want to invest in building such framework.)
Do you see any bottlenecks currently?
Not for me.
Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat)
We still have 4 years left to implement the migration :) There is still some time.
But anyone can send me their wishlist or suggestions and then I will see what I can do (but not before Corona times are over). You can also join if you wish, then I will set up a git or so.
lode
On 26. May 2020, at 04:22, Tama MA <tamama@yotcopi.com> wrote:
Containerize ALPS-DWA ?
- To maintain DWA further requires some effort. - One simple option would be to containerize DWA of course. With podman / runc in RHEL-8, this overhead is almost negligible. - Other virtualization option include conda.
I must admit that at the time DWA was written, we were at a different era. If given the time for another PhD with Matthias ( :) ), I would revamp this design to incorporate Kafka and Elasticsearch (++) - that supports built-in visualization and native-ETL support for modern big-data infrastructure.
The question now is really whether this code is worth our effort to improve further? (I wrote DWA, so I would be the best person to be assigned this task…)
Well, I leave this question to the community, and Matthias would have a bigger vote :)
If yes, how many man-days would we assign to this task?
Kind regards, Tama MA
On 26 May 2020, at 12:41 AM, Mateusz Łącki <mateusz.lacki@gmail.com> wrote:
Hi, I want to compile DWA from source on Linux. As far as I know "old" DWA verions from ALPS 2.2 work fine, but require ancient GCC (gcc at most 5.5). Newer DWA from ALPS 2.3 compiles ok with some of new compilers (at the very least with clang), but it quickly crashes with segmentation fault (Oskar Prosniak email from from 22/02/2018).
The problem is that it is not so easy to install gcc on a new machines these days, as gcc5 uses some deprecated header files which have been removed from glibc 2.28 and newer (ustat.h). Downgrading glibc is not really an option.
Is there any option these days to compole any version DWA from source, preferably 7650 (with the intent of make a few small changes to the source code) in such a way the updated DWA code can be deployed eg. on cluster (that is without using old linux distribution, without downgrading glibc, and preferably without gcc-5).
??
Best, Mateusz Łącki
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
Hey Synge, Would you provide some stacktrace? I hope to find some time to fix this.
On 27 May 2020, at 4:18 PM, Synge Todo <wistaria@phys.s.u-tokyo.ac.jp> wrote:
Dear all,
I have a binary of recent ALPS (2019-08-07) compile on Debian Jessie (GCC 4.9.2). Unfortunately, dwa-01-bosons tutorial crashes with segmentation fault just after the thermalization.
Best, Synge
On May 27, 2020, at 14:12, Mateusz Łącki <mateusz.lacki@gmail.com> wrote:
Hi, As for downgrading gcc to a version able to compile old version of alps it turs out that it is still compilable, provided someone i willing to comment out lines that give compiler error (in general this is nasty approach, but ustat.h seems hardly used in gcc after all). It seems they are activated via some extra compiler options, and at the very least one can compile gcc 55 with c,c++,fortran support and this allows to compile alps.
Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat)
We still have 4 years left to implement the migration :) There is still some time.
To compile DWA and alps in general it seems that no work is necessary as it compiles great with e.g compilers available in repositories of current stable Debian (provided one uses a recent version of alps). I guess is possible that is is the new version of compiler that is responsible for the segmentation fault. Then I am wrong :)
Best, Mateusz Łącki
On Tue, 26 May 2020 at 09:40, Tama MA <tamama@yotcopi.com> wrote:
Hey Lode,
Really nice to hear from you! It has been ages!
Answers inline.
On 26 May 2020, at 2:13 PM, Lode Pollet <Lode.Pollet@physik.uni-muenchen.de> wrote:
hi Tama,
how are you?
We are having a difficult year :)
Business-wise, trading execution services are fine. - HK exchange is still functioning. Political-wise, we are facing more squeeze from Beijing :(
I've been thinking about a light version migrated to ALPSCore for some time but never started doing it although it would be a good time to clean up the code again. Based on my existing code this would not take too long. I have enough experience with ALPSCore for that. ALPSCore does not incorporate the applications so it is independent of a worm algorithm code. It would only be for Bose-Hubbard and (in-plane ferromagnetic) spin models on very simple lattices. Visualization should be split again from the core code, as this (for us) is an application from the past. There are too many parts of this code that are model-dependent (nowadays it is always a variant of those models that you need to simulate, but it requires some small changes to the code), and that makes me hesitant because I am not sure it will be that useful. So this is why I think that something light and flexible is better.
I need your help (and Matthias’s help) for the physics. I already lost touch with physics.
Regarding integration into current middleware / virtualization, I would be happy to help. We just need to have a plan to spend this amount of man-days to implement this change.
Maybe Matthias has a bigger plan for this?
I don't know about Kafka or elasticsearch. What do you have in mind on native ETL support for big data infrastucture?
Elasticsearch - for fast-search NoSQL DB (Really fast for non-aggregated search). Kafka - for real-time ETL streaming.
If you need lower-latency, then you need gRPC + Protobuf. That’s another story.
(We don’t want to invest in building such framework.)
Do you see any bottlenecks currently?
Not for me.
Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat)
We still have 4 years left to implement the migration :) There is still some time.
But anyone can send me their wishlist or suggestions and then I will see what I can do (but not before Corona times are over). You can also join if you wish, then I will set up a git or so.
lode
On 26. May 2020, at 04:22, Tama MA <tamama@yotcopi.com> wrote:
Containerize ALPS-DWA ?
- To maintain DWA further requires some effort. - One simple option would be to containerize DWA of course. With podman / runc in RHEL-8, this overhead is almost negligible. - Other virtualization option include conda.
I must admit that at the time DWA was written, we were at a different era. If given the time for another PhD with Matthias ( :) ), I would revamp this design to incorporate Kafka and Elasticsearch (++) - that supports built-in visualization and native-ETL support for modern big-data infrastructure.
The question now is really whether this code is worth our effort to improve further? (I wrote DWA, so I would be the best person to be assigned this task…)
Well, I leave this question to the community, and Matthias would have a bigger vote :)
If yes, how many man-days would we assign to this task?
Kind regards, Tama MA
On 26 May 2020, at 12:41 AM, Mateusz Łącki <mateusz.lacki@gmail.com> wrote:
Hi, I want to compile DWA from source on Linux. As far as I know "old" DWA verions from ALPS 2.2 work fine, but require ancient GCC (gcc at most 5.5). Newer DWA from ALPS 2.3 compiles ok with some of new compilers (at the very least with clang), but it quickly crashes with segmentation fault (Oskar Prosniak email from from 22/02/2018).
The problem is that it is not so easy to install gcc on a new machines these days, as gcc5 uses some deprecated header files which have been removed from glibc 2.28 and newer (ustat.h). Downgrading glibc is not really an option.
Is there any option these days to compole any version DWA from source, preferably 7650 (with the intent of make a few small changes to the source code) in such a way the updated DWA code can be deployed eg. on cluster (that is without using old linux distribution, without downgrading glibc, and preferably without gcc-5).
??
Best, Mateusz Łącki
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
Dear all, I did some more research on the problem of dwa. It works until Rev. 7698 (at least for dwa-01-bosons tutorial), but Rev. 7699 crashes just after the thermalization. I think there is a problem in the following modification to dwa.c. Best, Synge --- $ svn diff -c 7699 Index: applications/qmc/dwa/dwa.cpp =================================================================== --- applications/qmc/dwa/dwa.cpp (revision 7698) +++ applications/qmc/dwa/dwa.cpp (revision 7699) @@ -149,7 +149,7 @@ // regarding lattice , is_periodic_ (std::find((this->lattice().boundary()).begin(), (this->lattice().boundary()).end(), "open") == (this->lattice().boundary()).end()) // regarding model - , is_diagonal_onsite_ (static_cast<double>(parms_.value_or_default("V",0.)) < 1e-6 ? true : false) + , is_diagonal_onsite_ (std::abs(static_cast<double>(parms_.value_or_default("V",0.))) < 1e-6 ? true : false) // regarding experiment , finite_tof (is_periodic_ ? false : (parms_.defined("tof_phase"))) , finite_waist (parms_.defined("waist")) @@ -1036,8 +1036,10 @@ for (unsigned int site=0; site < num_sites(); ++site) { total_particle_number += wl.site_state(site); - const double this_energy_vertex = -static_cast<double>(wl.num_kinks(site)-1)/2; - const double this_energy_onsite = onsite_energy(site, wl.site_state(site)); + double this_energy_vertex = -static_cast<double>(wl.num_kinks(site)-1)/2; + double this_energy_onsite = onsite_energy(site, wl.site_state(site)); + for (neighbor_bond_iterator it=neighbor_bonds(site).first; it != neighbor_bonds(site).second; ++it) + this_energy_onsite += onbond_energy(*it, wl.site_state(site), wl.site_state(target(*it)))/2.; total_energy_vertex += this_energy_vertex; total_energy_onsite += this_energy_onsite; if (measure_density2_)
On May 27, 2020, at 17:18, Synge Todo <wistaria@phys.s.u-tokyo.ac.jp> wrote:
Dear all,
I have a binary of recent ALPS (2019-08-07) compile on Debian Jessie (GCC 4.9.2). Unfortunately, dwa-01-bosons tutorial crashes with segmentation fault just after the thermalization.
Best, Synge
On May 27, 2020, at 14:12, Mateusz Łącki <mateusz.lacki@gmail.com> wrote:
Hi, As for downgrading gcc to a version able to compile old version of alps it turs out that it is still compilable, provided someone i willing to comment out lines that give compiler error (in general this is nasty approach, but ustat.h seems hardly used in gcc after all). It seems they are activated via some extra compiler options, and at the very least one can compile gcc 55 with c,c++,fortran support and this allows to compile alps.
Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat)
We still have 4 years left to implement the migration :) There is still some time.
To compile DWA and alps in general it seems that no work is necessary as it compiles great with e.g compilers available in repositories of current stable Debian (provided one uses a recent version of alps). I guess is possible that is is the new version of compiler that is responsible for the segmentation fault. Then I am wrong :)
Best, Mateusz Łącki
On Tue, 26 May 2020 at 09:40, Tama MA <tamama@yotcopi.com> wrote:
Hey Lode,
Really nice to hear from you! It has been ages!
Answers inline.
On 26 May 2020, at 2:13 PM, Lode Pollet <Lode.Pollet@physik.uni-muenchen.de> wrote:
hi Tama,
how are you?
We are having a difficult year :)
Business-wise, trading execution services are fine. - HK exchange is still functioning. Political-wise, we are facing more squeeze from Beijing :(
I've been thinking about a light version migrated to ALPSCore for some time but never started doing it although it would be a good time to clean up the code again. Based on my existing code this would not take too long. I have enough experience with ALPSCore for that. ALPSCore does not incorporate the applications so it is independent of a worm algorithm code. It would only be for Bose-Hubbard and (in-plane ferromagnetic) spin models on very simple lattices. Visualization should be split again from the core code, as this (for us) is an application from the past. There are too many parts of this code that are model-dependent (nowadays it is always a variant of those models that you need to simulate, but it requires some small changes to the code), and that makes me hesitant because I am not sure it will be that useful. So this is why I think that something light and flexible is better.
I need your help (and Matthias’s help) for the physics. I already lost touch with physics.
Regarding integration into current middleware / virtualization, I would be happy to help. We just need to have a plan to spend this amount of man-days to implement this change.
Maybe Matthias has a bigger plan for this?
I don't know about Kafka or elasticsearch. What do you have in mind on native ETL support for big data infrastucture?
Elasticsearch - for fast-search NoSQL DB (Really fast for non-aggregated search). Kafka - for real-time ETL streaming.
If you need lower-latency, then you need gRPC + Protobuf. That’s another story.
(We don’t want to invest in building such framework.)
Do you see any bottlenecks currently?
Not for me.
Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat)
We still have 4 years left to implement the migration :) There is still some time.
But anyone can send me their wishlist or suggestions and then I will see what I can do (but not before Corona times are over). You can also join if you wish, then I will set up a git or so.
lode
On 26. May 2020, at 04:22, Tama MA <tamama@yotcopi.com> wrote:
Containerize ALPS-DWA ?
- To maintain DWA further requires some effort. - One simple option would be to containerize DWA of course. With podman / runc in RHEL-8, this overhead is almost negligible. - Other virtualization option include conda.
I must admit that at the time DWA was written, we were at a different era. If given the time for another PhD with Matthias ( :) ), I would revamp this design to incorporate Kafka and Elasticsearch (++) - that supports built-in visualization and native-ETL support for modern big-data infrastructure.
The question now is really whether this code is worth our effort to improve further? (I wrote DWA, so I would be the best person to be assigned this task…)
Well, I leave this question to the community, and Matthias would have a bigger vote :)
If yes, how many man-days would we assign to this task?
Kind regards, Tama MA
On 26 May 2020, at 12:41 AM, Mateusz Łącki <mateusz.lacki@gmail.com> wrote:
Hi, I want to compile DWA from source on Linux. As far as I know "old" DWA verions from ALPS 2.2 work fine, but require ancient GCC (gcc at most 5.5). Newer DWA from ALPS 2.3 compiles ok with some of new compilers (at the very least with clang), but it quickly crashes with segmentation fault (Oskar Prosniak email from from 22/02/2018).
The problem is that it is not so easy to install gcc on a new machines these days, as gcc5 uses some deprecated header files which have been removed from glibc 2.28 and newer (ustat.h). Downgrading glibc is not really an option.
Is there any option these days to compole any version DWA from source, preferably 7650 (with the intent of make a few small changes to the source code) in such a way the updated DWA code can be deployed eg. on cluster (that is without using old linux distribution, without downgrading glibc, and preferably without gcc-5).
??
Best, Mateusz Łącki
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
If I comment out the following two lines in dwa.cpp, it seems that the tutorial works (both for GCC 4.9 and GCC 8.3), though the results should be incorrect... for (neighbor_bond_iterator it=neighbor_bonds(site).first; it != neighbor_bonds(site).second; ++it) this_energy_onsite += onbond_energy(*it, wl.site_state(site), wl.site_state(target(*it)))/2.; Synge
I think that the bug may be the same bug that was commented on in emails from Fri, 23 Feb 2018, 03:07 on this group (conversation between O. Prosniak and Tama Ma). Best, Mateusz Łącki On Thu, 28 May 2020 at 08:35, Synge Todo <wistaria@phys.s.u-tokyo.ac.jp> wrote:
If I comment out the following two lines in dwa.cpp, it seems that the tutorial works (both for GCC 4.9 and GCC 8.3), though the results should be incorrect...
for (neighbor_bond_iterator it=neighbor_bonds(site).first; it != neighbor_bonds(site).second; ++it) this_energy_onsite += onbond_energy(*it, wl.site_state(site), wl.site_state(target(*it)))/2.;
Synge
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
Short-term / Quick-fix: (1-2 weeks) - We find a base image that works - preferably Redhat for OS stability, and get them running behind containerisation. - Let’s not waste time in finding why this version of compiler won’t work with that version of library, etc. Long-term nice-to-have features: - I would decouple the storage layer (hdf5, …) from the process layer (simulation, …) into micro services. - Then, bind these into a pod of containers, push them in k8s. - Then, pump these statistics into some-form of middleware. - Seriously, I don’t want to maintain storage middlewares / dependencies that break the DWA code. - Neither would I want to bother with individual node crash for whatever reasons. That is to say, if simulation nodes fail, deployments should recover themselves within k8s, and data-recovery from some-form of NoSQL DB. / MQ etc.
On 27 May 2020, at 1:12 PM, Mateusz Łącki <mateusz.lacki@gmail.com> wrote:
Hi, As for downgrading gcc to a version able to compile old version of alps it turs out that it is still compilable, provided someone i willing to comment out lines that give compiler error (in general this is nasty approach, but ustat.h seems hardly used in gcc after all). It seems they are activated via some extra compiler options, and at the very least one can compile gcc 55 with c,c++,fortran support and this allows to compile alps.
Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat)
We still have 4 years left to implement the migration :) There is still some time.
To compile DWA and alps in general it seems that no work is necessary as it compiles great with e.g compilers available in repositories of current stable Debian (provided one uses a recent version of alps). I guess is possible that is is the new version of compiler that is responsible for the segmentation fault. Then I am wrong :)
Best, Mateusz Łącki
On Tue, 26 May 2020 at 09:40, Tama MA <tamama@yotcopi.com> wrote:
Hey Lode,
Really nice to hear from you! It has been ages!
Answers inline.
On 26 May 2020, at 2:13 PM, Lode Pollet <Lode.Pollet@physik.uni-muenchen.de> wrote:
hi Tama,
how are you?
We are having a difficult year :)
Business-wise, trading execution services are fine. - HK exchange is still functioning. Political-wise, we are facing more squeeze from Beijing :(
I've been thinking about a light version migrated to ALPSCore for some time but never started doing it although it would be a good time to clean up the code again. Based on my existing code this would not take too long. I have enough experience with ALPSCore for that. ALPSCore does not incorporate the applications so it is independent of a worm algorithm code. It would only be for Bose-Hubbard and (in-plane ferromagnetic) spin models on very simple lattices. Visualization should be split again from the core code, as this (for us) is an application from the past. There are too many parts of this code that are model-dependent (nowadays it is always a variant of those models that you need to simulate, but it requires some small changes to the code), and that makes me hesitant because I am not sure it will be that useful. So this is why I think that something light and flexible is better.
I need your help (and Matthias’s help) for the physics. I already lost touch with physics.
Regarding integration into current middleware / virtualization, I would be happy to help. We just need to have a plan to spend this amount of man-days to implement this change.
Maybe Matthias has a bigger plan for this?
I don't know about Kafka or elasticsearch. What do you have in mind on native ETL support for big data infrastucture?
Elasticsearch - for fast-search NoSQL DB (Really fast for non-aggregated search). Kafka - for real-time ETL streaming.
If you need lower-latency, then you need gRPC + Protobuf. That’s another story.
(We don’t want to invest in building such framework.)
Do you see any bottlenecks currently?
Not for me.
Regarding DWA, that was developed in 2012 with GCC-4.x. We know that it would be supported till 2024 by RHEL-7. (10 years promise by Redhat)
We still have 4 years left to implement the migration :) There is still some time.
But anyone can send me their wishlist or suggestions and then I will see what I can do (but not before Corona times are over). You can also join if you wish, then I will set up a git or so.
lode
On 26. May 2020, at 04:22, Tama MA <tamama@yotcopi.com> wrote:
Containerize ALPS-DWA ?
- To maintain DWA further requires some effort. - One simple option would be to containerize DWA of course. With podman / runc in RHEL-8, this overhead is almost negligible. - Other virtualization option include conda.
I must admit that at the time DWA was written, we were at a different era. If given the time for another PhD with Matthias ( :) ), I would revamp this design to incorporate Kafka and Elasticsearch (++) - that supports built-in visualization and native-ETL support for modern big-data infrastructure.
The question now is really whether this code is worth our effort to improve further? (I wrote DWA, so I would be the best person to be assigned this task…)
Well, I leave this question to the community, and Matthias would have a bigger vote :)
If yes, how many man-days would we assign to this task?
Kind regards, Tama MA
On 26 May 2020, at 12:41 AM, Mateusz Łącki <mateusz.lacki@gmail.com> wrote:
Hi, I want to compile DWA from source on Linux. As far as I know "old" DWA verions from ALPS 2.2 work fine, but require ancient GCC (gcc at most 5.5). Newer DWA from ALPS 2.3 compiles ok with some of new compilers (at the very least with clang), but it quickly crashes with segmentation fault (Oskar Prosniak email from from 22/02/2018).
The problem is that it is not so easy to install gcc on a new machines these days, as gcc5 uses some deprecated header files which have been removed from glibc 2.28 and newer (ustat.h). Downgrading glibc is not really an option.
Is there any option these days to compole any version DWA from source, preferably 7650 (with the intent of make a few small changes to the source code) in such a way the updated DWA code can be deployed eg. on cluster (that is without using old linux distribution, without downgrading glibc, and preferably without gcc-5).
??
Best, Mateusz Łącki
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
---- Comp-phys-alps-users Mailing List for the ALPS Project http://alps.comp-phys.org/
List info: https://lists.phys.ethz.ch//listinfo/comp-phys-alps-users Archive: https://lists.phys.ethz.ch//pipermail/comp-phys-alps-users
Unsubscribe by writing a mail to comp-phys-alps-users-leave@lists.phys.ethz.ch.
participants (3)
-
Mateusz Łącki
-
Synge Todo
-
Tama MA