Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiling Machida fails #1925

Closed
avinash010 opened this issue Jan 8, 2018 · 17 comments
Closed

Compiling Machida fails #1925

avinash010 opened this issue Jan 8, 2018 · 17 comments

Comments

@avinash010
Copy link

avinash010 commented Jan 8, 2018

I am getting the below error message when i try to compile machinda. I am trying to set up Wallaroo in my Linux box. Since i was not able to reopen the previous(#1876) issue i am creating a new issue.

Stack dump:
0.      Running pass 'Function Pass Manager' on module 'machida'.
1.      Running pass 'Loop Vectorization' on function '@Array_t2_U128_val_U128_val_Serialise'
Segmentation fault (core dumped)
../machida/Makefile:68: recipe for target '/home/ubuntu/wallaroo-tutorial/wallaroo/machida/build/machida' failed
make: *** [/home/ubuntu/wallaroo-tutorial/wallaroo/machida/build/machida] Error 1

I am using a Linux AWS box (Instance Type: m5.large)
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial

Terminal Output

ubuntu@ip-:~/wallaroo-tutorial/wallaroo/machida$ make
From https://github.com/WallarooLabs/pony-kafka
 * branch            HEAD       -> FETCH_HEAD
Updating f738312..b26d8ae
Fast-forward
 examples/performance/main.pony                              |  460 ++++++++++++++++++++++++++++++++++
 examples/simple/main.pony                                   |   13 +-
 misc/kafka/cleanup_data_files.sh                            |    9 +
 misc/kafka/create_replicate_topic.sh                        |   12 +
 misc/kafka/download_kafka_java.sh                           |   16 ++
 misc/kafka/kafka-server-0.properties                        |  125 ++++++++++
 misc/kafka/kafka-server-1.properties                        |  125 ++++++++++
 misc/kafka/kafka-server-2.properties                        |  125 ++++++++++
 misc/kafka/start_kafka_0.sh                                 |    9 +
 misc/kafka/start_kafka_1.sh                                 |    9 +
 misc/kafka/start_kafka_2.sh                                 |    9 +
 misc/kafka/start_zookeeper.sh                               |    8 +
 misc/kafka/zookeeper.properties                             |   20 ++
 pony-kafka/_test.pony                                       |  118 +++++----
 pony-kafka/compression/lz4.pony                             |   12 +-
 pony-kafka/compression/snappy.pony                          |   24 +-
 pony-kafka/compression/zlib.pony                            |   16 +-
 pony-kafka/custombuffered/_test.pony                        |   93 ++++++-
 pony-kafka/custombuffered/codecs/big_endian_decoder.pony    |  172 +++++++------
 pony-kafka/custombuffered/codecs/little_endian_decoder.pony |  188 +++++++-------
 pony-kafka/custombuffered/codecs/varint_decoder.pony        |   22 +-
 pony-kafka/custombuffered/iso_reader.pony                   |  226 +++++++++++++++++
 pony-kafka/custombuffered/reader.pony                       |  487 +-----------------------------------
 pony-kafka/custombuffered/val_reader.pony                   |  471 +++++++++++++++++++++++++++++++++++
 pony-kafka/custombuffered/writer.pony                       |   31 +--
 pony-kafka/customlogger/logger.pony                         |   10 +-
 pony-kafka/customnet/custom_tcp_connection_handler.pony     |   23 +-
 pony-kafka/customnet/custom_tcp_connection_notify.pony      |    8 +
 pony-kafka/customnet/mock_tcp_connection_handler.pony       |   21 ++
 pony-kafka/customnet/tcp_connection_handler.pony            |   18 ++
 pony-kafka/kafka_api.pony                                   |  653 ++++++++++++++++++++++++++++---------------------
 pony-kafka/kafka_broker_connection.pony                     |   62 +++--
 pony-kafka/kafka_config.pony                                |  475 +++++++++++++++++++++++++----------
 pony-kafka/kafka_config_cli_parser.pony                     |  395 ++++++++++++++++++++++++++++++
 pony-kafka/kafka_errors.pony                                |  371 ++++++++++++++++++++++------
 pony-kafka/kafka_handler.pony                               | 1656 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------
 36 files changed, 4698 insertions(+), 1794 deletions(-)
 create mode 100644 examples/performance/main.pony
 create mode 100755 misc/kafka/cleanup_data_files.sh
 create mode 100755 misc/kafka/create_replicate_topic.sh
 create mode 100755 misc/kafka/download_kafka_java.sh
 create mode 100644 misc/kafka/kafka-server-0.properties
 create mode 100644 misc/kafka/kafka-server-1.properties
 create mode 100644 misc/kafka/kafka-server-2.properties
 create mode 100755 misc/kafka/start_kafka_0.sh
 create mode 100755 misc/kafka/start_kafka_1.sh
 create mode 100755 misc/kafka/start_kafka_2.sh
 create mode 100755 misc/kafka/start_zookeeper.sh
 create mode 100644 misc/kafka/zookeeper.properties
 create mode 100644 pony-kafka/custombuffered/iso_reader.pony
 create mode 100644 pony-kafka/custombuffered/val_reader.pony
 create mode 100644 pony-kafka/kafka_config_cli_parser.pony
Previous HEAD position was b26d8ae... Fix invalid throttling edge case
HEAD is now at f738312... Compatiblity with ponyc master and exhaustive match changes.
Building builtin -> /usr/lib/pony/0.21.0-4301.acd811b/packages/builtin
Building . -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida
Building signals -> /usr/lib/pony/0.21.0-4301.acd811b/packages/signals
Building ponytest -> /usr/lib/pony/0.21.0-4301.acd811b/packages/ponytest
Building time -> /usr/lib/pony/0.21.0-4301.acd811b/packages/time
Building collections -> /usr/lib/pony/0.21.0-4301.acd811b/packages/collections
Building wallaroo_labs/mort -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/mort
Building wallaroo_labs/options -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/options
Building wallaroo -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo
Building wallaroo/ent/spike -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/ent/spike
Building wallaroo/ent/network -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/ent/network
Building wallaroo/core/common -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/common
Building wallaroo/core/boundary -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/boundary
Building buffered -> /usr/lib/pony/0.21.0-4301.acd811b/packages/buffered
Building net -> /usr/lib/pony/0.21.0-4301.acd811b/packages/net
Building wallaroo_labs/bytes -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/bytes
Building promises -> /usr/lib/pony/0.21.0-4301.acd811b/packages/promises
Building wallaroo_labs/time -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/time
Building wallaroo/ent/watermarking -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/ent/watermarking
Building wallaroo/core/invariant -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/invariant
Building wallaroo/core/routing -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/routing
Building wallaroo_labs/guid -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/guid
Building random -> /usr/lib/pony/0.21.0-4301.acd811b/packages/random
Building wallaroo/core/messages -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/messages
Building serialise -> /usr/lib/pony/0.21.0-4301.acd811b/packages/serialise
Building wallaroo/ent/data_receiver -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/ent/data_receiver
Building wallaroo/core/data_channel -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/data_channel
Building files -> /usr/lib/pony/0.21.0-4301.acd811b/packages/files
Building capsicum -> /usr/lib/pony/0.21.0-4301.acd811b/packages/capsicum
Building wallaroo/ent/recovery -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/ent/recovery
Building wallaroo/core/topology -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/topology
Building assert -> /usr/lib/pony/0.21.0-4301.acd811b/packages/assert
Building wallaroo/ent/rebalancing -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/ent/rebalancing
Building wallaroo/core/initialization -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/initialization
Building wallaroo/ent/cluster_manager -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/ent/cluster_manager
Building wallaroo/core/metrics -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/metrics
Building wallaroo_labs/hub -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/hub
Building json -> /usr/lib/pony/0.21.0-4301.acd811b/packages/json
Building format -> /usr/lib/pony/0.21.0-4301.acd811b/packages/format
Building wallaroo/ent/router_registry -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/ent/router_registry
Building wallaroo/core/source -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/source
Building wallaroo/core/sink/tcp_sink -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/sink/tcp_sink
Building options -> /usr/lib/pony/0.21.0-4301.acd811b/packages/options
Building wallaroo/core/sink -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/sink
Building wallaroo/core/source/tcp_source -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/source/tcp_source
Building wallaroo_labs/dag -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/dag
Building wallaroo_labs/equality -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/equality
Building wallaroo_labs/messages -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/messages
Building wallaroo_labs/queue -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/queue
Building debug -> /usr/lib/pony/0.21.0-4301.acd811b/packages/debug
Building wallaroo_labs/thread_count -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/thread_count
Building wallaroo/core/state -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/state
Building wallaroo_labs/weighted -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/weighted
Building wallaroo_labs/collection_helpers -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/collection_helpers
Building wallaroo_labs/conversions -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo_labs/conversions
Building itertools -> /usr/lib/pony/0.21.0-4301.acd811b/packages/itertools
Building net/http -> /usr/lib/pony/0.21.0-4301.acd811b/packages/net/http
Building net/ssl -> /usr/lib/pony/0.21.0-4301.acd811b/packages/net/ssl
Building encode/base64 -> /usr/lib/pony/0.21.0-4301.acd811b/packages/encode/base64
Building pony-kafka -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/WallarooLabs/pony-kafka/pony-kafka
Building custombuffered -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/WallarooLabs/pony-kafka/pony-kafka/custombuffered
Building codecs -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/WallarooLabs/pony-kafka/pony-kafka/custombuffered/codecs
Building ../../utils/bool_converter -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/WallarooLabs/pony-kafka/pony-kafka/utils/bool_converter
Building customnet -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/WallarooLabs/pony-kafka/pony-kafka/customnet
Building customlogger -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/WallarooLabs/pony-kafka/pony-kafka/customlogger
Building fsm -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/WallarooLabs/pony-kafka/pony-kafka/fsm
Building compression -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/WallarooLabs/pony-kafka/pony-kafka/compression
Building wallaroo/core/sink/kafka_sink -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/sink/kafka_sink
Building wallaroo/core/source/kafka_source -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo/core/source/kafka_source
Generating
 Reachability
 Selector painting
 Data prototypes
 Data types
 Function prototypes
 Functions
 Descriptors
Optimising
Stack dump:
0.      Running pass 'Function Pass Manager' on module 'machida'.
1.      Running pass 'Loop Vectorization' on function '@Array_t2_U128_val_U128_val_Serialise'
Segmentation fault (core dumped)
../machida/Makefile:68: recipe for target '/home/ubuntu/wallaroo-tutorial/wallaroo/machida/build/machida' failed
make: *** [/home/ubuntu/wallaroo-tutorial/wallaroo/machida/build/machida] Error 1
@SeanTAllen
Copy link
Contributor

Hi @avinash010. Sorry for have run into an issue.

It appears you first ran make using master then switched to building a release, that correct?

This bit:

Previous HEAD position was b26d8ae... Fix invalid throttling edge case
HEAD is now at f738312... Compatiblity with ponyc master and exhaustive match changes.

Indicates it wasn't a build that follows the installation directions. I'm interested what steps you took so I can reproduce and figure out what the problem is.

I suspect that you run make clean at the root of the wallaroo repository then

cd machida
make

things should work for you.

@avinash010
Copy link
Author

Hi @SeanTAllen . Thanks for helping me out here

I tried the steps mentioned in the https://docs.wallaroolabs.com/book/getting-started/linux-setup.html

git clone https://github.com/WallarooLabs/wallaroo
cd wallaroo
git checkout 0.3.2
cd ~/wallaroo-tutorial/wallaroo/machida
make

I also tried the steps suggested to rectify the problem by running make clean, but still couldn't get past the issue

@SeanTAllen
Copy link
Contributor

hi @avinash010 could you do the make clean one more time then do a make on machida again and past the terminal output here? thanks.

@avinash010
Copy link
Author

hi @SeanTAllen
Please find the terminal output below

ubuntu@ip-:~/wallaroo-tutorial/wallaroo/machida$ make
ar: creating /home/ubuntu/wallaroo-tutorial/wallaroo/machida/build/libpython-wal                                                                                                                laroo.a
a - /home/ubuntu/wallaroo-tutorial/wallaroo/machida/build/python-wallaroo.o
Cloning into '/home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/WallarooLabs                                                                                                                /pony-kafka'...
remote: Counting objects: 372, done.
remote: Total 372 (delta 0), reused 0 (delta 0), pack-reused 372
Receiving objects: 100% (372/372), 443.11 KiB | 0 bytes/s, done.
Resolving deltas: 100% (235/235), done.
Checking connectivity... done.
Note: checking out '0.3.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at f738312... Compatiblity with ponyc master and exhaustive match ch                                                                                                                anges.
Building builtin -> /usr/lib/pony/0.21.0-4301.acd811b/packages/builtin
Building . -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida
Building signals -> /usr/lib/pony/0.21.0-4301.acd811b/packages/signals
Building ponytest -> /usr/lib/pony/0.21.0-4301.acd811b/packages/ponytest
Building time -> /usr/lib/pony/0.21.0-4301.acd811b/packages/time
Building collections -> /usr/lib/pony/0.21.0-4301.acd811b/packages/collections
Building wallaroo_labs/mort -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/walla                                                                                                                roo_labs/mort
Building wallaroo_labs/options -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wa                                                                                                                llaroo_labs/options
Building wallaroo -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallaroo
Building wallaroo/ent/spike -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/walla                                                                                                                roo/ent/spike
Building wallaroo/ent/network -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wal                                                                                                                laroo/ent/network
Building wallaroo/core/common -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wal                                                                                                                laroo/core/common
Building wallaroo/core/boundary -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/w                                                                                                                allaroo/core/boundary
Building buffered -> /usr/lib/pony/0.21.0-4301.acd811b/packages/buffered
Building net -> /usr/lib/pony/0.21.0-4301.acd811b/packages/net
Building wallaroo_labs/bytes -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wall                                                                                                                aroo_labs/bytes
Building promises -> /usr/lib/pony/0.21.0-4301.acd811b/packages/promises
Building wallaroo_labs/time -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/walla                                                                                                                roo_labs/time
Building wallaroo/ent/watermarking -> /home/ubuntu/wallaroo-tutorial/wallaroo/li                                                                                                                b/wallaroo/ent/watermarking
Building wallaroo/core/invariant -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/                                                                                                                wallaroo/core/invariant
Building wallaroo/core/routing -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wa                                                                                                                llaroo/core/routing
Building wallaroo_labs/guid -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/walla                                                                                                                roo_labs/guid
Building random -> /usr/lib/pony/0.21.0-4301.acd811b/packages/random
Building wallaroo/core/messages -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/w                                                                                                                allaroo/core/messages
Building serialise -> /usr/lib/pony/0.21.0-4301.acd811b/packages/serialise
Building wallaroo/ent/data_receiver -> /home/ubuntu/wallaroo-tutorial/wallaroo/l                                                                                                                ib/wallaroo/ent/data_receiver
Building wallaroo/core/data_channel -> /home/ubuntu/wallaroo-tutorial/wallaroo/l                                                                                                                ib/wallaroo/core/data_channel
Building files -> /usr/lib/pony/0.21.0-4301.acd811b/packages/files
Building capsicum -> /usr/lib/pony/0.21.0-4301.acd811b/packages/capsicum
Building wallaroo/ent/recovery -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wa                                                                                                                llaroo/ent/recovery
Building wallaroo/core/topology -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/w                                                                                                                allaroo/core/topology
Building assert -> /usr/lib/pony/0.21.0-4301.acd811b/packages/assert
Building wallaroo/ent/rebalancing -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib                                                                                                                /wallaroo/ent/rebalancing
Building wallaroo/core/initialization -> /home/ubuntu/wallaroo-tutorial/wallaroo                                                                                                                /lib/wallaroo/core/initialization
Building wallaroo/ent/cluster_manager -> /home/ubuntu/wallaroo-tutorial/wallaroo                                                                                                                /lib/wallaroo/ent/cluster_manager
Building wallaroo/core/metrics -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wa                                                                                                                llaroo/core/metrics
Building wallaroo_labs/hub -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallar                                                                                                                oo_labs/hub
Building json -> /usr/lib/pony/0.21.0-4301.acd811b/packages/json
Building format -> /usr/lib/pony/0.21.0-4301.acd811b/packages/format
Building wallaroo/ent/router_registry -> /home/ubuntu/wallaroo-tutorial/wallaroo                                                                                                                /lib/wallaroo/ent/router_registry
Building wallaroo/core/source -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wal                                                                                                                laroo/core/source
Building wallaroo/core/sink/tcp_sink -> /home/ubuntu/wallaroo-tutorial/wallaroo/                                                                                                                lib/wallaroo/core/sink/tcp_sink
Building options -> /usr/lib/pony/0.21.0-4301.acd811b/packages/options
Building wallaroo/core/sink -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/walla                                                                                                                roo/core/sink
Building wallaroo/core/source/tcp_source -> /home/ubuntu/wallaroo-tutorial/walla                                                                                                                roo/lib/wallaroo/core/source/tcp_source
Building wallaroo_labs/dag -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wallar                                                                                                                oo_labs/dag
Building wallaroo_labs/equality -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/w                                                                                                                allaroo_labs/equality
Building wallaroo_labs/messages -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/w                                                                                                                allaroo_labs/messages
Building wallaroo_labs/queue -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wall                                                                                                                aroo_labs/queue
Building debug -> /usr/lib/pony/0.21.0-4301.acd811b/packages/debug
Building wallaroo_labs/thread_count -> /home/ubuntu/wallaroo-tutorial/wallaroo/l                                                                                                                ib/wallaroo_labs/thread_count
Building wallaroo/core/state -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/wall                                                                                                                aroo/core/state
Building wallaroo_labs/weighted -> /home/ubuntu/wallaroo-tutorial/wallaroo/lib/w                                                                                                                allaroo_labs/weighted
Building wallaroo_labs/collection_helpers -> /home/ubuntu/wallaroo-tutorial/wall                                                                                                                aroo/lib/wallaroo_labs/collection_helpers
Building wallaroo_labs/conversions -> /home/ubuntu/wallaroo-tutorial/wallaroo/li                                                                                                                b/wallaroo_labs/conversions
Building itertools -> /usr/lib/pony/0.21.0-4301.acd811b/packages/itertools
Building net/http -> /usr/lib/pony/0.21.0-4301.acd811b/packages/net/http
Building net/ssl -> /usr/lib/pony/0.21.0-4301.acd811b/packages/net/ssl
Building encode/base64 -> /usr/lib/pony/0.21.0-4301.acd811b/packages/encode/base                                                                                                                64
Building pony-kafka -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/Wal                                                                                                                larooLabs/pony-kafka/pony-kafka
Building custombuffered -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps                                                                                                                /WallarooLabs/pony-kafka/pony-kafka/custombuffered
Building codecs -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/Wallaro                                                                                                                oLabs/pony-kafka/pony-kafka/custombuffered/codecs
Building ../../utils/bool_converter -> /home/ubuntu/wallaroo-tutorial/wallaroo/m                                                                                                                achida/.deps/WallarooLabs/pony-kafka/pony-kafka/utils/bool_converter
Building customnet -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/Wall                                                                                                                arooLabs/pony-kafka/pony-kafka/customnet
Building customlogger -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/W                                                                                                                allarooLabs/pony-kafka/pony-kafka/customlogger
Building fsm -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/WallarooLa                                                                                                                bs/pony-kafka/pony-kafka/fsm
Building compression -> /home/ubuntu/wallaroo-tutorial/wallaroo/machida/.deps/Wa                                                                                                                llarooLabs/pony-kafka/pony-kafka/compression
Building wallaroo/core/sink/kafka_sink -> /home/ubuntu/wallaroo-tutorial/wallaro                                                                                                                o/lib/wallaroo/core/sink/kafka_sink
Building wallaroo/core/source/kafka_source -> /home/ubuntu/wallaroo-tutorial/wal                                                                                                                laroo/lib/wallaroo/core/source/kafka_source
Generating
 Reachability
 Selector painting
 Data prototypes
 Data types
 Function prototypes
 Functions
 Descriptors
Optimising
Stack dump:
0.      Running pass 'Function Pass Manager' on module 'machida'.
1.      Running pass 'Loop Vectorization' on function '@Array_t2_U128_val_U128_val_Serialise'
Segmentation fault (core dumped)
../machida/Makefile:68: recipe for target '/home/ubuntu/wallaroo-tutorial/wallaroo/machida/build/machida' failed
make: *** [/home/ubuntu/wallaroo-tutorial/wallaroo/machida/build/machida] Error 1

@SeanTAllen
Copy link
Contributor

Thanks @avinash010

@SeanTAllen
Copy link
Contributor

@avinash010 on this end @slfritchie will be looking into this. he might have questions for you if he isn't able to reproduce.

@slfritchie
Copy link
Contributor

I can't replicate the problem using a Xenial 64-bit VM plus the Pony 0.21.2-63911b5 [release] compiler. I'll try using the exact 0.21.0-4301.acd811b version that's specified by https://docs.wallaroolabs.com/book/getting-started/linux-setup.html ... and if that works alright, then try moving to an EC2 instance.

@avinash010
Copy link
Author

fine @SeanTAllen. @slfritchie : Thanks for the update

@slfritchie
Copy link
Contributor

slfritchie commented Jan 8, 2018

Very odd: I can reproduce it if using an m5.large EC2 instance in Oregon using the AMI ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20171121.1 (ami-0def3275), and installing the prerequisites via sh -x doit.sh with the script at https://gist.github.com/slfritchie/0b9932a50f159269bf5f0edbe6df522f. (Using the if'ed steps to upgrade GCC 5 to the package manager's latest version has no effect. Nor does an apt-get update ; apt-get upgrade ; reboot.)

After the "Stack dump:" message appeared, the make process appeared to hang, waiting for some Ubuntu crash reporting thingie apport ran at 100% CPU. After waiting 5 minutes, I killed apport.

@slfritchie
Copy link
Contributor

slfritchie commented Jan 8, 2018

I'm sorry, but I cannot recommend using the m5.large instance type. After compiling the Pony compiler myself on an EC2 m5.large instance in US West Oregon, the compiler does not crash ... but it also has not finished after 26 minutes of allegedly running at 100% CPU.

If I use an m4.large instance in US West Oregon with the same AMI, plus the same doit.sh script mentioned in the gist above, then I can successfully compile machida in about one minute forty seconds.

@slfritchie
Copy link
Contributor

The same bad behavior appears with m5.xlarge and m5.4xlarge instances.

@slfritchie
Copy link
Contributor

slfritchie commented Jan 8, 2018

Thanks to research by @dipinhora ... the c5 instance types are also affected. @avinash010, if you could avoid both m5 and c5 instance types, you ought to be fine.

Meanwhile, I've started a conversation with the Pony developers for work-arounds and a fix. Current conjecture is that this is an LLVM bug that is tickled by AVX512 support by the CPUs.

dipinhora added a commit to dipinhora/ponyc that referenced this issue Jan 9, 2018
Prior to this commit, we could accidentally trigger llvm bug 30542
(https://bugs.llvm.org/show_bug.cgi?id=30542) on a cpu with the
`avx512f` feature. See WallarooLabs/wally#1925
for one particular example of this occurring.

This commit automagically disables `avx512f` if the LLVM in use is
below version 5.0.0 to work around the issue.
dipinhora added a commit to dipinhora/ponyc that referenced this issue Jan 9, 2018
Prior to this commit, we could accidentally trigger llvm bug 30542
(https://bugs.llvm.org/show_bug.cgi?id=30542) on a cpu with the
`avx512f` feature. See WallarooLabs/wally#1925
for one particular example of this occurring.

This commit automagically disables `avx512f` if the LLVM in use is
below version 5.0.0 to work around the issue.
@avinash010
Copy link
Author

@slfritchie : Thanks for the update. Will try to set up Wallaroo in some other AWS instance type

SeanTAllen pushed a commit to ponylang/ponyc that referenced this issue Jan 9, 2018
Prior to this commit, we could accidentally trigger llvm bug 30542
(https://bugs.llvm.org/show_bug.cgi?id=30542) on a cpu with the
`avx512f` feature. See WallarooLabs/wally#1925
for one particular example of this occurring.

This commit automagically disables `avx512f` if the LLVM in use is
below version 5.0.0 to work around the issue.
SeanTAllen added a commit that referenced this issue Jan 9, 2018
We recommend that all users use LLVM 3.9.1. However, there is a bug that
in LLVM version prior to LLVM 5 that can cause the pony compiler to
crash during optimization. As seen in issue #1925, this is triggered by
the usage of the axv512f optimization. This patch turns off axv512f when
building Wallaroo.

Closes #1925
SeanTAllen added a commit that referenced this issue Jan 9, 2018
We recommend that all users use LLVM 3.9.1. However, there is a bug that
in LLVM version prior to LLVM 5 that can cause the pony compiler to
crash during optimization. As seen in issue #1925, this is triggered by
the usage of the axv512f optimization. This patch turns off axv512f when
building Wallaroo.

Closes #1925
@SeanTAllen
Copy link
Contributor

I've opened a patch for this. Thanks for @dipinhora and @slfritchie for doing all the hard work. The patch will go out in our 0.3.3 release that will be happening this week (quite possibly today).

@SeanTAllen
Copy link
Contributor

@avinash010 a fix for this went out with 0.3.3 today. you should be good now. thanks you for patience. happy wallarooing!

@avinash010
Copy link
Author

I am able to install wallaroo in my m5 instance type box. Thanks @SeanTAllen @dipinhora and @slfritchie

@SeanTAllen
Copy link
Contributor

That's awesome @avinash010. If you run have any questions, you can find us on IRC and our user mailing list.

http://www.wallaroolabs.com/community/get-in-touch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants