Debug
I've executed python commands in interactive mode to identify where it fails:
root@05e99fc1ad94:/usr/local/bin# python Python 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import re >>> import sys >>> from magenta.models.melody_rnn.melody_rnn_generate import console_entry_point Illegal instruction (core dumped)
So is it failed because of Magenta ot TensorFlow?
root@05e99fc1ad94:/usr/local/bin# python Python 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import re >>> import sys >>> import tensorflow as tf Illegal instruction (core dumped)
Yes, it failed in TensorFlow.(1.10.0)
As per https://github.com/tensorflow/tensorflow/issues/17411 and https://superuser.com/questions/1398548/your-cpu-supports-instructions-that-this-tensorflow-binary-was-not-compiled-to-u… starting TF 1.6 it is using AVX CPU features. And CPU I have doesn't support AVX instructions.
tf@ubuntu:~$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 2 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 46 Model name: Intel(R) Xeon(R) CPU X7560 @ 2.27GHz Stepping: 6 CPU MHz: 2261.000 BogoMIPS: 4522.00 Hypervisor vendor: VMware Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 24576K NUMA node0 CPU(s): 0-7 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss ht syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf pni ssse3 cx16 sse4_1 sse4_2 popcnt hypervisor lahf_lm dtherm ida
I'll try to use TF 1.5.
root@05e99fc1ad94:/usr/local/bin# pip uninstall tensorflow
Uninstalling tensorflow-1.10.0:
Would remove:
/usr/local/bin/freeze_graph
/usr/local/bin/saved_model_cli
/usr/local/bin/tensorboard
/usr/local/bin/tflite_convert
/usr/local/bin/toco
/usr/local/bin/toco_from_protos
/usr/local/lib/python2.7/dist-packages/tensorflow-1.10.0.dist-info/*
/usr/local/lib/python2.7/dist-packages/tensorflow/*
Proceed (y/n)? y
Successfully uninstalled tensorflow-1.10.0
root@05e99fc1ad94:/usr/local/bin# pip install tensorflow==1.5
Collecting tensorflow==1.5
Downloading https://files.pythonhosted.org/packages/69/6d/09d4fbeedbafbc6768a94901f14ace4153adba4c2e2c6e6f2080f4a5d1a7/tensorflow-1.5.0-cp27-cp27mu-manylinux1_x86_64.whl (44.4MB)
100% |################################| 44.4MB 8.4kB/s
Requirement already satisfied: protobuf>=3.4.0 in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.5) (3.6.0)
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.5) (1.11.0)
Requirement already satisfied: wheel in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.5) (0.31.1)
Requirement already satisfied: absl-py>=0.1.6 in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.5) (0.3.0)
Requirement already satisfied: backports.weakref>=1.0rc1 in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.5) (1.0.post1)
Requirement already satisfied: numpy>=1.12.1 in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.5) (1.14.5)
Collecting tensorflow-tensorboard<1.6.0,>=1.5.0 (from tensorflow==1.5)
Downloading https://files.pythonhosted.org/packages/cd/ba/d664f7c27c710063b1cdfa0309db8fba98952e3a1ba1991ed98efffe69ed/tensorflow_tensorboard-1.5.1-py2-none-any.whl (3.0MB)
100% |################################| 3.0MB 2.8MB/s
Requirement already satisfied: enum34>=1.1.6 in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.5) (1.1.6)
Requirement already satisfied: mock>=2.0.0 in /usr/local/lib/python2.7/dist-packages (from tensorflow==1.5) (2.0.0)
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf>=3.4.0->tensorflow==1.5) (39.1.0)
Requirement already satisfied: werkzeug>=0.11.10 in /usr/local/lib/python2.7/dist-packages (from tensorflow-tensorboard<1.6.0,>=1.5.0->tensorflow==1.5) (0.14.1)
Requirement already satisfied: futures>=3.1.1; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from tensorflow-tensorboard<1.6.0,>=1.5.0->tensorflow==1.5) (3.2.0)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python2.7/dist-packages (from tensorflow-tensorboard<1.6.0,>=1.5.0->tensorflow==1.5) (2.6.11)
Collecting html5lib==0.9999999 (from tensorflow-tensorboard<1.6.0,>=1.5.0->tensorflow==1.5)
Downloading https://files.pythonhosted.org/packages/ae/ae/bcb60402c60932b32dfaf19bb53870b29eda2cd17551ba5639219fb5ebf9/html5lib-0.9999999.tar.gz (889kB)
100% |################################| 890kB 5.0MB/s
Collecting bleach==1.5.0 (from tensorflow-tensorboard<1.6.0,>=1.5.0->tensorflow==1.5)
Downloading https://files.pythonhosted.org/packages/33/70/86c5fec937ea4964184d4d6c4f0b9551564f821e1c3575907639036d9b90/bleach-1.5.0-py2.py3-none-any.whl
Requirement already satisfied: pbr>=0.11 in /usr/local/lib/python2.7/dist-packages (from mock>=2.0.0->tensorflow==1.5) (4.2.0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /usr/local/lib/python2.7/dist-packages (from mock>=2.0.0->tensorflow==1.5) (1.0.2)
Building wheels for collected packages: html5lib
Running setup.py bdist_wheel for html5lib ... done
Stored in directory: /root/.cache/pip/wheels/50/ae/f9/d2b189788efcf61d1ee0e36045476735c838898eef1cad6e29
Successfully built html5lib
magenta 0.3.11 has requirement tensorflow>=1.8.0, but you'll have tensorflow 1.5.0 which is incompatible.
Installing collected packages: html5lib, bleach, tensorflow-tensorboard, tensorflow
Found existing installation: html5lib 1.0.1
Uninstalling html5lib-1.0.1:
Successfully uninstalled html5lib-1.0.1
Found existing installation: bleach 2.1.3
Uninstalling bleach-2.1.3:
Successfully uninstalled bleach-2.1.3
Successfully installed bleach-1.5.0 html5lib-0.9999999 tensorflow-1.5.0 tensorflow-tensorboard-1.5.1
You are using pip version 18.0, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
I've noticed an error:
magenta 0.3.11 has requirement tensorflow>=1.8.0, but you'll have tensorflow 1.5.0 which is incompatible.
root@05e99fc1ad94:/magenta# python
Python 2.7.12 (default, Dec 4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> import sys
>>> from magenta.models.melody_rnn.melody_rnn_generate import console_entry_point
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "magenta/__init__.py", line 33, in <module>
import magenta.music.abc_parser
File "magenta/music/__init__.py", line 17, in <module>
from magenta.music.abc_parser import parse_abc_tunebook
File "magenta/music/abc_parser.py", line 33, in <module>
from magenta.protobuf import music_pb2
ImportError: No module named protobuf
So, now I need to find Magenta version, which can work with TF 1.5. TF 1.5 was released on Jan 26, 2018 and I.6 was released on Feb 28, 2018. So I should try install Magenta version, which was released around that time. Magenta 0.3.6 was released on Feb 28, 2018 as well. I'll try to install it first:
root@f0f77d90b611:/magenta-data# pip install magenta==0.3.6
Collecting magenta==0.3.6
Downloading https://files.pythonhosted.org/packages/6a/c7/285fc2338e0cdcf4435d056c8c949be3e123ed2a1e56c321a0b889ebd38b/magenta-0.3.6-py2.py3-none-any.whl (1.0MB)
100% |################################| 1.0MB 2.7MB/s
Requirement already satisfied: matplotlib>=1.5.3 in /usr/local/lib/python2.7/dist-packages (from magenta==0.3.6) (2.2.2)
Requirement already satisfied: python-rtmidi in /usr/local/lib/python2.7/dist-packages (from magenta==0.3.6) (1.1.1)
Requirement already satisfied: Pillow>=3.4.2 in /usr/local/lib/python2.7/dist-packages (from magenta==0.3.6) (5.2.0)
Requirement already satisfied: wheel in /usr/local/lib/python2.7/dist-packages (from magenta==0.3.6) (0.31.1)
Requirement already satisfied: scipy>=0.18.1 in /usr/local/lib/python2.7/dist-packages (from magenta==0.3.6) (1.1.0)
Requirement already satisfied: numpy>=1.11.0 in /usr/local/lib/python2.7/dist-packages (from magenta==0.3.6) (1.14.5)
Requirement already satisfied: tensorflow>=1.4.0 in /usr/local/lib/python2.7/dist-packages (from magenta==0.3.6) (1.5.0)
...
Installing collected packages: magenta, joblib
Found existing installation: joblib 0.11
Uninstalling joblib-0.11:
Successfully uninstalled joblib-0.11
Successfully installed joblib-0.13.2 magenta-0.3.6
Let's test it.
root@f0f77d90b611:/magenta-data# python Python 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import re >>> import sys >>> >>> from magenta.models.melody_rnn.melody_rnn_generate import console_entry_point >>>
It's a good sign! No core dump was detected this time.
Generating music
root@f0f77d90b611:/magenta# melody_rnn_generate --config=lookback_rnn --bundle_file=/magenta-models/lookback_rnn.mag --output_dir=/magenta-data/lookback_rnn/generated --num_outputs=10 --num_steps=128 --primer_melody="[60]" 2019-05-26 11:30:49.117489: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 WARNING:tensorflow:The saved meta_graph is possibly from an older release: 'model_variables' collection should be of type 'byte_list', but instead is of type 'node_list'. INFO:tensorflow:Restoring parameters from /tmp/tmp2GVxTU/model.ckpt 2019-05-26 11:30:49.589955: W tensorflow/core/framework/op_def_util.cc:343] Op TensorArray is deprecated. It will cease to work in GraphDef version 16. Use TensorArrayV3. 2019-05-26 11:30:49.590786: W tensorflow/core/framework/op_def_util.cc:343] Op TensorArraySize is deprecated. It will cease to work in GraphDef version 16. Use TensorArraySizeV3. 2019-05-26 11:30:49.591090: W tensorflow/core/framework/op_def_util.cc:343] Op TensorArrayScatter is deprecated. It will cease to work in GraphDef version 19. Use TensorArrayGradV3. 2019-05-26 11:30:49.591247: W tensorflow/core/framework/op_def_util.cc:343] Op TensorArrayRead is deprecated. It will cease to work in GraphDef version 16. Use TensorArrayReadV3. 2019-05-26 11:30:49.592265: W tensorflow/core/framework/op_def_util.cc:343] Op TensorArrayWrite is deprecated. It will cease to work in GraphDef version 16. Use TensorArrayWriteV3. 2019-05-26 11:30:49.592458: W tensorflow/core/framework/op_def_util.cc:343] Op TensorArrayGather is deprecated. It will cease to work in GraphDef version 16. Use TensorArrayGatherV3. INFO:tensorflow:Beam search yields sequence with log-likelihood: -79.070923 INFO:tensorflow:Beam search yields sequence with log-likelihood: -35.156246 INFO:tensorflow:Beam search yields sequence with log-likelihood: -58.394318 INFO:tensorflow:Beam search yields sequence with log-likelihood: -75.590439 INFO:tensorflow:Beam search yields sequence with log-likelihood: -38.718128 INFO:tensorflow:Beam search yields sequence with log-likelihood: -110.032028 INFO:tensorflow:Beam search yields sequence with log-likelihood: -115.888031 INFO:tensorflow:Beam search yields sequence with log-likelihood: -142.042130 INFO:tensorflow:Beam search yields sequence with log-likelihood: -123.102356 INFO:tensorflow:Beam search yields sequence with log-likelihood: -143.292236 INFO:tensorflow:Wrote 10 MIDI files to /magenta-data/lookback_rnn/generated
And I can see these generated midi files now:
root@f0f77d90b611:/magenta# ls -la /magenta-data/lookback_rnn/generated total 48 drwxr-xr-x 2 root root 4096 May 26 11:31 . drwxr-xr-x 3 root root 4096 May 24 02:09 .. -rw-r--r-- 1 root root 295 May 26 11:30 2019-05-26_113049_01.mid -rw-r--r-- 1 root root 168 May 26 11:30 2019-05-26_113049_02.mid -rw-r--r-- 1 root root 145 May 26 11:30 2019-05-26_113049_03.mid -rw-r--r-- 1 root root 361 May 26 11:30 2019-05-26_113049_04.mid -rw-r--r-- 1 root root 230 May 26 11:30 2019-05-26_113049_05.mid -rw-r--r-- 1 root root 266 May 26 11:30 2019-05-26_113049_06.mid -rw-r--r-- 1 root root 237 May 26 11:30 2019-05-26_113049_07.mid -rw-r--r-- 1 root root 519 May 26 11:31 2019-05-26_113049_08.mid -rw-r--r-- 1 root root 241 May 26 11:31 2019-05-26_113049_09.mid -rw-r--r-- 1 root root 354 May 26 11:31 2019-05-26_113049_10.mid
And I can play them too (listen to the attached file).
Now I need to generate music using my sensors data.