Installing Gateway Engine on a Wind River Linux Gateway


#1

I am trying to install Gateway Engine on a Wind River Linux 7.0 XT gateway but there are errors with missing dependencies.
Follow instructions here: http://docs.exosite.com/exositeready/gwe/solution-app/

Firstly, it comes with python 2.7.3 by default, but Gateway Engine throws errors and fails to install with this version. I do have a build of python 3.3.3 that I can install on the gateway, but then there are more decencies that would need to be custom compiled for the Wind River Linux. For example trying to use python3 to install pip fails for pkgutil not found. What are all the dependencies and modules needed to have Gateway Engine run on Wind River Linux? Those would need to be created as RPM files if not already available to be installed.

First error thrown using python 2.7.3:

Building wheels for collected packages: exosite, watchdog, ndg-httpsclient, PyYAML, pathtools, cryptography, pycparser
  Running setup.py bdist_wheel for exosite ... error
  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-HnHplh/exosite/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpOF9OcYpip-wheel- --python-tag cp27:
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-build-HnHplh/exosite/setup.py", line 37, in <module>
      data_files = data_files
    File "/usr/lib64/python2.7/distutils/core.py", line 138, in setup
      ok = dist.parse_command_line()
    File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/setuptools/dist.py", line 232, in parse_command_line
      result = _Distribution.parse_command_line(self)
    File "/usr/lib64/python2.7/distutils/dist.py", line 467, in parse_command_line
      args = self._parse_command_opts(parser, args)
    File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/setuptools/dist.py", line 558, in _parse_command_opts
      nargs = _Distribution._parse_command_opts(self, parser, args)
    File "/usr/lib64/python2.7/distutils/dist.py", line 523, in _parse_command_opts
      cmd_class = self.get_command_class(command)
    File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/setuptools/dist.py", line 362, in get_command_class
      ep.require(installer=self.fetch_build_egg)
    File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py", line 2029, in require
      working_set.resolve(self.dist.requires(self.extras),env,installer))
    File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py", line 2239, in requires
      dm = self._dep_map
    File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py", line 2468, in _dep_map
      self.__dep_map = self._compute_dependencies()
    File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py", line 2501, in _compute_dependencies
      common = frozenset(reqs_for_extra(None))
    File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py", line 2498, in reqs_for_extra
      if req.marker_fn(override={'extra':extra}):
    File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/_markerlib/markers.py", line 100, in marker_fn
      return eval(compiled_marker, environment)
    File "<environment marker>", line 1, in <module>
  NameError: name 'sys_platform' is not defined

#2

Hi @wRiver -

Thanks for posting! Right now Gateway engine does not support Python 3, so we can focus on getting Gateway Engine working with Python 2. Is there a chance that you can upgrade your Gateway’s version of Python? We typically test versions 2.7.9+ so we won’t be able to guarantee the behavior of Gateway Engine on an older version of Python.

Though to fix your immediate problems, you may need to install setuptools and to uninstall the distribute module. You should be able to do this with the following commands:

pip install setuptools

pip uninstall distribute 

Let me know how that works out for you. I am happy to help,
-Martin


#3

setuptools installed with no issues. The pip command to uninstall distribute failed as it couldn’t “remove entriles from nonexistent file …/easy-instal.pth.” ‘pip install exosite’ still threw an error and I could see distribute was installed so I just blanked out the path to it in the setuptools.pth file in site-packages directory. ‘pip install exosite’ then had an error saying libffi was not installed, so I made it from source.

Now the errors I’m seeing to install exosite fails on building wheel for cryptography, snipped below:

x86_64-wrs-linux-gcc -m64 -march=corei7 -mtune=corei7 -mfpmath=sse -msse4.2 -fno-strict-aliasing -O2 -pipe -g -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/_openssl.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_openssl.o
build/temp.linux-x86_64-2.7/_openssl.c:18:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
error: command ‘x86_64-wrs-linux-gcc’ failed with exit status 1


Failed building wheel for cryptography


#4

@wRiver

Are you trying to install Gateway Engine or the Exosite Solution CLI on your gateway?

I am not sure you will be wanting to deploy and configure Murano Solutions form your gateway, and that is what the ‘exosite’ tool does.

-Martin


#5

To clarify my understanding now, the Exosite Solution CLI is just another way of managing and configure your Murano Solution, in case you don’t want to manage it from the website UI? So the GWE is what goes on the gateway itself and can be configured with MrMurano or the web UI, and the Exosite CLI can go onto the dev machine for example.

Going from there, the CLI installed fine on a Unix machine I have, but installing GatewayEngine-1.4.5 on the gateway throws some errors on install looking for exo.device.

********************* Installation Summary : ******************************
GWE Installed Successfully
’ez_setup.py’ execution failed. Exiting…
Traceback (most recent call last):
File “/usr/bin/gwe”, line 9, in
load_entry_point(‘GatewayEngine==1.4.5’, ‘console_scripts’, ‘gwe’)()
File “/usr/lib/python2.7/site-packages/GatewayEngine-1.4.5-py2.7.egg/GatewayEngine/cli.py”, line 609, in main
parser = parse_args(sys.argv[1:])
File “/usr/lib/python2.7/site-packages/GatewayEngine-1.4.5-py2.7.egg/GatewayEngine/cli.py”, line 338, in parse_args
from GatewayEngine import GWE_VERSION
File “/usr/lib/python2.7/site-packages/GatewayEngine-1.4.5-py2.7.egg/GatewayEngine/GatewayEngine.py”, line 44, in
from exo.device import Device, DeviceCfgSect, DeviceCfgOpt
ImportError: No module named exo.device
Gateway Engine appears to have installed, but the command-line has a non-zero exit code.
Installation for elementtree.v126-20050316.tar.gz passed.
Installation for gmq.v1.1.38.tar.gz passed.
Installation for meld3.v102.tar.gz passed.
Installation for supervisor.v30.tar.gz passed.
Running self test…
Linux WR-IDP-E2F1 3.14.58_IDP-XT_3.1-WR7.0.0.13_idp #1 SMP PREEMPT Thu Feb 25 00:57:17 PST 2016 x86_64 x86_64 x86_64 GNU/Linux
x86_64
WR-IDP-E2F1
3.14.58_IDP-XT_3.1-WR7.0.0.13_idp
Linux
x86_64
#1 SMP PREEMPT Thu Feb 25 00:57:17 PST 2016
…FTraceback (most recent call last):
File “/usr/bin/gwe”, line 9, in
load_entry_point(‘GatewayEngine==1.4.5’, ‘console_scripts’, ‘gwe’)()
File “/usr/lib/python2.7/site-packages/GatewayEngine-1.4.5-py2.7.egg/GatewayEngine/cli.py”, line 609, in main
parser = parse_args(sys.argv[1:])
File “/usr/lib/python2.7/site-packages/GatewayEngine-1.4.5-py2.7.egg/GatewayEngine/cli.py”, line 338, in parse_args
from GatewayEngine import GWE_VERSION
File “/usr/lib/python2.7/site-packages/GatewayEngine-1.4.5-py2.7.egg/GatewayEngine/GatewayEngine.py”, line 44, in
from exo.device import Device, DeviceCfgSect, DeviceCfgOpt
ImportError: No module named exo.device
FTraceback (most recent call last):
File “/usr/bin/gmq”, line 5, in
from pkg_resources import load_entry_point
File “/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py”, line 2807, in
working_set.require(requires)
File “/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py”, line 690, in require
needed = self.resolve(parse_requirements(requirements))
File “/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py”, line 588, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: twisted>=13.0.2
F

FAIL: test_08_gdc (test.self_test.SelfTest)

Traceback (most recent call last):
File “test/self_test.py”, line 106, in test_08_gdc
self.fail(“Test ‘{}’ FAILED”.format(cmd))
AssertionError: Test ‘gdc --version’ FAILED

======================================================================
FAIL: test_09_gwe (test.self_test.SelfTest)

Traceback (most recent call last):
File “test/self_test.py”, line 113, in test_09_gwe
self.fail(“Test ‘{}’ FAILED”.format(cmd))
AssertionError: Test ‘gwe --version’ FAILED

======================================================================
FAIL: test_10_gmq (test.self_test.SelfTest)

Traceback (most recent call last):
File “test/self_test.py”, line 121, in test_10_gmq
self.fail(“Test ‘{}’ FAILED”.format(cmd))
AssertionError: Test ‘gmq --version’ FAILED


Ran 10 tests in 0.674s

FAILED (failures=3)
uname -a
uname -a
uname -m
uname -m
uname -n
uname -n
uname -r
uname -r
uname -s
uname -s
uname -p
uname -p
uname -v
uname -v
import json
Test ‘import json’ PASSED
import requests
import twisted
import exo
import GatewayEngine
Test ‘import GatewayEngine’ PASSED
import gmq
Test ‘import gmq’ PASSED
********************* Installation Complete ******************************


#6

Have you had a chance to look at the above error?


#7

Sorry about the delayed response.

Each time you make an attempt to install gateway engine, it creates a file to capture the entire installation log, it is called gateway-engine/gwe_install_${TIMESTAMP}.log. Could you email that log to gwesupport@exosite.com with a reference link to this ticket?

It looks as though most, if not all, of the libraries are failing to install. This usually indicates there is an issue with setuptools and distribute on older versions of Python, but having the full log will help clear that up.


#8

I received your installation log file and saw that pip was having trouble installing:

+ '[' -e pip-8.1.2.tar.gz ']'
+ python -c 'import tarfile ; tarfile.open('\''pip-8.1.2.tar.gz'\'').extractall()'
+ cd pip-8.1.2
+ '[' 0 -ne 0 ']'
+ python setup.py install
running install
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 2] No such file or directory: '/usr/lib/python2.7/site-packages/test-easy-install-5095.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/lib/python2.7/site-packages/

This directory does not currently exist.  Please create it and try again, or
choose a different installation directory (using the -d or --install-dir
option).

Can you try installing as root or use sudo when running the installer?

cd gateway-engine/
sudo ./install.sh

http://docs.exosite.com/development/exositeready/gwe/getting_started/#install-gwe-onto-your-gateway

Please let me know if this helps to resolve your issue.


#9

Solved! Learned root doesn’t have full permissions (was installing as root previously). Using sudo worked though.

Thank you!


#10

You’re welcome :slight_smile:

Out of curiosity, what hardware platform were you testing on?

Thanks!


#11

An Advantech Gateway.

So the gmq service did not install properly. I added some missing python header files manually and was able to install twisted and the gmq service is running now (haven’t tested that it works though).

Looking over the install log though, it appears there are a number of errors around finding sys_platform. I believe this is an issue with setuptools but the system shows it as installed and being up to date. And despite the errors there is only one error in the install summary:

********************* Installation Summary : ******************************
GWE Installation Failed : Error Code 1
’ez_setup.py’ installation successful.
device-client: 1.4.22
gateway-engine: 1.4.5
Installation for elementtree.v126-20050316.tar.gz passed.
Installation for gmq.v1.1.38.tar.gz passed.
Installation for meld3.v102.tar.gz passed.
Installation for supervisor.v30.tar.gz passed.
Running self test…
Linux WR-IDP-E2F1 3.14.58_IDP-XT_3.1-WR7.0.0.13_idp #1 SMP PREEMPT Thu Feb 25 00:57:17 PST 2016 x86_64 x86_64 x86_64 GNU/Linux
x86_64
WR-IDP-E2F1
3.14.58_IDP-XT_3.1-WR7.0.0.13_idp
Linux
x86_64
#1 SMP PREEMPT Thu Feb 25 00:57:17 PST 2016
…1.4.22
.device-client: 1.4.22
gateway-engine: 1.4.5
.1.1.38
.


#12

Well after some searching it sounds to me like the kernel preempted the installation process.

I am out of my depth here. @willcharlton have you seen similar output on a different image/architecture?


@wRiver

What is the affect of this installation complaint? Is GWE running, and writing to Exosite, on your machine?

-Martin


#13

Good question. Can you send me the latest installation log?

Thanks!


#14

I don’t see any effect during my current testing with errors, I was able to get the gmq-sine-demo to send data from the gateway to murano cloud.

I sent the install log to gwesupport@exosite.com on Weds 3/29 at 10:21am PST.


#15

I see two issues in the install log.

  1. The Python distribute package seems to be fighting with setuptools.
  2. Wind River Linux doesn’t seem to have the pidof callable.

The Python distribute is a known problem in the Python community and it can be a tricky one to solve. Here is the stacktrace of the issue that is repeated in the log file everytime setuptools is used to install a package:

Traceback (most recent call last):
  File "setup.py", line 119, in <module>
    "License :: Other/Proprietary License",
  File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
    _setup_distribution = dist = klass(attrs)
  File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/setuptools/dist.py", line 225, in __init__
    _Distribution.__init__(self,attrs)
  File "/usr/lib64/python2.7/distutils/dist.py", line 287, in __init__
    self.finalize_options()
  File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/setuptools/dist.py", line 257, in finalize_options
    ep.require(installer=self.fetch_build_egg)
  File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py", line 2029, in require
    working_set.resolve(self.dist.requires(self.extras),env,installer))
  File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py", line 2239, in requires
    dm = self._dep_map
  File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py", line 2468, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py", line 2501, in _compute_dependencies
    common = frozenset(reqs_for_extra(None))
  File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/pkg_resources.py", line 2498, in reqs_for_extra
    if req.marker_fn(override={'extra':extra}):
  File "/usr/lib64/python2.7/site-packages/distribute-0.6.32-py2.7.egg/_markerlib/markers.py", line 100, in marker_fn
    return eval(compiled_marker, environment)
  File "<environment marker>", line 1, in <module>
NameError: name 'sys_platform' is not defined

I think the version of distribute that your gateway has is causing a compatibility issue. Could you try to pip uninstall distribute and re-run the installer? If this works, then I can update the installer to look for bad/old versions of distribute and try to correct it before installing packages and dependencies.

With regards to the pidof issue, I have created a ticket to update our installer to look for pids in other ways.


#16

distribute might be fighting with setuptools because of 32 bit vs 64 bit versions, distribute is in usr/lib… and setuptools is in /usr/lib64… Hence pip uninstall was unhappy as the easy-install.pth was missing from /usr/lib64/python2.7/site-packages/easy-install.pth, it was in instead /usr/lib/python2.7/site-packages/easy-install.pth. I tried a force reinstall as pip install found the existing version (pip install --upgrade --no-deps --force-reinstall dstribute) but it still looked for that file so I just copied the easy-install.pth to the folder pip was looking in. Then pip was able to uninstall the old 0.6.32 install distribute 0.7.3 into usr/lib/…

From there the install of gateway had no more error messages in the logs about sys_platform.

********************* Installation Summary : ******************************
GWE Installed Successfully
’ez_setup.py’ installation successful.
device-client: 1.4.22
gateway-engine: 1.4.5
Installation for elementtree.v126-20050316.tar.gz passed.
Installation for gmq.v1.1.38.tar.gz passed.
Installation for meld3.v102.tar.gz passed.
Installation for supervisor.v30.tar.gz passed.
Running self test…
Linux WR-IDP-E2F1 3.14.58_IDP-XT_3.1-WR7.0.0.13_idp #1 SMP PREEMPT Thu Feb 25 00:57:17 PST 2016 x86_64 x86_64 x86_64 GNU/Linux
x86_64
WR-IDP-E2F1
3.14.58_IDP-XT_3.1-WR7.0.0.13_idp
Linux
x86_64
#1 SMP PREEMPT Thu Feb 25 00:57:17 PST 2016
…1.4.22
.device-client: 1.4.22
gateway-engine: 1.4.5
.1.1.38
.


#17

Nicely done! Thanks for posting your results.

I’m certain others will benefit from your work :smiley: