I have a Travis CI build that is failing; py.test seems to be trying to import the wrong module, though I cannot reproduce this locally. I expect it to import tools.lint.tests.test_lint, not lint.tests.test_lint, as you can see in the traceback, given that build has --full-trace! This leads to the error beneath it when it tries to do a relative import from the tools package.
The short trackback is:
___________________ ERROR collecting lint/tests/test_lint.py ___________________
.tox/py27/lib/python2.7/site-packages/py/_path/local.py:650: in pyimport
__import__(modname)
lint/__init__.py:1: in <module>
from . import lint
lint/lint.py:15: in <module>
from .. import localpaths
E ValueError: Attempted relative import beyond toplevel package
Given the name of the top level package is just the directory that the repo is in, I wouldn't be surprised to see that differ—but I'd still expect to see it there!
Take a look at the path Travis has that file at: /home/travis/build/w3c/wpt-tools/lint/tests/test_lint.py. The directory called tools on your computer is called wpt-tools on Travis, following the name of the repo on GitHub.
Vitally, wpt-tools isn't a valid Python package name, as Python packages cannot contain hyphens in the name. (They have to be an identifier). This leads py.test to conclude it isn't a package, despite the __init__.py contained within, and hence it doesn't include it in the import path, leading to the error when code tries a relative import from what is meant to be the top-level package.
There's a couple of solutions here:
The possibly simple one is renaming the repository so that it doesn't contain any hyphens, though obviously if you're an established repository this is likely undesirable.
Get Travis CI to run the code from some directory, by copying/moving the repository to a directory whose name doesn't contain a hyphen, at the start of before_install, using something like:
before_install:
- mv `pwd` /tmp/tools
- cd /tmp/tools
This will then run all the install and later steps from /tmp/tools, which will allow everything to run as expected.
(Note you cannot use a symbolic link here as os.getcwd() in Python will eliminate the link from the path, returning the real path, leading that seeming workaround not to work at all.)
Related
I am trying to run the AdaIN-style model in Google Colab. In order to do this, I need to install Lua. I saw this question and tried the solution given (reproduced below for convenience), both with the given distro and the official torch distro.
%cd /content/
!git clone https://github.com/nagadomi/distro.git torch --recursive
import os
os.chdir('./torch/')
!bash install-deps
!./install.sh
!. ./install/bin/torch-activate
Now it should work using absolute path to th:
!/content/torch/install/bin/th neural_style.lua -style_image examples/inputs/picasso_selfport1907.jpg -content_image examples/inputs/brad_pitt.jpg
However, the last line fails, giving me the below error
/bin/bash: . ./install/bin/torch-activate: No such file or directory
However, the binary for th seems to exist at the given filepath. Thus, I try to run the given command
th test.lua -content input/content/cornell.jpg -style input/style/woman_with_hat_matisse.jpg
but I get the following error
/content/torch/install/bin/luajit: /content/torch/install/share/lua/5.1/trepl/init.lua:389: module 'cudnn' not found:No LuaRocks module found for cudnn
I tried looking this up further, but wasn't able to make much progress.
I also tried this method, which seems similar to the above, but I get the same errors.
Any pointers would be appreciated.
Error that I'm struggling with
I'm trying to use MarsExplorer (https://github.com/dimikout3/MarsExplorer) in google colab, but I'm struggling with properly importing it. I was able to successfully clone the repo, but when I try to run the demo mars_explorer isn't imported. How can I fix this?
Indeed #Dr.Snoopy makes a great point here. This is the line in test.py where it is expected to work:
from mars_explorer.envs.settings import DEFAULT_CONFIG as conf
Now, notice that this is being called from your runtime path, usually root in Colab's case (I suppose), then it should return an error.
Now, if installed, for instance with pip install -e you can both import the module and edit it as you go.
Check out these docs.-,6.1.2.%20The%20Module%20Search%20Path,-%C2%B6), they may be helpful to understand this error.
apply these two commands in colab cell:
!git clone https://github.com/dimikout3/GeneralExplorationPolicy.git
the above command will save GeneralExplorationPolicy folders in your colab and inside this folder you can find mars-explorer folder then make copy of the path of the mars-explorer folder for example. /content/GeneralExplorationPolicy/mars-explorer
then apply this command for install mars-explorer:
!pip install -e /content/GeneralExplorationPolicy/mars-explorer
I have been using a google colab template for iterative LQR that uses the Pydrake, however, it seems like the code repository is removed and I can't reinstall it on google Colab:
try:
import pydrake
import underactuated
except ImportError:
!curl -s https://raw.githubusercontent.com/RussTedrake/underactuated/master/scripts/setup/jupyter_setup.py > jupyter_setup.py
from jupyter_setup import setup_underactuated
setup_underactuated()
# Setup matplotlib backend (to notebook, if possible, or inline).
from underactuated.jupyter import setup_matplotlib_backend
plt_is_interactive = setup_matplotlib_backend()
File "/content/jupyter_setup.py", line 1
404: Not Found
^
SyntaxError: invalid syntax
I tried clicking this link https://raw.githubusercontent.com/RussTedrake/underactuated/master/scripts/setup/jupyter_setup.py, and the page is not found... everything was working fine yesterday
Sorry. You're correct... I updated it this morning, and don't have a good deprecation policy in place on that repo, and this setup script is two versions ago. The path you want is https://raw.githubusercontent.com/RussTedrake/underactuated/master/setup/jupyter_setup.py
(remove the script from the directory). But if you look at that file, you'll see that even that is pointing to an updated setup script which you might want to point to.
This is actually all good news... we are on the path to a much better solution. You can now just pip install drake on colab (see the drake installation guide). Once I land the pip install underactuated (probably in time for my Spring offering of the class), then all of that nasty setup will be gone.
A C extension module that compiles fine on Travis-CI without anaconda fails when installed with anaconda. It appears to install just fine, but when I try to import it, I get the following error:
ImportError: /home/travis/anaconda/lib/python2.7/site-packages/quaternion/numpy_quaternion.so: undefined symbol: __log_finite
The full error can be seen here. Obviously, this looks like a linker error, where it can't find glibc (which I believe is where __log_finite is found). But why should it fail to find glibc?
When I run nm on that .so file (through Travis), it shows that __log_finite is indeed undefined, but shouldn't it find it through the usual process?
I've tried installing quaternion through pip and I've tried installing it by directly downloading it and running python setup.py install. Both seem to work, in the sense that it looks like all the files are where they should be. But both fail on import because they can't find that symbol.
I've even tried installing the full version of anaconda (rather than just miniconda, which is recommended). Nothing seems to work. How can I make Travis find that symbol, and is this something I'll have to worry about ordinarily with my distribution?
It appears to be a problem with a -ffast-math flag in my quaternion package. One thing that flag does is make the code assume that the numbers are finite, so that instead of using the log function, it uses some log_finite function, which for some reason Travis doesn't have --- or something. Anyway, I have my numba package set an environment variable in Travis builds, which the quaternion package then looks for on installation, and turns off fast-math. This is unfortunate, because it means I'm not actually testing the code as it's actually used. But it means my code builds and tests pass.
There seems to be about one mention of this on the internet. Or not; I can't tell.
The dart page http://pub.dartlang.org/doc/#adding-a-dependency describes how you can have a dart file (parser_test.dart) import files from its own package using the 'import "package:..." style of import. It seems to imply this is a good thing - better than using relative paths. This example shown is for a file in test which appears to be special. But, then why does it not make sense for importing same package lib files from a lib in the package. Maybe it does make sense, but if so the pub update does not make it convenient.
foo/
/lib/
foo_lib_1.dart
foo_lib_2.dart
src/
foo_lib_1/
foo_lib_1_impl.dart
foo_lib_2/
foo_lib_2_impl.dart
Assume foo_lib_2 uses foo_lib_1. There are two options for foo_lib_2.dart:
import "../foo_lib_1.dart";
import "packages:foo/foo_lib_1.dart";
My guess is the suggested approach is the first for any such import that resides under lib. The reason I think this is pub update seems to automagically provide a soft link in the packages folder of any of bin, test, or example to foo, like foo -> ../lib. Yet, it does not do the same for the packages folder in top level foo. This means to get the second type of import (i.e. the packages import) to work you need to add:
foo:
path: lib
to the dependencies of foo in the pubspec.yaml. Is there any advantage or disadvantage for a library to use the package style import to import another library (not in test, bin, or example) from its own package? Is there a reason for the apparent inconsistency?
After accepting the answer below, I still am not seeing it. Here is what I'm seeing in a shell session and I would like to reconcile this behavior with the answer. Any explanations appreciated. I am using emacs instead of DartEditor, thus the old-school command line approach here.
### Show all files, one dart library file and one yaml, plus empty
### lib and test folders
user#user-thinkpad:/tmp/uml_codegen_sample$ ls -R
.:
lib pubspec.yaml test
./lib:
plusauri.dart
./test:
### Show contents of pubspec
user#user-thinkpad:/tmp/uml_codegen_sample$ cat pubspec.yaml
name: domain_model
version: 0.0.1
description: >
Auto-generated support from /home/user/plusauri/modeling/plusauri.xmi.json
dependencies:
ebisu:
path: /home/user/open_source/codegen/dart/ebisu
### Run pub install and show the changes. Note there is a soft
### link to packages from test, but not lib.
user#user-thinkpad:/tmp/uml_codegen_sample$ pub install
Resolving dependencies...
Dependencies installed!
Some packages that were installed are not compatible with your SDK version 0.4.7+5.r21658 and may not work:
- 'pathos' requires >=0.5.0+1
You may be able to resolve this by upgrading to the latest Dart SDK
or adding a version constraint to use an older version of a package.
user#user-thinkpad:/tmp/uml_codegen_sample$ ls -R
.:
lib packages pubspec.lock pubspec.yaml test
./lib:
plusauri.dart
./packages:
domain_model ebisu pathos
./test:
packages
### Note here the program does not work, and suspiciously pub
### install put no packages link under lib like it did test
user#user-thinkpad:/tmp/uml_codegen_sample$ dart lib/plusauri.dart
Unable to open file: /tmp/uml_codegen_sample/lib/packages/ebisu/ebisu_utils.dart'file:///tmp/uml_codegen_sample/lib/plusauri.dart': Error: line 5 pos 1: library handler failed
import "package:ebisu/ebisu_utils.dart" as EBISU_UTILS;
^
### Copy the same dart file to test to show that it can run there
### just fine
user#user-thinkpad:/tmp/uml_codegen_sample$ cp lib/plusauri.dart test/
user#user-thinkpad:/tmp/uml_codegen_sample$ dart test/plusauri.dart
Main for library plusauri
user#user-thinkpad:/tmp/uml_codegen_sample$
### Finally, manually create the soft link in lib, to show it will
### then run
user#user-thinkpad:/tmp/uml_codegen_sample$ ln -s ../packages lib/packages
user#user-thinkpad:/tmp/uml_codegen_sample$ dart lib/plusauri.dart
Main for library plusauri
Actually, you can definitely import using the package:foo/foo_lib_1.dart syntax without needing to change your pubspec.yaml or even creating a pubspec.yaml in the first place!
You can see that this is true from a language level, in this test: https://github.com/dart-lang/bleeding_edge/blob/master/dart/tests/standalone/package/packages/package1.dart
and an example of this in the wild is: https://github.com/kevmoo/hop.dart/blob/master/lib/hop_tasks.dart#L17
I do not think there is any benefit from writing one way or another except that writing relative paths is slightly shorter.
From a project structure point of view, I would use relative path imports when I am drilling into subdirectories that are not going to be exposed to the user. src is generally seen as implementation specific details that won't be visible to external users, so use relative paths to your heart's content.
However, if you're working within multiple directories, then you should use package: imports to reinforce the idea that the parts are stand-alone and interchangeable. Within the lib directory itself, you want to say that these two libraries, although they might rely on each other, can live separately and are not bound by their physical location.
I would recommend not ever using ../ in your imports, as that is fragile and may break in strange ways if/when you modify directory structure or deploy.