I have a Python package that is cross-compiled against multiple Python runtimes (2.7, 3.3-6). I'm attempting to use TravisCI to build and deploy that package to PyPi. However, each runtime variant of a particular version tag produces an artifact of the same name: mypackage-0.11.10.tar.gz. As a result, PyPi only accepts the first runtime variant.
To reiterate: the build for a given version (say, 0.11.10) succeeds in each runtime (2.7, 3.3, ..., 3.6), but each of these has the same artifact: mypackage-0.11.10.tar.gz. What I'd like to get to is something like mypackage-0.11.10-py27.tar.gz, or whatever might be appropriate.
It's possible I'm missing something, and I can be confident that if I have the build work against every runtime I'm good to go... but I lack that confidence, and don't know a good way to test it.
A sub-component of this is simply how to change the filename of the artifact being sent to PyPi by TravisCI? I was unable to determine if there was any way to do that.
This is my .travis.yml file:
language: python
python:
- '2.7'
- '3.3'
- '3.4'
- '3.5'
- '3.6'
- pypy
install:
- pip install coveralls
- pip install 'coverage<4'
script: nosetests --with-coverage --cover-package=<my package name>
after_success:
- coveralls
deploy:
provider: pypi
user: <my pypi username>
password:
secure: <my encrypted pypi password>
on:
tags: true
distributions: sdist bdist_wheel
repo: <my repo>
branch: production
.tar.gz suggests it's a source distribution. You cannot have multiple source distributions for a version. But you can have many binary wheels, each one for a different Python version and architecture.
Related
I'm trying to create CI pipeline with GitHub, Travis CI and AWS ECS. When I'm push commit to master branch, I'm getting error in travis CI: 'Could not parse .travis.yml'. I can't figure out, where is the problem. Travis dosen't provide more information about error.
There is a code, which I'm using:
.travis.yml
language: csharp
dist: trusty
sudo: required
mono: none
dotnet: 2.0.0
branches:
only:
- master
before_script:
- chmod -R a+x scripts
script:
- ./scripts/dotnet-build.sh
- ./scripts/dotnet-publish.sh
- ./scripts/docker-publish-travis.sh
dotnet-build.sh
dotnet restore
dotnet build
dotnet-publish.sh
dotnet publish ./BookMeMobi2 -c Release -o ./bin/Docker
dotnet-publish-travis.sh
pip install --user awscli
eval $(aws ecr get-login --no-include-email --region eu-central-1)
docker build -t bookmemobi2 .
docker ps
docker tag bookmemobi2:latest 601510060817.dkr.ecr.eu-central-1.amazonaws.com/bookmemobi2:latest
docker push 601510060817.dkr.ecr.eu-central-1.amazonaws.com/bookmemobi2:latest
I don't know where is the problem. Could you help me?
Use yamllint, which you can install, or just copy&paste to a web-based version.
With the example in the question, I get:
(<unknown>): found character that cannot start any token while scanning for the next token at line 7 column 1
There's a tab on line 7. See "https://stackoverflow.com/q/19975954".
I had a similar problem. In my case I was using python to launch a couple of scripts. I placed them one after the other with a hyphen at the beginning, exactly as you. So I searched to found out that I could place all of them in one line with "&" between each script and I got rid of the hyphen.
What I had:
script:
- python test_Math_DC.py
- python test_Math_Moy.py
- python test_Math_Var.py
- python test_Math_SQRT.py
Changed to :
script: python test_Math_DC.py & python test_Math_Moy.py & python test_Math_Var.py & python test_Math_SQRT.py
In your case you could try :
script: ./scripts/dotnet-build.sh & ./scripts/dotnet-publish.sh & ./scripts/docker-publish-travis.sh
or something like this :
script: sh ./scripts/dotnet-build.sh & sh ./scripts/dotnet-publish.sh & sh ./scripts/docker-publish-travis.sh
And see how it works out.
The travis cli tool has a linter
gem install travis
However, it only gives warnings for the example. Also, it currently does not work with all features, for example stages.
$ travis lint
Warnings for .travis.yml:
[x] unexpected key mono, dropping
[x] unexpected key dotnet, dropping
I am running a CI pipeline to build firmware for ESP8266 using plaitformio and bitbucket pipelines, my code builds successfully and now I want to cache the directory that contains the platformio libraries (.piolibdeps). Here are the contains of my platform.ini file.
[env:nodemcuv2]
platform = espressif8266
board = nodemcuv2
framework = arduino
upload_port = 192.168.1.108
lib_deps =
ESPAsyncTCP#1.1.0
OneWire
Time
FauxmoESP
Blynk
DallasTemperature
ArduinoJson
Adafruit NeoPixel
How to cache this directory in BitBucket pipelines? Please see below the contents of bitbucket-pipelines.yml file, with this it is not caching the defined directory, what's wrong here?
image: eclipse/platformio
pipelines:
branches:
develop:
- step:
name: Build Project
caches: # caches the depende
- directories
script: # Modify the commands below to build your repository.
- pio ci --project-conf=./Code/UrbanAquarium.Firmware/platformio.ini ./Code/UrbanAquarium.Firmware/src
- pwd
definitions:
caches:
directories: ./Code/UrbanAquarium.Firmware/.piolibdeps
And here my folder structure.
in case you're still looking for an answer - I think you got it almost right, but probably need to specify a custom --build-dir (so that you can specify the same path for your cache) as well as --keep-build-dir (see https://docs.platformio.org/en/latest/userguide/cmd_ci.html). Also, I'm not sure why you specified a ./Code/UrbanAquarium.Firmware/ prefix.
That said, I've tried the above and it became quickly ugly - for now I'll only cache ~/.platformio, as well as the default pip cache:
image: python:2.7.16
pipelines:
default:
- step:
caches:
- pip
- pio
script:
- pip install -U platformio
- platformio update
- platformio ci src/ --project-conf=platformio.ini
definitions:
caches:
pio: ~/.platformio
I'd like Travis CI to build and deploy the following artefacts to PyPI whenever a new commit hits the master branch:
Python 2 wheel
Python 3 wheel
Source
To make this happen, I've added the following to .travis.yml:
language: python
python:
- '2.7'
- '3.5'
- '3.6'
deploy:
on:
branch: master
provider: pypi
distribution: bdist_wheel sdist
For normal build/test, the configuration works great. However, it introduces a race condition when deploying to PyPI:
Uploading distributions to https://upload.pypi.org/legacy/
Uploading PyOTA-2.0.0b1.tar.gz
HTTPError: 400 Client Error: File already exists. for url: https://upload.pypi.org/legacy/
What changes should I make to .travis.yml to get Travis CI to deploy the correct artefacts to PyPI?
I ran into this problem today and eventually found this under-documented gem:
deploy:
provider: pypi
skip_existing: true
...
I use skip_existing: true on a project to get source and wheels published once even though I test across a couple of different configurations and python versions. Handy. More details in this resolved github issue. I also submitted a documentation diff.
Some days I think outside the box; other days it's just a really big box.
Previously, this project needed separate wheels for Python 2 and Python 3, so I needed Travis CI to build wheels using different versions of Python.
But recently I got the project to build universal wheels correctly, so now Travis can build all of the deployment artefacts using any one version of Python.
I modified .travis.yml accordingly, and everything is working great:
deploy:
on:
branch: master
python: '3.6'
I'm building a python package which is mostly c++ code (think numpy)
My travis file is currently
language: cpp
compiler:
- gcc
- clang
os:
- linux
- osx
dist: trusty
script: "make pcst_fast_test && ./pcst_fast_test"
notifications:
...
But I've also written some python tests in a file called test_pcst_fast.py. Is there some way to call those from travis as well?
It seems ambiguous as to whether travis supports multiple languages in one file, but it seems like most people pull this off despite only listing one language under the language tag.
Travis doesn't support multiple languages per job yet.
Look at categories after_success and after_script in the Travis build lifecycle docs
Also, you can add more build scripts, they will run independently, just list them like this:
script:
- "make pcst_fast_test && ./pcst_fast_test"
- "./test_pcst_fast.py"
If there is no python installed (not sure about c builds), you can install it, check out this .travis.yml, it installs custom python interpreter as a dependency.
You can't add multiple languages, which explains the behavior you are seeing, and the node_js setting will only be recognized in a node language project.
What you can do is utilize the incorporated nvm in TravisCI.
For example, you can include
nvm install 0.10
nvm use 0.10
in your before_install section to download the latest v0.10.x release of node.
However,
On a Travis Java build environment, you can use nvm to manage Node.js runtimes:
.travis.yml
language: java
jdk:
- oraclejdk8
env:
- NODE_VERSION="0.12"
before_install:
- nvm install $NODE_VERSION
If your Node version is very recent, you might have to update nvm too.
To update nvm, write this in your .travis.yml:
before_install:
- wget https://raw.githubusercontent.com/creationix/nvm/v0.31.0/nvm.sh -O ~/.nvm/nvm.sh
- source ~/.nvm/nvm.sh
- nvm install 5 # for Node v5
- node --version
The above example shows how to first update to nvm v0.31, to then obtain Node v5.
I'm having troubles building a project with maven on travis-ci because travis automatically runs
mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V
which fails because of a timeout:
No output has been received in the last 10 minutes, this potentially
indicates a stalled build or something wrong with the build itself
According to the documentation I should be able to override it defining a custom script in .travis.yml but it does not work, here my configuration:
sudo: false
language:
- java
script: "travis_wait mvn -T4 -pl quickfixj-codegenerator install"
jdk:
- oraclejdk8
env:
- MAVEN_OPTS="-Xms2048m -Xmx=2048m"
branches:
only:
- travis-ci-build
Is there any way to avoid the automatic mvn install or to tweak it ?
This is mentioned in the documentation:
https://docs.travis-ci.com/user/job-lifecycle/#skipping-the-installation-phase
Skip the installation step entirely by adding the following to your .travis.yml:
install: skip
The install step runs before script step, and with Maven you don't usually need the install step, at least I personally haven't ever found it useful — Maven will download dependencies on script step anyway.
I had the same issue. It was solved after some discussion with the Travis CI support. Here is their response:
That maven command is run as part of the install section of your
build, it's the default.
If you want to skip this step, you can override it by adding this to
your .travis.yml file:
install: /bin/true
I have found it useful to have an install: mvn dependency:resolve step that downloads the build dependencies up front, so that the output of the actual build script is kept clean