I'm trying to set up a task in Airflow that creates a pod in Google Kubernetes from a Docker image in Registry. The task fails and returns:
Pod Launching failed: Pod took too long to start
Which does not say much. In a similar question, someone suggested adding a tag to the image version when using a private repository, but it did not solve the problem.
The operator is defined as:
start_pod_job = GKEStartPodOperator(task_id="start_pod_job",
project_id="my-project",
gcp_conn_id=GCP_CONN_ID,
location="us-east1-a",
cluster_name="us-east-cluster-name",
is_delete_operator_pod=True,
namespace="default",
image="us.gcr.io/my-project/my-image:v1",
cmds=['gs://my-bucket/my-file.csv'],
name="my-name")
The entrypoint of the image is [ "python", "./myScript.py"]. It takes as a parameter the URI of a file in Google Storage. I was able to run the image in my local machine successfully.
I'm currently using Airflow 1.10.15 and Composer 1.16.14.
Any suggestions?
Related
I just downloaded this docker image to set up a spark cluster with two worker nodes. Cluster is up and running however I want to submit my scala file to this cluster. I am not able to start spark-shell in this.
When I was using another docker image, I was able to start it using spark-shell.
Can someone please explain if I need to install scala separately in the image or there is a different way to start
UPDATE
Here is the error bash: spark-shell: command not found
bash: spark-shell: command not found
root#a7b0682ff17d:/opt/spark# ls /home/shangupta/Scripts/
ProfileData.json demo.scala queries.scala
TestDataGeneration.sql input.scala
root#a7b0682ff17d:/opt/spark# spark-shell /home/shangupta/Scripts/input.scala
bash: spark-shell: command not found
root#a7b0682ff17d:/opt/spark#
You're getting command not found because PATH isn't correctly established
Use the absolute path /opt/spark/bin/spark-shell
Also, I'd suggest packaging your Scala project as an uber jar to submit unless you have no external dependencies or like to add --packages/--jars manually
I'm running superset on MacOS in docker and I'm trying to get OAuth working.
I’ve edited the config file /docker/pythonpath_dev/superset_config.py and added the OAuth configuration.
One of the lines I added was
AUTH_TYPE = AUTH_OAUTH
This required me to import the auth types as below:
from flask_appbuilder.security.manager import (
AUTH_OID,
AUTH_REMOTE_USER,
AUTH_DB,
AUTH_LDAP,
AUTH_OAUTH,
)
When I try to start up superset with the following command: docker-compose -f docker-compose-non-dev.yml up
I get the following error:
File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/manager.py", line 250, in __init__
from authlib.integrations.flask_client import OAuth
ModuleNotFoundError: No module named 'authlib'
I'm fairly new to docker itself. How do I go about resolving this?
In case anybody else comes across this, the solution was to add the Authlib module to the python env on the docker image.
The process for adding a new python module to the docker image is documented here: https://github.com/apache/superset/blob/master/docker/README.md#local-packages
Quoted below in case that file changes:
If you want to add python packages in order to test things like DBs locally, you can simply add a local requirements.txt (./docker/requirements-local.txt) and rebuild your docker stack.
Steps:
1. Create ./docker/requirements-local.txt
2. Add your new packages
3. Rebuild docker-compose
a. docker-compose down -v
b. docker-compose up
Important was running docker-compose up and not docker-compose -f docker-compose-non-dev.yml up. The latter does not seem to rebuild the docker image.
I'm trying to deploy the official CGAL docker. From reading the README I understand that after downloading the specific image (e.g I want to open a docker with ubuntu16+CGAL and all of it's dependencies) using the following command:
docker pull cgal/testsuite-docker:ubuntu # get a specific image by replacing TAG with some tag
I need to install the cgal library itself using the
./test_cgal.py --user **** --passwd **** --images cgal-testsuite/ubuntu
The thing is that eventually I want to start the docker with an interactive shell, i.e
docker run --rm -it -v $(pwd):/source somedocker
And I couldn't understand where is the generated image, after the CGAL installation script.
Those images are not for running CGAL. They are only images we use to define an environment for our testsuite, and run tests in it, including compiling CGAL.
test_cgal.py will download the integration branch, which is rarely working as it is the branch in which we merge our PR to test them nightly. Don't use this to get a working CGAL. To my knowledge, there is no such image as the one you are looking for. No official one anyways.
Furthermore, installing cgal at runtime in this image will not modify the image, once you close the container your installation will be lost. You need to specify how to install CGA in the Dockerfile of your image and
then build it if you want a "ready to use" image.
You can use the dockerfile of the image you found to write your own, as there should be all the dependencies specified in it, but you need to edit it to download CGAL and maybe build it if you don't want the header-only version. This is not done in test-cgal.py or anywhere in this docker repository.
I built an image with Google Cloud Build using Docker Compose. In my cloudbuild.yml file I have the following steps:
Build the docker image using docker compose
Tag the built image
Create an instance template
Create instance group
Now here is the problem every time a new instance gets built the created container from the image keeps restarting and never actually boots up. In spite of this I can build the image and start it as a container on the instance independent from the image from cloud build.
I managed to find some clues from the logs:
E1219 19:13:52 7f28dce6d700 api_server.cc:184 Metadata request unsuccessful: Server responded with 'Forbidden' (403): Transport endpoint is not connected
oauth2.cc:289 Getting auth token from metadata server docker
I also got some clue by running the following in the instance:
docker -a -i start <container_id>
Output: Unrecognized input header: 99
The cloudbuild.yml file looks like (I've replaced some variables with ...):
#cloudbuild.yaml
steps:
- name: 'docker/compose:1.22.0'
args: ['-f', 'docker/docker-compose.tb.prod.yml', 'up', '-d']
- name: 'gcr.io/cloud-builders/docker'
args: ['tag', 'tb:latest', '...']
- name: 'gcr.io/cloud-builders/gcloud'
args: [
'beta', 'compute', '--project=...', 'instance-templates', 'create-with-container',
'tb-app-staging-${COMMIT_SHA}',
'--machine-type=n1-standard-2', '--network=...', '--network-tier=PREMIUM', '--metadata=google-logging-enabled=true',
'--maintenance-policy=MIGRATE', '--service-account=...',
'--scopes=https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/devstorage.read_only,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/trace.append',
'--tags=http-server,https-server', '--image=cos-stable-69-10895-62-0', '--image-project=cos-cloud', '--boot-disk-size=20GB', '--boot-disk-type=pd-standard',
'--container-restart-policy=always', '--labels=container-vm=cos-stable-69-10895-62-0',
'--boot-disk-device-name=...',
'--container-image=...',
]
- name: 'gcr.io/cloud-builders/gcloud'
args: [
'beta', 'compute', '--project=...', 'instance-groups',
'managed', 'rolling-action', 'start-update',
'tb-app-staging',
'--version',
'template=...',
'--zone=europe-west1-b',
'--max-surge=20',
'--max-unavailable=9999'
]
images: ['...']
timeout: 1200s
I found the issue and I'll answer this question myself just incase someone else runs into the same issue.
The problem was that in my docker-compose.yml I have the configuration for stdin_open and tty set to true but my cloudbuild.yml file did not accept it and was failing silently (annoying!).
To fix the issue you will need to use the flags --container-stdin and --container-tty on the create-with-container command.
More details can be found on the google docs https://cloud.google.com/compute/docs/containers/configuring-options-to-run-containers
I has a similar issue the reason was setting USER in Dockerfile. I was using changing user to 'node' which is user available in official nodejs images. But does not work on Google cloud containers.
FROM node:current-buster-slim
USER node
I have a custom module that I want to install on a container running the bitnami/magento docker image within a kubernetes cluster.
I am currently trying to install the module from a local dir into the containers Dockerfile:
# run bitnami's magento container
FROM bitnami/magento:2.2.5-debian-9
# add magento_code directory to the bitnami magento install
# ./magento_data/code contains the module, i.e. Foo/Bar
ADD ./magento_data/code /opt/bitnami/magento/htdocs/app/code
After building and running this image the site pings back a 500 error. The pod logs show that Magento installs correctly but it doesn't know what to do with the custom module:
Exception #0 (UnexpectedValueException): Setup version for module 'Foo_Bar' is not specified
Therefore to get things working I have to open a shell to the container and run some commands:
$ php /opt/bitnami/magento/htdocs/bin/magento setup:upgrade
$ chown -R bitnami:daemon /opt/bitnami/magento/htdocs
The first sorts the magento set up issue, the second ensures the next time an http request comes in Magento is able to correctly generate any directories and files it needs.
This gives me a functioning container, however, kubernetes is not able to rebuild this container as I am manually running a bunch of commands after Magento has installed.
I thought about running the above commands within the containers readinessProbe however not sure if it would work as not 100% on the state of Magento when that is first called, alongside it seeming very hacky.
Any advice on how to best set up custom modules within a bitnami/magento container would be much appreciated.
UPDATE:
Since opening this issue I've been discussing it further on Github: https://github.com/bitnami/bitnami-docker-magento/issues/82
I've got it working via the use of composer instead of manually adding the module to the app/code directory.
I was able to do this by firstly adding the module to Packagist, then I stored my Magento Marketplace authentication details in auth.json:
{
"http-basic": {
"repo.magento.com": {
"username": <MAGENTO_MARKETPLACE_PUBLIC_KEY>,
"password": <MAGENTO_MARKETPLACE_PRIVATE_KEY>
}
}
}
You can get the public & private key values by creating a new access key within marketplace. Place the file in the modules root, alongside your composer.json.
Once I had that I updated my Dockerfile to use the auth.json and require the custom module:
# run bitnami's magento container
FROM bitnami/magento:2.2.5
# Require custom modules
WORKDIR /opt/bitnami/magento/htdocs/
ADD ./auth.json .
RUN composer require foo/bar
I then completed a new install, creating the db container alongside the magento container. However it should also work fine with an existing db so long as the modules versions are the same.