I´m using WSL2 on Windows with Ubuntu, I had problems with PIP, it gave me DNS problems, so I modified the file /ect/resolve.conf , after this I was able to use PIP, but still I have problems with the Dockerfile.
when using docker build, and trying to use pip install requirements.txt, it tries to download the packages from the requirements file and got DNS problems.
so, how can I add DNS servernames to the Dockerfile? or how can I solve this issue?
Regards.
In case any one else is having this issue, I'll share what worked for me. I've been in the Windows world so long that I forgot about one of the basic linux DNS config files. I just updated /etc/resolv.conf and was then able to resolve DNS names just fine.
Related
I've modified the docker-compose for Airflow (apache/airflow:2.5.0-python3.10) and added in a MariaDB service: to emulate the target DB for a project in dev.
In the _PIP_ADDITIONAL_REQUIREMENTS I've included pymsql and I am attempting to create a MySQL Connection under Admin | Connections and using the MySQL connector to bridge over to Maria (before asking I used host.docker.internal as the host name in the Connections| Add Connection field for host).
apache-airflow-providers-mysql is already installed as part of the official docker compose for Airflow (and I checked it manually on the container.)
However, every time I hit the Test button I get a No module named 'MySQLdb' which I assumed the pip extras install took care of.
Also, I am assuming here I can merely use MySQL over the wire to connect to the mariadb via python but if there additional libraries or installs needed (for example, libmariadb3 libmariadb-dev python3-mysqldb or similar to be installed, please let me know. I assume someone else has had to do this already, though perhaps not from docker... =] I am trying to avoid building my own image and use the existing docker-compose, but if it's unavoidable and I need to do a build ., lemme know.).
I am concerned this may be due to the fact I am on an M1 Mac laptop running this (as 2 requirements, I've now dug down and researched have a platform_machine != "aarch64" against them 8-/ Surely there is a way around this though, yeah? Esp with docker? (do I need to do a custom build?). Also, do not understand why these particular components would not work on arm64 at this point that macs have been chipped like that over 2 years.
thanks!
PS> Maria is not supported as a backend in Airflow but Connections to it as a database should work much like anything else that can connect via sqlalchemy etc.
Set up the Connection in Airflow and then installed the extra pip for pymysql. However, I still seem to be getting the same No module named 'MySQLdb' error on test connection.
You need to install the MySql provider for Airflow:
pip install apache-airflow-providers-mysql
or add it to your requirements file, it should install all what you need to use MySql/MariaDB as connection.
The issue appears to be one where Airflow's MySQL operators have a != aarch64 for the Apple OSX and my machine was an M1 chipped laptop.
How did I get around this? In my docker-compose for the airflow components I set a directive of platform: x86_64 in the service(s) and then rebuild the container. After that, I had no problem connecting with the MySQL connector to the MariaDB that we had as a target.
note: this greatly increased the startup time for the docker compose instance (from like 20s to 90s-ish, but works). As a solution for dev, worked great.
What is benefit of install annaconda over docker.
For python/pytorch programming. I installed annaconda docker image on Ubunut 20.4
But found many problems need to solve such as Display on opencv2 (already solved). and spyder error on libgl.so.1 cannot found and still no idea how to solve.
So, how about just install annacona on Ubuntu host. Becuase it should working as smoothly well on my job.
while deploying docker images on the production server using docker-compose, I got the following error.
Cannot create container for service xxx: open /var/lib/docker/overlay2/969rf5...fdf-init/merged/etc/resolv.conf: Operation not permitted
What I tried:
Changed permissions ( So much so that gave 777 for all the directories involved)
Upgraded kernel (Saw somewhere that it could be kernel issue)
Removed ACL using setfacl -Rb /var/lib/docker
Added the "graph: /var/lib/docker" line in daemon.json
Restarted docker service couple of times.
Tried running the images individually and not using docker-compose.
Tried running with sudo
Set the --storage-opt overlay2.override_kernel_check=1 (Since it is rhel and kernel version supported for docker overlay2 is >4.0, so was suggested to override it.)
I failed everytime!
Have no idea what the issue is or what it could be. Every time I run the docker-compose file, it creates a new directory in /var/lib/docker/overlay2 with old permissions even though I change the permissions for all the other directories in it. At this point I'm not even sure if it is permissions issue or not.
Any help would be appreciated! Thank you!
Finally after trying everything on the internet and nothing working out, the issue was resolved. It was due to the Antivirus that was installed on the server. Had it removed by the client and docker/docker-compose started working absolutely fine.
To give a little more information on the antivirus, it was some FIM antivirus that was installed on a rhel 3.10-xxx which was blocking docker. It was an older version and thank goodness newer versions are compatible with docker. So that's a wrap, thank you all!
Can I install Docker over a server with pre-installed cPanel and CentOS 7? Since I am not aware of Docker, I am not completely sure whether it will mess with cPanel or not. I already have a server with CentOS 7 and cPanel configured. I want to know if I can install Docker over this configuration I mentioned without messing up?
Yes you can install docker over cPanel/WHM just like installing it on any other CentOS server/virtual machine.
Just follow these simple steps (as root):
1) yum install -y yum-utils device-mapper-persistent-data lvm2 (these should be already installed...)
2) yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
3) yum install docker-ce
4) enable docker at boot (systemctl enable docker)
5) start docker service (systemctl start docker)
The guide above is for CentOS 7.x. Don't expect to find any references or options related to Docker in the WHM interface. You will be able to control docker via command line from a SSH shell.
I have some docker containers already running on my cPanel/WHM server and I have no issues with them. I basically use them for caching, proxying and other similar stuff.
And as long as you follow these instructions, you won't mess-up any of your cPanel/WHM services/settings or current cPanel accounts/settings/sites/emails etc.
Not sure why you haven't tried this already!
I've been doing research and working on getting Docker working on cPanel. It's not just getting it to work on a CentOS 7 box but rather making it palatable for the cPanel crowd in the form of a plugin. So far I can confirm that it's absolutely doable. Here's what I've accomplished and how:
Integrate Docker Compose with cPanel (which is somewhat a step
further from WHM)
Leverage the user-namespace kernel feature in Linux so Docker
services can't escalate their privileges (see userns remap)
Leverage Docker Compose so users can build complex services and
start ready apps from the store with a click
Make sure services starting via Docker run on a non-public IP on the
server. Everything gets routed via ProxyPass
cPanel has been gracious to provide a Slack channel for people to discuss this upcoming plugin. I'd be more than happy to invite you if you'd like to be kept updated or to contribute. Let me know!
FYI, there's more info here on https://www.unixy.net/docker if you're interested. Please note that this plugin is in private beta but more than happy to let people use it!
Yes you could, in fact someone else has done it already: https://github.com/mirhosting/cPanel-docker
I have three servers and I want to set up Ambari. Unfortunately I have CentOS7 which currently isn't supported by Ambari. So I decided to use docker to overcome the OS dependency. I have found various docker-based Ambari version like this and this. The second one seems to be really good and easy to install on one host creating a pseudo-cluster, but I failed to install it on different hosts. I followed the described steps one by one trying to adjust them to the fact that I am not one the same host but when I try to assign a new host I get an error that the hostname is not valid.
For the first solution on the other hand, I can't understand the following. How can I make the containers on each host be aware of each other? Because Ambari needs every host to can ssh each other without password.
Excuse me in advance for my lack of expertise in this domain. Any comment would be very useful.
Ambari 2.2+ can be installed successfully and works fine on CentOS 7. Then you can install HDP 2.0+.
The links in your question try to install 1.6 and 1.7.