I have a dockerfile to upload some python code on Azure. It has been working for a few months, but today it suddenly stopped working.
The relevant commands in the Dockerfile are:
FROM python:3.9.5
:
:
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/20.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN exit
RUN apt-get update && ACCEPT_EULA=Y apt-get install msodbcsql17
The error message is that appeared today is:
Err:1 https://packages.microsoft.com/ubuntu/20.04/prod focal/main amd64 msodbcsql17 amd64 17.7.2.1-1
404 Not Found [IP: 104.214.230.139 443]
E: Failed to fetch https://packages.microsoft.com/ubuntu/20.04/prod/pool/main/m/msodbcsql17/msodbcsql17_17.7.2.1-1_amd64.deb 404 Not Found [IP: 104.214.230.139 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get update && ACCEPT_EULA=Y apt-get install msodbcsql17' returned a non-zero code: 100
2021/06/16 20:50:56 Container failed during run: build. No retries remaining.
failed to run step ID: build: exit status 100
I believe this might be related to the .deb files being moved - or that some computer at Microsoft is down?
A good workaround would maybe be to download the relevant msodbcsql17 package directly, but I was unable to find this package in the normal repos?
There seems to be some ongoing trouble with microsoft repos for some linux distributions (including ubuntu and debian). Not clear when this will be fixed.
https://github.com/dotnet/core/issues/6381
https://github.com/MicrosoftDocs/sql-docs/issues/6494
The answer might be related to this post:
https://github.com/dotnet/core/issues/6381
It seems that some Ubuntu repositories are broken.
Hopefully it will be fixed soon...
I Will keep an eye on the resolve, but I have the same issue using:
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get update
RUN ACCEPT_EULA=Y apt-get install -y msodbcsql17
# optional: for bcp and sqlcmd
RUN ACCEPT_EULA=Y apt-get install -y mssql-tools
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
# optional: for unixODBC development headers
RUN apt-get install -y unixodbc-dev
I'm quite new to Docker. I have a Laravel project I did using Laragon. I would like to create a Docker Compose project out of it. My problem is, when I install composer, even if the terminal is showing me no error, I can't see anything inside of the container.
What I have:
I have a docker-compose.yml with PHP and NGINX services
I have a Dockerfile for the laravel app
Here is the Dockerfile:
FROM php:7.3.19-fpm-stretch
ENV ACCEPT_EULA=Y
# Set working directory
WORKDIR /var/www
# Install dependencies
RUN apt-get update && apt-get install -y \
build-essential \
libpng-dev \
gnupg \
libjpeg62-turbo-dev \
libfreetype6-dev \
locales \
zip \
libzip-dev \
jpegoptim optipng pngquant gifsicle \
vim \
unzip \
git \
curl
# Microsoft SQL Server Prerequisites
RUN apt-get update \
&& curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
&& curl https://packages.microsoft.com/config/debian/9/prod.list \
> /etc/apt/sources.list.d/mssql-release.list \
&& apt-get install -y --no-install-recommends \
locales \
apt-transport-https \
&& echo "en_US.UTF-8 UTF-8" > /etc/locale.gen \
&& locale-gen \
&& apt-get update \
&& apt-get -y --no-install-recommends install \
unixodbc-dev \
msodbcsql17
# Clear cache
RUN apt-get clean && rm -rf /var/lib/apt/lists/*
# Install extensions
RUN docker-php-ext-install pdo_mysql mbstring zip exif pcntl
RUN docker-php-ext-configure gd --with-gd --with-freetype-dir=/usr/include/ --with-jpeg-dir=/usr/include/ --with-png-dir=/usr/include/
RUN docker-php-ext-install gd
RUN pecl install sqlsrv pdo_sqlsrv xdebug \
&& docker-php-ext-enable sqlsrv pdo_sqlsrv xdebug
# Install composer
RUN curl -sS https://getcomposer.org/installer | \
php -- --install-dir=/usr/bin/ --filename=composer
# Copy composer.lock and composer.json
COPY composer.lock composer.json /var/www/
# Run composer install
RUN composer install --no-scripts --no-autoloader
# Add user for laravel application
RUN groupadd -g 1000 www
RUN useradd -u 1000 -ms /bin/bash -g www www
# Copy existing application directory contents
COPY . /var/www
RUN composer dump-autoload --optimize
# Copy existing application directory permissions
COPY --chown=www:www . /var/www
# Change current user to www
USER www
# Expose port 9000 and
EXPOSE 9000
# Start php-fpm server
CMD ["php-fpm"]
What I get when I run docker-compose build:
[...]
Step 11/21 : RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/bin/ --filename=composer
---> Running in 9ee162ed2895
All settings correct for using Composer
Downloading...
Composer (version 1.10.8) successfully installed to: /usr/bin/composer
Use it: php /usr/bin/composer
Removing intermediate container 9ee162ed2895
---> dc1df86b9a7b
Step 12/21 : COPY composer.lock composer.json /var/www/
---> 7bcf92a0a647
Step 13/21 : RUN composer install --no-scripts --no-autoloader
---> Running in f1695442d1db
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Package operations: 113 installs, 0 updates, 0 removals
- Installing doctrine/inflector (2.0.3): Downloading (100%)
[...] (all composer packages seem to be installed correctly)
Step 17/21 : RUN composer dump-autoload --optimize
---> Running in 18b8c992ace9
Generating optimized autoload files
> Illuminate\Foundation\ComposerScripts::postAutoloadDump
> #php artisan package:discover --ansi
Discovered Package: facade/ignition
Discovered Package: fideloper/proxy
Discovered Package: fruitcake/laravel-cors
Discovered Package: laravel/socialite
Discovered Package: laravel/tinker
Discovered Package: nesbot/carbon
Discovered Package: nunomaduro/collision
Discovered Package: socialiteproviders/manager
Package manifest generated successfully.
Generated optimized autoload files containing 4925 classes
Removing intermediate container 18b8c992ace9
---> d5075486971d
Step 18/21 : COPY --chown=www:www . /var/www
---> 15fcdde49457
Step 19/21 : USER www
---> Running in be9ab541543c
Removing intermediate container be9ab541543c
---> e17d1ae7a052
Step 20/21 : EXPOSE 9000
---> Running in 3e09f6279930
Removing intermediate container 3e09f6279930
---> 7a3649b9df62
Step 21/21 : CMD ["php-fpm"]
---> Running in 0db51df5de3c
Removing intermediate container 0db51df5de3c
---> fbb8e77bb8d1
Successfully built fbb8e77bb8d1
Successfully tagged digitalocean.com/php:latest
The docker-compose build sounds like it's working.
When I run docker-composer up -d and I go to http://localhost, I have a PHP error :
Fatal error: require(): Failed opening required '/var/www/public/../vendor/autoload.php' (include_path='.:/usr/local/lib/php') in /var/www/public/index.php on line 24
What I did so far:
Delete all the containers/images, rebuild
Edit the Dockerfile according to some tutorials I found on the internet
Verified that there's a vendor directory in www. Result : no.
Tried to execute composer install directly into the container (within docker exec -it <container-name> /bin/sh) : Result : /bin/sh: composer: not found.
I can't see where I'm wrong, and I don't understand where exactly does composer had installed its packages, and how to run my project...
Thanks forehand.
My Dockerfile
FROM debian:jessie-20180831 as builder
ENV BUILD_DEPS "curl git"
COPY . /go/src/chaochaogege.com/onlinecode
WORKDIR /go/src/chaochaogege.com/onlinecode
RUN apt-get update && apt-get install curl git sudo --no-install-recommends -y
RUN curl -sL https://deb.nodesource.com/setup_8.x | sudo -E sh -;\
apt-get install nodejs -y --no-install-recommends;
RUN cd ./client-side \
&& npm install && npm run build;
COPY ./sql ./client-side/dist/
RUN apt-get update && apt-get install ${BUILD_DEPS} -y --no-install-recommends;\
curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh; \
dep ensure -update;\
go install;
FROM golang:1.11.1-alpine3.7
RUN mkdir -p /go/src/chaochaogege.com/onlinecode
WORKDIR /go/src/chaochaogege.com/onlinecode
COPY --from=builder /go/bin/onlinecode .
COPY --from=builder /go/src/chaochaogege.com/onlinecode/client-side/dist/* .
EXPOSE 8086
ENTRYPOINT ["onlinecode"]
I don't know my happened.
From my view, I install nodejs in RUN layer, and then use npm in another layer after it.
But why not work?
There are some dupicate questions I am sure, but all of them not solve my problem.
I guess maybe npm only can be used in same docker layer? So I change Dockerfile to follow:
FROM debian:jessie-20180831 as builder
ENV BUILD_DEPS "curl git"
COPY . /go/src/chaochaogege.com/onlinecode
WORKDIR /go/src/chaochaogege.com/onlinecode
RUN apt-get update && apt-get install curl git sudo --no-install-recommends -y
RUN curl -sL https://deb.nodesource.com/setup_8.x | sudo -E sh -;\
apt-get install nodejs -y --no-install-recommends;\
npm --version; \
cd ./client-side \
&& npm install && npm run build;
COPY ./sql ./client-side/dist/
RUN apt-get update && apt-get install ${BUILD_DEPS} -y --no-install-recommends;\
curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh; \
dep ensure -update;\
go install;
FROM golang:1.11.1-alpine3.7
RUN mkdir -p /go/src/chaochaogege.com/onlinecode
WORKDIR /go/src/chaochaogege.com/onlinecode
COPY --from=builder /go/bin/onlinecode .
COPY --from=builder /go/src/chaochaogege.com/onlinecode/client-side/dist/* .
EXPOSE 8086
ENTRYPOINT ["onlinecode"]
But also get result
npm not found
Error log
Step 6/15 : RUN curl -sL https://deb.nodesource.com/setup_8.x | sudo -E sh -; apt-get install nodejs -y --no-install-recommends; npm --version; cd ./client-side && npm install && npm run build;
---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
---> Running in 2788863ccf8a
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
libc-ares2 libv8-3.14.5
The following NEW packages will be installed:
libc-ares2 libv8-3.14.5 nodejs
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 1990 kB of archives.
After this operation, 7495 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian/ jessie/main libc-ares2 amd64 1.10.0-2+deb8u2 [72.5 kB]
Get:2 http://deb.debian.org/debian/ jessie/main libv8-3.14.5 amd64 3.14.5.8-8.1 [1269 kB]
Get:3 http://deb.debian.org/debian/ jessie/main nodejs amd64 0.10.29~dfsg-2 [648 kB]
[91mdebconf: delaying package configuration, since apt-utils is not installed
[0mFetched 1990 kB in 1s (1874 kB/s)
Selecting previously unselected package libc-ares2:amd64.
(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 9841 files and directories currently installed.)
Preparing to unpack .../libc-ares2_1.10.0-2+deb8u2_amd64.deb ...
Unpacking libc-ares2:amd64 (1.10.0-2+deb8u2) ...
Selecting previously unselected package libv8-3.14.5.
Preparing to unpack .../libv8-3.14.5_3.14.5.8-8.1_amd64.deb ...
Unpacking libv8-3.14.5 (3.14.5.8-8.1) ...
Selecting previously unselected package nodejs.
Preparing to unpack .../nodejs_0.10.29~dfsg-2_amd64.deb ...
Unpacking nodejs (0.10.29~dfsg-2) ...
Setting up libc-ares2:amd64 (1.10.0-2+deb8u2) ...
Setting up libv8-3.14.5 (3.14.5.8-8.1) ...
Setting up nodejs (0.10.29~dfsg-2) ...
update-alternatives: using /usr/bin/nodejs to provide /usr/bin/js (js) in auto mode
Processing triggers for libc-bin (2.19-18+deb8u10) ...
[91m/bin/sh: 1: npm: not found
With apt-get install nodejs you install only node, but not npm itself. You can check the nodejs files in the debian package index.
To install npm you need to install the npm package.
So, to fix your Dockerfile edit this line:
apt-get install nodejs -y --no-install-recommends;
With:
apt-get install nodejs npm -y --no-install-recommends;
I currently have the following Dockerfile to create my Docker image.
FROM python:3.6.6-alpine3.8
# Add dependencies for Python packages pandas, numpy and pyodbc
RUN apk add --no-cache curl gcc g++ unixodbc-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
# Project files
ARG PROJECT_DIR=/srv/scripts
RUN mkdir -p $PROJECT_DIR
WORKDIR $PROJECT_DIR
COPY requirements.txt ./
# Install Python dependencies
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
I would like to include various ODBC drivers in this image so that I can use them to connect to different databases from the Python program running in my container.
The Python program is using Pyodbc to connect to databases.
The ODBC drivers I need to install are:
PostgreSQL
MySQL
Ms SQL Server
Teradata
Oracle
Hive
Impala
I wanted to start with PostgreSQL thinking it would be the easiest one but I could not find any package on the Alpine Linux Package manager. Do you have any idea how I should install such a driver?
I was facing the same issue. I solved this issue by adding RUN apk update before RUN apk add commands.(I was using python:3.6-alpine)
Dockerfile
FROM python:3.6-alpine
RUN apk update
RUN apk add gcc libc-dev g++ libffi-dev libxml2 unixodbc-dev mariadb-dev postgresql-dev
As the OP ended moving away from Alpine- to a Debian-base image, and another answer has a small snapshot of a working Dockerfile, I will flesh out a full Dockerfile that builds SQL Server ODBC Driver 17 into a Debian-base image.
# load python 3.8 dependencies using slim debian 10 image.
FROM python:3.8-slim-buster
# build variables.
ENV DEBIAN_FRONTEND noninteractive
# install Microsoft SQL Server requirements.
ENV ACCEPT_EULA=Y
RUN apt-get update -y && apt-get update \
&& apt-get install -y --no-install-recommends curl gcc g++ gnupg unixodbc-dev
# Add SQL Server ODBC Driver 17 for Ubuntu 18.04
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
&& curl https://packages.microsoft.com/config/debian/10/prod.list > /etc/apt/sources.list.d/mssql-release.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends --allow-unauthenticated msodbcsql17 mssql-tools \
&& echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile \
&& echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
# upgrade pip and install requirements.
COPY /requirements.txt /requirements.txt
RUN pip install --upgrade pip
RUN pip install -r /requirements.txt
# clean the install.
RUN apt-get -y clean
# copy all files to /app directory and move into directory.
COPY . /app
WORKDIR /app
ENTRYPOINT ["some", "python", "command"]
if keeping it lightweight is your concerns, specifically in case of odbc, i recommend using debian:stretch image.
I'm trying to connect a pyodbc python script running in a docker container to login to a MSSQL database I have tried all sorts of docker files, but not been able to make the connection (fails when bulding the docker or when python tries to connect), Does anyone have a working dockerfile, using pyodbc:
Dockerfile:
# Use an official Python runtime as a parent image
FROM python:2.7-slim
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
ADD . /app
# Install any needed packages specified in requirements.txt
RUN pip install -r requirements.txt
# Run app.py when the container launches
CMD ["python", "App.py"]
requirements.TXT
pyodbc
App.Py
import pyodbc
connection = pyodbc.connect('Driver={SQL Server};'
'Server=xxxx;'
'Database=xxx;'
'UID=xxxx;'
'PWD=xxxx')
cursor = connection.cursor()
cursor.execute("SELECT [Id],[Name] FROM [DCMM].[config].[Models]")
for row in cursor.fetchall():
print(row.Name)
connection.close()
Bulding the container
docker build -t sqltest .
Output:
Sending build context to Docker daemon 4.096kB
Step 1/5 : FROM python:2.7-slim
---> 426d65ab9a72
Step 2/5 : WORKDIR /app
---> Using cache
---> 725f35122880
Step 3/5 : ADD . /app
---> 3feb8b7744f7
Removing intermediate container 4214091a111a
Step 4/5 : RUN pip install -r requirements.txt
---> Running in 27aa4dcfe738
Collecting pyodbc (from -r requirements.txt (line 1))
Downloading pyodbc-4.0.17.tar.gz (196kB)
Building wheels for collected packages: pyodbc
Running setup.py bdist_wheel for pyodbc: started
Running setup.py bdist_wheel for pyodbc: finished with status 'error'
Failed building wheel for pyodbc
Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-EfWsmy/pyodbc/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpa3S13tpip-wheel- --python-tag cp27:
running bdist_wheel
running build
running build_ext
building 'pyodbc' extension
creating build
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPYODBC_VERSION=4.0.17 -DSQL_WCHART_CONVERT=1 -I/usr/local/include/python2.7 -c src/cursor.cpp -o build/temp.linux-x86_64-2.7/src/cursor.o -Wno-write-strings
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
Running setup.py clean for pyodbc
Failed to build pyodbc
Installing collected packages: pyodbc
Running setup.py install for pyodbc: started
Running setup.py install for pyodbc: finished with status 'error'
Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-EfWsmy/pyodbc/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-BV4sRM-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_ext
building 'pyodbc' extension
creating build
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPYODBC_VERSION=4.0.17 -DSQL_WCHART_CONVERT=1 -I/usr/local/include/python2.7 -c src/cursor.cpp -o build/temp.linux-x86_64-2.7/src/cursor.o -Wno-write-strings
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-EfWsmy/pyodbc/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-BV4sRM-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-EfWsmy/pyodbc/
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1
Need to Run:
sudo apt-get install gcc
need to add a odbcinst.ini file containing:
[FreeTDS]Description=FreeTDS Driver Driver=/usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so Setup=/usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
need to add folowing to docker file
ADD odbcinst.ini /etc/odbcinst.ini
RUN apt-get update
RUN apt-get install -y tdsodbc unixodbc-dev
RUN apt install unixodbc-bin -y
RUN apt-get clean -y
need to change connection in .py to
connection = pyodbc.connect('Driver={FreeTDS};'
'Server=xxxxx;'
'Database=DCMM;'
'UID=xxxxx;'
'PWD=xxxxx')
Now the container compiles, and gets data from SQL server
Running through this recently I found it was necessary to additionally include the following line (note that it did not build without this step):
RUN apt-get install --reinstall build-essential -y
The full Dockerfile looks as follows:
# parent image
FROM python:3.7-slim
# install FreeTDS and dependencies
RUN apt-get update \
&& apt-get install unixodbc -y \
&& apt-get install unixodbc-dev -y \
&& apt-get install freetds-dev -y \
&& apt-get install freetds-bin -y \
&& apt-get install tdsodbc -y \
&& apt-get install --reinstall build-essential -y
# populate "ocbcinst.ini"
RUN echo "[FreeTDS]\n\
Description = FreeTDS unixODBC Driver\n\
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so\n\
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so" >> /etc/odbcinst.ini
# install pyodbc (and, optionally, sqlalchemy)
RUN pip install --trusted-host pypi.python.org pyodbc==4.0.26 sqlalchemy==1.3.5
# run app.py upon container launch
CMD ["python", "app.py"]
Here's one way to then actually establish the connection inside app.py, via sqlalchemy (and assuming port 1433):
import sqlalchemy as sa
args = (username, password, server, database)
connstr = "mssql+pyodbc://{}:{}#{}/{}?driver=FreeTDS&port=1433&odbc_options='TDS_Version=8.0'"
engine = sa.create_engine(connstr.format(*args))
Based on Kåre Rasmussen's answer, here's a complete dockerfile for further use.
Make sure to edit the last two lines according to your architecture! They should reflect the actual paths to libtdsodbc.so and libtdsS.so.
If you're not sure about the paths to libtdsodbc.so and libtdsS.so, try dpkg --search libtdsodbc.so and dpkg --search libtdsS.so.
FROM python:3
#Install FreeTDS and dependencies for PyODBC
RUN apt-get update && apt-get install -y tdsodbc unixodbc-dev \
&& apt install unixodbc-bin -y \
&& apt-get clean -y
RUN echo "[FreeTDS]\n\
Description = FreeTDS unixODBC Driver\n\
Driver = /usr/lib/arm-linux-gnueabi/odbc/libtdsodbc.so\n\
Setup = /usr/lib/arm-linux-gnueabi/odbc/libtdsS.so" >> /etc/odbcinst.ini
Afterwards, install PyODBC, COPY your app and run it.
I was unable to use all of the above resolutions, I was keeping al kind of errors relating to the pyodbc package, in particular:
ImportError: libodbc.so.2: cannot open shared object file: No such file or directory.
I ended up with another resolution which defines the ODBC SQL Server Driver specifically for an Ubuntu 18.04 Docker image, in this case ODBC Driver 17 for SQL Server. In my specific use case I needed to make the connection to my MySQL database server on Azure via Flask SQLAlchemy, but the latter is not a necessity for the Docker configuration.
Dockerfile, with most important part adding the Microsoft repository and installing msodbcsql17 and unixodbc-dev:
# Ubuntu 18.04 base with Python runtime and pyodbc to connect to SQL Server
FROM ubuntu:18.04
WORKDIR /app
# apt-get and system utilities
RUN apt-get update && apt-get install -y \
curl apt-utils apt-transport-https debconf-utils gcc build-essential g++-5\
&& rm -rf /var/lib/apt/lists/*
# adding custom Microsoft repository
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/18.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
# install SQL Server drivers
RUN apt-get update && ACCEPT_EULA=Y apt-get install -y msodbcsql17 unixodbc-dev
# install SQL Server tools
RUN apt-get update && ACCEPT_EULA=Y apt-get install -y mssql-tools
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
RUN /bin/bash -c "source ~/.bashrc"
# python libraries
RUN apt-get update -y && \
apt-get install -y python3-pip python3-dev
# install necessary locales, this prevents any locale errors related to Microsoft packages
RUN apt-get update && apt-get install -y locales \
&& echo "en_US.UTF-8 UTF-8" > /etc/locale.gen \
&& locale-gen
# copy requirements and install packages, I added this for general use
COPY ./requirements.txt > ./requirements.txt
RUN pip3 install -r ./requirements.txt
# you can also use regular install of the packages
RUN pip3 install pyodbc SQLAlchemy
# and if you are also planning to use Flask and Flask-SQLAlchemy
Run pip3 install Flask Flask-SQLAlchemy
COPY ..
# run your app via entrypoint or change the CMD command to your regular command
COPY docker-entrypoint.sh wsgi.py ./
CMD ["./docker-entrypoint.sh"]
This should build without any errors in Docker.
My database url looked like this:
import urllib.parse
# name the sepcific ODBC driver by version number, we installed msodbcsql17
params = urllib.parse.quote_plus("DRIVER={ODBC Driver 17 for SQL Server};SERVER=<your.database.windows.net>;DATABASE=<your-db-name>;UID=<username>;PWD=<password>")
db_uri = "mssql+pyodbc:///?odbc_connect={PARAMS}".format(PARAMS=params)
And for the bonus if you are using Flask-SQLAlchemy, your app config should contain something like this:
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
app.config["SQLALCHEMY_DATABASE_URI"] = db_uri # from above
Happy coding!
How to install the necessary dependencies for pyodbc is related to the linux distribution and its version (in docker case, that is the base image of your docker image). If none of the above work for you, you can figure out the commands by trying in the docker container instance.
First, exec into the docker container
docker exec -it <container id> bash
Try various ways to get the distribution name and version of your linux. Then try different instructions in Install the Microsoft ODBC driver for SQL Server (Linux)
Here is a working example for Debian 9 based images, deriving exactly as the document instructions.
# Install pyodbc dependencies
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/debian/9/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get update
RUN ACCEPT_EULA=Y apt-get -y install msodbcsql17
RUN apt-get -y install unixodbc-dev
RUN pip install pyodbc
For me to solve this issue I also had to add the following 2 lines in the dockerfile:
RUN echo MinProtocol = TLSv1.0 >> /etc/ssl/openssl.cnf
RUN echo CipherString = DEFAULT#SECLEVEL=1 >> /etc/ssl/openssl.cnf
For those who wanted to do official microsoft approach to install odbc driver and use python:slim docker image, you can use this as DockerFile:
FROM python:3.9-slim
RUN apt-get -y update && apt-get install -y curl gnupg
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
# download appropriate package for the OS version
# Debian 11
RUN curl https://packages.microsoft.com/config/debian/11/prod.list \
> /etc/apt/sources.list.d/mssql-release.list
RUN exit
RUN apt-get -y update
RUN ACCEPT_EULA=Y apt-get install -y msodbcsql18
Then for sqlalchemy's this can be called:
con_str = f"mssql+pyodbc://{username}:{password}#{host}/{db}?" \
"driver=ODBC+Driver+18+for+SQL+Server&TrustServerCertificate=yes"
engine = create_engine(con_str)
I created a Gist on GitHub on how to do this. I hope it helps. I had to piece things together from what I found on different resources.
https://gist.github.com/joshatxantie/4bcf5d0243fba63845fce7cc40365a3a
Goodluck!
I fixed this problem by using pypyodbc instead of pyodbc.
pip install pypyodbc==1.3.5
https://pypi.org/project/pypyodbc/
Found the hint here:
https://github.com/Azure/azure-functions-python-worker/issues/249
For not more problem use library for python
pymssql this not need install driver
pip install pymssql
import pymssql
conn = pymssql.connect(server, user, password, "tempdb")
cursor = conn.cursor(as_dict=True)
cursor.execute('SELECT * FROM persons WHERE salesrep=%s', 'John Doe')
for row in cursor:
print("ID=%d, Name=%s" % (row['id'], row['name']))
conn.close()
and work in docker