I'm trying to set the following option flag to 1: experimental_allow_extended_storage_definition_syntax to be able to test the new partitions features.
But I don't find where this can be set. Is it in a config file, while opening a session?
I'm using:
Clickhouse in Docker under Ubuntu 16.04 LTS
Tabix in docker
If you have the exact command line to pass to be able to set that up with Docker that would be great.
It is user settings, which could be set for a particular session or globally using users.xml.
Let's set the setting for default user (settings of all users are inherited from default user settings).
We will not modify /etc/clickhouse-server/users.xml directly, just add special file experimental_allow_extended_storage_definition_syntax.xml in users.d subdirrectory. It will be merged into main users config file.
So, Docker file commands:
RUN mkdir -p /etc/clickhouse-server/users.d/
RUN chown -R clickhouse /etc/clickhouse-server/users.d/
RUN echo '<yandex><profiles><default><experimental_allow_extended_storage_definition_syntax>1</experimental_allow_extended_storage_definition_syntax></default></profiles></yandex>' > /etc/clickhouse-server/users.d/experimental_allow_extended_storage_definition_syntax.xml
You could see an example of the Dockerfile here
Related
I'm trying to spin up an ArangoDB server via docker compose.
It all works out fine with default configuration, but I'm struggling to make the server in the container pick up a custom configuration file with minimal setup
I've tried with overriding the startup sequence with the following:
command: >
arangod --configuration /arango.conf
I've checked and the file is present in the container, but when I check the configuration file via query through arangoshell it still references the default settings and the arangod.conf file placed in the /tmp folder.
Any hints?
--config /tmp/arangod.conf is hardcoded in docker-entrypoint.sh at line 185
it is also overwritten upon every start of container with /etc/arangodb3/arangod.conf at line 42
so to run arangod with your custom.conf file you need map it as volume to /etc/arangodb3/arangod.conf
via docker compose
volumes:
/custom.conf:/etc/arangodb3/arangod.conf
via docker cli
-v /custom.conf:/etc/arangodb3/arangod.conf
UPDATE: as per comments
custom.conf have to contain defaults from /etc/arangodb3/arangod.conf
easiest way is to save default config to custom.conf via docker cli and add/update options in that file
docker run --rm arangodb/arangodb:3.7.5 sh -c 'cat /etc/arangodb3/arangod.conf' > custom.conf
it gonna be also required to track updates of default options in /etc/arangodb3/arangod.conf in new versions of ArangoDB and reflect them in your custom.conf
I am trying to get the Superset running on ubuntu server. I have referenced the steps from Superset page as below:
git clone https://github.com/apache/incubator-superset/
cd incubator-superset/contrib/docker
# prefix with SUPERSET_LOAD_EXAMPLES=yes to load examples:
docker-compose run --rm superset ./docker-init.sh
# you can run this command everytime you need to start superset now:
docker-compose up
I have fixed the initial issues around right version of docker-compose and postgress address bind issue on port 5432. So after fixing those my docker compose run command
docker-compose run --rm superset ./docker-init.sh
works fine and it asks me to set up a user name and password.
Finally to get the container running I run the final command
docker-compose up.
On my mac, it would run redis, postrgre container and then give me a localhost:8088 for me to get access to Superset UI with login info.
However on Ubuntu, when I run that, I first get this:
So looks it is running redis and postgres containers fine.
But then it is giving me Permission denied errors to create some mkdir directory.
Pls note I am running it as root user.
Also, my docker compose version is fine with 1.23.2 and my docker along with docker-compose is installed under
/usr/bin/docker and not /usr/local/bin/docker.
But I think that shouldn't be an issue.
Any help where it is going wrong and how can I fix it?
Thanks
Edit:
Ok I looked at the same issue mentioned on Github. And used a suggestion of using it only for Production and not development in docker-compose.yml file.
It seems to not throw the same error now when I do
docker-compose up.
However when I open localhost:8088 it does not connect to the UI.
try this:
mkdir ../../assets
chmod -R 777 ../../superset/assets/
as set in docker-compose.yml#L64, it is using ../../superset as volume when in develop. However the container does not have any permission in the host so the solution is to make a directory by yourself and grant the necessary permissions on to it.
I use Docker Toolbox on Windows 7 in a corporate environment. My workflow requires pulling containers from one artifactory and pushing them to a different one (eg. external and internal). Each artifactory requires a different proxy to access it. Is there a way to configure Docker daemon to select proxy based on a URL? Or, if not, what else can I do to make this work?
Since, as Pierre B. mentioned, Docker daemon does not support URL-based proxy selection, the solution is to point it to a local proxy configured to select the proper upstream proxy based on the URL.
While any HTTP[S] proxy capable of upstream selection would do, (pac4cli project being particularly interesting for it's advertised capability to select the upstream based on proxy-auto-discovery protocol used by most Web browsers a in corporate setting), I've chosen to use tinyproxy, as more mature and light-weight solution. Furthermore, I've decided to run my proxy inside the docker-machine VM in order to simplify it's deployment and make sure the proxy is always running when the Docker daemon needs it.
Below are the steps I used to set up my system. I'm especially grateful to phoenix for providing steps to set up Docker Toolbox on Windows behind a corporate proxy, and will borrow heavily from that answer.
From this point on I will assume either Docker Quickstart Terminal or GitBash, with docker in the PATH, as your command line console and that "username" is your Windows user name.
Step 1: Build tinyproxy on your target platform
Begin by pulling a clean Linux distribution, I used CentOS, and run bash inside it:
docker run -it --name=centos centos bash
Next, install the tools we'll need:
yum install -y make gcc
After that we pull the latest release of Tinyproxy from it's GitHub repository and extract it inside root's home directory (at the time of this writing the latest release was 1.10.0):
cd
curl -L https://github.com/tinyproxy/tinyproxy/releases/download/1.10.0/tinyproxy-1.10.0.tar.gz \
| tar -xz
cd tinyproxy-1.10.0
Now let's configure and build it:
./configure --enable-upstream \
--disable-filter\
--disable-reverse\
--disable-transparent\
--disable-xtinyproxy
make
While --enable-upstream is obviously required, disabling other default features is optional but a good practice. To make sure it actually works run:
./src/tinyproxy -h
You should see something like:
Usage: tinyproxy [options]
Options are:
-d Do not daemonize (run in foreground).
-c FILE Use an alternate configuration file.
-h Display this usage information.
-v Display version information.
Features compiled in:
Upstream proxy support
For support and bug reporting instructions, please visit
<https://tinyproxy.github.io/>.
We exit the container by pressing Ctrl+D and copy the executable to a special folder location accessible from the docker-machine VM:
docker cp centos://root/tinyproxy-1.10.0/src/tinyproxy \
/c/Users/username/tinyproxy
Substitute "username" with your Windows user name. Please note that double slash — // before "root" is required to disable MINGW path conversion.
Now we can delete the container:
docker rm centos
Step 2: Point docker daemon to a local proxy port
Choose a TCP port number to run the proxy on. This can be any port that is not in use on the docker-machine VM. I will use number 8618 in this example.
First, let's delete the existing default Docker VM:
WARNING: This will permanently erase all currently stored containers and images
docker-machine rm -f default
Next, we re-create the default machine setting HTTP_PROXY and HTTPS_PROXY environment variables to the local host and the port we selected, and then refresh our shell environment:
docker-machine create default \
--engine-env HTTP_PROXY=http://localhost:8618 \
--engine-env HTTPS_PROXY=http://localhost:8618
eval $(docker-machine env)
Optionally, we could also set NO_PROXY environment variable to list hosts and/or wildcards (separated by ;) to which the daemon should connect directly, bypassing the proxy.
Step 3: Set up tinyproxy inside docker-machine VM
First, we will create two files in the /c/Users/username directory (this is where our tinyproxy binary should reside after Step 1 above) and then we'll copy them to the VM.
The first file is tinyproxy.conf, the exact syntax is documented on the Tinyproxy website, but the example below should have all the settings need:
# These settings can be customized to your liking,
# the port though must be the same we used in Step 2
listen 127.0.0.1
port 8618
user nobody
group nogroup
loglevel critical
syslog on
maxclients 50
startservers 2
minspareServers 2
maxspareServers 5
disableviaheader yes
# Here is the actual proxy selection, rules apply from top
# to bottom, and the last one is the default. More info on:
# https://tinyproxy.github.io/
upstream http proxy1.corp.example.com:80 ".foo.example.com"
upstream http proxy2.corp.example.com:80 ".bar.example.com"
upstream http proxy.corp.example.com:82
In the example above:
http://proxy1.corp.example.com:80 will be used to connect to URLs that end with "foo.example.com", such as http://www.foo.example.com
http://proxy2.corp.example.com:80 will be used to connect to URLs that end with "bar.example.com", such as http://www.bar.example.com, and
http://proxy.corp.example.com:80 will be used to connect all other URLs
It is also possible to match exact host names, IP addresses, subnets and hosts without domains.
The second file is as the shell script that will launch the proxy, its name must be bootlocal.sh:
#! /bin/sh
# Terminate on error
set -e
# Switch to the script directory
cd $(dirname $0)
# Launch proxy server
./tinyproxy -c tinyproxy.conf
Now, let's connect to the docker VM, get root, and switch to boot2docker directory:
docker-machine ssh
sudo -s
cd /var/lib/boot2docker
Next, we'll copy all three files over and a set their permissions:
cp /c/Users/username/boot2docker/{tinyproxy{,.conf},bootlocal.sh} .
chmod 755 tinyproxy bootlocal.sh
chmod 644 tinyproxy.conf
Exit VM session by pressing Ctrl+D twice and restart it:
docker-machine restart default
That's it! Now docker should be able pull and push images from different URLs automatically selecting the right proxy server.
I am new with selenium docker. I want to create a Chrome/Firefox node with capabilities (Selenium Grid). How to add capabilities when I add a Selenium Node docker container?
I found this command so far...
docker run -d --link selenium-hub:hub selenium/node-firefox:2.53.0
but I don't know how to add capabilities on it. Already use this command but not working.
docker run -d --link selenium-hub:hub selenium/node-firefox:2.53.0 -browser browserName=firefox,version=3.6,maxInstances=5,platform=LINUX
Solved... adding SE_OPTS will help you to set capabilites
docker run -d -e SE_OPTS="-browser browserName=chromeku,version=56.0,maxInstances=3,platform=WINDOWS" --link selenium-hub:hub selenium/node-chrome:2.53.0
There are multiple ways of doing this and SE_OPTS is one of them, however for me it complicated what I was trying to accomplish. Using SE_OPTS forced me to set capabilities I didn't want to change, otherwise they would be reset to blank/null
I wanted to do:
SE_OPTS=-browser applicationName=Testing123
but I was forced to do:
SE_OPTS=-browser applicationName=Testing123,browserName=firefox,maxInstances=1,version=59.0.1
Another way to set capabilities is to supply your own config.json
-nodeConfig /path/config.json
You can find a default config.json
Or you can start the node container and copy the current one from it
docker cp <containerId>:/opt/selenium/config.json /host/path/target
You can also take a look at entry_point.sh, either on github or on the running container:
/opt/bin/entry_point.sh
You can run bash on the node container via:
sudo docker exec -i -t <container> bash
This will let you see how SE_OPTS is used and how config.json is generated. Note config.json is generated only if you don't supply one.
/opt/bin/generate_config
By examining generate_config you can see quite a few ENV vars such as:
FIREFOX_VERSION, NODE_MAX_INSTANCES, NODE_APPLICATION_NAME etc.
This leads to the third way to set capabilities which is to set the environment variables being used by generate_config, in my case APPLICATION_NODE_NAME
docker run -d -e "NODE_APPLICATION_NAME=Testing123"
Finally, when using SE_OPTS be careful not to accidentally change values. Specifically, the browser version. You can see by looking at entry_point.sh the browser version is calculated.
FIREFOX_VERSION=$( firefox -version | cut -d " " -f 3 )
If you change it to something else you will not get the results you are looking for.
I'm newbie in docker. And i tried to create a Dockerfile to run a website was written byrails, postgresql on apache+passenger.
But when i run Dockerfile, it run successfully, but it had a problem with permission denied. I found problem that folder web must belongs to apache user. Then i tried to change apache user to source web (on container). And it run ok.
But every time i modified a file on local. It always ask password when i saved this file.
And i checked permission source on local. It changed all role to weird role.
How can i solved this problem ?
This is my Dockerfile.
And i used two commands to run.
docker build -t wics .
docker run -v /home/khanhpn/Project/wics:/home/abc -p 80:80 -it wics /bin/bash
After a mount of time, i found a solution to solve this problem.
I just add this line in Dockerfile, the problem was solved.
RUN usermod -u 1000 apache