I have a solr setup with two cores. I want to schedule a core(core1, backend) for full import frequently(e.g. after every 5 mins), then swap with the live(core0, serving) core from shell command through a shceduler.
For full-import command, I am using following shell command
wget -o - -q -t 1 http://localhost:8080/solr/core1/dataimport?command=full-import
Which works fine. If I do a core swap from browser by hitting
http://localhost:8080/solr/admin/cores?action=SWAP&core=core1&other=core0, I get latest update instantly on search. But if I schedule this URL as shell command similar to dataimport, it doesn't do that swap.
Did you try with
curl
"http://'localhost':8080/solr/admin/cores?action=SWAP&core=core1&other=core0"
from shell?
There is catch with the SWAPs
Apache Solr allows to swap two cores around for non-Cloud configurations. They take each other’s name, so it is a good way to push an updated core into a production without downtime.
But an interesting question is how this is achieved. Normally, core name is it’s directory name too. So, does Solr rename the directory on the filesystem too?
Not really! Instead name property in the core.properties file is updated to use the name of the other core. Usually that property is used to give an alternave name of the core for when the directory naming conventions are not suitable.
The gotcha is - of course - that you still have two directories with right looking names for the cores you see in the Admin UI. So, it is very easy to forget that extra redirection/rename step when troubleshooting somebody else’s - or even your own old - setup.
Related
I'm new to Docker, and I'm not sure how to quite deal with this situation.
So I'm trying to run a docker container in order to replicate some results from a research paper, specifically from here: https://github.com/danhper/bigcode-tools/blob/master/doc/tutorial.md
(image link: https://hub.docker.com/r/tuvistavie/bigcode-tools/).
I'm using a windows machine, and every time I try to run the docker image (via: docker run -p 80:80 tuvistavie/bigcode-tools), it instantly closes. I've tried running other images, such as the getting-started, but that image doesn't close instantly.
I've looked at some other potential workarounds, like using -dit, but since the instructions require setting an alias/doskey for a docker run command, using the alias and chaining it with other commands multiple times results in creating a queue for the docker container since the port is tied to the alias.
Like in the instructions from the GitHub link, I'm trying to set an alias/doskey to make api calls to pull data, but I am unable to get any data nor am I getting any errors when performing the calls on the command prompt.
Sorry for the long question, and thank you for your time!
Going in order of the instructions:
0. I can run this, it added the image to my Docker Desktop
1.
Since I'm using a windows machine, I had to use 'set' instead of 'export'
I'm not exactly sure what the $ is meant for in UNIX, and whether or not it has significant meaning, but from my understanding, the whole purpose is to create a directory named 'bigcode-workspace'
Instead of 'alias,' I needed to use doskey.
Since -dit prevented my image from instantly closing, I added that in as well, but I'm not 100% sure what it means. Running docker run (...) resulted in the docker image instantly closing.
When it came to using the doskey alias + another command, I've tried:
(doskey macro) (another command)
(doskey macro) ^& (another command)
(doskey macro) $T (another command)
This also seemed to be using github api call, so I also added a --token=(github_token), but that didn't change anything either
Because the later steps require expected data pulled from here, I am unable to progress any further.
Looks like this image is designed to be used as a command-line utility. So it should not be running continuously, but you run it via alias docker-bigcode for your tasks.
$BIGCODE_WORKSPACE is an environment variable expansion here. So on a Windows machine it's %BIGCODE_WORKSPACE%. You might want to set this variable in Settings->System->About->Advanced System Settings, because variables set with SET command will apply to the current command prompt session only. Or you can specify the path directly, without environment variable.
As for alias then I would just create a batch file with the following content:
docker run -p 6006:6006 -v %BIGCODE_WORKSPACE%:/bigcode-tools/workspace tuvistavie/bigcode-tools %*
This will run the specified command appending the batch file parameters at the end. You might need to add double quotes if BIGCODE_WORKSPACE path contains spaces.
We have some local installers and zip files that we use to build our docker images. It is easy to get this to work in a Dockerfile:
FROM mcr.microsoft.com/windows/nanoserver
COPY myinstaller.exe .
RUN myinstaller.exe; \
del myinstaller.exe
The problem here is that it produces a layer for the COPY line, which increases the size of the image. A common work-around for this is to have one RUN line, that downloads the file from the Internet, runs commands, and then deletes the installation file. The problem, as written above, is that the installers are on the local filesystem.
I found that there is a --squash command for docker:
docker build --squash -t mytestimage .
This does exactly what I want: It gives me an image without this extra installer file that is not necessary. To run this command, you need to enable experimental features though. There is also an open issue to simply remove this feature:
https://github.com/moby/moby/issues/34565
Is there some alternative way of using local installers in a Dockerfile when running on Windows, that doesn't involve setting up a server to provide the files?
We ended up setting up nginx to provide files when building. On our build server, the machine building our docker images and the server that has the installer files have a very good connection between them, so downloading huge files is not a real problem.
When it comes to --squash, it is bugged for Docker on Windows. Here is the relevant issue for it:
https://github.com/moby/moby/issues/31468
There is an issue to move --squash out of experimental, but it doesn't seem to have a lot of support:
https://github.com/moby/moby/issues/38657
The alternative that some people propose instead of --squash is multi stage build, discussion here:
https://github.com/moby/moby/issues/34565
There is an alternative to --squash, if you have local installer files, you don't want to set up a web server, and you would like your docker image to be small, and you are running Windows: Use mapped drives.
In Windows, you can share folders with other users on your network. Docker containers are like another computer that is running on your physical machine, and it can access these network drives.
First set up a new user, for example username share and password password1. Create a folder somewhere on your computer. Then right click it, click properties, and then go to the Sharing tab and click "Share". Find the user that you have just created, using the little dropdown menu and Find people ..., and share the folder with this user.
Create a folder somewhere for your test project. Create a batch file setupshare.bat that looks like this:
#echo off
for /f "tokens=2 delims=:" %%i in ('ipconfig ^| findstr "Default Gateway"') do (
set hostip=%%i
goto :end
)
:end
set hostip=%hostip: =%
net use O: \\%hostip%\vms /USER:share password1
The first part of this file is only to find the ip address that the docker container can use to access its host computer. It is not the most pretty thing I've ever put together, so let me know if there's a better way!
It uses a for-loop, as that is the way to save the output of a command to a variable in batch files. The command is ipconfig, and we pipe it to findstr and searches for Default Gateway. We need to use ^| instead of just | because it is in a for-loop. The first part of the for-loop divides each line from the command on the delimiter, which is : in this case, and we only take the second token. The for-loop only handles the first line, if there are multiple entries with a Default Gateway. This script doesn't work if there are multiple entries and the first one is not the correct one.
The line set hostip=%hostip: =% is to remove a space at the start of the string.
We then have the IP address that we want to use stored in hostip. We use this in the net use command, which will map O:\ to shared folder vms on the machine with IP hostip. We use the username share and the password password1. Note that this is a very bad way of handling passwords, as they kind of should be secret!
With a batch file like this, we can set up a Dockerfile in this way:
# escape=`
FROM mcr.microsoft.com/dotnet/core/sdk:3.0
COPY setupshare.bat .
RUN setupshare.bat && `
copy O:\file.txt file.txt
The RUN command will first call setupshare.bat that sets up the network share properly. We can then use any file that we shared, for example a huge installer, and install the things that we want. In this case I have only shared a test file file.txt to see that it works, so just change that line.
I would still advice everyone to just set up a little web server, for example nginx, and use the standard way of writing Dockerfiles, with downloading files and running it in the same RUN command. That's what people expect when they see a Dockerfile, and it should be a more robust solution.
We can also hope that the Docker people either makes a COPY command that can copy, run, and delete installers in the same layer, or that --squash is implemented properly.
I'm working on an application which needs to be initialized the first time it is run.
Practically, what this will do is initialize a database with some starter values, and save some files in a persistent volume. If I stop the container and then restart it, I don't want to re-run that bootstrapping routine. In other words, if the container is present and populated - skip the initialization routine.
The way I was going to implement this was have an entry-point script which checks if the configuration files are present, and if so will skip the bootstrapping routine, however, I was wondering if there is a better way to do it?
For example, is there a way to run a script which is specifically triggered by the need to create a volume? If I could d that, the only circumstance under which I'd run the bootstrapper would be when the application was initializing for the first time.
Or, is there a better, more Dockerish pattern that defines how I should go about this problem?
"Do the initialization in an entrypoint script if the files don't already exist" seems to be reasonably idiomatic. For example, the standard postgres:9.6 image checks for a $PGDATA/PG_VERSION file.
Hypothetically this can look something like:
#!/bin/sh
if [ ! -f /data/config.ini ]; then
/opt/myapp/setup-data.sh /data
fi
exec "$#"
Remember that it's very routine to delete and recreate containers for a variety of reasons (IME stop and start as actions are rare, but some of this is habits born of an earlier age of Docker);this ties well into your intuition to use the entrypoint for this, since it will get launched on every docker run. From within your container you can't really tell if a directory is or isn't a volume and there aren't any hooks you can tie into; at the point the entrypoint begins, the container environment is fully set up, with whatever networks and volumes already attached.
Is there a way of setting up some type of CRON job on the Netscaler VPX running firmware 11.0 to automatically clear Persistence Session Records on a daily basis?
https://docs.citrix.com/en-us/netscaler/12/load-balancing/load-balancing-persistence/clearing-persistence.html
in the /nsconfig folder you have a rc.netscaler file.
Make a entry in the rc.netscaler where you add the cron job.
When you reboot the rc.netscaler will be executed.
You can also put files in the /var directory (survives reboot) and use an entry in /nsconfig/rc.netscaler to copy to /etc/....
setup a cron job on the netscaler freebsd. use the following structure for your command:
nscli -U xxx.xxx.xxx.xxx:nsroot:nsroot "clear lb persistentSessions"
edit to add the following note: keep in mind you will most likely impact anyone connected at the time when running this command (depends on your application)
I am trying to contribute more with couchdb code, but I have really no idea how it is done the right way.
I have cloned the source from apache git repository and built it with
./configure
make && sudo make install
Then I wanted to change a file from the source called couch_httpd_show.erl
Do I need to run make && sudo make install again for every change I make to the source code and want to see how it behaves?
I am sure there's a more practical way to do it, because this approach is a bit time and patience consuming right?
Yes, there is a shortcut.
./configure
make dev
./utils/run
This builds and runs CouchDB entirely in the current directory. Instead of running as a background daemon, CouchDB will run in the foreground and output log messages to the terminal. It uses some local directories to store stuff: ./tmp/log for logs, ./tmp/lib for databases, and (if I remember correctly) ./etc/couch/local_dev.ini for configuration.
If you run this instead:
./utils/run -i
then you will also have an interactive Erlang prompt, which you can use to help debug.
When I work on CouchDB, I run this in the shell:
make dev && ./utils/run -i
After I change some code, I press ^C, up-arrow, return.
When I joined Couchio, I was responsible for production CouchDB deployments. I asked Chris Anderson for advice about something and he said, "Sorry, ask Jan. I've been just using utils/run for years!"
You can rebuild that one file and drop the output beam in place and restart.
erlc <file.erl>
& then copy the .beam file into place. To restart couchdb use either init:restart(). in the erlang shell or POST /_restart to CouchDB.
Although you might want to consider using the commandline erlang & javascript test suite also to ensure you didn't break anything.