I have dockerbeats set up on a local cluster that is running ELK stack and some other misc. dockers (all containers controlled via kubernetes). I set up the dashboard from Ingensi (Ingensi dockerbeat Dashboard) for kibana and ran into an issue with the containerNames field while setting up the graphs. Now, for context, my docker containers have names like these:
k8s_dockerbeats.79c42f90_dockerbeats-796n9_default_472faa11-1b3a-11e6-8bf4-28924a2bffbf_2832ea88
(as well as supporting containers for kubernetes with similar container names) [2]: http://i.stack.imgur.com/hvIUG.png
k8s_POD.6d00e006_dockerbeats-796n9_default_472faa11-1b3a-11e6-8bf4-28924a2bffbf_3ddcfe44
When I set up the dashboard in kibana I get an issue where I get multiple containerNames from the same container. For example instead of a single containerName output I get the containerName split up into smaller segments:
k8s_dockerbeats
79c42f90_dockerbeats
796n9
28924a2bffbf_3ddcfe44
and so on...
I assume that the format of the container name is confusing the dashboard (maybe in the way that it parses the name information) and I could probably go around renaming every container to a more sensible name.
But before I do that, is there a way to configure the dashboard in such a way that I read in the entire container name string so that it does not break up like it does in the first image? (assuming I'll have to dig into the .json files from the repository mentioned above)
Thanks in advance if anyone answers this.
It sounds like the container name is being analyzed by Elasticsearch. You need to make sure that the container name field is marked as not_analyzed in the Elasticsearch index template. You can do this by installing the index template provided by Dockerbeat.
Marking the field as not_analyzed ensures that the data is not tokenized and it gets indexed as is. It will only be searchable by specifying the exact string.
You will need to delete your current indexes after installing the new index template in order to change the mappings.
Install the provided index template:
curl -XPUT 'http://elasticsearch:9200/_template/dockerbeat' -d#dockerbeat.template.json
Delete your the existing indexes:
curl -XDELETE 'http://elasticsearch:9200/dockerbeat-*'
You can view your current mappings by querying Elasticearch:
curl http://elasticsearch:9200/dockerbeat-*/_mapping
Related
I'm new to cassandra and wanted to understand and implement the NetworkTopology Stratergy.
I want to create a cassandra cluster with NetworkTopology stratergy with multiple data centers. How to do it?
I tried creating a docker bridge network and three cassandra nodes: cas1, cas2, cas3. When used nodetools to check status a cluster with single datacentre is only getting created. But I want to create 2 datacenters.
There's a document which walks you through this: Initializing a multiple node cluster (multiple datacenters). It's for Cassandra 3.x, but the procedure is pretty much the same for 4.x as well.
But if I had to guess, I'd say there's two things you're probably missing:
In the cassandra.yaml set the endpoint_snitch to GossipingPropertyFileSnitch.
endpoint_snitch: GossipingPropertyFileSnitch
That tells Cassandra to check the cassandra-rackdc.properties file for data center and rack information. Inside that file, you'll find the following settings (by default).
dc=dc1
rack=rack1
This is where you can set the name of the new DC. Then you can use those data center names to specify replication on keyspaces using NetworkTopologyStrategy.
I am migrating a MariaDB database into a Linux docker container.
I am using mariadb:latest in Ubuntu 20 LTS via Windows 10 WSL2 via VSCode Remote WSL.
I have copied the sql dump into the container and imported it into the InnoDB database which has DEFAULT CHARACTER SET utf8. It does not report any errors:
> source /test.sql
That file does this (actual data truncated for this post):
USE `mydb`;
DROP TABLE IF EXISTS `opsitemtest`;
CREATE TABLE `opsitemtest` (
`opId` int(11) NOT NULL AUTO_INCREMENT,
`opKey` varchar(50) DEFAULT NULL,
`opName` varchar(200) DEFAULT NULL,
`opDetails` longtext,
PRIMARY KEY (`opId`),
KEY `token` (`opKey`)
) ENGINE=InnoDB AUTO_INCREMENT=4784 DEFAULT CHARSET=latin1;
insert into `opsitemtest`(`opId`,`opKey`,`opName`,`opDetails`) values
(4773,'8vlte0755dj','VTools addin for MSAccess','<p>There is a super helpful ...'),
(4774,'8vttlcr2fTA','BAS OLD QB','<ol>\n<li><a href=\"https://www.anz.com/inetbank/bankmain.asp\" ...'),
(4783,'9c7id5rmxGK','STP - Single Touch Payrol','<h1>Gather data</h1>\n<ol style=\"list-style-type: decimal;\"> ...');
If I source a subset of 12 records of the table in question all the columns are correctly populated.
If I source the full set of data for the same table ( 4700 rows ) where everything else is the same, many of the opDetails long text fields have a length showing in sqlYog but no data is visible. If I run a SELECT on that column there are no errors but some of the opDetails fields are "empty" (meaning: you can't see any data), and when I serialize that field, the opDetails column of some records (not all) has
"opDetails" : "\u0000\u0000\u0000\u0000\u0000\u0000\",
( and many more \u0000 ).
The opDetails field contains HTML fragments. I am guessing it is something to do with that content and possibly the CHARSET, although that doesn't explain why the error shows up only when there are a large number of rows imported. The same row imported via a set of 12 rows works correctly.
The same test of the full set of data on a Windows box with MariaDB running on that host (ie no Ubuntu or WSL etc) all works perfectly.
I tried setting the table charset to utf8 to match the database default but that had no effect. I assume it is some kind of Windows WSL issue but I am running the source command on the container all within the Ubuntu host.
The MariaDB data folder is mapped using a volume, again all inside the Ubuntu container:
volumes:
- ../flowt-docker-volumes/mariadb-data:/var/lib/mysql
Can anyone offer any suggestions while I go through and try manually removing content until it works? I am really in the dark here.
EDIT: I just ran the same import process on a Mac to a MariaDB container on the OSX host to check whether it was actually related to Windows WSL etc and the OSX database has the same issue. So maybe it is a MariaDB docker issue?
EDIT 2: It looks like it has nothing to do with the actual content of opDetails. For a given row that is showing the symptoms, whether or not the data gets imported correctly seems to depend on how many rows I am importing! For a small number of rows, all is well. For a large number there is missing data, but always the same rows and opDetails field. I will try importing in small chunks but overall the table isn't THAT big!
EDIT 3: I tried a docker-compose without a volume and imported the data directly into the MariaDB container. Same problem. I was wondering whether it was a file system incompatibility or some kind of speed issue. Yes, grasping at straws!
Thanks,
Murray
OK. I got it working. :-)
One piece of info I neglected to mention, and it might not be relevant anyway, is that I was importing from an sql dump from 10.1.48-MariaDB-0ubuntu0.18.04.1 because I was migrating a legacy app.
So, with my docker-compose:
Version
Result
mysql:latest
data imported correctly
mariadb:latest
failed as per this issue
mariadb:mariadb:10.7.4
failed as per this issue
mariadb:mariadb:10.7
failed as per this issue
mariadb:10.6
data imported correctly
mariadb:10.5
data imported correctly
mariadb:10.2
data imported correctly
Important: remember to completely remove the external volume mount folder content between tests!
So, now I am not sure whether the issue was some kind of sql incompatibility that I need to be aware of, or whether it is a bug that was introduced between v10.6 and 10.7. Therefore I have not logged a bug report. If others with more expertise think this is a bug, I am happy to make a report.
For now I am happy to use 10.6 so I can progress the migration- the deadline is looming!
So, this is sort of "solved".
Thanks for all your help. If I discover anything further I will post back here.
Murray
I run neo4j Version: 3.3.2 in a docker container and tried to configure a default style as described above.
Grass file is accessible via HTTP (http://somedomain.com/resources/style.grass)
CORS rules are set
Command added to config: browser.post_connect_cmd=style http://somedomain.com/resources/style.grass
When i start my container with this config and run match (n) return n i see nothing.
Then i run :style http://somedomain.com/resources/style.grass and it works... Why?
That leads me to conclude that the grass file ifself (content) and the accessible of this file are not the problem.
It seems that there is an error with the default style configuration.
Do you have any tips to debug this? The logs are clean.
I found the problem. We added this property at the end of the neo4j.conf. After startup neo4j reorder the properties, put it at the 2nd place and accidentally add the 3rd property at the end of the 2nd, so that it’s not valid. We put it manually at the 2nd place and now it works.
I read the following post which does what I want for all jobs (and I made it works)
List all keep-forever builds in Jenkins?
here is what I have now :
http://jenkinsmediaprocessor:7880/api/xml?depth=2&xpath=/hudson/job/build[keepLog=%22true%22]/url&wrapper=forever
But I only want to display a specific task. However I don't know where to set the job name,
I tried the followings
1-) job/MYJOBNAME
2-) JOBPATH/job/MYJOBNAME
3-) replace job with MYJOBNAME
I can't make it works.
Also would it be possible to only display a list of number? Instead of having to use a grep on the *.xml.
This works for me:
JENKINS_URL/job/JOB_NAME/api/xml?depth=2&xpath=//build[keepLog=%22true%22]/number&wrapper=forever
Sample output:
<forever>
<number>688</number>
<number>687</number>
</forever>
forever is a wrapper parameter. You will find more details if refer to JENKINS_URL/api:
For XPath that matches multiple nodes, you need to also specify the "wrapper" query parameter to specify the name of the root XML element to be created so that the resulting XML becomes well-formed.
I have elasticsearch cluster, and I add indices.memory.index_buffer_size parameter in elasticsearch.yml file.
I set this parameter be 70%. And then the cluster began to indexing. After all, I reset this parameter be 10%, and I restart the elasticsearch.
My question is how can I check my settings are working?
I think (but I'm not positive) that it should show up in the nodes information API. Try this:
curl -XGET "http://localhost:9200/_nodes?clear&all"
You can use the /_nodes API, but what I found puzzling was that the value for the setting is not returned if you are using the default. But once you change it (in elasticsearch.yml) and rebounce your cluster, it >will< appar in the output of the /_nodes API call.
to illustrate.
install elastic search and start cluster.
curl 'http://localhost:9200/_nodes' | grep index_buffer_size # no match
change value of index_buffer_size to other than default and bounce cluster
curl 'http://localhost:9200/_nodes' | grep index_buffer_size # Match !
My guess is that this is the case for any .yml setting, not just this one (although i have not verified that theory).