SaltStack: getting No top file or master_tops data matches found - devops

I am new to SaltStack and following some tutorials and trying to execute state.apply but getting below error:
# salt "host2" state.apply
host2:
----------
ID: states
Function: no.None
Result: False
Comment: No Top file or external nodes data matches found
Started:
Duration:
Changes:
Summary for host2
------------
Succeeded: 0
Failed: 1
------------
Total states run: 1
I am able to test.ping successfully to host.
here is directory structure:
/etc/salt/srv/salt/states
|-top.sls
|-installations
|-init.sls
file root entry in master config
file_roots:
base:
- /srv/salt/states
top.sls ->
base:
'*':
- installations
init.sls->
install_apache:
pkg.installed:
- name: apache2

You need to change the path to your states, or move them to the path set in file_roots.
The file_roots option is where you should place your files, you should have the following tree:
# tree /srv/salt/
/srv/salt/
|-- installations
`-- init.sls
`-- top.sls
Or you could change your file_roots, but I wouldn't do it, since /srv/salt/ seems to be a sort of "standard".
Have a look at the tutorials, if you haven't already: https://docs.saltstack.com/en/getstarted/fundamentals/

I changes the
file_root:
base:
- /etc/salt/srv/salt/state
and it works for me. looks it wasn't picking path correctly

Related

Using ansible to grep then format the output [duplicate]

I am using ansible to gather information from remote nodes and will then use this information to update relevant RPMs.
The issue I am having is collection version number of various applications and writing them to a file.
Playbook:
---
- name: Check Application Versions
hosts: kubernetes
tasks:
- name: Check K8S version.
shell: kubectl --version
register: k8s_version
- debug: msg="{{ k8s_version.stdout }}"
Inventory file:
[kubernetes]
172.29.219.102
172.29.219.105
172.29.219.104
172.29.219.103
Output:
TASK [debug] *******************************************************************
ok: [172.29.219.102] => {
"msg": "Kubernetes v1.4.0"
}
ok: [172.29.219.103] => {
"msg": "Kubernetes v1.4.0"
}
ok: [172.29.219.105] => {
"msg": "Kubernetes v1.4.0"
}
ok: [172.29.219.104] => {
"msg": "Kubernetes v1.4.0"
}
The above portion is simple and works. Now I want to write the output to file.
Now im trying to write this information to a file.I want something like:
Kubernetes v1.4.0
Kubernetes v1.4.0
Kubernetes v1.4.0
Kubernetes v1.4.0
So I added the below line:
- local_action: copy content={{ k8s_version.stdout_lines }} dest=/tmp/test
My /tmp/test looks like :
# cat /tmp/test
["Kubernetes v1.4.0"]
There is only one value here.
I tried to do something different then.
- local_action: lineinfile dest=/tmp/foo line="{{ k8s_version.stdout }}" insertafter=EOF
This resulted in:
# cat /tmp/foo
Kubernetes v1.4.0
Im trying to figure out why I only see one value whereas I should see the versions of every node in my inventory file. What am I doing wrong?
What am I doing wrong ?
lineinfile module does not perform the action "add a line to a file", instead it ensures a given line is present in the file. If all your target nodes have the same version, it won't add the same line multiple times.
On the other hand, copy module was overwriting the file.
If you need to register values for all hosts, you can for example create a template which will loop over hosts in the kubernetes group:
- copy:
content: "{% for host in groups.kubernetes %}{{ hostvars[host].k8s_version }}\n{% endfor %}"
dest: /tmp/test
delegate_to: localhost
run_once: true
Another way would be to extract the values with map from hostvars, but given you want the values from kubernetes host group only, I'm not sure it would be prettier. And having a for in the template allows you to easily add host names.
According to this post
Ansible register result of multiple commands
your desired variable is in k8s_version.results To access it you need to work with a template where you just iterate over it:
- local_action: template src=my_nodes.j2 dest=/tmp/test
And the template templates/my_nodes.j2 :
{% for res in k8s_version.results %}
{{ res.stdout }}
{% endfor %}
The complete playbook would then be:
---
- name: Check Application Versions
hosts: kubernetes
tasks:
- name: Check K8S version.
shell: kubectl --version
register: k8s_version
- local_action: template src=my_nodes.j2 dest=/tmp/test

Docker, ENOENT: no such file or directory

I have a Storage constant that is used in a file called listingController.js
const storage = Storage({
keyFilename: "../key/keyname.json"
});
Everything works fine when I'm not using Docker but after I create a Docker image and deploy it on server I get the following error:
ENOENT: no such file or directory, open '/key/keyname.json'
at wrapError (/app/node_modules/gcs-resumable-upload/build/src/index.js:17:12)
at /app/node_modules/gcs-resumable-upload/build/src/index.js:235:19
at getToken (/app/node_modules/google-auto-auth/index.js:27:9)
at getAuthClient (/app/node_modules/google-auto-auth/index.js:233:9)
at <anonymous>
Here I see a problem that the '..' is ignored in front of the path which is why I think that file is not found.
Here is my project structure:
src
--- key
----- keyname.json
----- firebasekeyfilename.json
--- controller
----- listingController.js
----- firebaseController.js
I have tried all different combinations of file names and paths but I cannot get it to find that file.
Does anyone have a clue why this is happening?
In my firebaseController I have the following reference to a similar file in the same folder and it works fine.
var serviceAccount = require("../key/firebasekeyfilename");
The only difference is that the path is inside require() and I guess that requires a different path.
Been stuck with this for a couple of days now, any pointers would be appreciated, thank!

Not able to find curator.yml (elasticsearch-curator) in linux

Official site of elasticsearch says the default config file exists in /home/username/.curator/curator.yml
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/command-line.html
But there is no such folder.
Also, I tried creating curator.yml and give path using --config option. But, it throws me error
curator --config ./curator.yml
Error: no such option: --config
Installation was done using apt
sudo apt-get update && sudo apt-get install elasticsearch-curator
Help me create a config file as I want to delete my log-indexes
Please note that the documentation does not say it that file exists after creation, it says:
If --config and CONFIG.YML are not provided, Curator will look in ~/.curator/curator.yml for the configuration file.
The file must be created by the end user.
Also, if you installed via:
sudo apt-get update && sudo apt-get install elasticsearch-curator
but did not add the official Elastic repository for Curator, then you installed an older version. Please check which version you are running with:
$ curator --version
curator, version 5.4.1
If you do not see the current version (5.4.1 at the time this answer was added), then you do not have the appropriate repository installed.
The official documentation provides an example client configuration file here.
There are also many examples of action files in the examples
Yes, one needs to create both the curator.yml as well as action.yml files.
Since I am on centos 7, I happened to install curator from RPM, and in its default /opt/elastic-curator' I could follow up this good blog (but badly formatted!) : https://anchormen.nl/blog/big-data-services/monitoring-aws-redshift-with-elasticsearch/ to ge the files as follows(you may modify according to your needs) :
curator.yml
---
# Remember, leave a key empty if there is no value. None will be a string,
# not a Python "NoneType"
client:
hosts:
- <host1>
- <host2, likewise upto hostN >
port: 9200
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth:
timeout: 30
master_only: False
logging:
loglevel: INFO
logfile: /var/log/curator.log
logformat: default
blacklist: []
and an action.yml as follows :
---
# Remember, leave a key empty if there is no value. None will be a string,
# not a Python "NoneType"
#
# Also remember that all examples have 'disable_action' set to True. If you
# want to use this action as a template, be sure to set this to False after
# copying it.
actions:
1:
action: rollover
description: Rollover the index associated with index 'name', which should be in the form of prefix-000001 (or similar),or prefix-YYYY.MM.DD-1.
options:
disable_action: False
name: redshift_metrics_daily
conditions:
max_age: 1d
extra_settings:
index.number_of_shards: 2
index.number_of_replicas: 1
2:
action: rollover
description: Rollover the index associated with index 'name' , which should be in the form of prefix-000001 (or similar), or prefix-YYYY.MM.DD-1.
options:
disable_action: False
name: redshift_query_metadata_daily
conditions:
max_age: 1d
extra_settings:
index.number_of_shards: 2
index.number_of_replicas: 1

How do I configure SaltStack to transfer a file (or install a package) for the first time?

I am running two instances of RedHat. I have SaltMaster installed on one machine and SaltMinion installed on another. I am using a free version of Salt. I want to test SaltStack to do a basic configuration management task. If it can transfer a file from SaltMaster to SaltMinion, that would be great. If it can install Apache web server on SaltMinion, that would be great. Either task will help me learn. My learning goal is semi-flexible.
I can use salt '*' test.ping. The response is True. I tried this command: salt '*' state.apply
I got this error:
> hostname.fqdn:
> Data failed to compile:
> ----------
> No matching salt environment for environment 'qa' found
> ----------
> No matching sls found for 'qa1' in env 'qa'
> ----------
> No matching sls found for 'base1' in env 'base'
> ----------
> No matching salt environment for environment 'dev' found
> ----------
> Specified SLS base1 in saltenv dev is not available on the salt master or through a configured fileserver
I modified the /etc/salt/master file. I uncommented these lines:
fileserver_backend:
- git
- roots
I tried this command again: salt '*' state.apply
I received this error:
> [ERROR ] Error parsing configuration file: /etc/salt/master -
> expected '<document start>', but found '<block mapping start>' in
> "<string>", line 547, column 1:
> fileserver_backend:
> ^ [ERROR ] Error parsing configuration file: /etc/salt/master - expected '<document start>', but found '<block mapping start>' in
> "<string>", line 547, column 1:
> fileserver_backend:
> ^
I have been following these directions here:
https://docs.saltstack.com/en/latest/topics/tutorials/states_pt1.html
I created a webserver.sls file.
I inserted these lines as the content:
apache: # ID declaration
pkg: # state declaration
- installed # function declaration
I do not see how the three lines in the directions above would be enough to configure SaltStack to work. Where would the apache installation media need to be? Where would the transfer happen from? Am I supposed to download the media to SaltMaster? I would assume so. But where would I put it? I have a satellite server for yum commands to work.
Alternatively, how do I get SaltStack to transfer a file from SaltMaster to SaltMinion?
The first error ([...]No matching sls found for 'qa1' in env 'qa'[...]) indicates that you have configured a lot of different environments (file_roots), which are not present on your master's filesystem. Your approach to solve this goes in the correct direction, but leads to this error:
[ERROR ] Error parsing configuration file: /etc/salt/master - expected '', but found '' in "", line 547, column 1: fileserver_backend: ^ [ERROR ] Error parsing configuration file: /etc/salt/master - expected '', but found '' in "", line 547, column 1: fileserver_backend: ^
You should no longer be able to test.ping your minion, as the salt master should not run anymore, does it? To solve it just read the error message. It tells you with which point in your salt master configuration file salt is unhappy.
The fileserver_backend configures which types of backend should be available. You should check the file_roots configuration to actually define which roots are available. Roots refer to salt states folders in your filesystem.
A very simple config might look like that:
file_roots:
base:
- /srv/salt
It assumes that /srv/salt is the root of your state tree - which effectively means, that your webserver.sls should be located in this folder.
Your webserver.sls looks promising - it should install apache2 on a minion, when you apply it.
Managing configuration files on the master and transferring them to the minions is something salt can easily achieve. A simple state might look like:
/etc/myawesomeconfigurationfile.conf:
file.managed:
source: salt://myawesomefile # refers to /srv/salt/myawesomefile
user: root
group: root
mode: 640
You also asked for media files that you want to manage. If you talk about application related data it is not a good idea to use salt to move them around. IMO other approaches like NFS, GlusterFS or anything else that decouples user content from your application would be a better approach.

reviewboard: post-review error uploading diff with basedir configuration error

I'm working in Ubuntu and I want to use post-review to create a new review request for reviewboard with a manually-generated .diff file but failed.
I worked on post-review for most time of today but find myself stuck on this postbasedir problem. I asked three of my colleagues to sit on my seat to solve this problem but they too found themselves run out of effective solutions.
here is the problem:
I set repoistory and reviewboard_url in the .reviewboardrc file, I'm sure the two settings are ok because the request was successfully created in review board and the repository points to the directory where the target file resides.
here is the content of .reviewboardrc:
============================= .reviewboardrc ===============================
REVIEWBOARD_URL = "http://10.22.200.166:8088"
REPOSITORY = "http://svn.d.xiaonei.com/wap/renren/renren-wap-talk/trunk"
============================================================================
I run post-review as follows but failed on uploading the Doxyfile.diff file and here is the content of it:
============================== Doxyfile.diff ================================
Index: Doxyfile
===================================================================
--- Doxyfile (revision 107236)
+++ Doxyfile (working copy)
## -1,3 +1,5 ##
+#POST_REVIEW_TEST
+POST_REVIEW_TEST =
# Doxyfile 1.7.6
# This file describes the settings to be used by the documentation system
# doxygen (www.doxygen.org) for a project.
=============================================================================
and the output error message with debug info enabled
====================== err msg ===============================================
houchenxi#houchenxi-PC:~/renren-wap-talk$ post-review -dp --diff-filename=Doxyfile.diff
>>> RBTools 0.4.2
>>> Python 2.7.2+ (default, Oct 4 2011, 20:06:09)
[GCC 4.6.1]
>>> Running on Linux-3.0.0-22-generic-x86_64-with-Ubuntu-11.10-oneiric
>>> Home = /home/houchenxi
>>> Current Directory = /home/houchenxi/renren-wap-talk
>>> Checking the repository type. Errors shown below are mostly harmless.
DEBUG:root:Checking for a CVS repository...
DEBUG:root:Checking for a ClearCase repository...
DEBUG:root:Checking for a Git repository...
DEBUG:root:Running: git rev-parse --git-dir
DEBUG:root:Command exited with rc 128: ['git', 'rev-parse', '--git-dir']
fatal: Not a git repository (or any parent up to mount parent )
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
---
DEBUG:root:Checking for a Mercurial repository...
DEBUG:root:Running: hg showconfig
DEBUG:root:Running: hg root
DEBUG:root:Command exited with rc 255: ['hg', 'root']
abort: no repository found in '/home/houchenxi/renren-wap-talk' (.hg not found)!
---
DEBUG:root:Checking for a Perforce repository...
DEBUG:root:Checking for a Plastic repository...
DEBUG:root:Checking for a Subversion repository...
DEBUG:root:Running: svn info http://svn.d.xiaonei.com/wap/renren/renren-wap-talk/trunk
DEBUG:root:Running: diff --version
DEBUG:root:repository info: Path: http://svn.d.xiaonei.com/wap, Base path: /renren/renren-wap-talk/trunk, Supports changesets: False
>>> Finished checking the repository type.
>>> HTTP GETting api/
>>> HTTP GETting http://ourhost:8088/api/info/
>>> Using the new web API
>>> HTTP GETting http://ourhost:8088/api/repositories/
>>> Attempting to create review request on http://svn.d.xiaonei.com/wap/renren/renren-wap-talk/trunk for None
>>> HTTP POSTing to http://ourhost:8088/api/review-requests/: {'repository': 'http://svn.d.xiaonei.com/wap/renren/renren-wap-talk/trunk'}
>>> Review request created
>>> Uploading diff, size: 335
>>> HTTP POSTing to http://ourhost:8088/api/review-requests/130/diffs/: {'basedir': '/renren/renren-wap-talk/trunk'}
>>> Got API Error 207 (HTTP code 400): The file was not found in the repository
>>> Error data: {u'stat': u'fail', u'file': u'/renren/renren-wap-talk/trunk/Doxyfile', u'err': {u'msg': u'The file was not found in the repository', u'code': 207}, u'revision': u'107236'}
Error uploading diff
Your review request still exists, but the diff is not attached.
==========================================================================
In the err msg above we found that the basedir parameter {'basedir': '/renren/renren-wap-talk/trunk'} is incorrect, which is supposed to be the full path from "http",namely "http://svn.d.xiaonei.com/wap/renren/renren-wap-talk/trunk" , but no matter how we try to modify the .reviewboardrc or add command line arguments, we just cannot correct this value to the expected right value, we searched in the help info by running "post-review --help", but there just isn't the "--basedir" option available.
All I have to do is to correct the basedir value to the correct full path , either by modifying the .reviewboardrc or changing post-review arguments.
Does anyone have any similar experience for solving this problem, and suggestion is highly appreciated.
I believe your problem may be the correlation between the path of the SVN repository:
http://svn.d.xiaonei.com/wap/renren/renren-wap-talk/trunk
and the relative path in your diff file (the level at which it was generated in the hierarchy of folders):
Index: Doxyfile
Unless the file Doxyfile is situated directly in the trunk, you should try to re-create your diff at such a level so that the diff will contain the missing relative path between the trunk and Doxyfile:
Index: /missing/path/to/Doxyfile
..or to update your SVN URL in Reviewboard so that the URL contains itself the missing path.
You will find a detailed explanation of this correlation in this answer for the post: review board diff not uploading
HTH!

Resources