I have a yaml file(config.yaml) with tags/structure similar to what is mentioned below. I need to add a new tenant(tenant3) to the list of the existing tenants. How do I achieve it using the pipeline/groovy script? Any help/lead would be appreciated.
consumer_services:
- security
- token
id: 10000
tenants:
tenant_1:
state: all
web_token: true
cluster_pairs:
- cluster1
datacenter: local
client: CLIENT_TEST
tenant_2:
state: all
web_token: true
cluster_pairs:
- cluster2
datacenter: local
client: CLIENT_TEST
base_network:
subnets:
- 10.160.10.10
- 10.179.1.09
I think you need to do something like that:
#Grab('org.yaml:snakeyaml:1.17')
import org.yaml.snakeyaml.DumperOptions
import org.yaml.snakeyaml.Yaml
def options = new DumperOptions()
options.setDefaultFlowStyle(DumperOptions.FlowStyle.BLOCK)
Yaml yaml = new Yaml(options)
// load existing structure
def structure = yaml.load(new File("original.yml").text)
// modify the structure
structure.tenants.tenant_3 =
[
state : 'all',
web_token : true,
cluster_pairs: ['cluster3'],
datacenter : 'local',
client : 'CLIENT_TEST'
]
// save to the new file
new File("modified.yml").write(yaml.dump(structure))
So the steps are:
load the data from file into the memory
modify the structure in a way you like
store the modified structure to the file
I hope it will help.
Related
I have a yaml file with the following structure:
transfers:
- name: xyz
cloud: aws
subheading:
impact: Low
reason: ---
artifacts:
- name: name1
type: type1
source:
hash: a1b2C3dd4 ---> VALUE TO OVERWRITE
I would like to overwrite the existing hash value with a value of the latest GIT_COMMIT.
I have tried the method from the following question: write yaml file in jenkins with groovy. However, the value of hash[0][0] remains unchanged. This is the case even when I replace env.GIT_COMMIT with a test hash string "testHash123". I'm unsure why this is the case?
def filename = ('path/to/file.yaml')
def datas = readYaml file: filename
//change hash
datas.transfers['artifacts'].source.hash[0][0] = env.GIT_COMMIT
writeYaml file: filename, data: datas, overwrite: true
Please try the following.
datas.transfers[0]['artifacts'][0]['source'].hash = env.GIT_COMMIT
The easiest way to figure this out is by printing, so you can understand the structure.
[transfers:[[name:xyz, cloud:aws, subheading:[impact:Low, reason:xxxx], artifacts:[[name:name1, type:type1, source:[hash:a1b2C3dd4]]]]]]
As you can see above the transfer is a sequence, so you need to extract the correct segment with an index.
All,
I am trying to get the list of all the files that are in a particular repo in TFS GIT using REST API.
I found the below one but it only display the contents of the specific file name mentioned after "scopePath=/buld.xml", it only display the contents of file build.xml.
But I am trying, only to list all the files that are in a particular repository with out mentioning the particular file name.
Please help me.
https://{accountName}.visualstudio.com/{project}/_apis/git/repositories/{repositoryId}/items?items?scopePath=/&api-version=4.1
You can use the api below:
https://{accountName}.visualstudio.com/{project}/_apis/git/repositories/{repositoryId}/items?recursionLevel=Full&api-version=4.1
Also that could be achieved using VisualStudioOnline libs (at the date of writing comment it becomes AzureDevOps): Microsoft.TeamFoundationServer.Client, Microsoft.VisualStudio.Services.Client.
First, you need to create access token. Then just use code below:
VssBasicCredential credintials = new VssBasicCredential(String.Empty, "YOUR SECRET CODE HERE");
VssConnection connection = new VssConnection(new Uri("https://yourserverurl.visualstudio.com/"), credintials);
GitHttpClient client = connection.GetClient<GitHttpClient>();
List<GitRepository> repositories = await client.GetRepositoriesAsync(true); // or use GetRepositoryAsync()
var repo = repositories.FirstOrDefault(r => r.Name == "Some.Repo.Name");
GitVersionDescriptor descriptor = new GitVersionDescriptor()
{
VersionType = GitVersionType.Branch,
Version = "develop",
VersionOptions = GitVersionOptions.None
};
List<GitItem> items = await client.GetItemsAsync(repo.Id, scopePath: "/", recursionLevel: VersionControlRecursionType.Full, versionDescriptor: descriptor);
Under the hood it's using the REST API. So if you try the same effect using c# lang, better delegate it to lib.
You need to call the items endpoint first, which gives you an objectId (the gitObjectType should be "tree"):
http://{tfsURL}/tfs/{collectionId}/{teamProjectId}/_apis/git/repositories/{repositoryId}/items?recursionLevel=Full&api-version=4.1
Then call the trees end point to list the objects in the tree:
http://{tfsURL}/tfs/{collectionId}/{teamProjectId}/_apis/git/repositories/{repositoryId}/trees/{objectId}?api-version=4.1
test
I integrated my rails application with MongoDB using mongo gem. I would like to know how I can store metadata about each collection.
def initialize(db_params)
db = Mongo::Connection.new(connection_uri[:host], connection_uri[:port])
#collection = db.db(db_params[:key]).collection(db_params[:collection])
end
More precisely I want to add some metadata during creating new collection and then be able to read that before other database actions like find.
in this case we could go for two solutions:
using flexibility of dynamic schema create a meta doc with _id="meta", and store there all requred data
create meta collection and in each document store info about collection,, _id="collection name"
per manual:
db.createCollection(<name>, { capped: <boolean>,
autoIndexId: <boolean>,
size: <number>,
max: <number>,
storageEngine: <document>,
validator: <document>,
validationLevel: <string>,
validationAction: <string>,
indexOptionDefaults: <document> } )
Any comments welcome!
I am working on a Grails project. In this I need to do two things related Migration Scripts as follows :
I made a script for boot strap data and also having BootStrap.groovy for boot strap data. My Code of Both scripts is as follows :
BootStrape.groovy
Mine DatabaseInit.groovy Script
Now I want to implement(or move the code of) both of these scripts in my migration script. How to achieve that, please give me some hint.
I need to do some SQL operations for removing some duplicate values from database as follow
UPDATE DatabaseName.TableName1 SET ColumName = 2 WHERE ColumName IN (
SELECT ColumName FROM DatabaseName.TableName2 st WHERE st.ColumName = 'XYZ');
DELETE FROM DatabaseName.TableName2 WHERE ColumName NOT IN (2);
How to write migration script for achieving this task.
It is a advisable idea that you keep your bootstrap file tiny ...no too much code inside and that good approach to follow , yours.
So , what you need is try to create a new migration scrip file even if you didn't have a change in your DOMAIN . Since ,this will help you add your sql script and then later will not execute them at every start up if they are once successfully created.
TODO on grails command line :
grails dbm-create-changelog
grails prod dbm-generate-gorm-changelog --add changelog-1.0.whatevername.groovy
Then ,
add the sql script inside changelog-1.0.whatevername.groovy file as follow :
1.Using sql file like
//sqlFile( path: "${ path to SQL file relative to changelog.groovy }")
//Used in a changeset, it looks like this:
changeSet(author: "username", id: "1305821637932-1") { //id:is to be changed
sqlFile( path: "books.sql")
}
2.Using by writing the scrip inside
changeSet(author: "username", id: "1305821637932-2") { //id : is to be changed
addColumn(tableName: "book") {
column(name: "flammable", type: "BIT") {
constraints(nullable: "false")
}
}
SQL("UPDATE DatabaseName.TableName1 SET ColumName = 2 WHERE ColumName IN (
SELECT ColumName FROM DatabaseName.TableName2 st WHERE st.ColumName = 'XYZ'))";
SQL("DELETE FROM DatabaseName.TableName2 WHERE ColumName NOT IN (2)");
}
Happy coding!! please refer this and this
Are variables within YAML files possible? For example:
theme:
name: default
css_path: compiled/themes/$theme.name
layout_path: themes/$theme.name
In this example, how can theme: name: default be used in other settings? What is the syntax?
I had this same question, and after a lot of research, it looks like it's not possible.
The answer from cgat is on the right track, but you can't actually concatenate references like that.
Here are things you can do with "variables" in YAML (which are officially called "node anchors" when you set them and "references" when you use them later):
Define a value and use an exact copy of it later:
default: &default_title This Post Has No Title
title: *default_title
{ or }
example_post: &example
title: My mom likes roosters
body: Seriously, she does. And I don't know when it started.
date: 8/18/2012
first_post: *example
second_post:
title: whatever, etc.
For more info, see this section of the wiki page about YAML: http://en.wikipedia.org/wiki/YAML#References
Define an object and use it with modifications later:
default: &DEFAULT
URL: stooges.com
throw_pies?: true
stooges: &stooge_list
larry: first_stooge
moe: second_stooge
curly: third_stooge
development:
<<: *DEFAULT
URL: stooges.local
stooges:
shemp: fourth_stooge
test:
<<: *DEFAULT
URL: test.stooges.qa
stooges:
<<: *stooge_list
shemp: fourth_stooge
This is taken directly from a great demo here: https://gist.github.com/bowsersenior/979804
After some search, I've found a cleaner solution wich use the % operator.
In your YAML file :
key : 'This is the foobar var : %{foobar}'
In your ruby code :
require 'yaml'
file = YAML.load_file('your_file.yml')
foobar = 'Hello World !'
content = file['key']
modified_content = content % { :foobar => foobar }
puts modified_content
And the output is :
This is the foobar var : Hello World !
As #jschorr said in the comment, you can also add multiple variable to the value in the Yaml file :
Yaml :
key : 'The foo var is %{foo} and the bar var is %{bar} !'
Ruby :
# ...
foo = 'FOO'
bar = 'BAR'
# ...
modified_content = content % { :foo => foo, :bar => bar }
Output :
The foo var is FOO and the bar var is BAR !
This is an old post, but I had a similar need and this is the solution I came up with. It is a bit of a hack, but it works and could be refined.
require 'erb'
require 'yaml'
doc = <<-EOF
theme:
name: default
css_path: compiled/themes/<%= data['theme']['name'] %>
layout_path: themes/<%= data['theme']['name'] %>
image_path: <%= data['theme']['css_path'] %>/images
recursive_path: <%= data['theme']['image_path'] %>/plus/one/more
EOF
data = YAML::load("---" + doc)
template = ERB.new(data.to_yaml);
str = template.result(binding)
while /<%=.*%>/.match(str) != nil
str = ERB.new(str).result(binding)
end
puts str
A big downside is that it builds into the yaml document a variable name (in this case, "data") that may or may not exist. Perhaps a better solution would be to use $ and then substitute it with the variable name in Ruby prior to ERB. Also, just tested using hashes2ostruct which allows data.theme.name type notation which is much easier on the eyes. All that is required is to wrap the YAML::load with this
data = hashes2ostruct(YAML::load("---" + doc))
Then your YAML document can look like this
doc = <<-EOF
theme:
name: default
css_path: compiled/themes/<%= data.theme.name %>
layout_path: themes/<%= data.theme.name %>
image_path: <%= data.theme.css_path %>/images
recursive_path: <%= data.theme.image_path %>/plus/one/more
EOF
This is how I was able to configure yaml files to refer to variable.
I have values.yaml where we have root level fields which are used as template variables inside values.yaml
values.yaml
.....
databaseUserPropName: spring.datasource.username
databaseUserName: sa
.....
secrets:
type: Opaque
name: dbservice-secrets
data:
- name: "{{ .Values.databaseUserPropName }}"
value: "{{ .Values.databaseUserName }}"
.....
When referencing these values in secret.yaml, we would use tpl function using syntax {{ tpl TEMPLATE_STRING VALUES }}
secret.yaml
when using inside range i:e iteration
{{ range .Values.deployments.secrets.data }}
{{ tpl .name $ }}: "{{ tpl .value $ }}"
{{ end }}
when directly referring as variable
{{ tpl .Values.deployments.secrets.data.name . }}
{{ tpl .Values.deployments.secrets.data.value . }}
$ - this is global variable and will always point to the root context
. - this variable will point to the root context based on where it used.
if your requirement is like parsing an replacing multiple variable and then use it as a hash/or anything then you can do something like this
require 'yaml'
require 'json'
yaml = YAML.load_file("xxxx.yaml")
blueprint = yaml.to_json % { var_a: "xxxx", var_b: "xxxx"}
hash = JSON.parse(blueprint)
inside the yaml just put variables like this
"%{var_a}"
Rails / ruby frameworks are able to do some templating ... it's frequently used to load env variables ...
# fooz.yml
foo:
bar: <%= $ENV[:some_var] %>
No idea if this works for javascript frameworks as I think that YML format is superset of json and it depends on what reads the yml file for you.
If you can use the template like that or the << >> or the {{ }} styles depending on your reader, after that you just ...
In another yml file ...
# boo.yml
development:
fooz: foo
Which allows you to basically insert a variable as your reference that original file each time which is dynamically set. When reading I was also seeing you can create or open YML files as objects on the fly for several languages which allows you to create a file & chain write a series of YML files or just have them all statically pointing to the dynamically created one.