I have a file with the following lines in it:
bash$ cat blah.txt
<smsDeliveryStatus value="Provider Malfunction"/>
<smsDeliveryStatus value="Provider Malfunction" id="23434"/>
<smsDeliveryStatus value="Delivery Failure"/>
<smsDeliveryStatus value="Delivery Successful" id="2"/>
bash$
I want to extract value and id from the file for each line and where either value or id do not exist I want to print unknown. I wrote the following code which seems to fail some of the time on setting id to unknown and some of the time it fails:
bash$ cat blah.txt | sed -nr "/smsDeliveryStatus /{h; /value/ {s/.*value=\"([^\"]*)?\".*/value: \1/}; /value/! {s/.*/value: Unknown/}; p; x; /id/ {s/.*id=\"([^\"]+)\".*/id: \1/g}; /id/! {s/.*/id: Unknown/g}; p}"
This yields the following result from the above file:
value: Provider Malfunction
<smsDeliveryStatus value="Provider Malfunction"/>
value: Provider Malfunction
id: 23434
value: Delivery Failure
id: Unknown
value: Delivery Successful
id: 2
Bizarrely the first line with id missing is printed out in full and the second line with id missing sets id to unknown as expected. Can anyone shed any light on why this is happening? What is the difference between the first time /id/! is read and the second time?
A
I added multiple lines to the file like so:
bash$ cat blah.txt
<smsDeliveryStatus value="Provider Malfunction"/>
<smsDeliveryStatus value="Provider Malfunction" id="23434"/>
<smsDeliveryStatus value="Delivery Failure"/>
<smsDeliveryStatus value="Delivery Successful" id="2"/>
<smsDeliveryStatus value="Provider Malfunction"/>
<smsDeliveryStatus value="Delivery Failure"/>
<smsDeliveryStatus value="Delivery Successful" id="2"/>
<smsDeliveryStatus value="Provider Malfunction" id="23434"/>
<smsDeliveryStatus value="Delivery Failure"/>
<smsDeliveryStatus value="Provider Malfunction"/>
bash$
When I ran the code again I got the following:
bash$ cat blah.txt | sed -nr "/smsDeliveryStatus /{h; /value/ {s/.*value=\"([^\"]*)?\".*/value: \1/}; /value/! {s/.*/value: Unknown/}; p; x; /id/ {s/.*id=\"([^\"]*)\".*/id: \1/g}; /id/! {s/.*/id: Unknown/g}; p}"
value: Provider Malfunction
<smsDeliveryStatus value="Provider Malfunction"/>
value: Provider Malfunction
id: 23434
value: Delivery Failure
id: Unknown
value: Delivery Successful
id: 2
value: Provider Malfunction
<smsDeliveryStatus value="Provider Malfunction"/>
value: Delivery Failure
id: Unknown
value: Delivery Successful
id: 2
value: Provider Malfunction
id: 23434
value: Delivery Failure
id: Unknown
value: Provider Malfunction
<smsDeliveryStatus value="Provider Malfunction"/>
bash$
Which led me to see that all of the unmatched lines had the letters id in them so I solved it using \b word boundaries around the id like so:
bash$ cat blah.txt | sed -nr "/smsDeliveryStatus /{h; /value/ {s/.*value=\"([^\"]*)?\".*/value: \1/}; /value/! {s/.*/value: Unknown/}; p; x; /\bid\b/ {s/.*id=\"([^\"]*)\".*/id: \1/g}; /\bid\b/! {s/.*/id: Unknown/g}; p}"
value: Provider Malfunction
id: Unknown
value: Provider Malfunction
id: 23434
value: Delivery Failure
id: Unknown
value: Delivery Successful
id: 2
value: Provider Malfunction
id: Unknown
value: Delivery Failure
id: Unknown
value: Delivery Successful
id: 2
value: Provider Malfunction
id: 23434
value: Delivery Failure
id: Unknown
value: Provider Malfunction
id: Unknown
bash$ cat blah.txt
So in the end I solved it myself. I hope this helps someone else though.
A
Related
I have a Jenkins pipeline That has parameters defined via active choice parameter,
defining a default value is done by:
defaultValue: '',
you can put a string there or leave it empty which will give you the default result of the groovyScript.
I am trying to change the default parameter using a script so it will take the value using a groovy script.
This is the snippet of the relevant part of the pipeline:
parameters([
extendedChoice(
bindings: '',
defaultValue: '',
groovyClasspath: '',
groovyScript:"""
def proc = ["bash","-c","/usr/local/bin/aws s3 ls s3://Spark-Jenkins-Clusters/"].execute() | ["bash","-c","cut -c32-"].execute()
proc.waitForOrKill(10000)
return proc.text.tokenize()
""",
multiSelectDelimiter: ',',
name: 'Choose_Cluster',
description: 'This parameter is nice',
quoteValue: false,
saveJSONParameterToFile: false,
type: 'PT_SINGLE_SELECT',
visibleItemCount: 5
),
So The way to do that is to use "defaultGroovyScript",
I didn't find it in the documentation I just saw an option in the UI and tried it and luckily it worked:
This is what I finally did:
parameters([
extendedChoice(
bindings: '',
defaultGroovyScript: """
def proc = ["bash","-c","/usr/local/bin/aws s3 ls s3://Spark-Jenkins-Clusters/"].execute() | \
["bash","-c","sort"].execute() | \
["bash","-c","sed 's/PRE//g'"].execute() | \
["bash","-c","grep main"].execute() | \
["bash","-c","tail -n 1"].execute() | \
["bash","-c","tr -d '/'"].execute()
proc.waitForOrKill(10000)
return proc.text.tokenize().reverse()
""",
groovyClasspath: '',
groovyScript:"""
def proc = ["bash","-c","/usr/local/bin/aws s3 ls s3://Spark-Jenkins-Clusters/"].execute() | ["bash","-c","cut -c32-"].execute()
proc.waitForOrKill(10000)
return proc.text.tokenize()
""",
multiSelectDelimiter: ',',
name: 'Choose_Cluster',
description: 'This parameter is nice',
quoteValue: false,
saveJSONParameterToFile: false,
type: 'PT_SINGLE_SELECT',
visibleItemCount: 5
),
Consider the following example
node {
stage('Build') {
echo "do buildy things"
}
stage('Deploy') {
hipchatSend(
color: "PURPLE",
message: "Holding for deployment authorization: ${env.JOB_NAME}, job ${env.BUILD_NUMBER}. Authorize or cancel at ${env.BUILD_URL}",
)
input('Push to prod?') //Block here until okayed.
echo "Deployment authorized by ${some.hypothetical.env.var}"
echo "do deploy things"
}
}
When responding to the input, the user name that clicked the button is stored in the build log.
Is this username made available in a variable that I could use in, say, another hipChatSend?
Supply the field submitterParameter to input:
def userName = input message: '', submitterParameter: 'USER'
echo "Accepted by ${userName}"
The value of submitterParameter doesn't matter if you doesn't have any parameters. But if you have parameters, then it will specify the name of the array element which holds the value:
def ret = input message: '', parameters: [string(defaultValue: '', description: '', name: 'para1')], submitterParameter: 'USER'
echo "Accepted by ${ret['USER']}"
I am using net/ssh gem in Ruby.
By the following code I can enter into the server from my local machine. But I want to execute the commands on the server by entering as a ROOT.
Normally, I enter into the server as a ROOT by the command
sudo su -
Following is my code.
require 'rubygems'
require 'net/ssh'
list_of_servers = "servers.csv"
username="XYZ"
IO.readlines(list_of_servers).each do |line|
line.chomp!
server = line.split(',')[0]
password = line.split(',')[1]
puts "---- " + server
Net::SSH.start(server, username, :password => password,:verbose => Logger::DEBUG) do |ssh|
result=ssh.exec!("sudo su -")
puts result
end
end
Output I get after entering into server.
D, [2015-11-21T21:48:26.005576 #56654] DEBUG -- io[3fce29cc9548]: received packet nr 15 type 97 len 12
I, [2015-11-21T21:48:26.005628 #56654] INFO -- net.ssh.connection.session[3fce29ce9834]: channel_close: 0
D, [2015-11-21T21:48:26.005796 #56654] DEBUG -- io[3fce29cc9548]: queueing packet nr 11 type 97 len 28
sudo: no tty present and no askpass program specified
Note:
I can successfully log into the server from my local machine but I cannot run the ROOT command (sudo su -).
You have to start tty for that
scenario = []
scenario << ""
scenario << "su"
#session = Net::SSH.start(ip , user,
:password => pass,
:auth_methods => ['password'],
:timeout => 10
)
#session.open_channel do |channel|
channel.request_pty do |ch, success|
ch.send_channel_request("shell") do |shell, success|
shell.on_data do |c, data|
#output << data
end
scenario.each do |s|
cmd = "#{s}\r\n"
shell.send_data(cmd)
end
end
end
end
According to man sudo, if you specify -S before the command, sudo takes the password from stdin.
I tried this both on a terminal and from ruby with success:
(insert your password, and the command you wish to run in the generic example below.)
echo "password" | sudo -S "command"
works for me.
I have got this haml/mustache template:
{{#data}}
ok
{{#items}}
{{#item}}
%b ID: {{id}}
{{/item}}
{{/items}}
{{/data}}
And I have got Illegal nesting: nesting within plain text is illegal Error.
I render it in Sinatra
Mustache.render(haml(:index), hash)
I'm not sure about rendering with Sinatra, but with this command:
cat example.yml foo.haml.mustache | mustache | haml -e
this data file example.yml
---
data:
- items:
- item:
- id: 1
- id: 2
- id: 3
---
and template (foo.haml.mustache ):
{{#data}}
#ok
{{#items}}
{{#item}}
%b ID: {{id}}
{{/item}}
{{/items}}
{{/data}}
I get following result:
<div id='ok'>
<b>ID: 1</b>
<b>ID: 2</b>
<b>ID: 3</b>
</div>
Pls pay attention to indentation level in *.mustache file. Hope this help you.
Every time I try to post to twitter in my rails app, the first post
fails. I will explain the issue further after I show a quick example.
Here is a live example from the console (with sensitive stuff edited):
>> u = User.find(7)
=> #<User id: 7, login: "corgan1003#gmail.com", email:
"corgan1003#gmail.com", crypted_password: "stuff", salt: "stuff",
twitter_token: "some_twitter_token", twitter_secret:
"some_twitter_secret", twitter_sn: "some_twitter_sn">
>> u.twitter_client
=> #<Twitter::Base:0x7f11672957a0 #client=#<Twitter::OAuth:
0x7f1167295728 #csecret="some_c_secret", #ctoken="some_c_token",
#asecret="some_a_secret", #consumer_options={},
#atoken="some_a_token">>
>> u.twitter_client.update("Hello World")
=> false
>> u.twitter_client
=> #<Twitter::Base:0x7f11672957a0 #client=#<Twitter::OAuth:
0x7f1167295728 #csecret="some_c_secret", #consumer=#<OAuth::Consumer:
0x7f1167290bd8 #uri=#<URI::HTTP:0x3f88b3948330 URL:http://
twitter.com>, #options={:access_token_path=>"/oauth/
access_token", :site=>"http://
twitter.com", :oauth_version=>"1.0", :scheme=>:header, :request_token_path=>"/
oauth/request_token", :signature_method=>"HMAC-
SHA1", :http_method=>:post, :authorize_path=>"/oauth/authorize"},
#http=#<Net::HTTP twitter.com:80 open=false>, #secret="secret",
#key="key">, #access_token=#<OAuth::AccessToken:0x7f1167290c78
#consumer=#<OAuth::Consumer:0x7f1167290bd8 #uri=#<URI::HTTP:
0x3f88b3948330 URL:http://twitter.com>, #options=
{:access_token_path=>"/oauth/access_token", :site=>"http://
twitter.com", :oauth_version=>"1.0", :scheme=>:header, :request_token_path=>"/
oauth/request_token", :signature_method=>"HMAC-
SHA1", :http_method=>:post, :authorize_path=>"/oauth/authorize"},
#http=#<Net::HTTP twitter.com:80 open=false>, #secret="secret",
#key="key">, #token="token", #response=#<Net::HTTPRequestTimeOut 408
Request Timeout readbody=true>, #secret="secret">, #ctoken="ctoken",
#asecret="asecret", #consumer_options={}, #atoken="atoken">>
>> u.twitter_client.update("Hello World")
=> <Mash created_at="Thu Sep 03 01:38:21 +0000 2009" favorited=false
id=3724099662 in_reply_to_screen_name=nil in_reply_to_status_id=nil
in_reply_to_user_id=nil source="<a href=\"http://www.url.com\" rel=
\"nofollow\">MyApp</a>" text="Hello World" truncated=false user=<Mash
created_at="Tue Aug 11 19:47:26 +0000 2009" description=nil
favourites_count=0 followers_count=4 following=false friends_count=21
id=64803633 location=nil name="some name" notifications=false
profile_background_color="9ae4e8" profile_background_image_url="http://
s.twimg.com/a/1251923748/images/themes/theme1/bg.gif"
profile_background_tile=false profile_image_url="http://s.twimg.com/a/
1251923748/images/default_profile_normal.png"
profile_link_color="0000ff" profile_sidebar_border_color="87bc44"
profile_sidebar_fill_color="e0ff92" profile_text_color="000000"
protected=false screen_name="screen_name" statuses_count=4
time_zone=nil url=nil utc_offset=nil verified=false>>
>> u.twitter_client
=> #<Twitter::Base:0x7f11672957a0 #client=#<Twitter::OAuth:
0x7f1167295728 #csecret="csecret", #consumer=#<OAuth::Consumer:
0x7f1167290bd8 #uri=#<URI::HTTP:0x3f88b3948330 URL:http://
twitter.com>, #options={:access_token_path=>"/oauth/
access_token", :site=>"http://
twitter.com", :oauth_version=>"1.0", :scheme=>:header, :request_token_path=>"/
oauth/request_token", :signature_method=>"HMAC-
SHA1", :http_method=>:post, :authorize_path=>"/oauth/authorize"},
#http=#<Net::HTTP twitter.com:80 open=false>, #secret="secret",
#key="key">, #access_token=#<OAuth::AccessToken:0x7f1167290c78
#consumer=#<OAuth::Consumer:0x7f1167290bd8 #uri=#<URI::HTTP:
0x3f88b3948330 URL:http://twitter.com>, #options=
{:access_token_path=>"/oauth/access_token", :site=>"http://
twitter.com", :oauth_version=>"1.0", :scheme=>:header, :request_token_path=>"/
oauth/request_token", :signature_method=>"HMAC-
SHA1", :http_method=>:post, :authorize_path=>"/oauth/authorize"},
#http=#<Net::HTTP twitter.com:80 open=false>, #secret="secret",
#key="key">, #token="token", #response=#<Net::HTTPOK 200 OK
readbody=true>, #secret="secret">, #ctoken="ctoken",
#asecret="asecret", #consumer_options={}, #atoken="atoken">>
It's obvious that doing u.twitter_client the first time is not
setting the client properly as #consumer is not set and the first time
I do u.twitter_client.update("Hello World"), if i look at the client
after that I see the 408 request time out issue which is really
strange to me. Does anyone know how I can fix this? I am including
my functions below that the above example used.
def twitter_oauth
#oauth ||= Twitter::OAuth.new(ConsumerConfig['twitter_token'],
ConsumerConfig['twitter_secret'])
end
def twitter_client
#client ||= begin
twitter_oauth.authorize_from_access(twitter_token,
twitter_secret)
Twitter::Base.new(twitter_oauth)
end
end
Update
I should also note that this issue only occurs in my production environment. This process would tweet successfully in my development environment.
Thanks!
Do you need to change the credentials for your production environment? When you setup your application on the Twitter site now you specify what URL you want it to return to and it skips whatever URL you pass in programatically as a return (security feature).