Unable to use aws cli from Jenkins Groovy - jenkins

Question

I am trying to write Jenkins post-initialisation scripts in Groovy that use the AWS CLI. My Jenkins lives behind a corporate proxy, and I configured it as myproxy port 3128 with a username and password, and a no_proxy of "10.*.*.*,ap-southeast-2.compute.internal,localhost,127.0.0.1,myothernoproxydomains.com".
The Groovy code I am trying is as follows:
def sg = "curl http://169.254.169.254/latest/meta-data/security-groups".execute().text
"aws ec2 describe-security-groups \
--region ap-southeast-2 \
--filters Name=group-name,Values=${sg} \
--query SecurityGroups[0].GroupId \
--output text".execute().text
If I comment out the second command, and run it in the Jenkins Script console, it runs fine and I can print the security group name. But if I allow the second command to run, I eventually get a message from my Chrome browser,
This page isn't working", myjenkins.mydomain.com took too long to respond. HTTP ERROR 504.
The Jenkins has no trouble using the HTTP proxy in other contexts, e.g. downloading packages, plugins etc.
I note that environment variables relating to the HTTP proxy do not appear in System.genenv:
System.getenv()
Result: {PATH=/sbin:/usr/sbin:/bin:/usr/bin, SHELL=/bin/bash, LOGNAME=jenkins, PWD=/, USER=jenkins, LANG=en_US.UTF-8, SHLVL=2, HOME=/var/lib/jenkins, _=/etc/alternatives/java}
I have seen Groovy code that calls the AWS CLI work on other Jenkinses at other sites. I think it might be somehow proxy-related?
Am I doing anything wrong? Any ideas on what the issue could be?

Answer 1

I think the issue is that the call to 169.254.169.254 is not passing through the proxy, so the CLI isn't timing out it is the AWS call to the meta data store. Add that into your NO_PROXY value and hopefully that should resolve the issue.
The other option is to turn off the proxy, they are evil :)

Answer 2

If you call it with Error Handling it won't cause a 504 Gateway Timed Out
def sg = "curl http://169.254.169.254/latest/meta-data/security-groups".execute().text
def sout = new StringBuilder(), serr = new StringBuilder()
def proc = 'aws ec2 describe-security-groups --region ap-southeast-2 --filters Name=group-name,Values=${sg} --query SecurityGroups[0].GroupId --output text'.execute()
proc.consumeProcessOutput(sout, serr)
proc.waitForOrKill(1000)
println "out> $sout\nerr> $serr"
This still doesn't work and returns empty. I can't get a simple aws ssm get-parameter to work so a proxy issue seems to be the culprit. I'll update if I get it working.