I have set my environment variables in 'Cypress.env.json' file.
While running the cypress test, it reads the Cypress.env variables successfully.
But to be more secure, rather than 'hard-cording' the values, my team asked me to keep these variables as separate 'parameters' which are read from Windows 10 Environment variables.
How do I achieve this ?
I need to set an environment variable in Windows level.
Take a look at this answer How to use process.env variables in browser running by Cypress - essentially the dotenv package will read any variables you have set on the Windows environment. Also works for other OS like Linux.
The answer given is a bit naive for what you want to do, as there will be a lot of variables you do not need in the test.
This modification would be better
cypress.config.js
const {defineConfig} = require('cypress')
require('dotenv').config()
const { secret1, secret2 } = process.env; // extract two secret variables
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
config.env = {
...config.env,
secret1,
secret2,
}
return config
}
}
})
Note the process.env values should come after the config.env values, so you are over-writing any defaults.
Those defaults would be in cypress.env.json, and stops the tests crashing if someone forgets to set their Windows environment values.
{
"host": "veronica.dev.local",
"api_server": "http://localhost:8888/api/v1/",
"secret1": "default-secret-value1",
"secret2": "default-secret-value2",
}
You can set operating system environment variable that's name starts with CYPRESS_ and it will be parsed and used in Cypress.
Example:
CYPRESS_HOST
will be available as
Cypress.env('HOST')
Related
I am learning AWS Cloud Development Kit (CDK).
As part of this learning, I am trying to understand how I am supposed to correctly handle production and development environment.
I know AWS CDK provides the environment parameter to allow deploying stacks to specific account.
But then, how to have specific options for development versus production stacks ? It does not seem to be provided by default by AWS CDK or am I missing/misunderstanding something ?
A very simple example could be that I want a S3 bucket called my-s3-bucket-dev for my development account and one named my-s3-bucket-prod for my production account. But then how to have e.g. a variable stage correctly handled in AWS CDK ?
I know I can add parameters in the cdk.json file but again, I don't know how to correctly use this file to depend upon the deployed stack i.e. production vs development.
Thanks for the support
Welcome to AWS CDK.
Enjoy the ride. ;)
Actually, there is no semantic (in your case the stage) in an account itself.
This has nothing to do with CDK or Cloud Formation.
You need to take care of this.
You're right, that you could use the CDK context in the cdk.json.
There's no schema enforcement in the context, except for some internally used variables by CDK.
You could define your dev and prod objects within.
There are other ways of defining the context.
Here is an example, what it could look like:
{
"app": "node app",
// usually there's some internal definition for your CDK project
"context": {
"dev": {
"accountId" : "prod_account",
"accountRegion" : "us-east-1",
"name": "dev",
"resourceConfig":
{
// here you could differentiate the config per AWS resource-type
// e.g. dev has lower hardware specs
}
},
"prod": {
"accountId" : "prod_account",
"accountRegion" : "us-east-1",
"name": "prod",
"resourceConfig":
{
// here you could differentiate the config per AWS resource-type
// prod has higher hardware specs or more cluster nodes
}
}
}
}
With this being defined, you need to run your CDK application with the -c flag to specify which configuration object (dev or prod), you want to have.
For instance, you could run it with cdk synth -c stage=prod.
This sets the stage variable in your context and makes it available.
When it was successful, you can re-access the context again and fetch the appropriate config object.
const app = new cdk.App();
const stage = app.node.tryGetContext('stage');
// the following step is only needed, if you have a different config per account
const stageConfig = app.node.tryGetContext(stage );
// ... do some validation and pass the config to the stacks as constructor argument
As I said, the context is one way of doing this.
However, there are drawbacks to it.
It's JSON and no code.
What I prefer is to have TypeScript types per resource configuration (e.g. S3) and wire them all together as a plain object.
The object maps the account/region information and the corresponding resource configurations.
I have my jenkins pipelines working and all in source code management, within my pipelines I have some constants which are variables that do not change/rarely change, so these would rarely change and the pipelines requires these values, 90% of it doesnt change but I have some that does change based on the environment type (production/pre-production/test etc)
The problem I have right now is that I would like to take thesame code from non-production to production without having to change things like the file server details, as production/non-production use different file servers, as it stands one has to remember to change the file server when promoting code to production, is it possible to have like a configuration file and the pipeline can read the values from the configuration file, I do not want to change the pipelines or make as little changes as possible when my code moves from non-production to production.
Thanks in advance.
Install the Pipeline Utility Steps plugin and create a properties file each in the repositories/branches for production and non-production respectively. Then read and populate the properties from the file in your pipeline during the build.
Sample server.properties
fileServerUrl=ftp://prod.company.net
fileServerPort=21
// more properties here
Sample Jenkinsfile
pipeline {
agent {
label 'production'
}
stages {
stage('prepare-env') {
script {
def props = readProperties file: 'server.properties'
env.fileServerUrl = props.fileServerUrl
env.fileServerPort = props.fileServerPort
// more properties here
}
}
stage('deploy') {
println("INFO: The file server is ${env.fileServerUrl}:${env.fileServerPort}")
// do stuff here
}
}
}
I've accessed environmental varibales with node apps before with process.env.VARIABLE_NAME, but I was curious to try Sails' alternative solution. It seems like I should be able to put in a dummy value (or nothing) in the /config/foo.js file, then overwrite it with a carefully named environmental variable. I modeled my setup on this example.
Unfortunately, CircleCI seems to be ignoring the environmental variable and using the dummy value instead. Have I set something up incorrectly? FYI, I'm using /config/local.js (no environment variables) to overwrite the password on my local machine and everything works fine...
/config/datastores.js:
module.exports.datastores = {
postgresqlTestDb: {
adapter: 'sails-postgresql',
host: 'test-postgres.myhost.com',
user: 'postgres',
password: 'PASSWORD',
database: 'my-db',
},
};
Environment Variables in CircleCI:
sails_datastores__postgresqlTestDb__password = theRealPassword
Error in CircleCI:
1) "before all" hook:
Error: done() invoked with non-Error: {"error":{"name":"error","length":104,"severity":"FATAL","code":"28P01","file":"auth.c","line":"307","routine":"auth_failed"},"meta":{"adapter":"sails-postgresql","host":"test-postgres.myhost.com","user":"postgres","password":"PASSWORD","database":"","identity":"postgresqlTestDb","url":"postgres://postgres:PASSWORD#test-postgres.myhost.com:5432/my-db"}}
at sails.lift (test/lifecycle.test.js:46:23)
...
The Important part of the error:
"url":"postgres://postgres:PASSWORD#test-postgres.myhost.com:5432/my-db"
I want to connect to postgres://postgres:theRealPassword#test-postgres.myhost.com:5432/my-db instead...
I just set an ENV variable for the entire connection URL. looks something like this:
sails_datastores__default__url: postgresql://user:passwrod#host:port/databse
I think in your example you are missing the "default" part
In Jenkins configuration (http://JenkinsURL/configure) within "Global properties" I defined some "Environment variables".
How can I access them in the Groovy Script console (http://JenkinsURL/script)?
I've tried to find appropriate solution (for example the solutions mentioned in: Access to build environment variables from a groovy script in a Jenkins build step (Windows))
but it seems that none of them work for me.
I've tried for example:
System.getenv("myVar")
and
manager.build.getEnvironment(listener).get('myVar') //no manager error
and
import jenkins.model.Jenkins
Jenkins.instance.getProperty('myVar') //No signature of method: hudson.model.Hudson.getProperty() is applicable for argument types: (java.lang.String)
and
import jenkins.model.Jenkins
Jenkins.instance.ParameterValue("DEV_local")
You can get global properties like this:
import jenkins.model.Jenkins
def envVars = Jenkins.instance.getGlobalNodeProperties()[0].getEnvVars()
println envVars['myVar']
I referred to the link below, about how to set global properties programatically.
https://groups.google.com/forum/#!topic/jenkinsci-users/KgCGuDmED1Q
It's not as simple as you'd think, like everything in Jenkins. It doesn't appear to expose a simple API to get the final effective environment for the current execution context, at least not to the script console.
The final recipe
Here's a wrapped up version you can use directly, or you can adapt a little to bundle into a vars/ class in your pipeline global library.
import jenkins.model.Jenkins
import hudson.model.Node
import hudson.slaves.EnvironmentVariablesNodeProperty
import hudson.EnvVars
EnvVars getCombinedNodeEnvironment(Node node) {
/*
* Start with env-vars defined by the shell the JVM
* was started from and env-vars set as system properties.
*/
def combined = new EnvVars(node.toComputer().getEnvironment())
/*
* Apply environment variables from jenkins global settings
* ("Manage Jenkins" -> "Configure System" -> "Global Properties"
* -> "Environment Variables")
*/
combined.overrideExpandingAll(Jenkins.instance.
getGlobalNodeProperties().
get(EnvironmentVariablesNodeProperty).
getEnvVars() ?: new EnvVars())
/*
* Apply environment variables from node specific settings
* ("Manage Jenkins" -> "Manage Nodes and Clouds"
* -> {nodename} -> "Configure" -> "Node Properties"
* -> "Environment Variables")
*/
combined.overrideExpandingAll((node.
getNodeProperty(EnvironmentVariablesNodeProperty)?.
getEnvVars()) ?: new EnvVars())
/*
* Important: This environment map will NOT contain job-level,
* or run-level properties, nor anything set via build steps etc.
*/
return combined
}
EnvVars getCombinedNodeEnvironment(String nodename) {
if (nodename == 'master' || !nodename)
return getCombinedNodeEnvironment(Jenkins.instance)
else
return getCombinedNodeEnvironment(Jenkins.instance.getNode(nodename))
}
Usage:
getCombinedNodeEnvironment('somenode').expand('$JENKINS_HOME/$USER/$SOME_NODE_VARIABLE')
getCombinedNodeEnvironment('').SOME_ENV_VAR_ON_MASTER
Relevant classes:
hudson.model.Node
jenkins.model.Jenkins
hudson.EnvVars
hudson.slaves.EnvironmentVariablesNodeProperty
hudson.util.DescribableList
Problems with existing answer
The answer by arasio is a good start, but it's incorrect to assume the envvars properties will be at index 0 of the global properties. That approach also ignores environment variables set locally on specific nodes.
At minimum it should read
jenkins.instance.Jenkins.instance.
getGlobalNodeProperties().
get(hudson.slaves.EnvironmentVariablesNodeProperty).
getEnvVars()
i.e find the property by class within the DescribableList result, instead of assuming index.
However, that only gets you env vars from the "Environment Variables" list in the global jenkins configuration - it won't show system environment variables, nor will it show node-specific environment variables.
Read on.
Keep it simple if possible
If you're using the Groovy Pipeline, most of the time you can just use the env "variable" (see "Global Variable Reference" in pipeline help), which exposes the unified environment as properties. As noted above this won't work directly from the script console, but the rest of the time it's the appropriate way to do things.
You can also use env.getEnvironment() in Pipeline scripts to get a unified EnvVars instance, which be used for placeholder substitution of env-vars in strings e.g. env.getEnvironment().expand('${FOO} $BAR'). (You'll need script security permission for this, but it's better to put it in a helper in your global library's vars/ instead).
Most of the time this is sufficient.
I only landed up diving into the details of the environment structure because I needed to expand strings containing environment variables as they would be expanded on a different node. That's not a common use case.
Explanation of how it works and setup for examples
That's the final recipe, but how did we get there, where do the different sets of environment variables come from, and why?
For the following code examples assume this common prelude, mostly to save on repetition in each example.
/* Jenkins uses '' for the master node */
nodenames = ['', 'some-other-node-name']
/* Imports used in various examples */
import jenkins.model.Jenkins
import hudson.slaves.EnvironmentVariablesNodeProperty
import hudson.EnvVars
nodes = nodenames.collect { nodename ->
(!nodename || nodename == 'master') ?
Jenkins.instance : Jenkins.instance.getNode(nodename)
import static groovy.json.JsonOutput.toJson
import static groovy.json.JsonOutput.prettyPrint
def eachNode(Closure c) {
nodes.collectEntries { node -> [node.nodeName, c(node, node.nodeName) ] }
def fmtEnv(desc,m) {
print "\n\n${desc}\n----\n" + m.collect { k, v -> "${k?:'master'}:\n\t${trimKeys(v)}" }.join('\n')
}
def trimKeys(l) {
if (l == null)
return l
if (l in Map)
l = l.keySet()
l = l - ['_', 'OLDPWD', 'PWD', 'SSH_CLIENT', 'JAVA_HOME', 'LANG', 'LOGNAME', 'MAIL', 'MANPATH', 'S_COLORS', 'SHLVL', 'XDG_RUNTIME_DIR', 'XDG_SESSION_ID']
l.sort()
}
nodes now contains the jenkins.model.Jenkins master and a hudson.model.Node worker node.
eachNode produces a Map of node name to abbreviated list of environment variable keys, just to make the examples briefer and easier to read. Don't use it in your code.
To help clarify the results of these examples, I have configured NODE1_SPECIFIC_ENVVAR in the node setup for node1 under "Manage Jenkins" -> "Manage Nodes and Clouds" -> [nodename] -> Configure -> Environment Variables.
On the master node entry in the same place, I have configured MASTER_SPECIFIC_ENVVAR
In "Manage Jenkins" -> "Configure System" -> "Global Properties" -> "Environment Variables" I added "ALL_NODES_ENVVAR".
I didn't bother setting custom env-vars at the JVM level for the node and master.
Different views of the environment
Now, lets explore the environment different ways.
JVM-level environment variables (master)
On master, System.getenv() shows only env vars set when the JVM started or as system properties:
fmtEnv('System.getenv()', ['': System.getenv()])
/*
master:
[HOME, JENKINS_HOME, PATH, SHELL, USER]
*/
so nothing configured per-node, globally in jenkins itself, or per-job.
Base environment of nodes
Jenkins exposes the base environment variables set on each node in its API. I think this is the same as System.getEnv() would return when executed on the target node JVM:
fmtEnv('toComputer.getEnvironment()', eachNode() {
node, name -> node.toComputer().getEnvironment()
})
/*
master:
[HOME, JENKINS_HOME, PATH, SHELL, USER]
ci-node-qa-fsn1-01:
[HOME, PATH, SHELL, SSH_CONNECTION, USER]
*/
Note the absence of global or node-specific env-vars set in Jenkins.
Globally configured env vars
fmtEnv('master getGlobalNodeProperties', ['':
Jenkins.instance.
getGlobalNodeProperties().
get(EnvironmentVariablesNodeProperty).
getEnvVars()
])
/*
master getGlobalNodeProperties
----
master:
[ALL_NODES_ENVVAR]
*/
So here we see only the globally configured environment properties, not node-specific properties, system properties, or host environment vars.
Node specific environment variable overrides
fmtEnv('node getNodeProperty', eachNode() {
node, name -> node.getNodeProperty(EnvironmentVariablesNodeProperty)?.getEnvVars()
})
/*
master:
[MASTER_SPECIFIC_ENVVAR]
ci-node-qa-fsn1-01:
[NODE1_SPECIFIC_ENVVAR]
*/
Here we see the properties configured under each node in "manage nodes", but not host env-vars, vars from system properties, standard jenkins job vars, or vars configured in jenkins global config.
Important: getNodeProperty(EnvironmentVariablesNodeProperty) returns null if there are no custom environment variables configured on a node, so you must handle that.
Putting it together
The above shows how to get EnvVars instances for the main sources of environment variables that are meaningful on the script console.
There are other sources when running a job which I don't consider here, such as job properties (EnvInject plugin), automatic env-vars added to all jobs, withEnvironment steps, vars injected by the SCM plugin(s), etc. But they don't make sense for script console tasks.
So how do we get a unified environment?
First, collect up the EnvVars for each relevant piece of the environment:
def node_base_env = node.toComputer().getEnvironment()
def global_env_properties = Jenkins.instance.
getGlobalNodeProperties().
get(EnvironmentVariablesNodeProperty).
getEnvVars()
def node_env_properties = node.getNodeProperty(EnvironmentVariablesNodeProperty)?.getEnvVars() ?: new EnvVars()
def merged = new EnvVars(node_base_env)
merged.overrideExpandingAll(global_env_properties)
merged.overrideExpandingAll(node_env_properties)
merged
/*
master:
[ALL_NODES_ENVVAR, HOME, JENKINS_HOME, MASTER_SPECIFIC_ENVVAR, PATH, SHELL, USER]
ci-node-qa-fsn1-01:
[ALL_NODES_ENVVAR, HOME, NODE1_SPECIFIC_ENVVAR, PATH, SHELL, SSH_CONNECTION, USER]
*/
I'm pretty sure that's going to produce the right results. I have not tested the expansion handling, priority override order, or expansion order in detail.
(Note: I removed another example that used EnvironmentExpander).
You can use System to get environment variables.
def env = System.getenv()
println(env['JENKINS_HOME'])
(See also http://#myJenkHostname#/env-vars.html (where #myJenkHostname# is your Jenkins hostname) for a list of built-in environment variables.)
None of these answers worked for me, due to 'security' issues.
Instead just use the name of the environment variable e.g. the environment variable PATH you can use:
final path = PATH
you can also use
final path = env.PATH
Surprised it was so easy but it is...
I would like to load Environment specific configurations in my grails application so that depending on which JVM the grails application is running on, I can point to that environment specific urls. In my case, I have 4 different environments to work with (instead of the default 3 that grails app assumes) when my app goes from dev to prod.
My JVMs all have a System property defined that, when I do "System.getProperty()", tell me which environment that application is running on.
My question is, what is the best place to check and load the environment-specific configurations during run-time? Inside BootStrap.groovy? I do not have the option to build my war file using command line or grails {env_name} war.
Thanks.
Set the variable grailsEnv as a environment Java variable for Tomcat below is an example:
set CATALINA_OPTS=%CATALINA_OPTS% -Xms256m -Xmx1024m -Dgrails.env=development
On a grails command line you add the environment variable:
grails run-app -Dgrails.env=stage
You can use check the environment variable like this:
if (grails.util.Environment.current.name == "development") {
UsageCodeDefinition ucd = new UsageCodeDefinition()
ucd.setDescription("UFARSFileUpload Upload Development")
ucd.setFiscalYear("12-13")
ucd.setInstructions("Welcome to UFARSFileUpload Development were Open")
ucd.save(failOnError: true)
}
You can use the Enumerated values instead of the name variable but if you use custom environmental values then they are mapped to the enumeration custom and using the name works to distinguish between the custom values.
if (grails.util.Environment.current == grails.util.Environment.DEVELOPMENT) {
Without setting the JVM startup parameter:
-Dgrails.env=whatever
Your grails app will use the value set in
<yourapp>/WEB-INF/classes/application.properties
There will be a value set like this:
grails.env=development
This default environment value is determined by what options are used when building the war. You can build the war with
-Dgrails.env=development war
Then the application.properties will have grails.env=development, if you leave that off, it defaults to grails.env=production
As far as your question, you are not specific about what is being configured to use "environment specific urls". And it is not clear how you are storing these environment specific urls. If, for example, the URL variable is a member variable of a Grails service and you are storing the environment specific URLs in the Config.groovy, then you could
import grails.util.Environment
...
//inject the GrailsApplication Configuration in Config.groovy
def grailsApplication
//Hold the URL value from Config.groovy
String environmentUrl
...
Environment current = Environment.getCurrent()
if(Environment.PRODUCTION == current) {
environmentUrl = grailsApplication.config.PRODUCTION_URL
} else {
environmentUrl = grailsApplication.config.DEVELOPMENT_URL
}
Where Config.groovy has
PRODUCTION_URL = "http://blah.com/blah/"
DEVELOPMENT_URL = "http://blah.dev/blah"
Hope that helps.
If you have a System property available that tells you what environment you're in you can simply add if statements or a switch statement in your Config.groovy file, like this:
if (System.getProperty("foo") == "myTestEnvironment") {
myConfigSetting = "test"
} else if (System.getProperty("foo") == "myProductionEnvironment") {
myConfigSetting = "production"
}
This solution also works in other config files under grails-app/conf
Grails config files are parsed using groovy ConfigSlurper so you can put executable code in there without a problem.
Sorry this is way late, but another way is to inject a configuration property in BootStrap.groovy.
For Example:
if (currentEnv == Environment.DEVELOPMENT) {
...
grailsApplication.config.some.property = DEVELOPMENT_ENVRIONMENT
...
}
else if (currentEnv == Environment.TEST) {
...
grailsApplication.config.some.property = TEST_ENVIRONMENT
...
}
I have used this recently and it works really well. We are using Grails 2.5.2
As an addendum to the other answers:
You can use Environment.isDevelopmentMode() or in a groovier way Environment.developmentMode to check if the environment is set to development. This is useful when you take the aproach of only modifying settings for development on your code where production settings are default.