I've accessed environmental varibales with node apps before with process.env.VARIABLE_NAME, but I was curious to try Sails' alternative solution. It seems like I should be able to put in a dummy value (or nothing) in the /config/foo.js file, then overwrite it with a carefully named environmental variable. I modeled my setup on this example.
Unfortunately, CircleCI seems to be ignoring the environmental variable and using the dummy value instead. Have I set something up incorrectly? FYI, I'm using /config/local.js (no environment variables) to overwrite the password on my local machine and everything works fine...
/config/datastores.js:
module.exports.datastores = {
postgresqlTestDb: {
adapter: 'sails-postgresql',
host: 'test-postgres.myhost.com',
user: 'postgres',
password: 'PASSWORD',
database: 'my-db',
},
};
Environment Variables in CircleCI:
sails_datastores__postgresqlTestDb__password = theRealPassword
Error in CircleCI:
1) "before all" hook:
Error: done() invoked with non-Error: {"error":{"name":"error","length":104,"severity":"FATAL","code":"28P01","file":"auth.c","line":"307","routine":"auth_failed"},"meta":{"adapter":"sails-postgresql","host":"test-postgres.myhost.com","user":"postgres","password":"PASSWORD","database":"","identity":"postgresqlTestDb","url":"postgres://postgres:PASSWORD#test-postgres.myhost.com:5432/my-db"}}
at sails.lift (test/lifecycle.test.js:46:23)
...
The Important part of the error:
"url":"postgres://postgres:PASSWORD#test-postgres.myhost.com:5432/my-db"
I want to connect to postgres://postgres:theRealPassword#test-postgres.myhost.com:5432/my-db instead...
I just set an ENV variable for the entire connection URL. looks something like this:
sails_datastores__default__url: postgresql://user:passwrod#host:port/databse
I think in your example you are missing the "default" part
Related
I have set my environment variables in 'Cypress.env.json' file.
While running the cypress test, it reads the Cypress.env variables successfully.
But to be more secure, rather than 'hard-cording' the values, my team asked me to keep these variables as separate 'parameters' which are read from Windows 10 Environment variables.
How do I achieve this ?
I need to set an environment variable in Windows level.
Take a look at this answer How to use process.env variables in browser running by Cypress - essentially the dotenv package will read any variables you have set on the Windows environment. Also works for other OS like Linux.
The answer given is a bit naive for what you want to do, as there will be a lot of variables you do not need in the test.
This modification would be better
cypress.config.js
const {defineConfig} = require('cypress')
require('dotenv').config()
const { secret1, secret2 } = process.env; // extract two secret variables
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
config.env = {
...config.env,
secret1,
secret2,
}
return config
}
}
})
Note the process.env values should come after the config.env values, so you are over-writing any defaults.
Those defaults would be in cypress.env.json, and stops the tests crashing if someone forgets to set their Windows environment values.
{
"host": "veronica.dev.local",
"api_server": "http://localhost:8888/api/v1/",
"secret1": "default-secret-value1",
"secret2": "default-secret-value2",
}
You can set operating system environment variable that's name starts with CYPRESS_ and it will be parsed and used in Cypress.
Example:
CYPRESS_HOST
will be available as
Cypress.env('HOST')
I've been following along with the testdriven.io tutorial for setting up a FastAPI with Docker. The first test I've written using PyTest errored out with the following message:
TypeError: Settings(environment='dev', testing=True, database_url=AnyUrl('postgres://postgres:postgres#web-db:5432/web_test', scheme='postgres', user='*****', password='*****', host='web-db',host_type='int_domain', port='5432', path='/web_test')) is not a callable object.
Looking at the picture, you'll notice that the Settings object has a strange form; in particular, its database_url parameter seems to be wrapping a bunch of other parameters like password, port, and path. However, as shown below my Settings class takes a different form.
From config.py:
# ...imports
class Settings(BaseSettings):
environment: str = os.getenv("ENVIRONMENT", "dev")
testing: bool = os.getenv("TESTING", 0)
database_url: AnyUrl = os.environ.get("DATABASE_URL")
#lru_cache()
def get_settings() -> BaseSettings:
log.info("Loading config settings from the environment...")
return Settings()
Then, in the conftest.py module, I've overridden the settings above with the following:
import os
import pytest
from fastapi.testclient import TestClient
from app.main import create_application
from app.config import get_settings, Settings
def get_settings_override():
return Settings(testing=1, database_url=os.environ.get("DATABASE_TEST_URL"))
#pytest.fixture(scope="module")
def test_app():
app = create_application()
app.dependency_overrides[get_settings] = get_settings_override()
with TestClient(app) as test_client:
yield test_client
As for the offending test itself, that looks like the following:
def test_ping(test_app):
response = test_app.get("/ping")
assert response.status_code == 200
assert response.json() == {"environment": "dev", "ping": "pong", "testing": True}
The container is successfully running on my localhost without issue; this leads me to believe that the issue is wholly related to how I've set up the test and its associated config. However, the structure of the error and how database_url is wrapping up all these key-value pairs from docker-compose.yml gives me the sense that my syntax error could be elsewhere.
At this juncture, I'm not sure if the issue has something to do with how I set up test_ping.py, my construction of the settings_override, with the format of my docker-compose.yml file, or something else altogether.
So far, I've tried to fix this issue by reading up on the use of dependency overrides in FastApi, noodling with my indentation in the docker-compose, changing the TestClient from one provided by starlette to that provided by FastAPI, and manually entering testing mode.
Something I noticed when attempting to manually go into testing mode was that the container doesn't want to follow suit. I've tried setting testing to 1 in docker-compose.yml, and testing: bool = True in config.Settings.
I'm new to all of the relevant tech here and bamboozled. What is causing this discrepancy with my test? Any and all insight would be greatly appreciated. If you need to see any other files, or are interested in the package structure, just let me know. Many thanks.
Any dependency override through app.dependency_overrides should provide the function being overridden as the key and the function that should be used instead. In your case you're assigning the correct override, but you're assigning the result of the function as the override, and not the override itself:
app.dependency_overrides[get_settings] = get_settings_override()
.. this should be:
app.dependency_overrides[get_settings] = get_settings_override
The error message shows that FastAPI tried to call your dictionary as a function, something that hints to it expecting a function instead.
I have an aurora file which contains this docker parameters:
jobs = [
Service(cluster = 'mesos-fr',
environment = 'devel',
role = 'root',
instances = 1,
name = 'frontend_service',
task = run_frontend_service,
container=Docker(image='frontend_service', parameters=[{'name': 'frontend_service'}, {'publish': '{{thermos.ports[http]}}:3000'}])
)
]
Got this error:
Error loading configuration: Unknown schema attribute publish
Is there a solution for connecting the host port with a docker container port?
EDIT: Mustache variable replacements might not help since they happen after the container comes up.
It looks like there's a problem with the form of your Docker Parameters. An example of correct ones is container=Docker(image='nginx', parameters=[Parameter(name='env', value='HTTP_PORT={{thermos.ports[http]}')]).
There is a Parameter object, with name and value. Value can be a string with mustache variable (like ports) so you can throw that stuff in there.
This documentation (under Announcer Objects) might help too: http://aurora.apache.org/documentation/latest/reference/configuration/
I am trying to override dataSource.url value running grails from the command line, example
grails <set property> dbm-status
My first try was using -D command line parameter as
grails -DdataSource.url=jdbc:sqlserver://xx.xx.xx.xx;databaseName=db_name
I have tried to add an additional config file to grails.config.locations that get values from System.getProperty but does not seems to work.
Exist a built in way to override config values from the command line, otherwise how I can inject parameter from the command line to the grails configuration ?
EDIT: I don't want to use another environment/datasource to avoid datasource configuration duplication and the need to configure things for this new environment.
By including the following if in DataSource.groovy I'm able to override url,password and username property if url is provided. (Valid for Grails 2.x)
....
environments {
development {
dataSource {
url = "jdbc:postgresql://localhost/db"
username = "user"
password = "pass"
if (System.properties['dataSourceUrl']) {
println 'Taking dataSource url, password, username from command line overrides'
url = System.properties['dataSourceUrl']
password = System.properties['dataSourcePassword']
username = System.properties['dataSourceUsername']
}
}
}
...
Now when I run the command, the overrides get applied:
grails dev -DdataSourceUrl=newUrl -DdataSourcePassword=newPass -DdataSourceUsername=newUser run-app
Unfortunately if you want to be able to override on every environment you have to duplicate this code for every env block. If you pull it up to root it won't work since config merging kicks in and the last run will actually apply what's in the env {} block and not what's in the System properties.
Looking at it again something like that looks even better:
...
url = System.properties['dataSourceUrl'] ?: 'jdbc:postgresql://localhost/db'
//and for every property...
...
DATASOURCE_URL=jdbc:sqlserver://xx.xx.xx.xx;databaseName=db_name grials run-app
For any variable you want to set, you can set it in the environment. Change to upper case and replace dots with underscores. This is a feature of spring boot.
I would like to load Environment specific configurations in my grails application so that depending on which JVM the grails application is running on, I can point to that environment specific urls. In my case, I have 4 different environments to work with (instead of the default 3 that grails app assumes) when my app goes from dev to prod.
My JVMs all have a System property defined that, when I do "System.getProperty()", tell me which environment that application is running on.
My question is, what is the best place to check and load the environment-specific configurations during run-time? Inside BootStrap.groovy? I do not have the option to build my war file using command line or grails {env_name} war.
Thanks.
Set the variable grailsEnv as a environment Java variable for Tomcat below is an example:
set CATALINA_OPTS=%CATALINA_OPTS% -Xms256m -Xmx1024m -Dgrails.env=development
On a grails command line you add the environment variable:
grails run-app -Dgrails.env=stage
You can use check the environment variable like this:
if (grails.util.Environment.current.name == "development") {
UsageCodeDefinition ucd = new UsageCodeDefinition()
ucd.setDescription("UFARSFileUpload Upload Development")
ucd.setFiscalYear("12-13")
ucd.setInstructions("Welcome to UFARSFileUpload Development were Open")
ucd.save(failOnError: true)
}
You can use the Enumerated values instead of the name variable but if you use custom environmental values then they are mapped to the enumeration custom and using the name works to distinguish between the custom values.
if (grails.util.Environment.current == grails.util.Environment.DEVELOPMENT) {
Without setting the JVM startup parameter:
-Dgrails.env=whatever
Your grails app will use the value set in
<yourapp>/WEB-INF/classes/application.properties
There will be a value set like this:
grails.env=development
This default environment value is determined by what options are used when building the war. You can build the war with
-Dgrails.env=development war
Then the application.properties will have grails.env=development, if you leave that off, it defaults to grails.env=production
As far as your question, you are not specific about what is being configured to use "environment specific urls". And it is not clear how you are storing these environment specific urls. If, for example, the URL variable is a member variable of a Grails service and you are storing the environment specific URLs in the Config.groovy, then you could
import grails.util.Environment
...
//inject the GrailsApplication Configuration in Config.groovy
def grailsApplication
//Hold the URL value from Config.groovy
String environmentUrl
...
Environment current = Environment.getCurrent()
if(Environment.PRODUCTION == current) {
environmentUrl = grailsApplication.config.PRODUCTION_URL
} else {
environmentUrl = grailsApplication.config.DEVELOPMENT_URL
}
Where Config.groovy has
PRODUCTION_URL = "http://blah.com/blah/"
DEVELOPMENT_URL = "http://blah.dev/blah"
Hope that helps.
If you have a System property available that tells you what environment you're in you can simply add if statements or a switch statement in your Config.groovy file, like this:
if (System.getProperty("foo") == "myTestEnvironment") {
myConfigSetting = "test"
} else if (System.getProperty("foo") == "myProductionEnvironment") {
myConfigSetting = "production"
}
This solution also works in other config files under grails-app/conf
Grails config files are parsed using groovy ConfigSlurper so you can put executable code in there without a problem.
Sorry this is way late, but another way is to inject a configuration property in BootStrap.groovy.
For Example:
if (currentEnv == Environment.DEVELOPMENT) {
...
grailsApplication.config.some.property = DEVELOPMENT_ENVRIONMENT
...
}
else if (currentEnv == Environment.TEST) {
...
grailsApplication.config.some.property = TEST_ENVIRONMENT
...
}
I have used this recently and it works really well. We are using Grails 2.5.2
As an addendum to the other answers:
You can use Environment.isDevelopmentMode() or in a groovier way Environment.developmentMode to check if the environment is set to development. This is useful when you take the aproach of only modifying settings for development on your code where production settings are default.