OpsWorks overriding database.yml / ignoring custom JSON - ruby-on-rails

When I deploy a Rails app with OpsWorks, a new database.yml gets created in the shared directory. It ignores the existing database.yml (which rightfully shouldn't be in the repo), and I've also tried specifying custom JSON but nothing works. Maybe I have the structure wrong?
{
"deploy": {
"my-app-name": {
"database": {
"adapter": "mysql2",
"encoding": "unicode",
"host": "xxxxxx.rds.amazonaws.com",
"port": "3306",
"database": "db-name",
"pool": "5",
"username": "username",
"password": "password"
}
}
}
}

Look also here:
http://wojtek.ziniewi.cz/2013/06/10/custom-symlinks-in-amazon-opsworks-ror-application/
And always remember to examine your stack-json by logging into console of one of your opsworks servers and typing:
opsworks-agent-cli get_json

Using the short name for the app worked (without hyphens). I was using the full name before. See: https://forums.aws.amazon.com/message.jspa?messageID=444711

Related

Configuring vimspector with ruby on rails app running passenger

My question is how do i get vimspector running with a ruby on rails app running passenger (I think)
I'm trying to set up vimspector with some ruby on rails projects I'm working on. Basically the problem is that vimspector doesn't stop at breakpoints in some apps. the apps are using passenger. In a very basic sample app vimspector works fine, but not in the ones where i actually need to debug.
Description:
when i call vimspector#Launch all the splits/windows come up, and a passenger application server gets launched. I don't work alone on some projects so i cannot really change this to another one.
I set some breakpoints and do actions that should stop at them but they run through like without a debugger. Also nothing except for threads running and the console output of the rails server shows up in the splits. In my processes 2 processes show up. Once the Passenger Ruby app and once a node process going to my cust_vscode-ruby directory which is needed for the debugger. The node process has the subprocess rdebug-ide and rdebug-ide has passenger start.
As vimspector works fine with the sample rails app, I don't think its a problem of the install or something like that.
I tried switching to attach but when configuring it for attach i get an endless loading without getting a timeout in the browser. the terminal window where i run rdebug-ide starts of without any repsonse, i launch vimspector and the server is being booted up. It stops at this however and never goes past booting Run rails server -h for more startup options. After which passenger normally starts up. With switching to attach mode only the rdebug controlled rails server shows up in my processes.
I think this should be possible as i'm usind ruby-debug-ide which is being used by RubyMine on which i could debug the apps running passenger just fine. I'm pretty sure that is has to do with passenger as calling it up in launch config leads to it escaping and using attach config leads to it blocking the rails server boot. But I don't know how to fix it.
.vimspector.json
same for both sample app and actual app :
{
"configurations": {
"rails": {
"adapter": "cust_vscode-ruby",
"default": true,
"configuration": {
"name": "Debug Rails server",
"type": "Ruby",
"request": "launch",
"cwd": "${workspaceRoot}",
"pathToBundler": "/Users/username/.rvm/rubies/ruby-version/bin/bundler",
"pathToRDebugIDE": "/Users/username/.rvm/gems/ruby-version/gems/ruby-debug-ide-0.7.3",
"program": "${workspaceRoot}/bin/rails",
"args": [
"server"
]
}
}
}
}
vimspector configuration output
Vimspector Debug Info
--------------------------------------------------------------------------------
ConnectionType: job
Adapter:
--------------------------------------------------------------------------------
{
"command": [
"node",
"/Users/username/.vim/plugged/vimspector/gadgets/macos/cust_vscode-ruby/dist/debugger/main.js"
],
"configuration": {
"cwd": "/Users/username/Folder/app-name",
"showDebugOutput": false,
"trace": false
},
"name": "vscode-ruby-debug",
"env": {},
"cwd": "/Users/username/Folder/app-name"
}
--------------------------------------------------------------------------------
Configuration:
--------------------------------------------------------------------------------
{
"adapter": "cust_vscode-ruby",
"default": true,
"configuration": {
"name": "Debug Rails server",
"type": "Ruby",
"request": "launch",
"cwd": "/Users/username/Folder/app-name",
"pathToBundler": "/Users/username/.rvm/rubies/ruby-version/bin/bundler",
"pathToRDebugIDE": "/Users/username/.rvm/gems/ruby-version/gems/ruby-debug-ide-0.7.3",
"program": "/Users/username/Folder/app-name/bin/rails",
"args": [
"server"
]
}
}
--------------------------------------------------------------------------------
API Prefix:
Launch/Init: True / True
Workspace Root: /Users/username/Folder/app-name
Launch Config:
--------------------------------------------------------------------------------
{
"cwd": "/Users/username/Folder/app-name",
"showDebugOutput": false,
"trace": false,
"name": "Debug Rails server",
"type": "Ruby",
"request": "launch",
"pathToBundler": "/Users/username/.rvm/rubies/ruby-version/bin/bundler",
"pathToRDebugIDE": "/Users/username/.rvm/gems/ruby-version/gems/ruby-debug-ide-0.7.3",
"program": "/Users/username/Folder/app-name/bin/rails",
"args": [
"server"
]
}
--------------------------------------------------------------------------------
Server Capabilities:
--------------------------------------------------------------------------------
{
"supportsConfigurationDoneRequest": true,
"supportsConditionalBreakpoints": true
}
--------------------------------------------------------------------------------
Line Breakpoints:
--------------------------------------------------------------------------------
{
"/Users/username/Folder/app-name/app/controllers/.../somethings_controller.rb": [
{
"state": "ENABLED",
"line": 83,
"options": {},
"sign_id": 1,
"server_bp": {
"verified": true,
"line": 83,
"id": 1
}
}
]
}
--------------------------------------------------------------------------------
Func Breakpoints:
--------------------------------------------------------------------------------
[]
--------------------------------------------------------------------------------
Ex Breakpoints:
None
so i got it to work not with passenger itself.
i changed my vimspector config to the following, switching up some paths to the used binaries by trial and error (i think the one that matters is calling the rails binary in the project root). The important thing is to call in the args server -u puma. Make sure to write each part of the command as a single argument. puma being the default rails server (i found it to be in my sample app), so this gets picked up by rdebug-ide and debase.
{
"configurations": {
"rails": {
"adapter": "cust_vscode-ruby",
"default": true,
"configuration": {
"name": "Debug Rails server",
"type": "Ruby",
"request": "launch",
"cwd": "${workspaceRoot}",
"pathToBundler": "/Users/username/.rvm/rubies/ruby-RUBYVERSION/bin/bundler",
"pathToRDebugIDE": "/Users/username/.rvm/gems/ruby-RUBYVERSION/gems/ruby-debug-ide-RUBYDEBUGIDEVERSION",
"program": "${workspaceRoot}/bin/rails",
"args": [
"server", "-u", "puma"
]
}
}
}
}
There are some options I found for using passenger, it probably has to do something with writing the config as attach.
What seemed closest (but it doesn't work) to the answer seemed to copy the rubymine config that calls:
/bin/bash -c "~/.rvm/bin/rvm _RUBYVERSION_ do ~/.rvm/rubies/ruby-_RUBYVERSION_/bin/ruby ~/.rvm/gems/ruby-_VERSION_/gems/ruby-debug-ide-_RUBYDEBUG-IDEVERSION_/bin/rdebug-ide --key-value --disable-int-handler --evaluation-timeout 10 --evaluation-control --time-limit 100 --memory-limit 0 --rubymine-protocol-extensions --port 42371 --host 0.0.0.0 --dispatcher-port 38403 -- ~/Folder/project/bin/rails server -b 0.0.0.0 -p 3000 -e development"
I think a point where it could be failing, and why it's not working is that passenger needs to be set to allow debugging (PassengerDebugger on), although i might just be confusing everything here. However this passenger option is reserved for the Enterprise version (https://www.phusionpassenger.com/library/admin/apache/debugging_console/ruby/).
Where this left me off was i launched this debug config but couldn't connect to the socket. Maybe this is to some security measure i made to tight for my own good on my system or more likely a fault in the config.
I think it's possible to attach to the passenger server properly somehow some ideas could maybe be found here but i didn't get anywhere with them.
https://mondotondo.com/2012/08/23/how-to-remotely-debug-rails-in-apache-passenger-rubymine/
How can I get ruby-debug-ide to work?
If you are struggeling like me with this, hang in there you will get it some day!

Migrating Elasticsearch logs to a different cluster

I have an Elasticsearch deployment on Kubernetes (AKS). I'm using the official Elastic's docker images for deployments. Logs are being stored in a persistent Azure Disk. How can I migrate some of these logs to another cluster with a similar setup? Only those logs that matches a filter condition based on datetime of the logs needs to be migrated.
Please use Reindex API for achieving the same
POST _reindex
{
"source": {
"remote": {
"host": "http://oldhost:9200",
"username": "user",
"password": "pass"
},
"index": "source",
"query": {
"match": {
"test": "data"
}
}
},
"dest": {
"index": "dest"
}
}
Note:
Run the aforementioned command on your target instance.
Make sure that the source instance is whitelisted in elasticsearch.yml
reindex.remote.whitelist: oldhost:9200
Run the process asynchronously using below query param
POST _reindex?wait_for_completion=false

How to get RSpec working with debugger IDE in VSCode

Despite having spent about the last three hours trying to get this working I cannot for the life of my get RSpec to work with the debugger on VSCode. I can get Rspec to run in the terminal on VSCode but that doesn't give me any of the IDE's debugging functionality for inspecting and stepping.
This is what I've got in my launch.json file:
{
"version": "0.2.0",
"configurations": [
{
"name": "Run RSpec - all",
"type": "Ruby",
"request": "launch",
"cwd": "${workspaceRoot}",
"program": "/Users/my-home-dir/.rvm/gems/ruby-2.6.5#project-gemset-name/wrappers/rspec",
"pathToRDebugIDE": "/Users/my-home-dir/.rvm/gems/ruby-2.6.5#project-gemset-name/gems/ruby-debug-ide-0.7.2",
"args": [
"--pattern",
"${workspaceRoot}/spec/**/*_rspec.rb"
]
}
]
}
And my gemfile contains:
gem 'ruby-debug-ide', '~>0.7.2'
gem 'debase', '~>0.2.4.1'
I've got a feeling that the errors may be coming about due to in incompatibility between RVM and VSCode but I've no idea how to unwind that issue.
This was all setup as per the Microsoft recipe here: https://github.com/Microsoft/vscode-recipes/tree/master/debugging-Ruby-on-Rails
Every time I run this setup I get the following error in the debug console:
Debugger terminal error: Process failed: spawn rdebug-ide ENOENT
Is there any way to get this to run? Also is there any way to get it to use guard so that it runs automatically?
I got it working. Unfortunately, I don't use RVM. So, my solution involves rbenv. I'm sharing it here anyway in case it helps you, or someone else.
which rspec pointed me to the shim (shell script) that rbenv uses to execute the version of rspec installed under the current version of Ruby. When I configured launch.json with that path rdebug-ide didn't like the shim. I assume it was expecting the executable.
So, I ran rbenv which rspec and got the actual path to the executable. Once I plugged that into launch.json it worked fine. Of course, if I change the version of Ruby I'm running, I'll have to update the file to point to the version of RSpec installed under the new version of Ruby.
Given the prevalence of Ruby version managers among the community, I would think ruby-debug-ide would have considered this. Perhaps it's worth an issue on their GitHub: https://github.com/ruby-debug/ruby-debug-ide.
I added a preLaunchTask to launch rdebug-ide server then looked for a pattern before attaching the debugger.
I used the focus flag for controlling which tests are run or not. Am able to set breakpoints in main script files as well as *_spec.rb files.
Gemfile:
group :test, :development do
gem "ruby-debug-ide", "~> 0.7.2"
gem "debase", "~> 0.2.4"
end
launch.json:
{
"version": "0.2.0",
"configurations": [
{
"name": "Listen for rdebug-ide rspec",
"type": "Ruby",
"request": "attach",
"remoteHost": "127.0.0.1",
"remotePort": "1234",
"preLaunchTask": "run-rdebug-for-rspec",
"remoteWorkspaceRoot": "${workspaceRoot}",
"cwd": "${workspaceFolder}"
}
]
}
tasks.json
{
"version": "2.0.0",
"tasks": [{
"label": "run-rdebug-for-rspec",
"command": "bundle",
// could update these are you see fit
// my tests were located in /spec folder
"args": [
"exec","rdebug-ide",
"--host","0.0.0.0",
"--port","1234",
"--dispatcher-port","26162",
// needed to specify the full path of bundle
// can be found by `which bundle`
"--", "/usr/local/bin/bundle", "exec", "rspec", "spec/"
],
"type": "shell",
"isBackground": true,
// this will look for a pattern to attach to rdebug server
// attaching to the server will start running the tests
"problemMatcher": [
{
"pattern": [
{
"regexp": ".",
"file": 1,
"location": 2,
"message": 3
}
],
"background": {
"activeOnStart": true,
// once server is up and running it'll display something like:
// Fast Debugger (ruby-debug-ide 0.7.2, debase 0.2.4.1, file filtering is supported) listens on 0.0.0.0:1234
"beginsPattern": "^Fast Debugger.*1234$",
"endsPattern": ".",
}
}
],
// open up a new task window everytime
// if set to default `shared` then previous task window needs to be closed
// otherwise, was having issues getting patternMatcher to work
// and for rdebug to attach within a shared terminal on successive runs, or restarts.
"presentation": {
"panel": "new"
}
}]
}
While the answer above may work, there will be problems as soon as you upgrade your ruby version because the paths are hard coded.
Here is a less brittle approach that should be able to survive a version upgrade without any problem.
I'm on ruby 3.0.3, managed by rvm on mac osx.
Gemfile includes:
group :development do
gem 'ruby-debug-ide'
gem 'debase', "~> 0.2.5.beta2"
end
(the stable version of debase didn't support ruby v3, hence the beta)
Launch config use the GEM_HOME environment variable to specify the rspec binary (vscode default configuration puts this in workspace root for some reason):
{
"name": "RSpec - all",
"type": "Ruby",
"request": "launch",
"program": "${env:GEM_HOME}/bin/rspec",
"args": [
"-I",
"${workspaceRoot}"
]
}

Using environment variables in Ghost v1 config

In Ghost 0.x, config was provided via a single config.js file with keys for each env.
In Ghost 1.0, config is provided via multiple config.json files
How do you provide environment variables in Ghost 1.0?
I would like to dynamically set the port value using process.env.port on Cloud9 IDE like so.
config.development.json
{
"url": "http://localhost",
"server": {
"port": process.env.port,
"host": process.env.IP
}
}
When I run the application using ghost start with the following config, it says You can access your publication at http://localhost:2368, but when I go to http://localhost:2368 in http://c9.io it gives me an error saying No application seems to be running here!
{
"url": "http://localhost:2368",
"server": {
"port": 2368,
"host": "127.0.0.1"
}
}
I managed to figure out how to do this.
Here is the solution incase if someone else is also trying to figure out how to do the same thing.
In your config.development.json file, add the following.
{
"url": "http://{workspace_name}-{username}.c9users.io:8080",
"server": {
"port": 8080,
"host": "0.0.0.0"
}
}
Alternatively, run the following command in the terminal. This will dynamically get the value for the port and host environment variable and add the above content to the config.development.json file.
ghost config url http://$C9_HOSTNAME:$PORT
ghost config server.port $PORT
ghost config server.host $IP

Losing Access to Server After Provisioning With Chef

I am using the rails-server-template available here (https://github.com/TalkingQuickly/rails-server-template) to provision a Rails server (Ubuntu 12.04) using Chef. When I am setting up a new server, I copy my public ssh key ssh-copy-id -i ~/.ssh/id_rsa.pub ubuntu#my-server.amazonaws.com and am able to enter my server fine.
But after I download a new copy of this template (updating the nodes/my-server.json file to this:)
{
"environment": "production",
"authorization": {
"sudo": {
"users": ["deploy", "vagrant"]
}
},
"run_list": [
"role[server]",
"role[postgres-server]"
],
"automatic": {
"ipaddress": "my-server.amazonaws.com"
},
"postgresql": {
"password": {
"postgres": "password"
}
}
}
And also updating the deploy.json user in data_bags/users:
{
"id": "deploy",
// generate this with: openssl passwd -1 "plaintextpassword"
"password": "password",
"ssh_keys": [ "ssh-rsa my-public-key from ~/.ssh/id_rsa.pub"
],
"groups": [ "sysadmin"],
"shell": "\/bin\/bash"
}
For some weird reason, after provisioning the server with bundle exec knife solo bootstrap ubuntu#my-server.com, I get a Permission denied (publickey) error. When trying to log-in using ssh, I get asked for the password for the ubuntu user, which I don't know. I can't even log in with my key pair .pem file from Amazon EC2 anymore.
Am I missing something? I didn't change the server.json role, and I can't seem to figure out what is going on. Has something changed my ssh configuration during provisioning?
Turns out when I was trying to ssh into my server, the user I was using was ubuntu, whereas in the data_bags, I set up a new user with the id deploy. I needed to ssh in as the deploy user.

Resources