Can I do setup of flume on my local machine?
I can only see setup guides for flume on a cluster environment.
I have to setup flume and have to integrate it with morphline.
Installing Flume in local is straightforward. Just download the flume distribution from CDH repo and set all the environment variables like HADOOP_HOME,JAVA_HOME etc and configure the flume configuration file.
Related
How can I deploy aws resources using external jenkins and terraform. (I don`t like my jenkins running in ec2 or in aws) because it may terminate at any time and every time I have to build from ami or all steps that I do on first time. I mean to say save all settings and credentials etc. So, I looking for some solution to install it on my VM/virtual box and then run pipeline job there and build aws resources/ services using terraform.
You can run terraform or jenkins from anywhere to create resources in AWS.
Jenkins is just an orchestrator tool which will use terraform to create resources.
We only need to change how terraform interact with your AWS environment.
if you are having terraform on one of the AWS EC2 you can utilize EC2 metadata to interact/authenticate with AWS.
now as you move towards your local system or VM you have to change the way how you authenticate with terraform.
you can use below code in terraform to authenticate with AWS
provider "aws" {
region = "us-west-2"
access_key = "my-access-key"
secret_key = "my-secret-key"
}
please refer terraform documentation for more authentication methods
https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration
I am very new to devops and I could really use some help to understand the concept of this.
So I am trying to develop a continuous integration environment using VirtualBox and Vagrant. I've read some examples of how to build such an environment to pull a maven project from github, build it and deploy to the nexus artefact repository.
I have managed to configure a VM with Ubuntu and installed Tomcat on it.
What I don't understand is where should I configure the Jenkins jobs to build the project and to deploy it to nexus and to make it run in Tomcat Server. On my local machine or in the virtual machine ?
Thanks.
If you are using bridged/hostonly networking for ubuntu, then you can run the jenkins in your host machine. If it is NAT/Private networking, use guest machine to run jenkin jobs.
I'm new to Google Cloud Platform and want to build a deployment pipeline via Jenkins. I have a virtual machine up and running that I connect via SSH button given on the dashboard and do deployments.
Unlike AWS where I get pem file to connect... I don't have a file to connect here. Now, in Jenkins when I want to connect via shell script and deploy latest code on VM - I don't know how to do it.
something like
ssh -i #some-file name:ip
Kindly help as to how can I connect/ssh gcp vm via shell in Jenkins and make deployments. Step by step answer needed. Thanks!
There are still PEM files stored in the VM instance, you can locate them in $HOME/.ssh.
See this stack post about how to retrieve SSH keys on GCE
I am trying to automate deployment proceeses using Jenkins. I am trying to deploy some config files and some EAR files to remote VM. Normally I would open a Remote Desktop connection and deploy manually.
Is there a way FROM JENKINS through scripting or plugin where I can deploy files to a remote VM?
You can use this framework which automates the management and deployment of files on remote VM's running in a VMWare environment.
Just install RI on your Jenkins server and add batch steps to execute the RI executable with a XML configuration that should do what you want.
Hi I am very new to Flume. I have installed flumes 0.9.3 version and able to start the node in window OS.
But to move forward I need certain basic things. Can any one help me to setup flumes completely?
How to setup flumes agent and configure with any source like fileshare or server log
How to setup flumes collector
How the sink in agent will push the files to collector
How to install flumes master and configure with multiple agents and collector
How to integrate flumes collectors with HDFS.