Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
Improve this question
Am new in the Apache Flume. I need to install the flume on top of the HDFS cluster environment. I did Google it, all are saying using the cloudera distribution but I need to install and configure from the source.
So can anyone please suggest me, where to start and how to customize the flume agent and sink services?
I have just installed Apache Flume 1.3 on Ubuntu.
You need to download the binary zip for your OS, extract it and create a config file which is similar to properties file in Java.
The installation and running of agents is a dumb/easy process, just read this
installation steps
#step1
goto http://flume.apache.org/download.html
#click following link
apache-flume-1.4.0-bin.tar.gz
http://apache.mesi.com.ar/flume/1.4.0/apache-flume-1.4.0-bin.tar.gz
#Download apache-flume by double clicking on the above link
tar -xvzf apache-flume-1.4.0-bin.tar.gz
mv apache-flume-1.4.0-bin.tar.gz flume
chown -R hduser:hadoop flume
#go to flume conf path
../../flume/conf
vi flume-env.sh.template
#mention JAVA Path
#mantion flume Classpath
Here is an example tutorial that I've put together that uses flume, apache, flask and kibana in a vagrant environment.
There is a video at the bottom to give a better intro:
https://github.com/mross462/Flume-Elasticsearch
In the course Moving Data into Hadoop offered by Big Data University at :http://bigdatauniversity.com/bdu-wp/bdu-course/moving-data-into-hadoop/ teaches you intro to flume and a lab exercise which helps you with both configuring and running flume.
FOR CENT OS
*step1 Download
[https://downloads.apache.org/flume/1.9.0/apache-flume-1.9.0-bin.tar.gz][1]
*Install apache flume
"tar -xvf apache-flume-1.9.0-bin.tar.gz"
*Moving
"mv apache-flume-1.9.0-bin.tar.gz usr/local/flume"
*Setting path
"vim /etc/profile"
"export FLUME_HOME=/usr/local/flume
export PATH=$PATH:$FLUME_HOME/bin"
*Version
"flume-ng version"
*Conf
"usr/local/flume/conf"
in flume-conf.properties
set your hadoop properties
Related
Closed. This question is not about programming or software development. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 2 days ago.
Improve this question
can't run docker run command and docker desktop is stopped...
i tried reinstalling and upgraded wsl 1 to 2
Post "http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.24/containers/create": open //./pipe/docker_engine: The system cannot find the file specified. See 'docker run --help'.
The error suggests that docker is not running or properly installed, if you installed it, check the following.
From the search bar, type "Turn Windows features on or off" and make sure you enable the following:
Windows subsystem for Linux
virtual machine platform
windows hypervisor platform
You may also check the PATH of the docker if it is properly added in the environment variable.
From the search bar type "Edit the system environment variable" then click on "Environment variable" in the bottom right corner then under the System variable section choose "path" in the "variable" column then "edit"
A new window will open. Make sure that the following path is added
C:\Program Files\Docker\Docker\resources\bin
If you have done all these steps. You should make sure that the docker desktop app is running, then try to run docker commands.
If the problem still persists, you may uninstall the app and reinstall it again and it should work
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
Improve this question
I knew that docker won't work on windows 10 home edition. It will work in windows 10 pro or enterprise as it need hyper v and it is available in these two. But I found some script for hyper v and able to install and enable hyper v in window 10 home. So is it better to install ubuntu as a virtual environment and install docker over there. or the windows 10 home with hyper v enabled will be good enough to install docker.
Please help me to understand this.
Thanks.
Docker does work on Windows 10 home (I'm using it now).
Install Docker Desktop (wsl2 integration will be disabled)
Install wsl 2.0
Restart.
Install your distro of choice through the Microsoft Store (I used Ubuntu).
Initialise Ubuntu and configure your user account.
If you run Docker in Ubuntu at this point you'll see a request to integrate with wsl2, open the Docker Desktop settings and enable integration with WSL2 for your linux distribution.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
We have a private docker registry (Sonatype nexus) which holds all our private docker images. I was looking for an open source vulnerability and security scanner for scanning all the images on the private registry also I want to install the tool on the linux box and also integrate with Jenkins.
I came across Twistlock, Anchore, Dagda. None of these seems to provide a tool which can be installed and used without any license.
Any inputs?
You can use Clair:
https://github.com/coreos/clair
Simply follow the instruction:
https://github.com/arminc/clair-scanner#run
replace the date on the clair-db image with either latest or a specific date from
https://hub.docker.com/r/arminc/clair-db/tags/
you can get the scanner binaries here:
https://github.com/arminc/clair-scanner/releases
I too would say to use CoreOS Clair for this purpose. However a few things to consider,
How you gonna run Clair i.e. As Docker compose or in your kube cluster?
Clair provides Static Scanning only (only the container image, not the running container)
What sort of integration tools you are using?
If you thinking to integrate with your CI/CD pipelines, the best Clair client I can recommend is Klar because it is much simple and straightforward to use. clairctl can also be used but it gives some hard time when troubleshooting with permission errors.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
I have a server with centos running rails application each time when I reboot my system I have doing following tasks manually. How to enable all the following tasks in start up
To stop and start my nginx I am using
sudo fuser -k 80/tcp # to stop
sudo /opt/nginx/sbin/nginx # to start
And to start my postgresql I am using following command
service postgresql-9.2 start/stop/restart
Once I connect my server using ssh and try some postgresql command ill get error as
postgresql-9.2: unrecognized service
To avoid this am using command
export PATH=/usr/pgsql-9.2/bin:$PATH
And then ill use postgresql(Doing export command each time when I connect through ssh).
And I am using ghost blog facing same issue whenever I reboot my server I need to run following command to start my ghost blog
NODE_ENV=production forever start index.js
How to solve all these. Can any one help
You can add the process in your init script that is at /etc/init.d/
.You can further read run script on startup
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 9 years ago.
Improve this question
This isn't exactly a production deployment of rails, or I'd go the standard Passenger route - I simply want it to start on port 3003 and sit there.
I'm running the following .sh script from rc.local
source /usr/local/rvm/scripts/rvm
/usr/local/rvm/rubies/ruby-2.0.0-p247/bin/ruby /root/rails_app/script/rails server -d --port 3003
I seem to recall the first line is required by rvm to enable ruby\rails commands in the shell. Unfortunately, this doesn't seem to do the trick.
I'm having a hard time diagnosing the issue.
Is there a way to see the error log of scripts that are executed via rc.local?
you can not source RVM in sh shell, you need to use a wrapper:
/usr/local/rvm/wrappers/ruby-2.0.0-p247/ruby \
/root/rails_app/script/rails server -d --port 3003
you can find more information about it here - http://rvm.io/integration/init-d