I am trying to append twitter data into HDFS using flume. I am not able to set the Java classpath in flume-env.sh file. It is showing that you don't have permissions to do it. How can I edit this file?
Take a look at Configuring Flume to use Twitter as Source for step by step instructions on how to configure flume to ingest Twitter data
Before edit this flume-env.sh using nano command in Linux OS use sudo command.
ex: sudo nano flume-env.sh
It will ask for user current user password provide the same to make changes.
Firstly, you need to confirm you have the right permissions for editing "flume-env.sh" file. File must be in $APACHE_FLUME_HOME/conf/flume-env.sh
Then you can edit the file (It is my own configuration) for set additional things to the classpath
JAVA_HOME=/usr/lib/jvm/java-1.8-openjdk
JAVA_OPTS="-Xmx16392m"
FLUME_CLASSPATH="/path/to/the/flume/classpath"
Related
My goal is to put my telegraf config into source control. To do so, I have a repo in my user's home directory with the appropriate config file which has already been tested and proven working.
I have added the path to the new config file in the "default" environment variables file:
/etc/default/telegraf
like this:
TELEGRAF_CONFIG_PATH="/home/ubuntu/some_repo/telegraf.conf"
... as well as other required variables such as passwords.
However, when I attempt to run
telegraf --test
It says No config file specified, and could not find one in $TELEGRAF_CONFIG_PATH etc.
Further, if I force it by
telegraf --test --config /home/ubuntu/some_repo/telegraf.conf
Then the process fails because it is missing the other required variables.
Questions:
What am I doing wrong?
Is there not also a way of specifying a config directory too (I would like to break my file down into separate input files)?
Perhaps as an alternative to all of this... is there not a way of specifying additional configuration files to be included from within the default /etc/telegraf/telegraf.conf file? (I've been unable to find any mention of this in documentation).
What am I doing wrong?
See what user:group owns /etc/default/telegraf. This file is better used when running telegraf as a service via systemd. Additionally, if you run env do you see the TELEGRAF_CONFIG_PATH variable? What about your other variables? If not, then you probably need to source the file first.
Is there not also a way of specifying a config directory too (I would like to break my file down into separate input files)?
Yes! Take a look at all the options of telegraf with telegraf --help and you will find:
--config-directory <directory> directory containing additional *.conf files
Perhaps as an alternative to all of this... is there not a way of specifying additional configuration files to be included from within the default /etc/telegraf/telegraf.conf file? (I've been unable to find any mention of this in documentation).
That is not the method I would suggest going down. Check out the config directory option above I mentioned.
Ok, after a LOT of trial and error, I figured everything out. For those facing similar issues, here is your shortcut to the answer:
Firstly, remember that when adding variables to the /etc/default/telegraf file, it must effectively be reloaded. So for example using ubuntu systemctl, that requires a restart.
You can verify that the variables have been loaded successfully using this:
$ sudo strings /proc/<pid>/environ
where <pid> is the "Main PID" from the telegraf status output
Secondly, when testing (eg telegraf --test) then (this is the part that is not necessarily intuitive and isn't documented) you will have to ALSO load the same environmental variables into the current user (eg: SET var=value) such that running
$ env
shows the same results as the previous command.
Hint: This is a good method for loading the current env file directly rather than doing it manually.
I am trying to train myself on Spring security and started looking at Baeldung.com website and examples posted for public download at github repository https://github.com/Baeldung/spring-security-registration. I downloaded the source code in this repository as ZIP and imported in netbeans. Following the instructions in README.md, I built it using mvn clean install. Application spring-security-login-and-registration downloaded all dependencies and built clean. But I don't have a clue how to configure the MySQL database needed for this application.
Do I need to download MySQL server and have a server instance on my localhost?
If yes, where would I get the scripts to create the schema/tables needed in the security application? I can't find the scripts anywhere in source code. Anything similar to hSQL going on here?
Any help is appreciated.
Yes you need to download MySql from here https://dev.mysql.com/downloads/windows/installer/8.0.html.
And need to configure few things during installation like username and password.
And update these fields inside you database properties file.
################### DataSource Configuration ##########################
jdbc.driverClassName=com.mysql.cj.jdbc.Driver
jdbc.url=jdbc:mysql://localhost:3306/YourDatabaseName?createDatabaseIfNotExist=true
jdbc.user=YourUserNameForMySql
jdbc.pass=PasswordHere
You can start you mysql on windows by executing this command:-
"C:\Program Files\MySQL\MySQL Server 8.0\bin\mysqld.exe" on command prompt
Meanwhile you can login to mysql shell using this below command on terminal:-
mysql -uroot -proot
I am building a dropwizard service which will connect to multiple data sources including mySQL and Elasticsearch. All the mySQL settings can be defined in the yaml config file which gets read in after running from the commandline.
But what about other settings that I need to read in for other data sources that I will connect with myself, for example Elasticsearch? Where can I define those settings?
I thought I could add another commandline Command - which I tried, but I can only run a single command (from the commandline) at a time - so I can't seem to run both the 'server' command as well as my custom command, 'custom' which is followed by the my own config file for elasticsearch.
How can I introduce settings either individually or from a file - which are defined at run time (not hard coded)?
Thanks
Anton
Check out the Dropwizard Core documentation on adding custom configuration.
You'd create an ElasticSearchFactory class similar to the MessageQueueFactory in the example, reference this in your Configuration (that's in turn referenced in your Application), and then the options you need can be added to your main yaml configuration.
I want to add a hook to ejabbered code. Steps followed:
added mod_filter.erl to src folder(mod_filter catches filter_packet event and returns modified packet)
added mod_filter in ejjbaerd.yml under modules section.
ran ./configure, make , make install
Ejabberd server runs successfully , with mod_filter.beam file in /ebin folder but the ejabberd.yml file located at /etc/ejabberd/ejabberd.yml does not get updated as per my second step.
I can manually update it and restart server but that does not seem to be the correct way. I think i might be missing a deployment step here, please suggest.
Change configuration file (ejabberd.yml) according to your need and reload configuration file like that- ejabberd_ctl reload_config or ejabberd_config:reload_file(). or ejabberd_config:load_file("/etc/ejabberd/ejabberd.yml"). you do not need to restart server.
https://www.ejabberd.im/tricks
https://docs.ejabberd.im/admin/api/
I tried to run grailsw, but the wrapper cannot connect to download grails-2.2.1-download.zip (creates a 0 byte file instead).
I need to use a proxy server to connect to the internet, where do I configure proxy settings for the Grails Wrapper?
After running grails wrapper, your project directory has a new subdirectory called wrapper, with a file grails-wrapper.properties. You can configure your proxy settings in there, with the following properties:
systemProp.http.proxyHost=
systemProp.http.proxyPort=
systemProp.http.proxyUser=
systemProp.http.proxyPassword=
systemProp.http.nonProxyHosts=
I solved this problem for myself.
It is a two step process
1.a) Back up your JRE_HOME\lib\security folder. This is essential because the below steps might corrupt cacerts file under jre.
1.b) You need to install the ssl public key of Github.com to your local file system. To do that you have to use the InstallCert.java program( Link to InstallCert.java )
It is supposed to be run as java InstallCert github.com
and when it asks to enter cert number you need to enter 1
It will create a file with name "jssecacerts" in the current directory
1.c) But this program will not work because it does not know about how to authenticate with proxy. For this you need the code from SSLSocketClientWithTunneling page
Use the above two and create a program that tunnels through the proxy retrieves the ssl key and writes a file called jssecerts
2) Update your grails.bat with addtional options. Add these options to the %JAVA_EXE% command line. Paste them after %DEFAULT_JVM_OPTS%
-Dhttp.proxyHost=YourproxyURL -Dhttp.proxyPort=YourproxyPort -Dhttps.proxyHost=YourproxyURL -Dhttps.proxyPort=YourproxyPort -Dhttp.proxyUser=YourProxyUserID -Dhttp.proxyPassword=YourProxyPassword -Dhttps.proxyUser=YourProxyUserID -Dhttps.proxyPassword=YourProxyPassword -Djavax.net.ssl.trustStore=path-to-your-jssecacerts-created-in-step-1