We are trying to enable SSL connection between Apache Ranger and Kafka cluster. After creating keystore and truststore for both Kafka and Ranger, we are unable to connect Kafka to Ranger and we are getting the following error message:
[2020-06-25 20:47:40,013] ERROR Unable to get the Credential Provider from the Configuration (org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider)
java.lang.IllegalArgumentException: The value of property hadoop.security.credential.provider.path must not be null
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1134)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1115)
at org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider.getCredentialProviders(RangerCredentialProvider.java:68)
at org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider.getCredentialString(RangerCredentialProvider.java:46)
at org.apache.ranger.plugin.util.RangerRESTClient.getCredential(RangerRESTClient.java:386)
at org.apache.ranger.plugin.util.RangerRESTClient.getKeyManagers(RangerRESTClient.java:272)
at org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:188)
at org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:176)
at org.apache.ranger.plugin.util.RangerRESTClient.getResource(RangerRESTClient.java:156)
at org.apache.ranger.admin.client.RangerAdminRESTClient.createWebResource(RangerAdminRESTClient.java:275)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:126)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:264)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:202)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:171)
[2020-06-25 20:47:40,013] ERROR PolicyRefresher(serviceName=KafkaTest): failed to refresh policies. Will continue to use last known version of policies (51) (org.apache.ranger.plugin.util.PolicyRefresher)
java.lang.IllegalArgumentException: TrustManager is not specified
at org.apache.commons.lang.Validate.notNull(Validate.java:192)
at org.apache.ranger.plugin.util.RangerRESTClient.getSSLContext(RangerRESTClient.java:369)
at org.apache.ranger.plugin.util.RangerRESTClient.buildClient(RangerRESTClient.java:190)
at org.apache.ranger.plugin.util.RangerRESTClient.getClient(RangerRESTClient.java:176)
at org.apache.ranger.plugin.util.RangerRESTClient.getResource(RangerRESTClient.java:156)
at org.apache.ranger.admin.client.RangerAdminRESTClient.createWebResource(RangerAdminRESTClient.java:275)
at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:126)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:264)
at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:202)
at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:171)
In Kafka-Ranger plugin, I have configured below properties in install.properites
COMPONENT_INSTALL_DIR_NAME=/home/ec2-user/kafka
POLICY_MGR_URL=https://public-dns-of-ec2:6182
REPOSITORY_NAME=KafkaTest
SSL_KEYSTORE_FILE_PATH=/etc/hadoop/conf/<keystore>.jks
SSL_KEYSTORE_PASSWORD=<password
>
SSL_TRUSTSTORE_FILE_PATH=/etc/hadoop/conf/<truststore>.jks
SSL_TRUSTSTORE_PASSWORD=<password>
Note: We are not using Ambari
Make sure following properties are set:-
xasecure.policymgr.clientssl.keystore.credential.file=jceks://file/{{credential_file}}
xasecure.policymgr.clientssl.truststore.credential.file=jceks://file/{{credential_file}}
xasecure.policymgr.clientssl.truststore=/path/to/truststore
Related
I attempted to connect neo4j aura database using Python but failed as "Unable to retrieve routing information".
from neo4j import GraphDatabase
from neo4j.debug import watch
uri = "neo4j+s://<id>.databases.neo4j.io"
driver = GraphDatabase.driver(uri, auth=("neo4j", "<password>"))
def workload(tx):
return tx.run("RETURN 1 as n").data()
with watch("neo4j"): # enable logging
with driver.session() as session:
session.write_transaction(workload)
driver.close()
Running above python scripts returned the following log:
Attempting to update routing table from IPv4Address(('<id>.databases.neo4j.io', 7687))
[#0000] C: <RESOLVE> <id>.databases.neo4j.io:7687
[#0000] C: <OPEN> xx.xxx.xxx.xxx:7687
[#C000] C: <SECURE> <id>.databases.neo4j.io
[#0000] C: <CONNECTION FAILED> BoltSecurityError: [SSLCertVerificationError] Connection Failed. Please ensure that your database is listening on the correct host and port and that you have enabled encryption if required. Note that the default encryption setting has changed in Neo4j 4.0. See the docs for more information. Failed to establish encrypted connection. (code 1: Operation not permitted)
Failed to fetch routing info 35.xxx.xxx.xxx:7687
[#0000] C: <ROUTING> Deactivating address IPv4Address(('<id>.databases.neo4j.io', 7687))
[#0000] C: <ROUTING> table={None: RoutingTable(database=None routers={}, readers={}, writers={}, last_updated_time=0.235748575, ttl=0)}
Attempting to update routing table from
Unable to retrieve routing information
Transaction failed and will be retried in 1.1281720312998946s (Unable to retrieve routing information)
I looked into neo4j documentation and searched other places but none of the possible resolutions can be found.
Version:
Python 3.7.4
neo4j 4.4.2
I very much appreciate your input if you have ever experienced the same issues and found any way to resolve the issue.
There is lot of help available. Not able to fix it. After I enter my user name and password.
Jenkins with Azure AD login is successful and it return the token also. After that it fails with "A problem occurred while processing the request". Login is successful when I see the Azure side and jenkins ui prints token also.
When I see the error logs, I see
javax.net.ssl|DEBUG|13|Handling POST /securityRealm/finishLogin from x.x.x.x : Jetty (winstone)-19|2021-07-15 19:36:53.374 EDT|Utilities.java:73|the previous server name in SNI (type=host_name (0), value=login.microsoftonline.com) was replaced with (type=host_name (0), value=login.microsoftonline.com)
2021-07-15 23:36:55.398+0000 [id=326] INFO c.m.a.a.AuthenticationAuthority#doInstanceDiscovery: [Correlation ID: e11160be-50c3-43d7-96a8-dc02c3cc2b2c] Instance discovery was successful
javax.net.ssl|ERROR|13|Handling POST /securityRealm/finishLogin from x.x.x.x : Jetty (winstone)-19|2021-07-15 19:36:55.769 EDT|TransportContext.java:344|Fatal **** (CERTIFICATE_UNKNOWN): PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (
"throwable" : {
Also
javax.net.ssl|DEBUG|13|Handling POST /securityRealm/finishLogin from x.x.x.x : Jetty (winstone)-19|2021-07-15 19:36:55.773 EDT|SSLSocketImpl.java:1569|close the underlying socket
javax.net.ssl|DEBUG|13|Handling POST /securityRealm/finishLogin from x.x.x.x : Jetty (winstone)-19|2021-07-15 19:36:55.773 EDT|SSLSocketImpl.java:1588|close the SSL connection (initiative)
2021-07-15 23:36:55.787+0000 [id=19] SEVERE c.m.j.azuread.AzureSecurityRealm#doFinishLogin: error
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
I have imported certs for login.microsoftonline.com and portal.azure.com, my jenkins.xml has
-Djavax.net.ssl.trustStore="C:\Program Files (x86)\Jenkins\.cacerts\jssecacerts" -Djavax.net.ssl.trustStorePassword=changeit
Not sure what website it says "unable to find valid certification path to requested target"
As posted in another entry in stack overflow I'd try to debug what is happening with your trustStore with something like:
java -Djavax.net.debug=all -Djavax.net.ssl.trustStore="C:\Program Files (x86)\Jenkins\.cacerts\jssecacerts" -Djavax.net.ssl.trustStorePassword=changeit
You may want to have a look at this post
I have a JAVA application which I want to monitor its JMX attributes using telegraf tool.
The tool provides jolikia plugin to monitor JMX attributes. I have added following dependencies to my app's pom.xml file regarding Maven section of Jolokia documentation:
<dependency>
<groupId>org.jolokia</groupId>
<artifactId>jolokia-core</artifactId>
<version>1.3.7</version>
</dependency>
<dependency>
<groupId>org.jolokia</groupId>
<artifactId>jolokia-client-java</artifactId>
<version>1.3.7</version>
</dependency>
This is my /etc/telegraf/telegraf.conf file:
[[inputs.jolokia]]
context = "/jolokia/"
[[inputs.jolokia.servers]]
name = "wr-core"
host = "192.168.100.175"
port = "1998"
[[inputs.jolokia.metrics]]
name = "send_success"
mbean = "wr-core:type=monitor,name=execution"
attribute = "MessageSendSuccessCount"
The application is up in the provided IP/port (I can connect to it with jconsole). The application has a monitoring section which its object name (as shown in jconsole) is wr-core:type=monitor,name=execution and has the attribute MessageSendSuccessCount. But when I start telegraf service, following error occurs:
Jan 14 14:30:32 ZiZi telegraf[17258]: 2018-01-14T11:00:32Z E! Error in plugin [inputs.jolokia]: error performing request: Error decoding JSON response: invalid character '\x00' looking for beginning of value:
Note that 1998 is my app's jmx port. I also tried using 8778 which is jolokia-agent port; got:
Jan 14 14:40:03 ZiZi telegraf[9150]: 2018-01-14T11:10:03Z E! Error in plugin [inputs.jolokia]: error performing request: Post http://192.168.100.175:8778/jolokia/: dial tcp 192.168.100.175:8778: getsockopt: connection refused
EDIT 1:
I have checked my CLASSPATH and both jolokia-client and jolokia-core were listed: ../lib/jolokia-client-java-1.3.7.jar:../lib/jolokia-core-1.3.7.jar.
EDIT 2:
I have put following lines into my app's execution file:
JOLOKIA_OPTS=-javaagent:$LIB_PATH/jolokia-core-java-1.3.7.jar=port=8778,host=0.0.0.0
JAVA_OPTS="-mx4096M $JAVA_OPTS $JACOCO_OPTS $JOLOKIA_OPTS"
But when I run the file, I get this error (even though ../lib/jolokia-core-java-1.3.7.jar has been listed in the CLASSPATH):
Error opening zip file or JAR manifest missing : ../lib/jolokia-core-java-1.3.7.jar
Error occurred during initialization of VM
agent library failed to init: instrument
Found the solution.
I have skipped maven solution and tried the javaagent approach, but I had misunderstood the usage of javaagent previously; I should address jolokia jvm agent (this helped):
JOLOKIA_OPTS=-javaagent:/root/jolokia-jvm-1.3.7-agent.jar=port=8778,host=0.0.0.0
JAVA_OPTS="-mx4096M $JAVA_OPTS $JACOCO_OPTS $JOLOKIA_OPTS"
Now, my app starts with this log (successfully):
I> No access restrictor found, access to any MBean is allowed
Jolokia: Agent started with URL http://192.168.100.175:8778/jolokia/
In the other side there is no error in the telegraf console for jolokia anymore.
All the observation are implying the jolokia jvm library has been started and works successfully.
I have also found jolokia jmx documentation for using it as dependencies in the project; but since I'm not a JAVA expert (I'm testing the app), I prefer to currently use the javaagent approach and leave it for future study/experience. BTW, it may help the others.
EDIT 1:
I have found and deployed jolokia jvm agent using its spring support.
Configuring it in the spring XML file, I can now have jolokia jvm agent start listening at my app's startup.
I am writing a complemental service for OpenAM for some features not available as RESTful services in default server. I am using OpenAM Client SDK (12 or 13). I get the folloing error:
DebugConfiguration:07/03/2017 04:13:12:530 PM IRDT:
Thread[main,5,main]
'/debugconfig.properties' isn't valid, the default configuration will be used instead: Can't find the configuration file
'/debugconfig.properties'.
amAuthContext:07/03/2017 04:13:12:564 PM IRDT: Thread[main,5,main]:
TransactionId[unknown]
ERROR: Failed to obtain auth service url from server: null://null:null
amNaming:07/03/2017 04:13:12:573 PM IRDT: Thread[main,5,main]:
TransactionId[unknown]
ERROR: Failed to initialize naming service
java.lang.Exception: Cannot find Naming Service URL.
at com.iplanet.services.naming.WebtopNaming.getNamingServiceURL(WebtopNaming.java:1254)
at com.iplanet.services.naming.WebtopNaming.initializeNamingService(WebtopNaming.java:272)
at com.iplanet.services.naming.WebtopNaming.updateNamingTable(WebtopNaming.java:1149)
at com.iplanet.services.naming.WebtopNaming.getNamingProfile(WebtopNaming.java:1070)
at com.iplanet.services.naming.WebtopNaming.getServiceAllURLs(WebtopNaming.java:494)
at com.sun.identity.authentication.AuthContext.login(AuthContext.java:654)
at com.sun.identity.authentication.AuthContext.login(AuthContext.java:584)
at com.sun.identity.authentication.AuthContext.login(AuthContext.java:386)
at MainKt.realmLogin(Main.kt:56)
at MainKt.main(Main.kt:144)
IdRepoSampleUtils: Failed to start login for default authmodule
Exception in thread "main"
com.sun.identity.authentication.spi.AuthLoginException: Failed to create new Authentication Context: null
at com.sun.identity.authentication.AuthContext.login(AuthContext.java:657)
at com.sun.identity.authentication.AuthContext.login(AuthContext.java:584)
at com.sun.identity.authentication.AuthContext.login(AuthContext.java:386)
at MainKt.realmLogin(Main.kt:56)
at MainKt.main(Main.kt:144)
The main error is SDk does not find STS server url. How can I fix it?
I found the solution with checking the Example SDK client. The solution is to use Java's well-knwon properties file. There is a AMConfig.properties in there which the SDK jar automatically tries to extract values from it. For the format of the file we can refer to Oracle OpenSSO, and use the AMConfig.properties.template within the OpenAM example client application.
I have WebSphere 8.5.5.9 and want to run Jenkins on it, because of that, I have installed jenkins.war (the last version 2.19.2), but when I try to start it, I get the error bellow and the application(Jenkins) doesn't start.
OS: Red Hat Enterprise Linux Server release 6.7 (Santiago) - 64 bits
[11/14/16 14:34:55:824 EST] 0000007c BootFailure E Failed to initialize Jenkins
hudson.util.HudsonFailedToLoad: java.lang.NullPointerException
at hudson.WebAppMain$3.run(WebAppMain.java:248)
Caused by: java.lang.NullPointerException
at com.ibm.ws.webcontainer.webapp.WebApp.getInitParameter(WebApp.java:2791)
at com.ibm.wsspi.webcontainer.facade.ServletContextFacade.getInitParameter(ServletContextFacade.java:264)
at jenkins.util.SystemProperties.tryGetValueFromContext(SystemProperties.java:322)
at jenkins.util.SystemProperties.getString(SystemProperties.java:121)
at jenkins.util.SystemProperties.getInteger(SystemProperties.java:258)
at jenkins.model.Jenkins.<init>(Jenkins.java:599)
at hudson.model.Hudson.<init>(Hudson.java:85)
at hudson.model.Hudson.<init>(Hudson.java:81)
at hudson.WebAppMain$3.run(WebAppMain.java:231)
Does somebody have any idea what could be the rouse cause?
Thanks
Looking into other log from WebSphere I found an erro like this:
restricted programmatic session cookie
looking new solutions to this new error, I found a solution:
Just remove on item from programmatic session cookie configuration
Cookie Domain:*
Cookie Name: "JSESSIONID"
Cookie Path: /
from:
Security > Global security > Programmatic session cookie configuration.
After that, I could start Jenkins with success.