This same question was asked but not with jenkins in a Docker container
Jenkins: allow local checkout
I had jenkins running in a docker container and was able to execute pipeline jobs with no problem.
After installing the container recently on a new machine I'm getting:
ERROR: Checkout of Git remote 'file:///usr/src/' aborted because it references a local directory, which may be insecure. You can allow local checkouts anyway by setting the system property 'hudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT' to true.
ERROR: Maximum checkout retry attempts reached, aborting
Finished: FAILURE
I tried this script in init.d
import jenkins.model.Jenkins
import java.util.logging.LogManager
/* Jenkins home directory */
def jenkinsHome = Jenkins.instance.getRootDir().absolutePath
def logger = LogManager.getLogManager().getLogger("")
/* Replace the Key and value with the values you want to set.*/
/* System.setProperty(key, value) */
System.setProperty("hudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT", "true")
logger.info("Jenkins Startup Script: set GitSCM.ALLOW_LOCAL_CHECKOUT to true . Script location : ${jenkinsHome}/init.groovy.d ")
I tried this inside the docker container
jenkins#3d5e0ebf919e:/$ java -D"hudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true" -jar /usr/share/jenkins/jenkins.war
Running from: /usr/share/jenkins/jenkins.warns.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true" -jar /usr/share/jenkins/jenkins.war
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
2022-05-28 20:26:27.644+0000 [id=1] INFO org.eclipse.jetty.util.log.Log#initialized: Logging initialized #758ms to org.eclipse.jetty.util.log.JavaUtilLog
2022-05-28 20:26:27.744+0000 [id=1] INFO winstone.Logger#logInternal: Beginning extraction from war file
2022-05-28 20:26:27.792+0000 [id=1] WARNING o.e.j.s.handler.ContextHandler#setContextPath: Empty contextPath
2022-05-28 20:26:27.876+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 11.0.15+10
2022-05-28 20:26:28.162+0000 [id=1] INFO o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
2022-05-28 20:26:28.204+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: DefaultSessionIdManager workerName=node0
2022-05-28 20:26:28.204+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: No SessionScavenger set, using defaults
2022-05-28 20:26:28.205+0000 [id=1] INFO o.e.j.server.session.HouseKeeper#startScavenging: node0 Scavenging every 600000ms
2022-05-28 20:26:28.694+0000 [id=1] INFO hudson.WebAppMain#contextInitialized: Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
2022-05-28 20:26:28.869+0000 [id=1] INFO o.e.j.s.handler.ContextHandler#doStart: Started w.#36c54a56{Jenkins v2.332.3,/,file:///var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war}
2022-05-28 20:26:28.892+0000 [id=1] INFO o.e.j.server.AbstractConnector#doStop: Stopped ServerConnector#206a70ef{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2022-05-28 20:26:28.893+0000 [id=1] INFO o.e.j.server.session.HouseKeeper#stopScavenging: node0 Stopped scavenging
2022-05-28 20:26:28.901+0000 [id=1] INFO hudson.WebAppMain#contextDestroyed: Shutting down a Jenkins instance that was still starting up
java.lang.Throwable: reason
at hudson.WebAppMain.contextDestroyed(WebAppMain.java:386)
at org.eclipse.jetty.server.handler.ContextHandler.callContextDestroyed(ContextHandler.java:1074)
at org.eclipse.jetty.servlet.ServletContextHandler.callContextDestroyed(ServletContextHandler.java:584)
at org.eclipse.jetty.server.handler.ContextHandler.contextDestroyed(ContextHandler.java:1037)
at org.eclipse.jetty.servlet.ServletHandler.doStop(ServletHandler.java:319)
at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:94)
at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:180)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:201)
at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:108)
at org.eclipse.jetty.security.SecurityHandler.doStop(SecurityHandler.java:437)
at org.eclipse.jetty.security.ConstraintSecurityHandler.doStop(ConstraintSecurityHandler.java:423)
at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:94)
at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:180)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:201)
at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:108)
at org.eclipse.jetty.server.session.SessionHandler.doStop(SessionHandler.java:520)
at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:94)
at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:180)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:201)
at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:108)
at org.eclipse.jetty.server.handler.ContextHandler.stopContext(ContextHandler.java:1060)
at org.eclipse.jetty.servlet.ServletContextHandler.stopContext(ServletContextHandler.java:386)
at org.eclipse.jetty.webapp.WebAppContext.stopWebapp(WebAppContext.java:1454)
at org.eclipse.jetty.webapp.WebAppContext.stopContext(WebAppContext.java:1420)
at org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:1114)
at org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:297)
at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:547)
at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:94)
at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:180)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:201)
at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:108)
at org.eclipse.jetty.server.Server.doStop(Server.java:470)
at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:94)
at winstone.Launcher.shutdown(Launcher.java:318)
at winstone.Launcher.<init>(Launcher.java:205)
at winstone.Launcher.main(Launcher.java:369)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at Main._main(Main.java:304)
at Main.main(Main.java:108)
2022-05-28 20:26:28.905+0000 [id=1] INFO o.e.j.s.handler.ContextHandler#doStop: Stopped w.#36c54a56{Jenkins v2.332.3,/,null,STOPPED}{/var/jenkins_home/war}
Exception in thread "Jenkins initialization thread" java.lang.NoClassDefFoundError: hudson/util/HudsonFailedToLoad
at hudson.WebAppMain$3.run(WebAppMain.java:261)
Caused by: java.lang.ClassNotFoundException: hudson.util.HudsonFailedToLoad
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:538)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 1 more
2022-05-28 20:26:28.908+0000 [id=1] INFO winstone.Logger#logInternal: Jetty shutdown successfully
java.io.IOException: Failed to start Jetty
at winstone.Launcher.<init>(Launcher.java:194)
at winstone.Launcher.main(Launcher.java:369)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at Main._main(Main.java:304)
at Main.main(Main.java:108)
Caused by: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8080
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:349)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:310)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.eclipse.jetty.server.Server.doStart(Server.java:401)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at winstone.Launcher.<init>(Launcher.java:192)
... 7 more
Caused by: java.net.BindException: Address already in use
at java.base/sun.nio.ch.Net.bind0(Native Method)
at java.base/sun.nio.ch.Net.bind(Net.java:459)
at java.base/sun.nio.ch.Net.bind(Net.java:448)
at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:344)
... 14 more
2022-05-28 20:26:28.909+0000 [id=1] SEVERE winstone.Logger#logInternal: Container startup failed
java.net.BindException: Address already in use
at java.base/sun.nio.ch.Net.bind0(Native Method)
at java.base/sun.nio.ch.Net.bind(Net.java:459)
at java.base/sun.nio.ch.Net.bind(Net.java:448)
at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:344)
Caused: java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:8080
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:349)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:310)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.eclipse.jetty.server.Server.doStart(Server.java:401)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at winstone.Launcher.<init>(Launcher.java:192)
Caused: java.io.IOException: Failed to start Jetty
at winstone.Launcher.<init>(Launcher.java:194)
at winstone.Launcher.main(Launcher.java:369)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at Main._main(Main.java:304)
at Main.main(Main.java:108)
In jenkins system properties i see hudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT true
Still the problem persists
Running Jenkins 2.332.3
Dockerfile:
FROM jenkins/jenkins:lts
ARG user
ARG password
ARG git_password
ARG branch
USER jenkins
RUN /usr/local/bin/install-plugins.sh \
cloudbees-folder \
antisamy-markup-formatter \
build-timeout \
credentials-binding \
timestamper \
ws-cleanup \
ant \
gradle \
workflow-aggregator \
github-organization-folder \
pipeline-stage-view \
git \
subversion \
ssh-slaves \
matrix-auth \
pam-auth \
ldap \
email-ext \
mailer \
ssh \
build-user-vars-plugin \
yet-another-build-visualizer \
rebuild
ENV JENKINS_USER ${user}
ENV JENKINS_PASS ${password}
ENV GIT_PASS ${git_password}
ENV CURRENT_BRANCH ${branch}
ENV JAVA_OPTS -Dhudson,plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true
ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false
COPY init.groovy.d/dev/*.groovy /usr/share/jenkins/ref/init.groovy.d/
VOLUME /var/jenkins_home
Still the problem exists:
Any ideas to fix this?
Any way to make my git directory not "a local directory"?
I just stumbled on this problem too. Based on this answer, you just need to add the env variable when starting the container.
docker run ... --env JAVA_OPTS="-Dhudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true"
The answer for me was in the init.groovy.d/dev/jobs.groovy file
After this line:
def scm = new GitSCM("file:///usr/src/");
I added this line
scm.ALLOW_LOCAL_CHECKOUT=true
Your ENV JAVA_OPTS are overwriting each other. Try putting them on the same line.
ENV JAVA_OPTS -Dhudson,plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true -Djenkins.install.runSetupWizard=false
The reason I say that is I added the below to my dockerfile and it works great.
ENV JAVA_OPTS -Dhudson,plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true
Related
I'm trying to turn my jenkins server to HTTPS.
I got the pfx file, and I created from it the jks file.
Afterward I've updated my jenkins.config file with the below line:
<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\jenkins.war" --httpPort=-1 --httpsPort=8080 --httpsKeyStore="C:\Users\aman\.jenkins\clientcert.jks" --httpsKeyStorePassword="######" --webroot="%BASE%\war"</arguments>
but getting this exception when trying to start jenkins:
java.io.IOException: Failed to start a listener: winstone.HttpsConnectorFactory Caused by: winstone.WinstoneException: Error getting the SSL context object Caused by: java.security.UnrecoverableKeyException: Cannot recover key 2022-08-17 18:44:56.770+0000 [id=1] SEVERE winstone.Logger#logInternal: Container startup failed
I tried to google but couldn't find any solution.
some more details:
OS - Windows server 2019
Jenkins version: 2.346.3 LTS
Installation: thought war file
java version: java 8u333
The httpsKeyStorePassword is the same password I wrote in the destination keystore password while running this command:
keytool -importkeystore -srckeystore my_cert.pfx -srcstoretype pkcs12 -destkeystore clientcert.jks -deststoretype JKS
I have deployed k8s cluster with 3 nodes, deployed hdfs.I've written a simple pyspark script and want to deploy it on k8s cluster, but dont know how to initialize spark context correctly: what to need to pass as a master to SparkConf().setMaster ??(When i set master as k8s://https://172.20.234.174:6443 i'm getting errors)
The command i'm using to deploy on k8s:
bin/spark-submit \
--name spark_k8s_hello_world_0 \
--master k8s://https://172.20.234.174:6443 \
--deploy-mode cluster \
--conf spark.executor.instances=2 \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.kubernetes.container.image=semenchukou/pyspark-k8s-example:0.1 \
--conf spark.kubernetes.pyspark.pythonVersion=3 \
local:///app/HelloWorldSpark.py
UPD
current script:
#!/usr/bin/env python
from pyspark import SparkContext
from pyspark import SparkConf
if __name__ == '__main__':
conf = SparkConf()
sc = SparkContext(conf = conf)
txt = sc.textFile('hdfs://172.20.234.174:1515/testing/testFile.txt')
first = txt.first()
sc.parallelize(first).saveAsTextFile('hdfs://172.20.234.174:9000/testing/result.txt')
I'm running the following command from a master machine in cluster:
bin/spark-submit \
--name spark_k8s_hello_world_0 \
--master k8s://https://172.20.234.174:6443 \
--deploy-mode cluster \
--conf spark.executor.instances=2 \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.kubernetes.container.image=semenchukou/pyspark-k8s-example:conf \
--conf spark.kubernetes.pyspark.pythonVersion=3 \
local:///app/HelloWorldSpark.py
and get following error stacktrace:
File "/app/HelloWorldSpark.py", line 8, in <module>
sc = SparkContext(conf = conf)
File "/opt/spark/python/lib/pyspark.zip/pyspark/context.py", line 136, in __init__
File "/opt/spark/python/lib/pyspark.zip/pyspark/context.py", line 198, in _do_init
File "/opt/spark/python/lib/pyspark.zip/pyspark/context.py", line 306, in _initialize_context
File "/opt/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1525, in __call__
File "/opt/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 328, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext.
: org.apache.spark.SparkException: External scheduler cannot be instantiated
at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2794)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:493)
at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:238)
at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)
at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [get] for kind: [Pod] with name: [sparkk8shelloworld0-1580920409707-driver] in namespace: [default] failed.
at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64)
at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:229)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:185)
at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator$$anonfun$1.apply(ExecutorPodsAllocator.scala:57)
at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator$$anonfun$1.apply(ExecutorPodsAllocator.scala:55)
at scala.Option.map(Option.scala:146)
at org.apache.spark.scheduler.cluster.k8s.ExecutorPodsAllocator.<init>(ExecutorPodsAllocator.scala:55)
at org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:89)
at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2788)
... 13 more
Caused by: java.net.UnknownHostException: kubernetes.default.svc: Try again
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
at java.net.InetAddress.getAllByName(InetAddress.java:1193)
at java.net.InetAddress.getAllByName(InetAddress.java:1127)
at okhttp3.Dns$1.lookup(Dns.java:39)
at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:171)
at okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:137)
at okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:82)
at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:171)
at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121)
at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:100)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:119)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at io.fabric8.kubernetes.client.utils.HttpClientUtils$2.intercept(HttpClientUtils.java:107)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185)
at okhttp3.RealCall.execute(RealCall.java:69)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:379)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:344)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:313)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:296)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:801)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:218)
... 20 more
20/02/05 16:33:55 INFO ShutdownHookManager: Shutdown hook called
20/02/05 16:33:55 INFO ShutdownHookManager: Deleting directory /tmp/spark-1399e509-6729-436d-9355-eecec6e58113
20/02/05 16:33:55 INFO ShutdownHookManager: Deleting directory /var/data/spark-3355ab9d-38f1-4083-b7af-5fd03dc1ae2f/spark-2a479391-3a9d-4f5b-93a0-707132c802cc
you shouldn't set anything with SparkConf().setMaster() in the code, master config value is propagated automatically from spark-submit.
While executing spark-submit --master ... you can pass:
Kubernetes API endpoint got from kubectl cluster-info with appended leading k8s://, eg.: k8s://https://xxx.xxx.xxx.xxx:443
Kubernetes API internal endpoint k8s://kubernetes.default.svc.cluster.local:443 if you are running spark-submit within the Kubernetes cluster network
Also please refer the official docs and try to run the Spark PI example first.
Hope it helps.
UPDATE 1:
Spark Driver uses another property spark.kubernetes.driver.master for setting Kubernetes client URL, when running in cluster mode, which defaults to https://kubernetes.default.svc (which is the default internal Kubernetes API endpoint, Kubernetes cluster by default has Service named kubernetes in default namespace): ref1, ref2.
In your case you may try to set additional --conf spark.kubernetes.driver.master=https://172.20.234.174:6443.
Also I would advice you to check if the Service named kubernetes exists in default namespace of your cluster, and it should also expose port 443. If so - there might be problems with DNS resolution in your cluster, which is another topic I guess.
UPDATE 2:
The config option described above is not yet available and introduced in PR [SPARK-30371]. Before that Spark Driver always uses https://kubernetes.default.svc:443 to call Kubernetes APIs. If you cannot resolve this address within the cluster - probably you have some problems with DNS or cluster setup.
I've installed Jenkins, based on Jetty server on Centos 7.
In the configuration file : /etc/sysconfig/jenkins default port is 8080, and everything works perfectly, but I need to move jenkins on 80 port.
When I change JENKINS_PORT="8080" to JENKINS_PORT="80" - jenkins no longer available. Firewall I turned off.
in log messages: /var/log/jenkins/jenkins.log
Aug 17, 2017 12:07:45 PM org.eclipse.jetty.util.log.JavaUtilLog warn
WARNING: FAILED ServerConnector#4082ba93{HTTP/1.1}{0.0.0.0:80}: java.net.SocketException: Permission denied
java.net.SocketException: Permission denied
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at winstone.Launcher.<init>(Launcher.java:152)
at winstone.Launcher.main(Launcher.java:352)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at Main._main(Main.java:293)
at Main.main(Main.java:132)
Can anyone help me with that problem?
Google led me to https://confluence.atlassian.com/confkb/permission-denied-error-when-binding-a-port-290750651.html
Ports below 1024 are called Privileged Ports and in Linux (and most UNIX flavors and UNIX-like systems), they are not allowed to be opened by any non-root user.
So as I run startup script as JENKINS_USER="jenkins" the Permission denied error has appeared.
As #Nimda answered https://stackoverflow.com/a/45754065/3471694
I got the same issue below
SEVERE: Container startup failed
java.io.IOException: Failed to start Jetty
at winstone.Launcher.<init>(Launcher.java:186)
at winstone.Launcher.main(Launcher.java:354)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at Main._main(Main.java:344)
at Main.main(Main.java:160)
Caused by: java.net.SocketException: Permission denied
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:339)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:395)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at winstone.Launcher.<init>(Launcher.java:184)
... 7 more
Solution: Edit file /etc/sysconfig/jenkins or any jenkins' configuration file
- Change #JENKINS_USER="jenkins" to any user that can run as root
example: JENKINS_USER="root"
Hope this help
I struggled for hours with this problem. What i did to solve it eventually was, on Ubuntu, I changed the JENKINS_USER variable value from $NAME to root in /etc/default/jenkins. Then, I did a daemon reload by running the command below.
$ systemctl daemon-reload
Finally, I restarted jenkins as root.
$ service jenkins restart
80 is not accessible to non-root user and jenkins is non-root user. I have changes port number and then it worked for me.
Could you try this Centos7 firewall setting?
Don't need to turn off firewall.
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-service=http --permanent
firewall-cmd --reload
firewall-cmd --list-all
Then starting Jenkins.
When trying to execute the example from https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki/Controlling-your-build-environment:
pipeline {
agent {
docker 'node'
}
stages {
stage("testing 123") {
steps {
sh 'node --version'
}
}
}
}
I get the following output:
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Agent Setup)
[Pipeline] sh
[XXXXX-ABOESSRH5FV5AM3VLFMZ4UZP722N63WT5EEAE2JPIXC5U5ZTMBYA] Running shell script
+ docker pull node
Using default tag: latest
latest: Pulling from library/node
Digest: sha256:a72f8cd9aba12ea3a19ada91e077c4d8822d3bd7dc3c4707b16630e5c2477845
Status: Image is up to date for node:latest
[Pipeline] }
[Pipeline] // stage
[Pipeline] sh
[XXXXX-ABOESSRH5FV5AM3VLFMZ4UZP722N63WT5EEAE2JPIXC5U5ZTMBYA] Running shell script
+ docker inspect -f . node
.
[Pipeline] withDockerContainer
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
GitHub has been notified of this commit’s build result
java.io.IOException: Cannot run program "docker": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at hudson.Proc$LocalProc.<init>(Proc.java:245)
at hudson.Proc$LocalProc.<init>(Proc.java:214)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:846)
at hudson.Launcher$ProcStarter.start(Launcher.java:384)
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:268)
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:249)
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:246)
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.version(DockerClient.java:218)
at org.jenkinsci.plugins.docker.workflow.WithContainerStep$Execution.start(WithContainerStep.java:136)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:184)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:126)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:18)
at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(jar:file:/Users/Shared/Jenkins/Home/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:128)
at org.jenkinsci.plugins.docker.workflow.Docker.node(jar:file:/Users/Shared/Jenkins/Home/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:63)
at org.jenkinsci.plugins.docker.workflow.Docker$Image.inside(jar:file:/Users/Shared/Jenkins/Home/plugins/docker-workflow/WEB-INF/lib/docker-workflow.jar!/org/jenkinsci/plugins/docker/workflow/Docker.groovy:116)
at org.jenkinsci.plugins.pipeline.modeldefinition.agent.impl.DockerPipelineScript.runImage(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/DockerPipelineScript.groovy:54)
at org.jenkinsci.plugins.pipeline.modeldefinition.agent.impl.AbstractDockerPipelineScript.configureRegistry(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/AbstractDockerPipelineScript.groovy:68)
at org.jenkinsci.plugins.pipeline.modeldefinition.agent.impl.AbstractDockerPipelineScript.run(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/AbstractDockerPipelineScript.groovy:54)
at org.jenkinsci.plugins.pipeline.modeldefinition.agent.CheckoutScript.checkoutAndRun(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-extensions/WEB-INF/lib/pipeline-model-extensions.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/agent/CheckoutScript.groovy:57)
at org.jenkinsci.plugins.pipeline.modeldefinition.agent.CheckoutScript.doCheckout(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-extensions/WEB-INF/lib/pipeline-model-extensions.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/agent/CheckoutScript.groovy:40)
at org.jenkinsci.plugins.pipeline.modeldefinition.agent.impl.LabelScript.run(jar:file:/Users/Shared/Jenkins/Home/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/agent/impl/LabelScript.groovy:44)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
at sun.reflect.GeneratedMethodAccessor268.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
at com.cloudbees.groovy.cps.Next.step(Next.java:74)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:165)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:328)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:80)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:240)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:228)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
at hudson.Proc$LocalProc.<init>(Proc.java:245)
at hudson.Proc$LocalProc.<init>(Proc.java:214)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:846)
at hudson.Launcher$ProcStarter.start(Launcher.java:384)
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:268)
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:249)
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.launch(DockerClient.java:246)
at org.jenkinsci.plugins.docker.workflow.client.DockerClient.version(DockerClient.java:218)
at org.jenkinsci.plugins.docker.workflow.WithContainerStep$Execution.start(WithContainerStep.java:136)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:184)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:126)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:18)
... 29 more
Finished: FAILURE
docker is installed on the server, and is in the PATH: the image is built :-) (docker pull command succeed at the beginning of the pipeline)
I added a logger on org.jenkinsci.plugins.docker.workflow.client.DockerClient with the following result:
Apr 07, 2017 3:56:18 PM FINE org.jenkinsci.plugins.docker.workflow.client.DockerClient
Executing docker command docker -v
I also tried launching a docker in a stage with a regular agent, same issue:
pipeline {
agent any
stages {
stage("testing 123") {
agent {
docker {
image 'ubuntu:16.04'
}
}
steps {
sh 'uname -a'
}
}
}
}
using Docker.Pipeline 1.10
This seems to be related to a PATH configuration issue on the node, but I can't find where.
If I run org.jenkinsci.plugins.docker.commons.tools.DockerTool.getExecutable("docker", Jenkins.getActiveInstance(), null, null) in the node console, I get the correct (existing, executable) docker path.
any idea ?
Thanks a lot!
You can run a shell command that is something like sh "docker 'node'"
What I've done is used a Jenkinsfile and Dockerfile (since I edit my images) and have the Jenkinsfile call a script that has
docker build --tag=image-name .
docker run --rm --name=image-name-container image-name \
./someScript.sh \
-j ${SCRIPT_PARAM1} \
-u ${SCRIPT_PARAM2} > ${DOCKER_OUTPUT_FILE_NAME}
inside of it.
The reason why you are facing this is that as per your Jenkins file you are not able to tell Jenkins that where node is? and that's why giving such exception,I can suggest some other way like but you can do is install node on your Jenkins master, configure the path in the configuration system by installing node plugin.Then, in the Jenkins file add below tools section at the top:
tools {
node 'name-configured-in-the-configuration-system'
}
In this way Jenkins will know that where your node is installed and would execute the command given in the stage.
To sperate stages and each stage its own container you can use the following style.
node() {
checkout scm
def String dockerImage = 'dockerImageName'
def String dockerArgs = "-v ${WORKSPACE}/test:/tmp/test"
stage ('Build') {
docker.image(dockerImage).inside(dockerArgs) {
echo 'Build them'
}
}
stage('Test') {
docker.image(dockerImage).inside(dockerArgs) {
sh 'node --version'
}
}
}
If nodejs is installed in your docker image then without any issue it should work
Adding ~/.zshenv for zshell (or other file) to put docker in the path. I did this and problem resolved:
$ cat ~/.zshenv
export PATH=/usr/local/bin:$PATH
It looks like the docker inspect not run in the same env as docker pull.
On my Mac, the docker pull failed, then I configured node/agent to add such Name: PATH, Value: /usr/local/bin:$PATH to the Environment variables, then the docker pull will run normally. And docker inspect failed.
Then I add ~/.zshenv as above (and restart node/agent), the docker inspect worked finally.
I am trying to run docker image of sonarqube with mysql db by below dockercommand:
sudo docker run -d --name hg-sonarqube \
-p 9000:9000 \
-e SONARQUBE_JDBC_USERNAME='sonar' \
-e SONARQUBE_JDBC_PASSWORD='sonar' \
-e SONARQUBE_JDBC_URL='jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance' \
sonarqube
But container is not running due to error:
2016.12.28 11:20:11 INFO web[][o.sonar.db.Database] Create JDBC data source for jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance
2016.12.28 11:20:11 ERROR web[][o.a.c.c.C.[.[.[/]] Exception sending context initialized event to listener instance of class org.sonar.server.platform.web.PlatformServletContextListener
java.lang.IllegalStateException: Can not connect to database. Please check connectivity and settings (see the properties prefixed by 'sonar.jdbc.').
at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:108)
MySQL service is running and sonar database. I use command to create database and give privileges in Ubuntu-14.04.
echo "GRANT ALL PRIVILEGES ON *.* TO 'root'#'%' IDENTIFIED BY 'welcome123'; flush privileges;" | mysql -u root -pwelcome123
echo "CREATE DATABASE sonar CHARACTER SET utf8 COLLATE utf8_general_ci; CREATE USER 'sonar' IDENTIFIED BY 'sonar';GRANT ALL PRIVILEGES ON sonar.* TO 'sonar'#'%' IDENTIFIED BY 'sonar'; GRANT ALL ON sonar.* TO 'sonar'#'localhost' IDENTIFIED BY 'sonar'; flush privileges;" | mysql -u root -pwelcome123
Full Log file:
2016.12.28 11:19:58 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
2016.12.28 11:19:58 INFO app[][o.s.p.m.JavaProcessLauncher] Launch process[es]: /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Djava.awt.headless=true -Xmx1G -Xms256m -Xss256k -Djna.nosys=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Djava.io.tmpdir=/opt/sonarqube/temp -javaagent:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/management-agent.jar -cp ./lib/common/*:./lib/search/* org.sonar.search.SearchServer /opt/sonarqube/temp/sq-process5713024831851311243properties
2016.12.28 11:19:59 INFO es[][o.s.p.ProcessEntryPoint] Starting es
2016.12.28 11:19:59 INFO es[][o.s.s.EsSettings] Elasticsearch listening on /127.0.0.1:9001
2016.12.28 11:19:59 INFO es[][o.elasticsearch.node] [sonarqube] version[2.3.5], pid[18], build[90f439f/2016-07-27T10:36:52Z]
2016.12.28 11:19:59 INFO es[][o.elasticsearch.node] [sonarqube] initializing ...
2016.12.28 11:19:59 INFO es[][o.e.plugins] [sonarqube] modules [], plugins [], sites []
2016.12.28 11:19:59 INFO es[][o.elasticsearch.env] [sonarqube] using [1] data paths, mounts [[/opt/sonarqube/data (/dev/sda1)]], net usable_space [24.2gb], net total_space [28.8gb], spins? [possibly], types [ext4]
2016.12.28 11:19:59 INFO es[][o.elasticsearch.env] [sonarqube] heap size [1007.3mb], compressed ordinary object pointers [true]
2016.12.28 11:20:03 INFO es[][o.elasticsearch.node] [sonarqube] initialized
2016.12.28 11:20:03 INFO es[][o.elasticsearch.node] [sonarqube] starting ...
2016.12.28 11:20:03 INFO es[][o.e.transport] [sonarqube] publish_address {127.0.0.1:9001}, bound_addresses {127.0.0.1:9001}
2016.12.28 11:20:03 INFO es[][o.e.discovery] [sonarqube] sonarqube/CPgnfx6NTe2aO07d6fR0Bg
2016.12.28 11:20:06 INFO es[][o.e.cluster.service] [sonarqube] new_master {sonarqube}{CPgnfx6NTe2aO07d6fR0Bg}{127.0.0.1}{127.0.0.1:9001}{rack_id=sonarqube, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
2016.12.28 11:20:06 INFO es[][o.elasticsearch.node] [sonarqube] started
2016.12.28 11:20:06 INFO es[][o.e.gateway] [sonarqube] recovered [0] indices into cluster_state
2016.12.28 11:20:06 INFO app[][o.s.p.m.Monitor] Process[es] is up
2016.12.28 11:20:06 INFO app[][o.s.p.m.JavaProcessLauncher] Launch process[web]: /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djruby.management.enabled=false -Djruby.compile.invokedynamic=false -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom -Djava.io.tmpdir=/opt/sonarqube/temp -javaagent:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/management-agent.jar -cp ./lib/common/*:./lib/server/*:/opt/sonarqube/lib/jdbc/mysql/mysql-connector-java-5.1.39.jar org.sonar.server.app.WebServer /opt/sonarqube/temp/sq-process6242669754365841464properties
2016.12.28 11:20:08 INFO web[][o.s.p.ProcessEntryPoint] Starting web
2016.12.28 11:20:08 INFO web[][o.s.s.a.TomcatContexts] Webapp directory: /opt/sonarqube/web
2016.12.28 11:20:08 INFO web[][o.a.c.h.Http11NioProtocol] Initializing ProtocolHandler ["http-nio-0.0.0.0-9000"]
2016.12.28 11:20:08 INFO web[][o.a.t.u.n.NioSelectorPool] Using a shared selector for servlet write/read
2016.12.28 11:20:09 INFO web[][o.e.plugins] [Bushwacker] modules [], plugins [], sites []
2016.12.28 11:20:11 INFO web[][o.s.s.e.EsClientProvider] Connected to local Elasticsearch: [127.0.0.1:9001]
2016.12.28 11:20:11 INFO web[][o.s.s.p.LogServerVersion] SonarQube Server / 6.2 / 4a28f29f95254b58f3cf0a0871bc632e998403f5
2016.12.28 11:20:11 INFO web[][o.sonar.db.Database] Create JDBC data source for jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance
2016.12.28 11:20:11 ERROR web[][o.a.c.c.C.[.[.[/]] Exception sending context initialized event to listener instance of class org.sonar.server.platform.web.PlatformServletContextListener
java.lang.IllegalStateException: Can not connect to database. Please check connectivity and settings (see the properties prefixed by 'sonar.jdbc.').
at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:108)
at org.sonar.db.DefaultDatabase.start(DefaultDatabase.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.invokeMethod(ReflectionLifecycleStrategy.java:110)
at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.start(ReflectionLifecycleStrategy.java:89)
at org.sonar.core.platform.ComponentContainer$1.start(ComponentContainer.java:320)
at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.start(AbstractInjectionFactory.java:84)
at org.picocontainer.behaviors.AbstractBehavior.start(AbstractBehavior.java:169)
at org.picocontainer.behaviors.Stored$RealComponentLifecycle.start(Stored.java:132)
at org.picocontainer.behaviors.Stored.start(Stored.java:110)
at org.picocontainer.DefaultPicoContainer.potentiallyStartAdapter(DefaultPicoContainer.java:1016)
at org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1009)
at org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:767)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:141)
at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:88)
at org.sonar.server.platform.Platform.start(Platform.java:216)
at org.sonar.server.platform.Platform.startLevel1Container(Platform.java:175)
at org.sonar.server.platform.Platform.init(Platform.java:90)
at org.sonar.server.platform.web.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:44)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4812)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5255)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1408)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1398)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
at org.sonar.db.profiling.NullConnectionInterceptor.getConnection(NullConnectionInterceptor.java:31)
at org.sonar.db.profiling.ProfiledDataSource.getConnection(ProfiledDataSource.java:323)
at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:106)
... 30 common frames omitted
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
Might be helpful for those who still facing this issue
You might be running mysql in another container. You can use below docker-compose file to make it in the same network
# Use sonar/sonar as user/password credentials
version: '3.1'
services:
sonarqube:
image: sonarqube:5.1.1
networks:
- sonarqube-network
ports:
- "9000:9000"
- "3306:3306"
environment:
- SONARQUBE_JDBC_USERNAME=sonar
- SONARQUBE_JDBC_PASSWORD=sonar
- SONARQUBE_JDBC_URL=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true
db:
image: mysql
networks:
- sonarqube-network
environment:
- MYSQL_ROOT_PASSWORD=sonar
- MYSQL_DATABASE=sonar
- MYSQL_USER=sonar
- MYSQL_PASSWORD=sonar
networks:
sonarqube-network:
Save the file as docker-compose.yml and run docker-compose up
Please note this entry,
- 3306:3306
After that try with,
mysql -u sonar -h localhost -p
to connect to MySQL