So im trying to run a Keycloak server on my Plesk server in a docker container.
The issue is that my keycloak server dies after a while(or after an action, im not sure because nothing gets logged), all im getting in the log is
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.wildfly.extension.elytron.SSLDefinitions (jar:file:/opt/jboss/keycloak/modules/system/layers/base/org/wildfly/extension/elytron/main/wildfly-elytron-integration-11.1.1.Final.jar!/) to method com.sun.net.ssl.internal.ssl.Provider.isFIPS()
WARNING: Please consider reporting this to the maintainers of org.wildfly.extension.elytron.SSLDefinitions
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
*** JBossAS process (340) received KILL signal ***
I have the required env setting PROXY_ADDRESS_FORWARDING to true, and im pretty sure i followed the instructions properly on https://github.com/keycloak/keycloak-containers/blob/10.0.2/server/README.md
I have tried with and without TLS settings and it still just dies after about 5 min
here is some extra log dump after setting it to extreme verbose logging
[0m16:51:30,688 TRACE [org.jboss.modules] (ServerService Thread Pool -- 63) Attempting to find all resources META-INF/jpa-changelog-1.2.0.Beta1-db2.xml in Module "org.keycloak.keycloak-model-jpa" version 10.0.2 from local module loader #797cf65c (finder: local module finder #31bcf236 (roots: /opt/jboss/keycloak/modules,/opt/jboss/keycloak/modules/system/layers/keycloak,/opt/jboss/keycloak/modules/system/layers/base))
[0m16:51:30,705 TRACE [org.jboss.modules] (ServerService Thread Pool -- 63) Attempting to find all resources liquibase/parser/core/xml/dbchangelog-3.1.xsd in Module "org.keycloak.keycloak-model-jpa" version 10.0.2 from local module loader #797cf65c (finder: local module finder #31bcf236 (roots: /opt/jboss/keycloak/modules,/opt/jboss/keycloak/modules/system/layers/keycloak,/opt/jboss/keycloak/modules/system/layers/base))
[0m16:51:30,770 TRACE [org.jboss.modules] (ServerService Thread Pool -- 63) Attempting to find all resources META-INF/jpa-changelog-1.2.0.CR1.xml in Module "org.keycloak.keycloak-model-jpa" version 10.0.2 from local module loader #797cf65c (finder: local module finder #31bcf236 (roots: /opt/jboss/keycloak/modules,/opt/jboss/keycloak/modules/system/layers/keycloak,/opt/jboss/keycloak/modules/system/layers/base))
[0m16:51:30,771 TRACE [org.jboss.modules] (ServerService Thread Pool -- 63) Attempting to find all resources liquibase/parser/core/xml/dbchangelog-3.1.xsd in Module "org.keycloak.keycloak-model-jpa" version 10.0.2 from local module loader #797cf65c (finder: local module finder #31bcf236 (roots: /opt/jboss/keycloak/modules,/opt/jboss/keycloak/modules/system/layers/keycloak,/opt/jboss/keycloak/modules/system/layers/base))
[0m16:51:30,882 TRACE [org.jboss.modules] (ServerService Thread Pool -- 63) Attempting to find all resources META-INF/jpa-changelog-1.2.0.CR1-db2.xml in Module "org.keycloak.keycloak-model-jpa" version 10.0.2 from local module loader #797cf65c (finder: local module finder #31bcf236 (roots: /opt/jboss/keycloak/modules,/opt/jboss/keycloak/modules/system/layers/keycloak,/opt/jboss/keycloak/modules/system/layers/base))
[0m16:51:30,904 TRACE [org.jboss.modules] (ServerService Thread Pool -- 63) Attempting to find all resources liquibase/parser/core/xml/dbchangelog-3.1.xsd in Module "org.keycloak.keycloak-model-jpa" version 10.0.2 from local module loader #797cf65c (finder: local module finder #31bcf236 (roots: /opt/jboss/keycloak/modules,/opt/jboss/keycloak/modules/system/layers/keycloak,/opt/jboss/keycloak/modules/system/layers/base))
[0m*** JBossAS process (148) received KILL signal ***
Edit: so def seems to have something to do with Java having an OOM crash
Maybe it is a bit late, as far as am concerned, this error happened to me while using jboss/keycloak v14.0.
Am using it with Plesk and Docker proxy https://<hostname>/auth.
It was working before (was using v13.0)
So, even v13.1 gave me the same error.
Finally, I decided to switch back to jboss/keycloak v13.0.
/!\ - Be ready to loose your data, or map container volume with the host while testing.
Related
I have a ktor app. I works fine when I run it in development mode. I package it in a docker image by copying over what the gradle application plugin provided. That also works fine on my local machine 8 cores. But now the strange part. When I do exactly the same thing on a rented V-Server also running Ubuntu-20.04 like my local system, ktor is incredible slow.
docker-compose logs server:
server | 2021-08-24 08:00:23.337 [main] INFO ktor.application - Autoreload is disabled because the development mode is off.
server | 2021-08-24 08:25:35.048 [main] INFO ktor.application - Autoreload is disabled because the development mode is off.
server | 2021-08-24 09:18:48.246 [main] INFO c.e.e.s.TemplateStore - Starting to parse Sentences
server | 2021-08-24 09:18:48.345 [main] INFO c.e.e.s.TemplateStore - Finished parsing sentences
server | 2021-08-24 09:18:48.346 [main] INFO ktor.application - Responding at http://0.0.0.0:8080
server | 2021-08-24 09:18:48.347 [main] INFO ktor.application - Application started in 3193.32 seconds.
Application started in 3193.32 seconds
The source code can be found here https://github.com/1-alex98/whatisthat . It has a docker-compose.yml defining the whole docker container being started.
Local system 32 gb ram + 8 cores . V-Server 4 gb Ram + 2 cores (htop shows pleinty of resources are free).
I am looking for ideas on what in the world could cause this behavior. Or ways to debug it.
Update:
Seems to read a file forever:
"main" #1 prio=5 os_prio=0 cpu=652.14ms elapsed=173.92s tid=0x00007f01d4016000 nid=0xe runnable [0x00007f01dace6000]
java.lang.Thread.State: RUNNABLE
at java.io.FileInputStream.readBytes(java.base#11.0.12/Native Method)
at java.io.FileInputStream.read(java.base#11.0.12/FileInputStream.java:279)
at java.io.FilterInputStream.read(java.base#11.0.12/FilterInputStream.java:133)
at sun.security.provider.NativePRNG$RandomIO.readFully(java.base#11.0.12/NativePRNG.java:424)
at sun.security.provider.NativePRNG$RandomIO.ensureBufferValid(java.base#11.0.12/NativePRNG.java:526)
at sun.security.provider.NativePRNG$RandomIO.implNextBytes(java.base#11.0.12/NativePRNG.java:545)
- locked <0x00000000c7571158> (a java.lang.Object)
at sun.security.provider.NativePRNG$Blocking.engineNextBytes(java.base#11.0.12/NativePRNG.java:268)
at java.security.SecureRandom.nextBytes(java.base#11.0.12/SecureRandom.java:751)
at kotlin.random.AbstractPlatformRandom.nextBytes(PlatformRandom.kt:47)
at kotlin.random.Random.nextBytes(Random.kt:260)
at com.example.routes.websocket.WebsocketRoutingKt.<clinit>(WebsocketRouting.kt:40)
at com.example.plugins.RoutingKt$routing$1.invoke(Routing.kt:13)
at com.example.plugins.RoutingKt$routing$1.invoke(Routing.kt:11)
at io.ktor.routing.Routing$Feature.install(Routing.kt:106)
at io.ktor.routing.Routing$Feature.install(Routing.kt:88)
at io.ktor.application.ApplicationFeatureKt.install(ApplicationFeature.kt:68)
at io.ktor.routing.RoutingKt.routing(Routing.kt:129)
at com.example.plugins.RoutingKt.routing(Routing.kt:11)
at com.example.ApplicationKt$main$1.invoke(Application.kt:18)
at com.example.ApplicationKt$main$1.invoke(Application.kt:14)
at io.ktor.server.engine.internal.CallableUtilsKt.executeModuleFunction(CallableUtils.kt:50)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$launchModuleByName$1.invoke(ApplicationEngineEnvironmentReloading.kt:317)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$launchModuleByName$1.invoke(ApplicationEngineEnvironmentReloading.kt:316)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartupFor(ApplicationEngineEnvironmentReloading.kt:341)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.launchModuleByName(ApplicationEngineEnvironmentReloading.kt:316)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.access$launchModuleByName(ApplicationEngineEnvironmentReloading.kt:30)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:304)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading$instantiateAndConfigureApplication$1.invoke(ApplicationEngineEnvironmentReloading.kt:295)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.avoidingDoubleStartup(ApplicationEngineEnvironmentReloading.kt:323)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:295)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:136)
at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:268)
at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:174)
at com.example.ApplicationKt.main(Application.kt:21)
at com.example.ApplicationKt.main(Application.kt)
It is a fresh rented server but I guess something is wrong with it
docker-compose being slow and my program not starting seemed to be due to insufficient(not good enough) input to /dev/urandom. Installing https://github.com/smuellerDD/jitterentropy-rngd resolved the problem.
I'm trying to setup tensorflow to use GPU acceleration with WSL 2 running Ubuntu 20.04. I'm following this tutorial and am running into the error seen here. However, when I follow the solution there and try to start docker with sudo service docker start I get told docker is an unrecognized service. However, considering I can access the help menu and whatnot, I know docker is installed. While I can get docker to work with the desktop tool, since it doesn't support Cuda as mentioned in the SO post from earlier, it's not very helpful. It's not really giving me error logs or anything, so please ask if you need more details.
Edit:
Considering the lack of details, here are a list of solutions I've tried to no avail. 1 2 3
Update: I used sudo dockerd to get the container started and tried running the nvidia benchmark container only to be met with
INFO[2020-07-18T21:04:05.875283800-04:00] shim containerd-shim started address=/containerd-shim/021834ef5e5600bdf62a6a9e26dff7ffc1c76dd4ec9dadb9c1fcafb6c88b6e1b.sock debug=false pid=1960
INFO[2020-07-18T21:04:05.899420200-04:00] shim reaped id=70316df254d6b2633c743acb51a26ac2d0520f6f8e2f69b69c4e0624eaac1736
ERRO[2020-07-18T21:04:05.909710600-04:00] stream copy error: reading from a closed fifo
ERRO[2020-07-18T21:04:05.909753500-04:00] stream copy error: reading from a closed fifo
ERRO[2020-07-18T21:04:06.001006700-04:00] 70316df254d6b2633c743acb51a26ac2d0520f6f8e2f69b69c4e0624eaac1736 cleanup: failed to delete container from containerd: no such container
ERRO[2020-07-18T21:04:06.001045100-04:00] Handler for POST /v1.40/containers/70316df254d6b2633c743acb51a26ac2d0520f6f8e2f69b69c4e0624eaac1736/start returned error: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"process_linux.go:432: running prestart hook 0 caused \\\"error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: driver error: failed to process request\\\\n\\\"\"": unknown
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"process_linux.go:432: running prestart hook 0 caused \\\"error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: driver error: failed to process request\\\\n\\\"\"": unknown.
ERRO[0000] error waiting for container: context canceled
Update 2: After installing windows insider and making everything as up to date as possible, I encountered a different error.
Run "nbody -benchmark [-numbodies=<numBodies>]" to measure performance.
-fullscreen (run n-body simulation in fullscreen mode)
-fp64 (use double precision floating point values for simulation)
-hostmem (stores simulation data in host memory)
-benchmark (run benchmark to measure performance)
-numbodies=<N> (number of bodies (>= 1) to run in simulation)
-device=<d> (where d=0,1,2.... for the CUDA device to use)
-numdevices=<i> (where i=(number of CUDA devices > 0) to use for simulation)
-compare (compares simulation results running once on the default GPU and once on the CPU)
-cpu (run n-body simulation on the CPU)
-tipsy=<file.bin> (load a tipsy model file for simulation)
NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.
Error: only 0 Devices available, 1 requested. Exiting.
I have a GTX 970, so I'm not sure why it's not being detected. After running sudo lshw -C display, it was confirmed that my graphics card isn't being detected. I got:
*-display UNCLAIMED
description: 3D controller
product: Microsoft Corporation
vendor: Microsoft Corporation
physical id: 4
bus info: pci#941e:00:00.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: bus_master cap_list
configuration: latency=0
I'm running my neo4j community edition 3.5.5 version with 8GB ram in aws instance.
Initially for few months it ran very fine and got results in millis of time, but now a days it's getting stopping automatically and starting automatically. Sometimes it's not at all starting for hours,even we started it manually also.
Can anyone please help me with this. I'm getting the below logs.
tail -100f /var/log/neo4j/neo4j.log
2019-07-29 13:17:52.570+0000 WARN The client is unauthorized due to authentication failure.
2019-09-04 05:33:52.328+0000 WARN The client is unauthorized due to authentication failure.
2019-10-17 15:18:14.652+0000 INFO Transaction with id 2683388 has been automatically rolled back due to transaction timeout.
nohup: ignoring input
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000006e5400000, 3670016000, 0) failed; error='Cannot allocate memory' (errno=12)
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (mmap) failed to map 3670016000 bytes for committing reserved memory.
An error report file with more information is saved as:
/home/ubuntu/hs_err_pid8965.log
nohup: ignoring input
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000006e5400000, 3670016000, 0) failed; error='Cannot allocate memory' (errno=12)
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (mmap) failed to map 3670016000 bytes for committing reserved memory.
An error report file with more information is saved as:
/home/ubuntu/hs_err_pid9050.log
nohup: ignoring input
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000006e5400000, 3670016000, 0) failed; error='Cannot allocate memory' (errno=12)
2019-10-17 17:14:44.651+0000 INFO Transaction with id 2689294 has been automatically rolled back due to transaction timeout.
this can be because you are running lot of merge operations and dont have proper indices created or try increasing the heap size in config file .
I am new to Jboss world and running JBOSS and getting following error while starting.
=========================================================================
JBoss Bootstrap Environment
JBOSS_HOME: /home/manish/Desktop/ometap/wildfly-10.0.0.Final
JAVA: java
JAVA_OPTS: -server -Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n
=========================================================================
Listening for transport dt_socket at address: 8787
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by __redirected.__SAXParserFactory (file:/home/manish/Desktop/ometap/wildfly-10.0.0.Final/jboss-modules.jar) to constructor com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl()
WARNING: Please consider reporting this to the maintainers of __redirected.__SAXParserFactory
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
13:34:35,267 INFO [org.jboss.modules] (main) JBoss Modules version 1.5.1.Final
13:34:35,627 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
13:34:35,798 INFO [org.jboss.as] (MSC service thread 1-7) WFLYSRV0049: WildFly Full 10.0.0.Final (WildFly Core 2.0.10.Final) starting
13:34:36,608 ERROR [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0055: Caught exception during boot: org.jboss.as.controller.persistence.ConfigurationPersistenceException: WFLYCTL0085: Failed to parse configuration
at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:131)
at org.jboss.as.server.ServerService.boot(ServerService.java:356)
at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:299)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javax.xml.stream.XMLStreamException: WFLYCTL0083: Failed to load module org.wildfly.extension.core-management
at org.jboss.as.controller.parsing.ExtensionXml.parseExtensions(ExtensionXml.java:155)
at org.jboss.as.server.parsing.StandaloneXml$DefaultExtensionHandler.parseExtensions(StandaloneXml.java:126)
at org.jboss.as.server.parsing.StandaloneXml_4.readServerElement(StandaloneXml_4.java:218)
at org.jboss.as.server.parsing.StandaloneXml_4.readElement(StandaloneXml_4.java:141)
at org.jboss.as.server.parsing.StandaloneXml.readElement(StandaloneXml.java:103)
at org.jboss.as.server.parsing.StandaloneXml.readElement(StandaloneXml.java:49)
at org.jboss.staxmapper.XMLMapperImpl.processNested(XMLMapperImpl.java:110)
at org.jboss.staxmapper.XMLMapperImpl.parseDocument(XMLMapperImpl.java:69)
at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:123)
... 3 more
Caused by: java.util.concurrent.ExecutionException: javax.xml.stream.XMLStreamException: WFLYCTL0083: Failed to load module
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at org.jboss.as.controller.parsing.ExtensionXml.parseExtensions(ExtensionXml.java:147)
... 11 more
Caused by: javax.xml.stream.XMLStreamException: WFLYCTL0083: Failed to load module
at org.jboss.as.controller.parsing.ExtensionXml.loadModule(ExtensionXml.java:196)
at org.jboss.as.controller.parsing.ExtensionXml.access$000(ExtensionXml.java:69)
at org.jboss.as.controller.parsing.ExtensionXml$1.call(ExtensionXml.java:127)
at org.jboss.as.controller.parsing.ExtensionXml$1.call(ExtensionXml.java:124)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
Caused by: org.jboss.modules.ModuleNotFoundException: org.wildfly.extension.core-management:main
at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:223)
at org.jboss.as.controller.parsing.ExtensionXml.loadModule(ExtensionXml.java:178)
... 8 more
13:34:36,611 FATAL [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0056: Server boot has failed in an unrecoverable manner; exiting. See previous messages for details.
13:34:36,615 INFO [org.jboss.as.server] (Thread-1) WFLYSRV0220: Server shutdown has been requested.
13:34:36,681 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0050: WildFly Full 10.0.0.Final (WildFly Core 2.0.10.Final) stopped in 34ms
How can i fix above error?
If you want to use Java 9+ I'd suggest using at least WildFly 15. WildFly 17.0.1.Final is the latest release and does work with Java 9+.
If you are stuck on WildFly 10, I'd suggest using Java 1.8 instead.
I have installed erlang/OTP 17.1 from source & I have installed ejabberd from GitHub. Both have installed successfully. But when i am starting ejabberd server, i am getting the following error in error.log.
2016-01-08 16:13:57.700 [error] <0.100.0> unable to load p1_yaml NIF: {error,{bad_lib,"Library version (2.8) not compatible (with 2.6)."}}
2016-01-08 16:13:57.700 [error] <0.99.0> CRASH REPORT Process <0.99.0> with 0 neighbours exited with reason: {{bad_lib,"Library version (2.8) not compatible (with 2.6)."},{p1_yaml_app,start,[normal,[]]}} in application_master:init/4 line 133
2016-01-08 16:13:57.769 [critical] <0.38.0>#ejabberd:exit_or_halt:133 failed to start application 'p1_yaml': {error,
{{bad_lib,
"Library version (2.8) not compatible (with 2.6)."},
{p1_yaml_app,start,[normal,[]]}}}
You likely have a p1_yaml.so file around that was build with a different version of Erlang runtime than the one you are using.
You should rebuild ejabberd (or just p1_yaml) with the correct Erlang version.