I'm a novice setting up a server for the first time to implement WebRTC
Linux is using Centos7 and has set up KMS and Coturn.
However, there is one problem.
The client and server are not connected on the screen, so I checked the logs of kms
docker logs --follow kms
0:00:01.206579656 1 0x56191aac5010 INFO KurentoServerMethods ServerMethods.cpp:90:ServerMethods: Using above 80% of system limits will throw NOT_ENOUGH_RESOURCES exception
0:00:01.206607827 1 0x56191aac5010 INFO KurentoServerMethods ServerMethods.cpp:109:ServerMethods: System limits: unlimited threads, 32768 files
0:00:01.206902099 1 0x56191aac5010 INFO KurentoWorkerPool WorkerPool.cpp:67:WorkerPool: Worker thread pool size: 2
0:00:01.207158442 1 0x56191aac5010 INFO KurentoServerMethods ServerMethods.cpp:144:ServerMethods: RPC Request Cache is ENABLED
0:00:01.207351433 1 0x56191aac5010 INFO KurentoWebSocketTransport WebSocketTransport.cpp:187:initWebSocket: WebSocket server (ws://) listening on address '::', port 8888
0:00:01.207411744 1 0x56191aac5010 INFO KurentoWebSocketTransport WebSocketTransport.cpp:88:WebSocketTransport: Secure WebSocket server (wss://) not enabled
0:00:01.208078290 1 0x56191aac5010 INFO KurentoMediaServer main.cpp:259:main: Kurento Media Server started
0:02:29.095818552 1 0x7f5070017630 INFO KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:164:generateDefaultCertificates: Unable to load the RSA certificate from file. Using the default certificate.
0:02:29.284074137 1 0x7f5070017630 INFO KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:174:generateDefaultCertificates: Unable to load the ECDSA certificate from file. Using the default certificate.
0:02:29.290405426 1 0x7f5070017630 INFO KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:110:remove_not_supported_codecs_from_array:<kmswebrtcendpoint0> Removing not supported codec 'AMR/8000'
0:02:29.515589312 1 0x7f5064039e00 INFO basertpendpoint kmsbasertpendpoint.c:1132:kms_base_rtp_endpoint_start_transport_send:<kmswebrtcendpoint0> Media 'video' has REMB
0:02:29.515721223 1 0x7f5064039e00 INFO basertpendpoint kmsbasertpendpoint.c:1078:kms_base_rtp_endpoint_create_remb_manager:<kmswebrtcendpoint0> Creating REMB for session ID 0 (kmswebrtcendpoint0-sess0) and remote video SSRC 3653849939
0:02:29.515746113 1 0x7f5064039e00 INFO basertpendpoint kmsbasertpendpoint.c:1089:kms_base_rtp_endpoint_create_remb_manager:<kmswebrtcendpoint0> REMB: Set RTCP min interval to 500 ms
0:02:29.519063004 1 0x7f5064007580 WARN kmswebrtcsession kmswebrtcsession.c:823:kms_webrtc_session_set_stun_server_info:<kmswebrtcsession0> STUN server not configured! NAT traversal requires STUN or TURN
0:02:29.519107324 1 0x7f5064007580 WARN kmswebrtcsession kmswebrtcsession.c:843:kms_webrtc_session_set_relay_info:<kmswebrtcsession0> TURN relay server not configured! NAT traversal requires STUN or TURN
0:02:29.522346434 1 0x7f50700054f0 INFO KurentoWorkerPool WorkerPool.cpp:67:WorkerPool: Worker thread pool size: 2
0:02:40.930306053 1 0x7f5050001630 INFO KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:110:remove_not_supported_codecs_from_array:<kmswebrtcendpoint1> Removing not supported codec 'AMR/8000'
0:02:40.951376487 1 0x7f5064018b30 INFO basertpendpoint kmsbasertpendpoint.c:1132:kms_base_rtp_endpoint_start_transport_send:<kmswebrtcendpoint1> Media 'video' has REMB
0:02:40.951898082 1 0x7f5064018b30 INFO basertpendpoint kmsbasertpendpoint.c:1078:kms_base_rtp_endpoint_create_remb_manager:<kmswebrtcendpoint1> Creating REMB for session ID 0 (kmswebrtcendpoint1-sess0) and remote video SSRC 3442416509
"NAT traversal requires STUN or TURN."
I don't know how to solve this part.
This is because the STUN server results from Trickle ICE were also successful.
If you know what I need to do, I'd appreciate it if you could tell me all the actions.
And please let me know if there is anything else I need to fill out!
STUN and TURN
You dont have to have coturn if you are doing local testing. The warning is saying if you want to go outside of your network (out of your router and to the web) you will need a STUN or TURN server.
Docker
Docker doesn't open the port 8888 by itself. You may need to open that port manually. To do this, add this -p 8888:8888 when creating your container.
Or if you are using the Desktop version you can enter it into Host port under the Optional settings when you first run it.
Related
I am using the docker image of guacamole (oznu/guacamole:amd64). When I try to connect to a RDP host, I get the message "You have been disconnected".
In the configuration, I set the following parameters:
Protocol = RDP
Network/Hostname = ...
Network/Port = 3389
Authentication/Username = ...
Authentication/Password = ...
Authentication/Security mode = any (also tried all the others)
Authentication/Ignore server certificate = Checked
When I look into the logs of the docker container after a failed connection attempt, I see the following error:
17:54:40.256 [http-nio-8080-exec-8] INFO o.a.g.tunnel.TunnelRequestService - User "guacadmin" connected to connection "1".
guacd[533]: INFO: Loading keymap "base"
guacd[533]: INFO: Loading keymap "en-us-qwerty"
guacd[533]: INFO: Connected to RDPDR 1.13 as client 0x0002
*** Error in `guacd': munmap_chunk(): invalid pointer: 0x000055d216529840 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x70bfb)[0x7f5f2e172bfb]
/lib/x86_64-linux-gnu/libc.so.6(+0x76fc6)[0x7f5f2e178fc6]
/usr/lib/x86_64-linux-gnu/libwinpr2.so.2(Stream_Free+0x2f)[0x7f5f2974577f]
/usr/lib/x86_64-linux-gnu/libfreerdp2.so.2(+0x7840d)[0x7f5f29d2c40d]
/usr/lib/x86_64-linux-gnu/libfreerdp2.so.2(+0x78a25)[0x7f5f29d2ca25]
/usr/lib/x86_64-linux-gnu/libfreerdp2.so.2(freerdp_channels_check_fds+0x35)[0x7f5f29d2d5f5]
/usr/lib/x86_64-linux-gnu/libfreerdp2.so.2(freerdp_check_event_handles+0x48)[0x7f5f29d2b128]
/usr/local/lib/libguac-client-rdp.so(guac_rdp_client_thread+0x287)[0x7f5f2a01e927]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x74a4)[0x7f5f2f6fb4a4]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f5f2e1ead0f]
======= Memory map: ========
...
The RDP host machine is running Windows 10. I tested it with another machine also running Windows 10.
I also verified:
Other machines can connect to that RDP host using the same hostname, port, username and password as supplied
The host can be pinged from within the docker container
If the credential are not correct I get a different error saying the log in has failed, so the credentials are actually verified by the RDP host
In Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp "UserAuthentication" is 0 and "SecurityLayer" is 1
I expected the RDP connection to work
Docker desktop was working fine but after a reboot the docker desktop doesn't start at all.
I've tried switchDaemon, Switch to Windows container etc., but none of them starts.
I'm using WSL2 and all my containers are linux based. If I re-install then I'll lose all data and images.
wsl --list
Windows Subsystem for Linux Distributions:
Ubuntu-20.04 (Default)
docker-desktop
docker-desktop-data
Every restart attempt results with the following log entires.
open \\.\pipe\dockerProcd: The system cannot find the file specified.
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
Log:
[2022-08-01T14:09:47.861542400Z][com.docker.backend.exe][I] (15e5c8d5) 4b373d33-BackendAPI S->C DockerDesktopElectron POST /nps (1.6312ms): OK
[2022-08-01T14:09:50.286711200Z][com.docker.backend.exe][I] (e474d45c) 4b373d33-BackendAPI S<-C DockerDesktopElectron POST /analytics/track
[2022-08-01T14:09:50.287235200Z][com.docker.backend.exe][I] (e474d45c) 4b373d33-BackendAPI S<-C DockerDesktopElectron bind: {"body":null,"event":"actionMenuSwitchWindowsDaemon"}
[2022-08-01T14:09:50.287758600Z][com.docker.backend.exe][I] (e474d45c) 4b373d33-BackendAPI S->C DockerDesktopElectron POST /analytics/track (1.0474ms): OK
[2022-08-01T14:09:50.288277300Z][com.docker.backend.exe][I] Usage statistics: actionMenuSwitchWindowsDaemon
[2022-08-01T14:09:50.288277300Z][com.docker.backend.exe][I] anonymous remaining time: 23h35m58.7117227s
[2022-08-01T14:09:51.767785100Z][IPCServer ][Info ] (3f58fd7b) acc5d626-CSharpAPI S<-C DockerDesktopElectron POST /desktop/switch-engine
[2022-08-01T14:09:51.773786400Z][IPCServer ][Info ] (3f58fd7b) acc5d626-CSharpAPI S->C DockerDesktopElectron POST /desktop/switch-engine (6ms): OK
[2022-08-01T14:09:53.056235800Z][WslKeepAlive ][Info ] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
[2022-08-01T14:09:53.059234500Z][WslKeepAlive ][Info ] wsl keep-alive stopped
[2022-08-01T14:09:53.059234500Z][WslKeepAlive ][Warning] stopped unexpectedly
[2022-08-01T14:09:53.244964600Z][vpnkit-bridge.exe][W] windows: still waiting for dns-forwarder, volume-contents, lifecycle-server, wsl2-bootstrap-expose-ports, devenv-volumes, procd, docker, debug-shell, diagnosticd, wsl2-cross-distro-service, log after 10.01406s
[2022-08-01T14:10:03.238970300Z][vpnkit-bridge.exe][W] windows: still waiting for dns-forwarder, volume-contents, lifecycle-server, wsl2-bootstrap-expose-ports, devenv-volumes, procd, docker, debug-shell, diagnosticd, wsl2-cross-distro-service, log after 20.0080657s
[2022-08-01T14:10:05.610766100Z][com.docker.backend.exe][W] 526c5971-PauseHDL /pause/events server not replying: Get "http://ipc/pause/events": open \\.\pipe\dockerProcd: The system cannot find the file specified.
[2022-08-01T14:10:13.241610100Z][vpnkit-bridge.exe][W] windows: still waiting for lifecycle-server, wsl2-bootstrap-expose-ports, devenv-volumes, procd, docker, dns-forwarder, volume-contents, wsl2-cross-distro-service, log, debug-shell, diagnosticd after 30.0105872s
I posted about this over on the Isaac forums, but listing it here for visibility as well. I am trying to get the Isaac Realsense examples working on a Jetson Nano with my 435i (firmware downgraded to 5.11.15 per the Isaac documentation), but I've been unable to so far. I've got a Nano flashed with Jetpack4.3 and have installed all dependencies on both the desktop and the Nano. The realsense-viewer works fine, so I know the camera is functioning properly and is being detected by the Nano. However, when I run ./apps/samples/realsense_camera/realsense_camera it throws an error:
ERROR engine/alice/components/Codelet.cpp#229: Component 'camera/realsense' of type 'isaac::RealsenseCamera' reported FAILURE:
No device connected, please connect a RealSense device
ERROR engine/alice/backend/event_manager.cpp#42: Stopping node 'camera' because it reached status 'FAILURE'
I've attached the log of this output as well. I get the same error running locally on my desktop, but that's running through WSL so I was willing to write that off. Any suggestions would be greatly appreciated!
0m2020-06-15 17:18:20.620 INFO engine/alice/tools/websight.cpp#166: Loading websight...0m
33m2020-06-15 17:18:20.621 WARN engine/alice/backend/application_json_loader.cpp#174: This application does not have an explicit scheduler configuration. One will be autogenerated to the best of the system's abilities if possible.0m
0m2020-06-15 17:18:20.622 INFO engine/alice/backend/redis_backend.cpp#40: Successfully connected to Redis server.
0m
33m2020-06-15 17:18:20.623 WARN engine/alice/backend/backend.cpp#201: This application does not have an execution group configuration. One will be autogenerated to the best of the systems abilities if possible.0m
33m2020-06-15 17:18:20.623 WARN engine/gems/scheduler/scheduler.cpp#337: No default execution groups specified. Attempting to create scheduler configuration for 4 remaining cores. This may be non optimal for the system and application.0m
0m2020-06-15 17:18:20.623 INFO engine/gems/scheduler/scheduler.cpp#290: Scheduler execution groups are:0m
0m2020-06-15 17:18:20.623 INFO engine/gems/scheduler/scheduler.cpp#299: __BlockerGroup__: Cores = [3], Workers = No0m
0m2020-06-15 17:18:20.623 INFO engine/gems/scheduler/scheduler.cpp#299: __WorkerGroup__: Cores = [0, 1, 2], Workers = Yes0m
0m2020-06-15 17:18:20.660 INFO engine/alice/backend/modules.cpp#226: Loaded module 'packages/realsense/librealsense_module.so': Now has 45 components total0m
0m2020-06-15 17:18:20.679 INFO engine/alice/backend/modules.cpp#226: Loaded module 'packages/rgbd_processing/librgbd_processing_module.so': Now has 51 components total0m
0m2020-06-15 17:18:20.696 INFO engine/alice/backend/modules.cpp#226: Loaded module 'packages/sight/libsight_module.so': Now has 54 components total0m
0m2020-06-15 17:18:20.720 INFO engine/alice/backend/modules.cpp#226: Loaded module 'packages/viewers/libviewers_module.so': Now has 83 components total0m
90m2020-06-15 17:18:20.720 DEBUG engine/alice/application.cpp#348: Loaded 83 components: isaac::RealsenseCamera, isaac::alice::BufferAllocatorReport, isaac::alice::ChannelMonitor, isaac::alice::CheckJetsonPerformanceModel, isaac::alice::CheckOperatingSystem, isaac::alice::Config, isaac::alice::ConfigBridge, isaac::alice::ConfigLoader, isaac::alice::Failsafe, isaac::alice::FailsafeHeartbeat, isaac::alice::InteractiveMarkersBridge, isaac::alice::JsonToProto, isaac::alice::LifecycleReport, isaac::alice::MessageLedger, isaac::alice::MessagePassingReport, isaac::alice::NodeStatistics, isaac::alice::Pose, isaac::alice::Pose2Comparer, isaac::alice::PoseFromFile, isaac::alice::PoseInitializer, isaac::alice::PoseMessageInjector, isaac::alice::PoseToFile, isaac::alice::PoseToMessage, isaac::alice::PoseTree, isaac::alice::PoseTreeJsonBridge, isaac::alice::PoseTreeRelink, isaac::alice::ProtoToJson, isaac::alice::PyCodelet, isaac::alice::Random, isaac::alice::Recorder, isaac::alice::RecorderBridge, isaac::alice::Replay, isaac::alice::ReplayBridge, isaac::alice::Scheduling, isaac::alice::Sight, isaac::alice::SightChannelStatus, isaac::alice::Subgraph, isaac::alice::Subprocess, isaac::alice::TcpPublisher, isaac::alice::TcpSubscriber, isaac::alice::Throttle, isaac::alice::TimeOffset, isaac::alice::TimeSynchronizer, isaac::alice::UdpPublisher, isaac::alice::UdpSubscriber, isaac::map::Map, isaac::map::ObstacleAtlas, isaac::map::OccupancyGridMapLayer, isaac::map::PolygonMapLayer, isaac::map::WaypointMapLayer, isaac::navigation::DistanceMap, isaac::navigation::NavigationMap, isaac::navigation::RangeScanModelClassic, isaac::navigation::RangeScanModelFlatloc, isaac::rgbd_processing::DepthEdges, isaac::rgbd_processing::DepthImageFlattening, isaac::rgbd_processing::DepthImageToPointCloud, isaac::rgbd_processing::DepthNormals, isaac::rgbd_processing::DepthPoints, isaac::rgbd_processing::FreespaceFromDepth, isaac::sight::AliceSight, isaac::sight::SightWidget, isaac::sight::WebsightServer, isaac::viewers::BinaryMapViewer, isaac::viewers::ColorCameraViewer, isaac::viewers::DepthCameraViewer, isaac::viewers::Detections3Viewer, isaac::viewers::DetectionsViewer, isaac::viewers::FiducialsViewer, isaac::viewers::FlatscanViewer, isaac::viewers::GoalViewer, isaac::viewers::ImageKeypointViewer, isaac::viewers::LidarViewer, isaac::viewers::MosaicViewer, isaac::viewers::ObjectViewer, isaac::viewers::OccupancyMapViewer, isaac::viewers::PointCloudViewer, isaac::viewers::PoseTrailViewer, isaac::viewers::SegmentationCameraViewer, isaac::viewers::SegmentationViewer, isaac::viewers::SkeletonViewer, isaac::viewers::TensorViewer, isaac::viewers::TrajectoryListViewer, 0m
33m2020-06-15 17:18:20.723 WARN engine/alice/application.cpp#164: The function Application::findComponentByName is deprecated. Please use `getNodeComponentOrNull` instead. Note that the new method requires a node name instead of a component name. (argument: 'websight/isaac.sight.AliceSight')0m
0m2020-06-15 17:18:20.723 INFO engine/alice/application.cpp#255: Starting application 'realsense_camera' (instance UUID: 'e24992d0-af66-11ea-8bcf-c957460c567e') ...0m
90m2020-06-15 17:18:20.723 DEBUG engine/gems/scheduler/execution_groups.cpp#476: Launching 0 pre-start job(s)0m
90m2020-06-15 17:18:20.723 DEBUG engine/gems/scheduler/execution_groups.cpp#485: Replaying 0 pre-start event(s)0m
90m2020-06-15 17:18:20.723 DEBUG engine/gems/scheduler/execution_groups.cpp#476: Launching 0 pre-start job(s)0m
90m2020-06-15 17:18:20.723 DEBUG engine/gems/scheduler/execution_groups.cpp#485: Replaying 0 pre-start event(s)0m
0m2020-06-15 17:18:20.723 INFO engine/alice/backend/asio_backend.cpp#33: Starting ASIO service0m
0m2020-06-15 17:18:20.727 INFO packages/sight/WebsightServer.cpp#216: Sight webserver is loaded0m
0m2020-06-15 17:18:20.727 INFO packages/sight/WebsightServer.cpp#217: Please open Chrome Browser and navigate to http://<ip address>:30000m
33m2020-06-15 17:18:20.727 WARN engine/alice/backend/codelet_canister.cpp#225: Codelet 'websight/isaac.sight.AliceSight' was not added to scheduler because no tick method is specified.0m
33m2020-06-15 17:18:20.728 WARN engine/alice/components/Codelet.cpp#53: Function deprecated. Set tick_period to the desired tick paramater0m
33m2020-06-15 17:18:20.728 WARN engine/alice/backend/codelet_canister.cpp#225: Codelet '_check_operating_system/isaac.alice.CheckOperatingSystem' was not added to scheduler because no tick method is specified.0m
33m2020-06-15 17:18:20.728 WARN engine/alice/components/Codelet.cpp#53: Function deprecated. Set tick_period to the desired tick paramater0m
33m2020-06-15 17:18:20.730 WARN engine/alice/components/Codelet.cpp#53: Function deprecated. Set tick_period to the desired tick paramater0m
1;31m2020-06-15 17:18:20.741 ERROR engine/alice/components/Codelet.cpp#229: Component 'camera/realsense' of type 'isaac::RealsenseCamera' reported FAILURE:
No device connected, please connect a RealSense device
0m
1;31m2020-06-15 17:18:20.741 ERROR engine/alice/backend/event_manager.cpp#42: Stopping node 'camera' because it reached status 'FAILURE'0m
33m2020-06-15 17:18:20.743 WARN engine/alice/backend/codelet_canister.cpp#225: Codelet 'camera/realsense' was not added to scheduler because no tick method is specified.0m
0m2020-06-15 17:18:21.278 INFO packages/sight/WebsightServer.cpp#113: Server connected / 10m
0m2020-06-15 17:18:30.723 INFO engine/alice/backend/allocator_backend.cpp#57: Optimized memory CPU allocator.0m
0m2020-06-15 17:18:30.724 INFO engine/alice/backend/allocator_backend.cpp#66: Optimized memory CUDA allocator.0m
I am trying to run a spark job with PySpark through Jupyter notebook running in Docker. Workers are located on separate machines in the same network. I am performing a take operation on RDD:
data.take(number_of_elements)
When the number_of_elements is 2000 everything works fine. When it is 20000 an exception occurs. From my point of view it breaks when the size of the result exceeds 2GB (or it seems for me so). The idea about 2GB comes from that spark can send results smaller than 2GB in one block and when the result is bigger than 2GB another mechanism starts to work and something breaks there (see here). Here is the exception from executor log:
19/11/05 10:27:14 INFO CodeGenerator: Code generated in 205.7623 ms
19/11/05 10:27:40 INFO PythonRunner: Times: total = 25421, boot = 3, init = 1751, finish = 23667
19/11/05 10:27:42 INFO MemoryStore: Block taskresult_4 stored as bytes in memory (estimated size 927.7 MB, free 6.4 GB)
19/11/05 10:27:42 INFO Executor: Finished task 0.0 in stage 3.0 (TID 4). 972788748 bytes result sent via BlockManager)
19/11/05 10:27:49 ERROR TransportRequestHandler: Error sending result ChunkFetchSuccess{streamChunkId=StreamChunkId{streamId=1585998572000, chunkIndex=0}, buffer=org.apache.spark.storage.BlockManagerManagedBuffer#4399ad49} to /10.0.0.9:56222; closing connection
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
at org.apache.spark.util.io.ChunkedByteBufferFileRegion.transferTo(ChunkedByteBufferFileRegion.scala:64)
at org.apache.spark.network.protocol.MessageWithHeader.transferTo(MessageWithHeader.java:121)
at io.netty.channel.socket.nio.NioSocketChannel.doWriteFileRegion(NioSocketChannel.java:355)
at io.netty.channel.nio.AbstractNioByteChannel.doWrite(AbstractNioByteChannel.java:224)
at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:382)
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:934)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:362)
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:901)
at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1321)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768)
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749)
at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:983)
at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:248)
at io.netty.channel.nio.AbstractNioByteChannel$1.run(AbstractNioByteChannel.java:284)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
at java.lang.Thread.run(Thread.java:748)
As we can see from the log executor tries to send result to 10.0.0.9:56222. It fails because the port is not opened in docker compose. 10.0.0.9 is an IP address of a master node but port 56222 is random though I explicitly set up all ports I can find in documentation to disable random port selection:
spark = SparkSession.builder\
.master('spark://spark.cyber.com:7077')\
.appName('My App')\
.config('spark.task.maxFailures', '16')\
.config('spark.driver.port', '20002')\
.config('spark.driver.host', 'spark.cyber.com')\
.config('spark.driver.bindAddress', '0.0.0.0')\
.config('spark.blockManager.port', '6060')\
.config('spark.driver.blockManager.port', '6060')\
.config('spark.shuffle.service.port', '7070')\
.config('spark.driver.maxResultSize', '14g')\
.getOrCreate()
I mapped these ports with docker compose:
version: "3"
services:
jupyter:
image: jupyter/pyspark-notebook:latest
ports:
- "4040-4050:4040-4050"
- "6060:6060"
- "7070:7070"
- "8888:8888"
- "20000-20010:20000-20010"
You should probably configure you spark driver memory to follow your docker container memory settings
I added
.config('spark.driver.memory', '14g')
as #ML_TN proposed and everything works now.
From my point of view it is strange that the memory setting affects the ports that spark uses.
I'm trying to use graylog2 to collect logs from docker containers. Docs says that only UDP GELF input is supported for this purpose.
I'm using docker-compose to run the graylog server. See gist for all files used: https://gist.github.com/olegabr/7f5190c453bb63c71dabf151d2373c2f.
And I'm using this command to test it:
sendip -p ipv4 -is 127.0.0.1 -p udp -us 5070 -ud 12201 -d '{"version": "1.1","host":"example.org","short_message":"Short message","full_message":"Backtrace here\n\nmore stuff","level":1,"_user_id":9001,"_some_info":"foo","_some_env_var":"bar"}' -v 127.0.0.1
Server receives this message, but it can not process it. I see following in the graylog2 logs:
2016-12-09 11:53:20,125 WARN : org.graylog2.bindings.providers.DefaultStreamProvider - Unable to load default stream, tried 1 times, retrying every 500ms. Processing is blocked until this succeeds.
2016-12-09 11:53:25,129 WARN : org.graylog2.bindings.providers.DefaultStreamProvider - Unable to load default stream, tried 11 times, retrying every 500ms. Processing is blocked until this succeeds.
e.t.c. many many similar lines.
The API call curl http://admin:123456#127.0.0.1:9000/api/count/total returns
{"events":0}
In the server logs I see that the default stream was initialized:
mongo_1 | 2016-12-09T11:51:12.522+0000 I INDEX [conn3] build index on: graylog.pipeline_processor_pipelines_streams properties: { v: 2, unique: true, key: { stream_id: 1 }, name: "stream_id_1", ns: "graylog.pipeline_processor_pipelines_streams" }
graylog_1 | 2016-12-09 11:51:13,408 INFO : org.graylog2.periodical.Periodicals - Starting [org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration] periodical, running forever.
graylog_1 | 2016-12-09 11:51:13,424 INFO : org.graylog.plugins.pipelineprocessor.periodical.LegacyDefaultStreamMigration - Legacy default stream has no connections, no migration needed.
graylog_1 | 2016-12-09 11:51:13,487 INFO : org.graylog2.migrations.V20160929120500_CreateDefaultStreamMigration - Successfully created default stream: All messages
graylog_1 | 2016-12-09 11:51:13,653 INFO : org.graylog2.migrations.V20161125142400_EmailAlarmCallbackMigration - No streams needed to be migrated.
graylog_1 | 2016-12-09 11:51:13,662 INFO : org.graylog2.migrations.V20161125161400_AlertReceiversMigration - No streams needed to be migrated.
graylog_1 | 2016-12-09 11:51:13,672 INFO : org.graylog2.migrations.V20161130141500_DefaultStreamRecalcIndexRanges - Cluster not connected yet, delaying migration until it is reachable.
So, why it can not be loaded when the message arrives? Why it is needed in the first place?
I've tried to find similar reports in web but with no success.
This has nothing to do with the UDP input per se.
Graylog 2.2.0-beta.1 is broken and shouldn't be used. Please downgrade to Graylog 2.1.2 (the latest stable version) or wait for Graylog 2.2.0-beta.2.
See https://groups.google.com/forum/#!searchin/graylog2/docker|sort:date/graylog2/gCycC3_K3vU/EL-Lz_uNDQAJ for a related post on the Graylog mailing list.
same trouble
just setup graylog and configure input gelf udp 12209 port
then test it twice by:
docker run --log-driver=gelf --log-opt gelf-address=udp://127.0.0.1:12209 busybox echo Hello Graylog
in UI i saw:
2 messages in process buffe
2 unprocessed messages are currently in the journal, in 1 segments.
0 messages have been appended in the last second, 0 messages have been read in the last second.
and still getting:
2016-12-09 12:41:23,715 INFO : org.graylog2.inputs.InputStateListener - Input [GELF UDP/584aa67308813b00010d009e] is now RUNNING
2016-12-09 12:41:43,666 WARN : org.graylog2.bindings.providers.DefaultStreamProvider - Unable to load default stream, tried 1 times, retrying every 500ms. Processing is blocked until this succeeds.
anyone have found solution ?