I have logging configured in my dropwizard yml file to log to a file and not to the console, however some logs are still being logged to the console.
service.yml
logging:
level: INFO
appenders:
- type: file
threshold: DEBUG
logFormat: "%-6level [%d{HH:mm:ss.SSS}] [%t] %logger{5} - %X{code} %msg %n"
currentLogFilename: /tmp/application.log
archivedLogFilenamePattern: /tmp/application-%d{yyyy-MM-dd}.log
archivedFileCount: 7
timeZone: UTC
When I execute I get my service logs in my log file but i get request logs on the console and not in the log file
127.0.0.1 - - [11/Nov/2015:22:31:52 +0000] "GET /api/v1/hello HTTP/1.1" 200 - "-" "curl/7.15.3 (x86_64-unknown-linux-gnu) libcurl/7.15.3 OpenSSL/0.9.8w" 1
Im using Dropwizard 0.7.1
You will need to configure an appender for request logs. The default appender is console.
Reference: https://dropwizard.github.io/dropwizard/0.7.1/docs/manual/configuration.html#request-log
Related
I have a Rails 5.2.3 application with Ruby 2.4.5.
I found a weird issue that the request info are logged twice in stdout.
Here is the log config in config/environments/product.rb
config.logger = ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDOUT))
config.log_tags = [ lambda {|req| "#{req.cookie_jar["_session_id"]}" }, :remote_ip, :uuid ]
Supposedly it will tag every log with the remote ip and request uuid, and it does for most of logs, except there is a weird additional log for the request. In following example, the last line is a duplicated log for the request without the tag:
[INFO] [2019-10-28 06:11:45 UTC] [127.0.0.1] [f6de1900-a7e5-4486-8b73-7095d0cacb35] Started GET "/api/v1/nodes?pageSize=20&pageNumber=1" for 127.0.0.1 at 2019-10-28 06:11:45 +0000
[INFO] [2019-10-28 06:11:45 UTC] [127.0.0.1] [f6de1900-a7e5-4486-8b73-7095d0cacb35] Processing by Api::V1::NodesController#index as XML
...
...
[INFO] [2019-10-28 06:20:02 UTC] [127.0.0.1] [993e0db4-3995-41ef-851a-bfea1bc25781] Completed 200 OK in 1084ms (Views: 341.9ms | ActiveRecord: 150.6ms)
127.0.0.1 - - [28/Oct/2019:06:20:02 +0000] "GET /api/v1/nodes?pageSize=20&pageNumber=1 HTTP/1.1" 200 - 1.1347
I checked configurations, there is no any other logger configured.
The consequence is that there is a timer in client side to check notification status by every 5 seconds. I added a log silencer to avoid log such request:
# config/application.rb
config.middleware.insert_before Rails::Rack::Logger, LogSilencer, silenced: /notification_messages/
# lib/log_silencer.rb
class LogSilencer
def initialize(app, opts = {})
#app = app
#silenced = opts.delete(:silenced)
end
def call(env)
if #silenced.match(env['PATH_INFO'])
Rails.logger.silence do
#app.call(env)
end
else
#app.call(env)
end
end
end
It does avoid logging from the tagged logger, but the duplicated request logs are still there, then the stdout is full of this request
127.0.0.1 - - [28/Oct/2019:07:04:21 +0000] "GET /api/v1/notification_messages/to_notify HTTP/1.1" 200 - 0.0118
127.0.0.1 - - [28/Oct/2019:07:04:26 +0000] "GET /api/v1/notification_messages/to_notify HTTP/1.1" 200 - 0.0140
127.0.0.1 - - [28/Oct/2019:07:04:31 +0000] "GET /api/v1/notification_messages/to_notify HTTP/1.1" 200 - 0.0137
127.0.0.1 - - [28/Oct/2019:07:04:36 +0000] "GET /api/v1/notification_messages/to_notify HTTP/1.1" 200 - 0.0149
...
Spent a whole day try to find out who generated this log, but no any clue...
Would like to ask for help on how to turn off this log so make the log data clear...
Thanks!
Seems the logs come from the web server used when up the Rails app.
When use WEBrick, the log is like
=> Booting WEBrick
=> Rails 5.2.3 application starting in development on http://localhost:4000
=> Run `rails server -h` for more startup options
[2019-10-30 07:01:13] INFO WEBrick 1.3.1
[2019-10-30 07:01:13] INFO ruby 2.4.5 (2018-10-18) [x86_64-linux]
[2019-10-30 07:01:13] INFO WEBrick::HTTPServer#start: pid=5812 port=4000
127.0.0.1 - - [30/Oct/2019:07:01:17 UTC] "GET /test HTTP/1.1" 304 0
- -> /test
When use unicorn, the log is like:
I, [2019-10-30T07:02:47.626962 #5956] INFO -- : Refreshing Gem list
I, [2019-10-30T07:02:48.289373 #5956] INFO -- : listening on addr=0.0.0.0:4000 fd=17
I, [2019-10-30T07:02:48.393598 #5956] INFO -- : master process ready
I, [2019-10-30T07:02:48.394853 #5965] INFO -- : worker=0 ready
I, [2019-10-30T07:02:48.399878 #5968] INFO -- : worker=1 ready
127.0.0.1 - - [30/Oct/2019:07:02:52 +0000] "GET /test HTTP/1.1" 304 - 0.0724
I can find that the request log format is a bit different (the time zone info, the time consumed etc).
I use Unicorn, its logger by default uses stderr, but seems the logger configuration does not work
logger Logger.new("#{rails_root}/log/unicorn.log")
so I have to set the stderr path
stderr_path "#{rails_root}/log/unicorn.stderr.log
Then the request logs are in the unicorn.stderr.log file, and STDOUT are the rails app logs.
But still dont know how to turn it off since it is a kind of duplicated and useless log...
I get a quite strange error message after executing Install-SecurityDaemon.
I deployed the IoT Edge module image to Azure Registry using Visual Studio Code. Then, I try to execute the IoT Edge solution in the simulator. Everything is ok.
I executed Install-SecurityDaemon. Everything went ok. But when I exec "iotedge list" command, I saw that only two main modules arrived but there is no module developed by me
PS C:\WINDOWS\system32> iotedge list
NAME STATUS DESCRIPTION CONFIG
edgeHub failed Failed (137) an hour ago mcr.microsoft.com/azureiotedge-hub:1.0
edgeAgent running Up 1 second mcr.microsoft.com/azureiotedge-agent:1.0
I'd seen the log and found only one error message:
11.11.2018 22:52:37 warn: edgelet_docker::runtime -- Attempt to pull image failed.
11.11.2018 22:52:42 info: edgelet_core::watchdog -- Checking edge runtime status
11.11.2018 22:52:42 info: edgelet_core::watchdog -- Edge runtime is running.
11.11.2018 22:52:43 warn: edgelet_docker::runtime -- Attempt to pull image failed.
11.11.2018 22:52:43 info: edgelet_http::logging -- [mgmt] - - - [2018-11-11 19:52:43.487711600 UTC] "POST /modules?api-version=2018-06-28 HTTP/1.1" 500 Internal Server Error 141 "-" "-" pid(any)
11.11.2018 22:52:43 warn: edgelet_utils::logging -- Get https://warlibregistry.azurecr.io/v2/iot-edge-engine-simulator/manifests/0.0.1-amd64: unauthorized: authentication required
11.11.2018 22:52:55 info: edgelet_http::logging -- [mgmt] - - - [2018-11-11 19:52:55.502141500 UTC] "POST /modules?api-version=2018-06-28 HTTP/1.1" 500 Internal Server Error 141 "-" "-" pid(any)
11.11.2018 22:52:55 warn: edgelet_utils::logging -- Get https://warlibregistry.azurecr.io/v2/iot-edge-engine-simulator/manifests/0.0.1-amd64: unauthorized: authentication required
Then I execute "docker login" command to be sure, that docker properly authorized in Azure registry. Everything was ok. Then, I reinstall SecurityDaemon. I got the same error.
Executing command
PS C:\WINDOWS\system32> docker pull warlibregistry.azurecr.io/iot-edge-engine-simulator:0.0.1-amd64
0.0.1-amd64: Pulling from iot-edge-engine-simulator
Digest: sha256:4ba6ae6442ca974b2c52459b85c0861e9664f26990c6e87f20829954f4d67d09
Status: Image is up to date for warlibregistry.azurecr.io/iot-edge-engine-simulator:0.0.1-amd64
return no errors. But if try to get manifest https://warlibregistry.azurecr.io/v2/iot-edge-engine-simulator/manifests/0.0.1-amd64 I really got the JSON with the error "unauthorized: authentication required".
In the Azure Portal after adding my module engineSimulator I saw another one iot_edge_engine_simulator with type "Module identity" which is not created by me. When I installed e.g. Microsoft module termoSensor - there is no such additional module.
So, I'm stuck, I don't know why the error occurs and what to do further.
Thank you for a help!
I found the mistake. When I set the module I missed filling "Container Registry Settings". In the case of using your own container registry (not Microsoft), you MUST fill authorization parameters.
In my case:
NAME: WarlibRegistry
ADDRESS: warlibregistry.azurecr.io
USERNAME & PASSWORD from the "Access keys" section of your container registry.
I want to send STDOUT log from Docker container to fluentd.
But, when one container outputs access logs and error logs, logs are mixed.
example
# rails access log
2017-04-07 12:10:01 +0000 6a51e389e724: {"log":"I, [2017-04-07T12:10:01.825923 #7] INFO -- : Started GET \"/users/new\" for 172.21.0.1 at 2017-04-07 12:10:01 +0000","container_id":"6a51e389e724c67be4e714402b69da192db4a304cbfdf638594de6cff9774c23","container_name":"/app","source":"stdout"}
# rails error log
2017-04-07 12:10:01 +0000 6a51e389e724: {"container_id":"6a51e389e724c67be4e714402b69da192db4a304cbfdf638594de6cff9774c23","container_name":"/app","source":"stdout","log":"E, [2017-04-07T12:10:01.830039 #7] ERROR -- : Invoke logger error"}
# rails access log
2017-04-07 12:10:03 +0000 6a51e389e724: {"log":"I, [2017-04-07T12:10:01.825923 #7] INFO -- : Started POST \"/users/create\" for 172.21.0.1 at 2017-04-07 12:10:01 +0000","container_id":"6a51e389e724c67be4e714402b69da192db4a304cbfdf638594de6cff9774c23","container_name":"/app","source":"stdout"}
Can I add a label for each log type?
Please tell me if you have a good solution.
Thanks you and best regards.
As your app's access and error logs are sent to STDOUT of container, then you have no way to separate them by logging driver. As solution you can send access log to STDOUT, error log to STDERR of container, then later differentiate logs by "source" field of each json message (this can be done if you connect fluentd to elasticsearch+kibana).
I am using Neo4j 2.3.0-M03 community version. I have created a database using neo4j import tool. Now when I am going to start the neo4j server, its failing. Any advice? Thanks in advance!
Starting Neo4j Server...WARNING: not changing user
process [21597]... waiting for server to be ready......... Failed to start within 120 seconds.
Neo4j Server may have failed to start, please check the logs.
The log is as follows:
2015-10-14 14:55:35.438-0400 INFO No SSL certificate found, generating a self-signed certificate..
14:55:35.936 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
14:55:36.105 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
14:55:36.106 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
14:55:36.107 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
14:55:36.107 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
14:55:36.108 [main] DEBUG i.n.util.internal.PlatformDependent - Java version: 7
14:55:36.108 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
14:55:36.109 [main] DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available
14:55:36.109 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
14:55:36.110 [main] DEBUG i.n.util.internal.PlatformDependent - Javassist: unavailable
14:55:36.110 [main] DEBUG i.n.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes. Please check the configuration for better performance.
14:55:36.110 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
14:55:36.110 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
14:55:36.110 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
14:55:36.119 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetectionLevel: simple
14:55:36.135 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
14:55:36.135 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
2015-10-14 14:55:41.341-0400 INFO Successfully started database
2015-10-14 14:55:41.405-0400 INFO Starting HTTP on port 7474 (32 threads available)
2015-10-14 14:55:41.659-0400 INFO Successfully shutdown Neo4j Server
2015-10-14 14:55:42.224-0400 INFO Successfully stopped database
2015-10-14 14:55:42.227-0400 ERROR Failed to start Neo4j: Starting Neo4j failed: tried to access field org.neo4j.server.rest.repr.RepresentationFormat.mediaType from class org.neo4j.server.rest.repr.RepresentationFormatRepository Starting Neo4j failed: tried to access field org.neo4j.server.rest.repr.RepresentationFormat.mediaType from class org.neo4j.server.rest.repr.RepresentationFormatRepository
org.neo4j.server.ServerStartupException: Starting Neo4j failed: tried to access field org.neo4j.server.rest.repr.RepresentationFormat.mediaType from class org.neo4j.server.rest.repr.RepresentationFormatRepository
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:67)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:234)
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:96)
at org.neo4j.server.CommunityBootstrapper.start(CommunityBootstrapper.java:48)
at org.neo4j.server.CommunityBootstrapper.main(CommunityBootstrapper.java:35)
Caused by: java.lang.IllegalAccessError: tried to access field org.neo4j.server.rest.repr.RepresentationFormat.mediaType from class org.neo4j.server.rest.repr.RepresentationFormatRepository
at org.neo4j.server.rest.repr.RepresentationFormatRepository.<init>(RepresentationFormatRepository.java:46)
at org.neo4j.server.AbstractNeoServer.createDefaultInjectables(AbstractNeoServer.java:641)
at org.neo4j.server.AbstractNeoServer.configureWebServer(AbstractNeoServer.java:360)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:216)
... 3 more
Seems like some permission issue on the lib folder of the Neo4j installation directory.
Try to give full permission to the neo4j installation folder.
Would also suggest to use stable version and not the milestone version.
I could fix it. There was some problem with my embedded database. I tried to run the server with its default graph.db and it run successfully. Now I have to identify what's wrong with the embedded database with I created using import tool.
I am trying to integrate Jenkins with Selenium. I have prepared the following command:
java -jar "D:\CI\TWCP\Selenium plugins\selenium-server-standalone-2.32.0.jar" -htmlSuite *firefox "http://www.google.co.in" "C:\Users\....\Desktop\Test suites\GoogleTestSuite.html" "C:\Users\.....\Desktop\results.html" -firefoxProfileTemplate "C:\....\Mozilla\Firefox\Profiles\s980v5zn.default"
When I execute the command in the command line, it works Perfectly fine. Then I called the same command through Jenkins and I got the following error:
Started by user anonymous
Building in workspace D:\CI\Jenkins\jobs\Google Test\workspace
[workspace] $ cmd /c call C:\Windows\TEMP\hudson600399625489206831.bat
D:\CI\Jenkins\jobs\Google Test\workspace>java -jar "D:\CI\TWCP\Selenium plugins\selenium-server-standalone-2.32.0.jar" -htmlSuite *firefox "http://www.google.co.in" "C:\Users\N_chandanKumar\Desktop\Test suites\GoogleTestSuite.html" "C:\Users\N_chandanKumar\Desktop\results.html" -firefoxProfileTemplate "C:\Users\N_chandanKumar\AppData\Roaming\Mozilla\Firefox\Profiles\s980v5zn.default"
Apr 24, 2013 10:54:07 AM org.openqa.grid.selenium.GridLauncher main
INFO: Launching a standalone server
10:54:08.647 INFO - Java: Sun Microsystems Inc. 1.6.0-b105
10:54:08.647 INFO - OS: Windows Vista 6.1 x86
10:54:08.662 INFO - v2.32.0, with Core v2.32.0. Built from revision 6c40c18
10:54:08.756 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
10:54:08.756 INFO - Version Jetty/5.1.x
10:54:08.756 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
10:54:08.756 INFO - Started HttpContext[/selenium-server,/selenium-server]
10:54:08.756 INFO - Started HttpContext[/,/]
10:54:08.787 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler#13c1b02
10:54:08.787 INFO - Started HttpContext[/wd,/wd]
10:54:08.787 INFO - Started SocketListener on 0.0.0.0:4444
10:54:08.787 INFO - Started org.openqa.jetty.jetty.Server#7ced01
jar:file:/D:/CI/TWCP/Selenium%20plugins/selenium-server-standalone-2.32.0.jar!/customProfileDirCUSTFFCHROME
10:54:09.941 INFO - Preparing Firefox profile...
10:54:12.041 INFO - Launching Firefox...
10:54:14.242 INFO - Checking Resource aliases
10:54:37.079 INFO - Checking Resource aliases
10:54:37.079 INFO - Received posted results
GoogleTestCase.html
GoogleTestCase</td></tr>
</tbody></table>
10:54:37.297 INFO - Killing Firefox...
Tests failed, see result file for details: C:\Users\N_chandanKumar\Desktop\results.html
10:54:37.921 INFO - Shutting down...
10:54:37.921 INFO - Stopping Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=4444]
D:\CI\Jenkins\jobs\Google Test\workspace>exit 1
Build step 'Execute Windows batch command' marked build as failure
Finished: FAILURE
result.html shows failure with below log
info: Starting test /selenium-server/tests/GoogleTestCase.html
info: Executing: |setTimeout | 1000000 | |
info: Executing: |openAndWait | http://www.google.co.in/ | |
info: Executing: |setTimeout | 1000000 | |
info: Executing: |pause | 20000 | |
info: Executing: |verifyTextPresent | India | |
warn: currentTest.recordFailure: false
info: Executing: |assertTitle | Google | |
error: Actual value 'Error Message' did not match 'Google'
warn: currentTest.recordFailure: Actual value 'Error Message' did not match 'Google'
If I run the same code adding debug tag i.e.
java -jar "D:\CI\TWCP\Selenium plugins\selenium-server-standalone-2.32.0.jar" -htmlSuite *firefox "http://www.google.co.in" "C:\Users\....\Desktop\Test suites\GoogleTestSuite.html" "C:\Users\....\Desktop\results.html" -firefoxProfileTemplate "C:\Users\.....\AppData\Roaming\Mozilla\Firefox\Profiles\s980v5zn.default" -debug
I get the following logs at the end
GoogleTestCase.html
GoogleTestCase</td></tr>
</tbody></table>
10:58:56.737 DEBUG - Handled by org.openqa.selenium.server.htmlrunner.SeleniumHTMLRunnerResultsHandler#14b7453
10:58:56.737 DEBUG - RESPONSE:
HTTP/1.1 200 OK
Date: Wed, 24 Apr 2013 05:28:56 GMT
Server: Jetty/5.1.x (Windows Vista/6.1 x86 java/1.6.0
Cache-Control: no-cache
Pragma: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/html
Transfer-Encoding: chunked
10:58:57.002 INFO - Killing Firefox...
10:58:57.018 DEBUG - java.net.SocketException: Connection reset
10:58:57.018 DEBUG - java.net.SocketException: Connection reset
10:58:57.018 DEBUG - java.net.SocketException: Connection reset
10:58:57.018 DEBUG - java.net.SocketException: Connection reset
10:58:57.033 DEBUG - java.net.SocketException: Connection reset
10:58:57.033 DEBUG - java.net.SocketException: Connection reset
Tests failed, see result file for details: C:\Users\N_chandanKumar\Desktop\results.html
D:\CI\Jenkins\jobs\Google Test\workspace>exit 1
Build step 'Execute Windows batch command' marked build as failure
Finished: FAILURE
I have jenkins 1.509 and seleniumHQ2.32.0
Not able to make out how I am getting "java.net.SocketException: Connection reset" Any help in this regard will be appreciated.
Thanks & Regards,
cK
To check on the issue I ran the jenkins server in Interactive mode. The setting can be seen when you go to services.msc -> Jenkins process -> properties-> click on logon tab and you can select Local system account and tick "Allow service to interact with desktop".
This made jenkins to run the selenium test interactively instead of running in background.
There I could observe browser failing to open the website. This was tackled by following settings services.msc -> Jenkins process -> properties-> click on logon tab --> Select this account and give the account using which you have logged into this system.