I want to run two separate and independent instances/installations of InfluxDB on the same machine (running on ubuntu). They should have different endpoints.
Is it possible? How do you do it?
Yes you can, just provide required configuration in corresponding influxdb.conf for each installation. The following attributes must be reconfigured to reference different targets:
Different data, wal and meta storage directories:
[data]
# The directory where the TSM storage engine stores TSM files.
dir = "/var/lib/influxdb/data"
# The directory where the TSM storage engine stores WAL files.
wal-dir = "/var/lib/influxdb/wal"
[meta]
# Where the metadata/raft database is stored
dir = "/var/lib/influxdb/meta"
Different HTTP (UDP) ports:
[http]
# Determines whether HTTP endpoint is enabled.
enabled = true
# The bind address used by the HTTP service.
bind-address = ":8086"
[[udp]]
enabled = false
bind-address = ":8089"
Also take a look, at other configuration settings to see if they not conflict.
Related
How can I tell if the settings files associated with a Mosquitto instance, have been properly applied?
I want to add a configuration file to the conf.d folder to override some settings in the default file, but I do not know how to check they have been applied correctly once the Broker is running.
i.e. change persistence to false (without editing the standard file).
Test it.
You can run mosquitto with verbose output enabled, which will generally give you feedback on what options were set, but don't just believe that.
To do that, stop running Mosquitto as a service (how you do this depends on you setup) and manually run it from the shell with the -v option. Be sure to point it at the correct configuration file with the -c option.
That's not enough to be sure that it's actually working properly. To do that you need to test it.
Options have consequences or we wouldn't use them.
If you configure Mosquitto to listen on a specific port, test it by trying to connect to that port.
If you configure Mosquitto to require secure connections on a port, test it by trying to connect to the port unsecured (this shouldn't work) and secured (this should work).
You should be able to devise relatively simple tests for any options you can set in the configuration file. If you care if it's actually working, don't just take it on faith; test it.
For extra credit you can bundle the tests up into a script so that you can run an entire test suite easily in the future and test your Mosquitto installation anytime you make changes to it.
Having duplicate configuration options with different values is a REALLY bad idea.
The behaviour of mosquitto is not defined in this case, which value should be honoured, the first found, the last? When using the conf.d directory, what order will the files be loaded in?
Also will you always remember that you have changed the value in a conf.d file in the future when you go looking?
If you want to change one of the defaults in the /etc/mosquitto/mosquitto.conf file then edit that file. (Any sensible package management system will notice the file has been changed and ask what to do at the point of upgrade)
The conf.d/ directory is intended for adding extra listeners.
Also be aware that there really isn't a default configuration file, you must always specify a configuration file with the -c command line option. The file at /etc/mosquitto/mosquitto.conf just happens to be the config file that is passed when mosquitto is started as a service when installed using most Linux package managers. (The default Fedora install doesn't even setup the /etc/mosquitto/conf.d directory)
I use typescript definition of serverless configuration (serverless.ts). How do I conditionally add resources using cloud formation templates ?
For example AWS cognito userpools, if I want to exclude these for offline mode, how can I specify it in serverless config file to not include them?
I solved it 2 ways. serverless-plugin-ifelse allowed me to exclude resources based on certain parameters.
Later I realized this may be problematic in the long run. So I created separate serverless config file for my offline use case to include all necessary resources. Prod/Staging environments used default serverless.ts file. Offline uses offline-serverless.ts file. Though there is some repetition of the resources config, this option ensures prod/staging config is not polluted with offline content. I could start offline using sls offline start --stage local --reloadHandler --config offline-serverless.ts
And the offline config reuses some code from main config. Sample offline-serverless.ts content is below:
import offlineplugins from "offlineplugins";
import pluginsconfig from "offlinepluginsconfig";
import plugins from "plugins";
const serverlessConfiguration: AWS = {
service: "my-offline-apis",
frameworkVersion: "3",
...pluginsconfig,
plugins: [...plugins, ...offlineplugins],
provider: {
name: "aws",
....
....
I am having issues enabling SSL configuration.
I walked through the guide provided on docs and searched through the other peoples posts but still cannot make it work. Therefore I kindly ask for your help :)
I am using neo4j 4.1.0 Enterprise on macOS Catalina ver. 10.15.5
When starting server I receive following error:
2020-07-06 16:54:16.837+0000 INFO Starting...
2020-07-06 16:54:24.804+0000 INFO ======== Neo4j 4.1.0 ========
2020-07-06 16:54:25.162+0000 ERROR Failed to start Neo4j on dbms.connector.http.listen_address, a socket address. If missing port or hostname it is acquired from dbms.default_listen_address. Error starting Neo4j database server at /Users/xxx/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-abc65513-dd91-4140-b69c-201fc2ec2aa8/installation-4.1.0/data/databases
java.lang.RuntimeException: Error starting Neo4j database server at /Users/xxx/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-abc65513-dd91-4140-b69c-201fc2ec2aa8/installation-4.1.0/data/databases
at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:198)
at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.build(DatabaseManagementServiceFactory.java:158)
at com.neo4j.server.enterprise.EnterpriseManagementServiceFactory.createManagementService(EnterpriseManagementServiceFactory.java:38)
at com.neo4j.server.enterprise.EnterpriseBootstrapper.createNeo(EnterpriseBootstrapper.java:20)
at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:117)
at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:87)
at com.neo4j.server.enterprise.EnterpriseEntryPoint.main(EnterpriseEntryPoint.java:25)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.bolt.BoltServer#29dcdd1c' failed to initialize. Please see the attached cause exception "io.netty.internal.tcnative.SSLPrivateKeyMethod".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:424)
at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:65)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:101)
at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:189)
... 6 more
Caused by: java.lang.NoClassDefFoundError: io/netty/internal/tcnative/SSLPrivateKeyMethod
at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:471)
at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:571)
at org.neo4j.ssl.SslPolicy.nettyServerContext(SslPolicy.java:82)
at org.neo4j.bolt.BoltServer.createSslContext(BoltServer.java:286)
at org.neo4j.bolt.BoltServer.createExternalProtocolInitializer(BoltServer.java:252)
at org.neo4j.bolt.BoltServer.init(BoltServer.java:154)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:403)
... 9 more
Caused by: java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSLPrivateKeyMethod
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 16 more
2020-07-06 16:54:25.162+0000 INFO Neo4j Server shutdown initiated by request
Server fails to start when line dbms.connector.bolt.tls_level=OPTIONAL is uncommented.
If I am not mistaken it is related to io/netty/internal/tcnative/SSLPrivateKeyMethod
I spent much time trying to figure out what is the problem and I have no clue how to proceed anymore.
I generated self-signed certificate with command:
openssl req \
-newkey rsa:2048 -nodes -keyout private.key \
-x509 -days 365 -out public.crt
certificate folder structure is following
└── default
├── private.key
├── public.crt
├── revoked
└── trusted
plugins are following:
netty-all-4.1.50.Final-sources.jar
netty-all-4.1.50.Final.jar
My configuration neo4j.conf is following:
#*****************************************************************
# Neo4j configuration
#
# For more details and a complete list of settings, please see
# https://neo4j.com/docs/operations-manual/current/reference/configuration-settings/
#*****************************************************************
# This setting constrains all `LOAD CSV` import files to be under the `import` directory. Remove or comment it out to
# allow files to be loaded from anywhere in the filesystem; this introduces possible security problems. See the
# `LOAD CSV` section of the manual for details.
dbms.directories.import=import
# Whether requests to Neo4j are authenticated.
# To disable authentication, uncomment this line
dbms.security.auth_enabled=true
#********************************************************************
# Memory Settings
#********************************************************************
# Java Heap Size: by default the Java heap size is dynamically calculated based
# on available system resources. Uncomment these lines to set specific initial
# and maximum heap size.
dbms.memory.heap.initial_size=512m
dbms.memory.heap.max_size=1G
# The amount of memory to use for mapping the store files.
# The default page cache memory assumes the machine is dedicated to running
# Neo4j, and is heuristically set to 50% of RAM minus the Java heap size.
dbms.memory.pagecache.size=512m
# Transaction state location. It is recommended to use ON_HEAP.
dbms.tx_state.memory_allocation=ON_HEAP
#*****************************************************************
# Network connector configuration
#*****************************************************************
# With default configuration Neo4j only accepts local connections.
# To accept non-local connections, uncomment this line:
dbms.default_listen_address=0.0.0.0
# Bolt connector
dbms.connector.bolt.enabled=true
dbms.connector.bolt.tls_level=OPTIONAL
#dbms.connector.bolt.listen_address=:7687
#dbms.connector.bolt.advertised_address=localhost:9000
# HTTP Connector. There can be zero or one HTTP connectors.
dbms.connector.http.enabled=true
#dbms.connector.http.listen_address=:7474
# HTTPS Connector. There can be zero or one HTTPS connectors.
dbms.connector.https.enabled=true
#dbms.connector.https.listen_address=:7473
#*****************************************************************
# SSL policy configuration
#*****************************************************************
# Bolt SSL configuration
dbms.ssl.policy.bolt.enabled=true
dbms.ssl.policy.bolt.base_directory=certificates/default
dbms.ssl.policy.bolt.private_key=private.key
dbms.ssl.policy.bolt.public_certificate=public.crt
dbms.ssl.policy.bolt.client_auth=NONE
dbms.ssl.policy.bolt.trust_all=true
# Https SSL configuration
dbms.ssl.policy.https.enabled=true
dbms.ssl.policy.https.base_directory=certificates/default
dbms.ssl.policy.https.private_key=private.key
dbms.ssl.policy.https.public_certificate=public.crt
dbms.ssl.policy.https.client_auth=NONE
dbms.ssl.policy.https.trust_all=true
#*****************************************************************
# Miscellaneous configuration
#*****************************************************************
# A comma separated list of procedures and user defined functions that are allowed
# full access to the database through unsupported/insecure internal APIs.
dbms.security.procedures.unrestricted=apoc.*
#********************************************************************
# JVM Parameters
#********************************************************************
# G1GC generally strikes a good balance between throughput and tail
# latency, without too much tuning.
dbms.jvm.additional=-XX:+UseG1GC
# Have common exceptions keep producing stack traces, so they can be
# debugged regardless of how often logs are rotated.
dbms.jvm.additional=-XX:-OmitStackTraceInFastThrow
# Make sure that `initmemory` is not only allocated, but committed to
# the process, before starting the database. This reduces memory
# fragmentation, increasing the effectiveness of transparent huge
# pages. It also reduces the possibility of seeing performance drop
# due to heap-growing GC events, where a decrease in available page
# cache leads to an increase in mean IO response time.
# Try reducing the heap memory, if this flag degrades performance.
dbms.jvm.additional=-XX:+AlwaysPreTouch
# Trust that non-static final fields are really final.
# This allows more optimizations and improves overall performance.
# NOTE: Disable this if you use embedded mode, or have extensions or dependencies that may use reflection or
# serialization to change the value of final fields!
dbms.jvm.additional=-XX:+UnlockExperimentalVMOptions
dbms.jvm.additional=-XX:+TrustFinalNonStaticFields
# Disable explicit garbage collection, which is occasionally invoked by the JDK itself.
dbms.jvm.additional=-XX:+DisableExplicitGC
#Increase maximum number of nested calls that are can be inlined from 9 (default) to 15
dbms.jvm.additional=-XX:MaxInlineLevel=15
# Restrict size of cached JDK buffers to 256 KB
dbms.jvm.additional=-Djdk.nio.maxCachedBufferSize=262144
# More efficient buffer allocation in Netty by allowing direct no cleaner buffers.
dbms.jvm.additional=-Dio.netty.tryReflectionSetAccessible=true
# Expand Diffie Hellman (DH) key size from default 1024 to 2048 for DH-RSA cipher suites used in server TLS handshakes.
# This is to protect the server from any potential passive eavesdropping.
dbms.jvm.additional=-Djdk.tls.ephemeralDHKeySize=2048
# This mitigates a DDoS vector.
dbms.jvm.additional=-Djdk.tls.rejectClientInitiatedRenegotiation=true
# Increase the default flight recorder stack sampling depth from 64 to 256, to avoid truncating frames when profiling.
dbms.jvm.additional=-XX:FlightRecorderOptions=stackdepth=256
# Allow profilers to sample between safepoints. Without this, sampling profilers may produce less accurate results.
dbms.jvm.additional=-XX:+UnlockDiagnosticVMOptions
dbms.jvm.additional=-XX:+DebugNonSafepoints
#********************************************************************
# Wrapper Windows NT/2000/XP Service Properties
#********************************************************************
# WARNING - Do not modify any of these properties when an application
# using this configuration file has been installed as a service.
# Please uninstall the service before modifying this section. The
# service can then be reinstalled.
# Name of the service
dbms.windows_service_name=neo4j
#********************************************************************
# Other Neo4j system properties
#********************************************************************
dbms.netty.ssl.provider=OPENSSL
Does anyone have clue what is the reason for my issues?
The error log is saying that the class SSLPrivateKeyMethod is not found. The error is not related to the content of the certificate.
Check the classpath that Neo4j Server refers to. For example, check if netty-all-4.1.50.Final.jar contains the SSLPrivateKeyMethod class.
What worked for me was replacing
netty-all-4.1.50.Final-sources.jar
netty-all-4.1.50.Final.jar
with
netty-tcnative-2.0.31.Final-osx-x86_64.jar
netty-tcnative-2.0.31.Final-sources.jar
I've setup a data pipeline using divolte.io to stream click data from website to a server. I'm not sure how can I do this for multiple websites because all the streams can get mixed up. Any ideas on how to do this?
On the same server, you need to bind to different ports
Create more than one config file, setting divolte.global.server.port to different values, then run the application with those configs.
In order to set a new config file, it actually needs to be in it's own directory
Divolte Collector will try to find configuration files at startup in the configuration directory. Typically this is the conf/ directory nested under the Divolte Collector installation. Divolte Collector will try to locate the configuration directory at ../conf relative to the startup script. The configuration directory can be overridden by setting the DIVOLTE_CONF_DIR environment variable. If set, the value will be used as configuration directory
Alternatively, you could run the exact same config within many containers/VMs, then use port mappings around that
I am having some confusion over configuring Sonar plugin on Jenkins. I went to Manage Jenkins -> Configure System and added Sonar.I am confused about what to put in the Database URL in the Sonar section.
I put
jdbc:mysql://10.4.1.206/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true
10.4.1.206 is the node I am connecting to.
However, the port is 3306.
Should I put
jdbc:mysql://10.4.1.206:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true or just leave it like before?
Also, since I am using MySQL, do I need to put com.mysql.jdbc.Driver in the Driver section? It says to leave it blank if I am using embedded default driver.
Please forgive me; this is my first time tampering with both Jenkins and Sonar.
In case you have configured your Sonar to use MySQL, you need to provide both the URL and the driver. The default, embedded database for Sonar is Derby - below you will find a sample of a default sonar configuration:
# Comment the 3 following lines to deactivate the default embedded database
sonar.jdbc.url: jdbc:derby://localhost:1527/sonar;create=true
sonar.jdbc.driverClassName: org.apache.derby.jdbc.ClientDriver
sonar.jdbc.validationQuery: values(1)
So, if you have configured your Sonar to use MySQL, and I can only assume that you had, let's analyze the configuration itself:
The driver that you need to explicitly declare is com.mysql.jdbc.Driver.
Yours URL string looks good to me. According to the MySQL Connect specification:
The JDBC URL format for MySQL Connector/J is as follows, with items in square brackets ([, ]) being optional:
jdbc:mysql://[host][,failoverhost...][:port]/[database] »
[?propertyName1][=propertyValue1][&propertyName2][=propertyValue2]...
If the host name is not specified, it defaults to 127.0.0.1. If the port is not specified, it defaults to 3306, the default port number for MySQL servers.
jdbc:mysql://[host:port],[host:port].../[database] »
[?propertyName1][=propertyValue1][&propertyName2][=propertyValue2]...
In my current setup the connection is as follows:
jdbc:mysql://localhost:3306/radical_sonar?useUnicode=true&characterEncoding=utf8
I tend to use the port number explicitly in order to avoid confusion rather than anything else - we do have a test MariaDB install running on a different port...
In Manage Jenkins > Configure System, your Sonar-Settings should be as follows:
Database URL should be:
jdbc:mysql://10.4.1.206:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true
Driver should be:
com.mysql.jdbc.Driver
If you need more information, you might also want to have a look at your "sonarqube/conf/sonar.properties" File and the following documentation link
http://docs.codehaus.org/display/SONAR/Configuring+SonarQube+Jenkins+Plugin
Good Luck with your configuration!