I'm trying to bound somehow off-heap memory consumption in ksqlDB server. I found an article about this: https://www.confluent.io/blog/bounding-ksqldb-memory-usage. In first approach, I set KSQL_OPTS:
-Dksql.streams.rocksdb.config.setter=io.confluent.ksql.rocksdb.KsqlBoundedMemoryRocksDBConfigSetter
but I got:
[2020-11-16 07:25:15,848] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:60)
org.apache.kafka.common.config.ConfigException: Invalid value io.confluent.ksql.rocksdb.KsqlBoundedMemoryRocksDBConfigSetter for configuration rocksdb.config.setter: Class io.confluent.ksql.rocksdb.KsqlBoundedMemoryRocksDBConfigSetter could not be found.
at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:728)
at io.confluent.ksql.config.ConfigItem$Resolved.parseValue(ConfigItem.java:125)
at io.confluent.ksql.util.KsqlConfig.lambda$resolveStreamsConfig$2(KsqlConfig.java:693)
at java.base/java.util.Optional.map(Optional.java:265)
at io.confluent.ksql.util.KsqlConfig.resolveStreamsConfig(KsqlConfig.java:693)
at io.confluent.ksql.util.KsqlConfig.lambda$applyStreamsConfig$0(KsqlConfig.java:675)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1746)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
at io.confluent.ksql.util.KsqlConfig.applyStreamsConfig(KsqlConfig.java:678)
at io.confluent.ksql.util.KsqlConfig.buildStreamingConfig(KsqlConfig.java:701)
at io.confluent.ksql.util.KsqlConfig.<init>(KsqlConfig.java:738)
at io.confluent.ksql.util.KsqlConfig.<init>(KsqlConfig.java:708)
at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:50)
I also tried to set this parameter in ksqldb-server.properties but result was the same. I'm using docker image: confluentinc/ksqldb-server:0.9.0. Is it correct way to do this in such a way or I miss something?
The ksqldb-rocksdb-config-setter jar isn't included in the Docker image. You can pull the jar in from confluent's maven repository.
See: https://github.com/confluentinc/ksql/issues/6644
I'm running through the upcase courses and they've introduced me to VIM which, with the help of tmux, is slowly replacing my use of Atom. I've been trying to use some simple plugins using Vundler and every time I run Vim . from my rails directory I get the following errors:
E121: Undefined variable: VundleInstalled
E15: Invalid expression: VundleInstalled == 1
I have been up and down this mean, mean internet and cannot find a solution to this issue.
My ~/.vimrc
1 set number
2 set nocompatible
3 filetype off
4
5 syntax on
6 colorscheme Tomorrow-Night
7
8 set rtp+=~/.vim/bundle/vundle.vim
9 call vundle#begin()
10
11 Plugin 'VundleVim/Vundle.vim'
12 Plugin 'tpope/vim-fugitive'
13 Plugin 'wincent/command-t'
14 Plugin 'rstacruz/sparkup', {'rtp': 'vim/'}
15 Plugin 'vim-scripts/L9', {'name': 'newL9'}
16 Plugin 'tpope/vim-rails'
17 Plugin 'tpope/vim-rake'
18 Plugin 'tpope/vim-bundler'
19
20 " Color Schemes
21 Plugin 'chriskempson/tomorrow-theme'
22 Plugin 'flazz/vim-colorschemes'
23 Plugin 'chriskempson/base16-vim'
The error I get occurs any time I try to open a file in VIM. On top of that, when I try and work in VIM I get the following message every time I try and open a directory or run a command:
Error detected while processing FileType Auto commands for "*": │Maximum connections set to 1024
E117: Unknown function: rails#buffer_setup
Any help you could offer would be fantastic. I'm really digging VIM and tmux but I'm also discovering there's a bit of a learning curve to both of them.
Thanks!
Update: I have adjusted any issues with matching file names and, as far as I can tell it's not disrupting functionality. All the same, I would like to figure out why I"m getting this error.
I've been using the Neo4j batch loader for a while now and tonight started running into issues building my graph from a fresh database export. Running it yields the following:
> java -servjava -server -Xmx4G -jar ~/Dev/github.com/jexp/batch-import/target/batch-import-jar-with-dependencies.jar ./graph.db nodes.csv rels.csv node_index entities exact entities_idx.csv
Usage: Importer data/dir nodes.csv relationships.csv [node_index node-index-name fulltext|exact nodes_index.csv rel_index rel-index-name fulltext|exact rels_index.csv ....]
Using: Importer ./graph.db nodes.csv rels.csv node_index entities exact entities_idx.csv
Using Existing Configuration File
........................
Importing 2412268 Nodes took 4 seconds
.....................
Total import time: 9 seconds
Exception in thread "main" org.neo4j.graphdb.NotFoundException: id=2412269
at org.neo4j.unsafe.batchinsert.BatchInserterImpl.getNodeRecord(BatchInserterImpl.java:917)
at org.neo4j.unsafe.batchinsert.BatchInserterImpl.createRelationship(BatchInserterImpl.java:471)
at org.neo4j.batchimport.Importer.importRelationships(Importer.java:136)
at org.neo4j.batchimport.Importer.doImport(Importer.java:214)
at org.neo4j.batchimport.Importer.main(Importer.java:78)
I was able to successfully run the batch loader for the nodes.csv and rels.csv that are included in its own repository, so I'm thinking that the issue is somewhere in my rels.csv file. However, it's a pretty big file and I would like to know what id=2412269 means, as it seems like the best starting point for diagnosing the failure.
Any ideas?
_howard
This means that in the rels.csv file, you are trying to create a relationship for a node referenced by id = 2412269 . But no such node has been created in your nodes.csv file.
After working through the issue with the author of the importer, it turns out that the issue was that I had single, unescaped quotes in my nodes.csv file. So the rels.csv record was pointing to a node that could not be created in nodes.csv. Unfortunately, the error reported on the console was not exactly the error causing the issue.
I compiled CouchDB and installed. It seems to work great except when I use views on the database, then it just spins the wheel and nothing happens and the cpu load spikes to 100% and slowly it eats away all memory and starts to swap a lot which in return increases the cpu load.
I have tried both with the js-1.70-12 that comes with centos 6.4, as well as build and install my own js-1.85-1. All erlang packages are installed from epel :
erlang-crypto-R14B-04.2.el6.x86_64
erlang-syntax_tools-R14B-04.2.el6.x86_64
erlang-mnesia-R14B-04.2.el6.x86_64
erlang-ssl-R14B-04.2.el6.x86_64
erlang-cosProperty-R14B-04.2.el6.x86_64
erlang-asn1-R14B-04.2.el6.x86_64
erlang-cosEventDomain-R14B-04.2.el6.x86_64
erlang-eunit-R14B-04.2.el6.x86_64
erlang-erl_docgen-R14B-04.2.el6.x86_64
erlang-toolbar-R14B-04.2.el6.x86_64
erlang-debugger-R14B-04.2.el6.x86_64
erlang-tools-R14B-04.2.el6.x86_64
erlang-typer-R14B-04.2.el6.x86_64
erlang-megaco-R14B-04.2.el6.x86_64
erlang-oauth-1.1.1-1.el6.x86_64
erlang-stdlib-R14B-04.2.el6.x86_64
erlang-hipe-R14B-04.2.el6.x86_64
erlang-kernel-R14B-04.2.el6.x86_64
erlang-runtime_tools-R14B-04.2.el6.x86_64
erlang-snmp-R14B-04.2.el6.x86_64
erlang-public_key-R14B-04.2.el6.x86_64
erlang-inets-R14B-04.2.el6.x86_64
erlang-ibrowse-2.2.0-4.el6.x86_64
erlang-cosEvent-R14B-04.2.el6.x86_64
erlang-cosNotification-R14B-04.2.el6.x86_64
erlang-edoc-R14B-04.2.el6.x86_64
erlang-otp_mibs-R14B-04.2.el6.x86_64
erlang-cosFileTransfer-R14B-04.2.el6.x86_64
erlang-cosTransactions-R14B-04.2.el6.x86_64
erlang-inviso-R14B-04.2.el6.x86_64
erlang-jinterface-R14B-04.2.el6.x86_64
erlang-erl_interface-R14B-04.2.el6.x86_64
erlang-diameter-R14B-04.2.el6.x86_64
erlang-gs-R14B-04.2.el6.x86_64
erlang-tv-R14B-04.2.el6.x86_64
erlang-appmon-R14B-04.2.el6.x86_64
erlang-odbc-R14B-04.2.el6.x86_64
erlang-wx-R14B-04.2.el6.x86_64
erlang-et-R14B-04.2.el6.x86_64
erlang-observer-R14B-04.2.el6.x86_64
erlang-sasl-R14B-04.2.el6.x86_64
erlang-dialyzer-R14B-04.2.el6.x86_64
erlang-common_test-R14B-04.2.el6.x86_64
erlang-os_mon-R14B-04.2.el6.x86_64
erlang-examples-R14B-04.2.el6.x86_64
erlang-compiler-R14B-04.2.el6.x86_64
erlang-erts-R14B-04.2.el6.x86_64
erlang-xmerl-R14B-04.2.el6.x86_64
erlang-orber-R14B-04.2.el6.x86_64
erlang-cosTime-R14B-04.2.el6.x86_64
erlang-ssh-R14B-04.2.el6.x86_64
erlang-docbuilder-R14B-04.2.el6.x86_64
erlang-percept-R14B-04.2.el6.x86_64
erlang-parsetools-R14B-04.2.el6.x86_64
erlang-ic-R14B-04.2.el6.x86_64
erlang-pman-R14B-04.2.el6.x86_64
erlang-webtool-R14B-04.2.el6.x86_64
erlang-test_server-R14B-04.2.el6.x86_64
erlang-reltool-R14B-04.2.el6.x86_64
erlang-R14B-04.2.el6.x86_64
erlang-mochiweb-1.4.1-5.el6.x86_64
Every thing configures and makes and installs as expected. You can dump data into the database, you can create documents and all that. But I can not run any view, temporary or not.
The only error I see in the logs is like this one, and it is a lot of these errors :
[Sun, 18 Aug 2013 23:10:38 GMT] [error] [<0.124.0>] {error_report,<0.30.0>,
{<0.124.0>,crash_report,
[[{initial_call,
{mochiweb_socket_server,init,['Argument__1']}},
{pid,<0.124.0>},
{registered_name,[]},
{error_info,
{exit,eaddrinuse,
[{gen_server,init_it,6},
{proc_lib,init_p_do_apply,3}]}},
{ancestors,
[couch_secondary_services,couch_server_sup,
<0.31.0>]},
{messages,[]},
{links,[<0.93.0>]},
{dictionary,[]},
{trap_exit,true},
{status,running},
{heap_size,987},
{stack_size,24},
{reductions,459}],
[]]}}
But I have no idea what they mean.
Do I need to compile and install erlang as well ? All the above packages or just erlang ?
Your compilation and installation looks fine. At least your error (note: eaddrinuse in traceback) is about that there is some process that listens same address and port as your CouchDB try to. Check other listening processes with netstat -anp command or change CouchDB's listen port to different.
I am seeing this error for quite sometime now.
I am running ant build on CYGWIN which inturn runs on WindowsXP.
The resolution(bad one) I found was to delete my gcct/first directory and run ant build again (which runs from another directory). It runs successfully but if I modify some code under gcct/first, I do not want to delete it because of this error.
I did see this link. The resolution here does not apply to me since I do not have .cvspass defined anywhere in the build.xml.
C:\svn\CEL_v3681\buildCore.xml:1883: cvs exited with error code 1
Command line was [Executing 'cvs' with arguments:
'checkout'
'-A'
'-rfirst_v2_126'
'gcct/first'
The ' characters around the executable and arguments are
not part of the command.
environment:
ALLUSERSPROFILE=C:\Documents and Settings\All Users
ANT_HOME=C:/Apps/Apache/apache-ant-1.7.0
APPDATA=C:\Documents and Settings\shankarc\Application Data
CLASSPATH=./;C:/Program Files/Java/jre1.5.0_07/lib/ext/QTJava.zip
COMMONPROGRAMFILES=C:\Program Files\Common Files
COMPUTERNAME=NYKPWM2035798
COMSPEC=C:\WINNT\system32\cmd.exe
CUSTPROF=Roaming700Live
CVSROOT=:pserver:shankarc#amcvs2.lehman.com:/home/eqcvs/cmte
CVS_RSH=/bin/ssh
FP_NO_HOST_CHECK=NO
HOME=C:\Apps\CYGWIN\home\shankarc
HOMEDRIVE=F:
HOMEPATH=\
HOSTNAME=nykpwm2035798
IDEA_PROPERTIES=C:\Documents and Settings\shankarc\idea.properties
INFOPATH=/usr/local/info:/usr/share/info:/usr/info:
JAVA_HOME=C:/Program Files/Java/jdk1.6.0_21/
JDK_HOME=C:\Program Files\Java\jdk1.6.0_21\
LOGONSERVER=\\NYKPSM00069
MANPATH=/usr/local/man:/usr/share/man:/usr/man::/usr/ssl/man
NUMBER_OF_PROCESSORS=2
OS=Windows_NT
PATH=C:\Apps\CYGWIN\usr\local\bin;C:\Apps\CYGWIN\bin;C:\Apps\CYGWIN\bin;C:\Apps\CYGWIN\usr\X11R6\bin;C:\Apps\Apache\apache-ant-1.7.0\bin;C:\Program Files\Java\jdk1.6.0_21\bin\;C:\Apps\CYGWIN\bin;C:\Program Files\VisualSVN Server\bin;C:\Program Files\Sudowin\Clients\Console;C:\Program Files\Fortify Software\Fortify 360 v2.5.0\bin
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.PSC1
PRINTER=\\NYKPSM04020\NYKLPR1301-03-03C05
PROCESSOR_ARCHITECTURE=x86
PROCESSOR_IDENTIFIER=x86 Family 6 Model 15 Stepping 6, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=0f06
PROFGROUP=FONP
PROGRAMFILES=C:\Program Files
PROMPT=$P$G
PWD=/cygdrive/c/svn/CEL_v3681/gcct/cel
QHOME=c:\q
QTJAVA=C:\Program Files\Java\jre1.5.0_07\lib\ext\QTJava.zip
SESSIONNAME=Console
SHLVL=1
SITECODE=NYK
SITEIDENT=NYK
SVN_ASP_DOT_NET_HACK=1
SYSTEMDRIVE=C:
SYSTEMROOT=C:\WINNT
TEMP=C:\TEMP
TERM=cygwin
TMP=C:\TEMP
UATDATA=C:\WINNT\system32\CCM\UATData\D9F8C395-CAB8-491d-B8AC-179A1FE1BE77
USER=shankarc
USERDNSDOMAIN=INTRANET.BARCAPINT.COM
USERDOMAIN=INTRANET
USERNAME=shankarc
USERPROFILE=C:\Documents and Settings\shankarc
WINDIR=C:\WINNT
CVS_PASSFILE=C:\Apps\CYGWIN\home\shankarc\.cvspass]
Total time: 58 seconds
How I resolve this?
I had the same issue and found that even though I was not using .cvspass I did have a build property of cvs.pass set which needed to be reset to OVERRIDE to function depending on how you set up your cvs access (though it looked similar from your post). This needed to be changed in build.properties and .build.properties. Hope this helps!