There is one and only one connection and user here.
d = l.descriptions.first
Language#descriptions 1200270ms MATCH language137, language137-[rel1:`DESCRIBED_IN`]->(result_descriptions:`Description`) WHERE (ID(language137) = {ID_language137}) RETURN result_descriptions | {:ID_language137=>137}
Faraday::TimeoutError: too many connection resets (due to Net::ReadTimeout - Net::ReadTimeout) after 0 requests on 70156828873380, last used 1438628883.105085 seconds ago
After that no other connection is allowed until server is restarted.
What is wrong here?
Here is in more detail what I am trying to do: selecting a language i.e. English. Getting the count of descriptions in English. Searching the first description in English. This never returns or delivers the connection error. During the long run of the last one no other connections can be open to the database.
irb(main):001:0> l = Language.find_by(iso_639_2_code: 'eng')
CYPHER 316ms MATCH (n:`Language`) WHERE (n.iso_639_2_code = {n_iso_639_2_code}) RETURN n LIMIT {limit_1} | {:n_iso_639_2_code=>"eng", :limit_1=>1}
=> #<Language uuid: nil, english_name_of_language: "English", french_name_of_language: "anglais", german_name_of_language: "Englisch", iso_639_1_code: "en", iso_639_2_code: "eng", spoken_in: "English, a West Germanic language is the first language for about 309–400 million people. See: Countries by Languages - English Speaking Countries.">
irb(main):002:0>
irb(main):005:0* n = l.descriptions.count
Language#descriptions 17749ms MATCH language137, language137-[rel1:`DESCRIBED_IN`]->(result_descriptions:`Description`) WHERE (ID(language137) = {ID_language137}) RETURN count(result_descriptions) AS result_descriptions | {:ID_language137=>137}
=> 2107041
irb(main):006:0> d = l.descriptions.first
I think that we fixed this is version 5.0 of the gems. Could you try upgrading?
The issue was moved to the neo4jrb git repo and solved by the neo4j maintainers by recommending to upgrade to the gem core version 5.0.11 from 5.0.9
You might see this error if your Neo4j database server becomes unresponsive. To fix this, restart your Neo4j database server.
bundle exec rake neo4j:stop
bundle exec rake neo4j:start
Related
I'd like to start over with errbit - there are millions of records in our mongodb database and we hadn't been cleaning them up. I'd like to start over, but I don't want to lose my user accounts.
I've tried to run these routines (https://mensfeld.pl/2015/01/making-errbit-work-faster-by-keeping-it-clean-and-tidy/):
bundle exec rake errbit:clear_resolved
desc 'Resolves problems that didnt occur for 2 weeks'
task :cleanup => :environment do
offset = 2.weeks.ago
Problem.where(:updated_at.lt => offset).map(&:resolve!)
Notice.where(:updated_at.lt => offset).destroy_all
end
but the second one (deleting problems and notices over 2 weeks old), just seems to run forever.
Querying problems and notices collections via mongo shell doesn't seem to show any being deleted... we're using errbit V 0.7.0-dev and mongodb version 3.2.22.
Fastest way would be to get a mongo console and drop most of the collections. I'd say stop your errbit server, get a mongo console, connect to the db you use and run:
> db.errs.drop()
true
> db.problems.drop()
true
> db.backtraces.drop()
true
> db.notices.drop()
true
> db.comments.drop()
Problem.where(:updated_at.lt => 2.months.ago).destroy_all
runs too long because of N+1 problem with recursive deletion of Err, Notice and Comment, also mongoid does not support nested eager loading, so only way to delete faster - is to manually take these ids and delete directly, without callbacks:
problem_ids = Problem.where(:updated_at.lt => 2.months.ago).pluck(:id)
err_ids = Err.where(problem_id: {:$in => problem_ids}).pluck(:id)
Notice.where(err_id:{:$in => err_ids}).delete_all
Err.where(id:{:$in => err_ids}).delete_all
Comment.where(problem_id: {:$in => problem_ids}).delete_all
Problem.where(id: {:$in => problem_ids}).delete_all
I have a DBIx::Class query that's taking too long to complete.
All SQL below were generated by DBIx::Class.
First scenario (DBIx Simple select):
SELECT me.pf_id, me.origin_id, me.event_time, me.proto_id FROM pf me ORDER BY event_time DESC LIMIT 10;
DBIx query time: 0.390221s (ok)
Second scenario (DBIx Simple select using where):
SELECT me.pf_id, me.origin_id, me.event_time, me.proto_id FROM pf me WHERE ( proto_id = 7 ) ORDER BY event_time DESC LIMIT 10;
DBIx query time: 29.27025s!! :(
Third scenario (Using pgadmin3 to run the query above):
SELECT me.pf_id, me.origin_id, me.event_time, me.proto_id FROM pf me WHERE ( proto_id = 7 ) ORDER BY event_time DESC LIMIT 10;
Pgadmin query time: 25ms (ok)
The same query is pretty fast using pgdamin.
Some info:
Catalyst 5.90091
DBIx::Class 0.082820 (latest)
Postgres 9.1
I did all tests on localhost, using Catalyst internal server.
I have no problems with any other table/column combination, it's specific with proto_id.
Database Schema automatically generated by DBIx::Class::Schema::Loader
proto_id definition:
"proto_id",
{ data_type => "smallint", is_foreign_key => 1, is_nullable => 0 },
Anybody have a clue why DBIx is taking so long to run this simple query?
Edit 1: Column is using index (btree).
Edit 2: This is a partitioned table, I'm checking if all the sub-tables have all the indexes, but still doesn't explain why the same query is slower on DBIx::Class.
Edit 3: I did a simple DBIx::Class script and I got the same results, just to make sure the problem is not the Catalyst Framework.
Edit 4: Using tcpdump I noticed postgres is taking too long to respond, still trying...
Edit 5: Using DBI with SQL seems pretty fast, I'm almost convinced this is a DBIx::Class problem.
After some tests, I found the problem:
When I do the query using DBI bind_param() (As DBIx::Class does) for some reason it became very slow.
SELECT me.pf_id, me.origin_id, me.event_time, me.proto_id FROM pf me WHERE ( proto_id = ? ) ORDER BY event_time DESC LIMIT ?;
my $sth = $dbh->prepare($sql);
$sth->bind_param(1, 80, { TYPE => SQL_INTEGER });
$sth->bind_param(2, 10, { TYPE => SQL_INTEGER });
$sth->execute();
So after some time searching CPAN I've noticed that my DBD::Pg was outdated (My bad). I downloaded the source from CPAN, compiled and the problem is gone. Must be some bug from older versions.
TL;DR: If you're having problems with DBI or DBIx::Class make sure your DBI database driver is updated.
I'm trying to use the Java API for Neo4j but I seem to be stuck at IndexHits. If I query the DB with Cypher using
START n=node:types(type="Process") RETURN n;
I get all 2087 nodes of type "Process".
In my application I have the following lines
Index<Node> nodeIndex = db.index().forNodes("types");
IndexHits<Node> hits = nodeIndex.get("type", "Process");
System.out.println("Node index size: " + hits.size());
which leads my console to spit out a value of 0. Here, db is of course an instance of GraphDatabaseService.
I expected an object that included all 2087 nodes. What am I doing wrong?
The .size() question is just the prelude to my iterator
for(Node process : hits) { ... }
but that does not much when hits.size() == 0. According to http://api.neo4j.org/1.9.2/org/neo4j/graphdb/index/IndexHits.html this should be possible, provided there is something in hits.
Thanks in advance for your help.
I figured it out. Man, I feel so embarrassed...
It so happens that I had set up the DB_PATH to my default data folder, whereas the default storage folder is the default data folder plus graph.db. When I tried to run the code from that corrected DB_PATH I got an error saying that a lock file was in place because the Neo4j server was running. After shutting it down it worked perfectly.
So, if you happen to see the following error, just stop the server and run the code again:
Caused by: org.neo4j.kernel.StoreLockException: Could not create lock file
at org.neo4j.kernel.StoreLocker.checkLock(StoreLocker.java:74)
at org.neo4j.kernel.StoreLockerLifecycleAdapter.start(StoreLockerLifecycleAdapter.java:40)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:491)
I found on several forums that you cannot run the Neo4j server and use the Java API to query it at the same time.
Hullo,
We are attempting to set up an Neo4j HA cluster in our Rails dev environment, much like what is explained here: https://github.com/andreasronge/neo4j/wiki/Neo4j%3A%3ARails-Config
We have two instances in the cluster. Server 1 is the app, Server 2 is the Rails console. They both start fine, but eventually one of them will fall over. Usually, it's one of the following:
1) java.io.FileNotFound: /server_1_path/path/to/some/RailsModel_exact/_2.fxm file. Somehow, the indexes expect a file to exist that does not exist. Sometimes, the file does not exist in EITHER server directory, and the only thing that helps is to make both sets of index files identical by copying one to the other.
2) Orphaned index.lock files. The error here will say that a certain index is locked, and removing the specific .lock file fixes the issue. Annoying.(maybe similar issue)
3) Add data in one instance, never shows up in the other instance. In this case, I create a node in the Rails console, and it never shows up in the app, or vice versa. In this case, it seems that both instances start up as master and will never sync. Usually have to delete one of the dbs and restart to get them working again.
I am not sure if the new 1.9 HA stuff isn't ready for prime time or we are being too nonchalant with how we quit the app/console and Neo4j is not shutting down cleanly.
This is a highly frustrating issue. We'd appreciate any help/pointers to get it working right.
We are using the 1.9 M03 version of the gem, and here is our config:
server_id = ((defined? Rails::Console)) ? 2 : 1
config.neo4j['enable_ha'] = true
config.neo4j['enable_remote_shell'] = "port=133#{server_id}"
config.neo4j['ha.server_id'] = server_id
config.neo4j['ha.server'] = "localhost:600#{server_id}"
config.neo4j['ha.pull_interval'] = '1s'
config.neo4j['ha.discovery.enabled'] = false
config.neo4j['ha.initial_hosts'] = [1,2,3].map{|id| ":500#{id}"}.join(',')
config.neo4j['ha.cluster_server'] = ":5001-5099" #"#{server_id}"
config.neo4j.storage_path = File.expand_path("db/ha_neo_#{server_id}", Object::Rails.root)
config.neo4j['online_backup_server']= "localhost:636#{server_id}"
config.neo4j['ha.cluster_server'] = "localhost:500#{server_id}"
config.neo4j['webserver.port'] = "747#{server_id}"
config.neo4j['webserver.https.port'] = "748#{server_id}"
config.neo4j['enable_remote_shell'] = "port=933#{server_id}"
config.neo4j['use_adaptive_cache'] = false
puts "Config HA cluster, ha.server_id: #{config.neo4j['ha.server_id']}, db: #{config.neo4j.storage_path}"
Thanks for any/all help/advice.
I am running a Rails 3.1.0 app and I have an odd problem. On our staging server, with VERY little activity we have 5 ruby processes CONSTANTLY pinging mySQL with the following:
poll([{fd=12, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
write(12, "\f\0\0\0\3SHOW TABLES", 16) = 16
select(13, [12], NULL, NULL, NULL) = 1 (in [12])
read(12, "\1\0\0\1\1D\0\0\2\3def\0\vTABLE_NAMES\0\31Tabl"..., 16384) = 637
poll([{fd=12, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
write(12, "\f\0\0\0\3SHOW TABLES", 16) = 16
select(13, [12], NULL, NULL, NULL)
That last line is incomplete, but we're talking a few times every single second (x5/6 processes). The server is a beast, it has 32GB of RAM and has been optimised somewhat (the mySQL setup that is) but its killing the server.
Like I say, the server has very little activity, so its not users, or a task.
(For admins thinking of moving this away from this forum, I believe this is a ruby/rails issue, I'm not sure if it was in a server forum it would have a good compatibility with answerers)
I would be incredibly grateful for any advice, I fear it might be a bit over my head. I'm not such a Linux/mySQL pro.
Thanks
I would look at the connection pool for your database. Does running this help?
ActiveRecord::Base.clear_active_connections!
Specifically, in your config/database.yml for this environment, try setting pool: 50 and restart rails, then see if this affects the result. The next question if your pool is exhausted would be to get to the specifics of why the database connection pool is getting used up (this command, or something running in resque). I think the pool default size is 4 or 5