I use the ETL Talend Open Studio (TOS). I want transfered a data base A into a data base B. I use a tMap component. When I use a tLogRow to look results, it's ok. TOS shows data correctly. But when I make the transfer, TOS writes "Lock wait timeout exceeded; try restarting transaction".
I don't understand this problem... It's ok for the reading of data but there is a problem for the writing of data.
Can you help me, please ?
Try running your job using a single connection to Mysql ( I assume you are using it as the error is a mysql error )
The error above can occur when you attempt to insert/update/delete from two or more connections concurrently.
to create a single connection and have all components share it you will need a pair of components: "tMysqlConnection" and "tMysqlCommit"
the Connection component should be placed before you attempt to query the database. Once you have it in the job, you can link the tMysqlInput components to it by selecting "use existing connection"
The Commit component will issue the commit command and close the transaction.
You will need Connection components for each separate DB server you are working with.
The base A contains 300 articles. I think that this problem is caused by Talend Open Studio. TOS can't execute more 100 articles. I have tried to "cut" the base A in three bases. Then, I run TOS. The error has missing. It's strange... but it works.
Related
Actual Situation:
We use IBM Data Replication (11.4) to replicate Data from an Informix Database to an SQL Server Database.
Now we have an instance with 45 different subscriptions. On the informix side, we have 30 different log files.
The Problem:
When we want to “refresh” all subscriptions at once, we get in trouble that some logs aren’t available anymore, because they were overwritten.
The problem is that these logs were not full to 100 percent, but instead only to approximately 0,5%.
I don’t know when exactly a new log will be created.
Is there any possibility to change the settings, at which time a new logfile will be created? or that a new logfile only will be created when it is full to 100% or something else? Or do you have another solution to that problem at all?
We have found the problem:
The parameter “log_api_switch_log_num_pages” has to be defined. It describes log switching after a refresh.
See details here:
http://www-01.ibm.com/support/docview.wss?uid=swg21997830
I am new to Neo4j , using Desktop (3.4.1 Enterprise Version).
I have used LOAD CSV utility executed from Cypher Shell Command close to 1 Million records in the file. I have monitored the load using Neo4j browser by monitoring the count of properties and relationships that was created. Every time the load stops with the error "BoltConnectionError: No connection found, did you connect to Neo4j?" . I have also tried monitoring through the browser localhost:7474 - the error is different "server connection time out.. " , but the end result is that the load CSV fails to completed. Could someone please advise or guide me what needs to be done to resolve this issue ?
You should be loading along with USING PERIODIC COMMIT when loading data to batch the load and avoid killing your heap.
Also, you may want to EXPLAIN your query and ensure your load is using index lookups, especially if you're doing MERGEs on node properties.
In your query plan, watch out for Eager operations, that will effectively kill your periodic commit approach (and the browser should warn you with that query if it's in the query box prior to executing). You should include your query here for analysis and troubleshooting (along with the query plan) if the previous advice isn't enough to help you pinpoint the issue.
I've read all post with the same or very close headline, but still can't find a proper solution or explanation to my problem.
I'm working with MySQL Workbench 6.3 CE. I have been able to create a database with several tables, and create a conexion with python to write data to it. Still, I had a problem related to a varchar field that needed to be set to more than 45 characters. When I try to set it to bigger limits, like VARCHAR(70), no matter how many times I try, wether I set higher limits for timeout, I get the 2013 error, saying my connection was closed during the query.
I'm using the above version of workbench, on windows 10, and I'm trying to modify that field from the workbench. Afer that first time, I can't drop a table either, nor can I connect from python.
What is happening?
Ok, apparently what was happening is that I had a block, and there where a lot of query waiting in a situation of "waiting for table metadata block".
I did the following in the console of workbench
Select concat('KILL ',id,';') from information_schema.processlist where user='root'
that generates a list of all those processes. I copy that list in a new tab, and execute a massive kill of processes. After that it worked again.
Can anybody explain me how did I arrive to that situation and what precautions to take in my python scripts so as to avoid it?
Thnak you
I'm using Neo4j over windows for testing purposes and I'm working with a db containing ~2 million relations and about the same amount of nodes. after I had an ungraceful shutdown of neo4j while writing a batch of relations the db got corrupted.
it seems like there are some broken nodes/relations in the db and whenever I try to read them I get this error (I'm using py2neo):
Error: NodeImpl#1292315 not found. This can be because someone else deleted this entity while we were trying to read properties from it, or because of concurrent modification of other properties on this entity. The problem should be temporary.
I tried rebooting but neo4j fails to recover from this error. I found this question:
Neo4j cannot read certain nodes. Throws NotFoundException. Corrupt database
but the answer he got is no good for me because it involved in going over the db and redo the indexing, and I can't even read those broken nodes/relations so I can't fix their index (tried it and got the same error).
In general I've had many stability issues with neo4j (and on multiple platforms, not just windows). if no decent solution is found for this problem I will have to switch to a different database.
thanks in advance!
I wrote a tool a while ago that allows you to copy a broken store and keeps the good records intact.
You might want to check it out. I assume you used the 2.1.x version of Neo4j.
https://github.com/jexp/store-utils/tree/21
For 2.0.x check out:
https://github.com/jexp/store-utils/tree/20
To verify if your datastore is consistent follow the steps mentioned in http://www.markhneedham.com/blog/2014/01/22/neo4j-backup-store-copy-and-consistency-check/.
Are you referring to batch inserter API when speaking of "while writing a batch of relations"?
If so, be aware that batch inserter API requires a clean shutdown, see the big fat red warning on http://docs.neo4j.org/chunked/stable/batchinsert.html.
Are the broken nodes schema indexed and are you attempting to read them via this indexed label/property? If so, it's possible you may have a broken index following the sudden shutdown.
Assuming this is the case, you could try deleting the schema subdirectory within the graph store directory while the server is not running and let the database rebuild the index on restart. While this isn't an official way to recover from a broken index, it can sometimes work. Obviously, I suggest you back up your store before trying this.
We have some Delphi code that uses the BDE to Access SQL Server 2008 through the SQL Server Native Client ODBC driver (2005 version). Our issue is that we're experiencing some deadlock issues in a loop doing inserts to multiple tables.
The whole loop is done within a [TDatabase].StartTransaction. Looking at the SQL Server Profiler we clearly see that at one point during the loop the SPID (Session ID?) change, and then we naturally end up with a deadlock. (Both SPID doing inserts to the same table)
It seems like the BDE at some point does a second connection to the DB...
(Although I would love to skip the BDE, it's currently not possible. )
Anyone with experiences to share?
In case your app is multithreaded: BDE is not threadsafe. You have to use a separate BDE session (explicitly created instance of TSession) for each thread; the global Session created automatically for the main thread is not sufficient. Also, all database access components (TDatabase, TQuery, etc.) can only be used in the context of the thread where their corresponding instance of TSession has been created.
Verify in the ODBC installation if SQL Server driver is configured to do connection pooling.
Appear that Native Client installation activates it for default... (At least, mine installation had connection pooling active and I don't activated it).
This probably comes too late for the asker, but maybe it helps others.
Everytime there is a cursor that doesn't get closed, the BDE/ODBC combo will establish a new connection for successive querys. The "spid change" is probably the result of a non-closed cursor.
To solve this problem you have to find the BDE-component that caused this stil-opened cursor. Then you call a method that will eventually close the cursor (TTable.Close, TTable.Last ...).
After that the "spid change" should be gone and therefore the deadlock.
Some tips to find that component:
During the lock, execute the following statement (for example using Management Studio):
EXEC sp_who2.
Look in column BlkBy. The blocked connection has a number in it.
This number is the spid (Server Process ID) of the blocking connection.
Then you execute DBCC INPUTBUFFER(spid).
In column EventInfo you will find the sql-statement that has been issued by your programm.
With that information you should be able to find the BDE-component that causes your trouble.