I have a Talend job that calls several Snowflake stored procedures in sequence. It uses tDBRow components connected by OnComponentOk to call the SPs. It works fine, except when an SP throws an error the OnComponentOk trigger still kicks off the next component - even though I see the error message in the Talend job log. What can I do to make sure the job terminates if any of the stored procedures fail?
Any suggestions are appreciated.
Related
Why would a Snowflake stored procedure hang on a statement that, when executed outside the stored procedure, works? Further info: I remove that statement from the stored procedure, then the SP also runs properly. How can this sort of thing be debugged?
(One more piece of info: running as a different user on a different schema, the SP works as intended.)
Update: running the SP on a different warehouse worked, so it might be a problem with the warehouse, not the schema.
Why would a Snowflake stored procedure hang on a statement that, when executed outside the stored procedure, works?
There can be multiple reasons: Query gets queued due to lack of resources, is awaiting a lock to free (if its a transactional query), etc.
How can this sort of thing be debugged?
Check the Query History UI page on Snowflake. If your procedure-executed statement is showing a queued status, you're likely running into a warehouse size limit or a maximum concurrency limit, which can be resolved by reconfiguring your warehouse (via auto-scaling and/or using higher warehouse sizes).
We have several data feeds that run every evening using SSIS packages with SQL table data sources. Part of this standard process is a data engine we've built using stored procedures that run for each data feed and returns that customers data based on their specific parameters. That engine dumps the data into a SQL table where it is retrieved by the package and then remains there until the next evenings run.
Maybe two weeks ago we started to intermittently get the following error running these stored procedures (which are executed using SQL Agent):
"INSERT EXEC failed because the stored procedure altered the schema of the target table. [SQLSTATE 42000] (Error 556). The step failed."
These stored procedures have been running for months, some even years, without being changed. These errors just started intermittently occurring. Inside the stored procedure we do have a temporary table being used that receives the engine data and a table that is dropped and re-created using a statement like this:
SELECT field1, field2 INTO sqlTable FROM #tempTable
I looked for a SQL updated or something that may have changed to cause these errors all of a sudden but can't find anything. It's occurred to several different stored procedures, intermittently, that all have this same kind of structure but I can't identify any particular reason. It will happen one night and not another, to one stored procedure and not another just like it. Any idea what could cause this?
We are running Microsoft SQL Server 2016 Standard 64-bit (13.0.4604.0) on Windows Server 2016 Datacenter 10.0 (Build 14393: ) (Hypervisor). This is all on a VM in the Azure environment.
If you are using "INSERT ... EXEC" and has enabled Query Store, it might be the reason.
The Query Data Store periodically runs auto-cleanup.
This has turned out to cause problems when a stored procedure makes a call to another stored procedure by using "INSERT…EXEC" syntax.
This is only an issue with SQL Server 2016
For more details and possible solution, see: https://support.microsoft.com/en-us/help/4465511/error-556-insert-exec-failed-stored-procedure-altered-table-schema
maybe I am a bad seeker but I couldn't find my answers in documentation, so I just want to try my luck here
So my question is that say I have a dataflow job that write to a BigQuery or BigTable and the job failed. Will dataflow will able to rollback to state before it started or there might simply be partial data in my table?
I know that write to GCS seems not atomic that there will be partial output partition produced along the way when the job is running.
However, I have tried dumping data into BQ by dataflow, and it seems that the output table will not be exposed to users until the job claimed success.
In Batch, Cloud Dataflow uses the following procedure for BigQueryIO.Write.to("some table"):
Write all data to a temporary directory on GCS.
Issue a BigQuery load job with an explicit list of all the temporary files containing the rows to be written.
If there are failures when the GCS writes are only partially complete, we will recreate the temp files on retry. Exactly one complete copy of the data will be produced by step 1 and used for loading in step 2, or the job will fail before step 2.
Each BigQuery load job, as in William V's answer, is atomic. The load job will succeed or fail, and if it fails there will be no data written to BigQuery.
For slightly more depth, Dataflow also uses a deterministic BigQuery job id (like dataflow_job_12423423) so that if the Dataflow code monitoring the load job fails and is retried we will still have exactly-once write semantics to BigQuery.
Together, this design means that each BigQueryIO.Write transform in your pipeline is atomic. In a common case, you have only one such write in your job, and so if the job succeeds the data will be in BigQuery and if the job fails there will be no data written.
However: Note that if you have multiple BigQueryIO.Write transforms in a pipeline, some of the writes may have successfully completed before the Dataflow job fails. The completed writes will not be reverted when the Dataflow job fails.
This means that you may need to be careful when rerunning a Dataflow pipeline with multiple sinks in order to ensure correctness in the presence of commited writes from the earlier failed job.
I can speak for Bigtable. Bigtable is atomic at the row level, not at the job level. A Dataflow job that fails part way will write partial data into Bigtable.
BigQuery jobs fail or succeed as a unit. From https://cloud.google.com/bigquery/docs/reference/v2/jobs
Each action is atomic and only occurs if BigQuery is able to complete the job successfully. Creation, truncation and append actions occur as one atomic update upon job completion.
Though, just to be clear, BigQuery is atomic at the level of the BigQuery job, not at the level of a Dataflow job that might have created the BigQuery job. E.g. if your Dataflow job fails but it has written to BigQuery before failing (and that BigQuery job is complete) then the data will remain in BigQuery.
I use the ETL Talend Open Studio (TOS). I want transfered a data base A into a data base B. I use a tMap component. When I use a tLogRow to look results, it's ok. TOS shows data correctly. But when I make the transfer, TOS writes "Lock wait timeout exceeded; try restarting transaction".
I don't understand this problem... It's ok for the reading of data but there is a problem for the writing of data.
Can you help me, please ?
Try running your job using a single connection to Mysql ( I assume you are using it as the error is a mysql error )
The error above can occur when you attempt to insert/update/delete from two or more connections concurrently.
to create a single connection and have all components share it you will need a pair of components: "tMysqlConnection" and "tMysqlCommit"
the Connection component should be placed before you attempt to query the database. Once you have it in the job, you can link the tMysqlInput components to it by selecting "use existing connection"
The Commit component will issue the commit command and close the transaction.
You will need Connection components for each separate DB server you are working with.
The base A contains 300 articles. I think that this problem is caused by Talend Open Studio. TOS can't execute more 100 articles. I have tried to "cut" the base A in three bases. Then, I run TOS. The error has missing. It's strange... but it works.
I've been running around circles the last 2 days, trying to figure a problem in our customers live environment. I figured I might as well post it here, since google gave me very limited information on the error message (5 results to be exact).
The error boils down to a timeout when requesting a certain report in SSRS2005, when a certain parameter is used.
The deployment scenario is:
Machine #1 Running reporting services (SQL2005, W2K3, IIS6)
Machine #2 Running datawarehouse database (SQL2005, W2K3) which is the data source for #1
Both machines are running on the same vm cluster and LAN.
The report requests a fairly simple SP - lets called it sp(param $a, param $b).
When requested with param $a filled, it executes correctly. When using param $b, it times out after the global timeout periode has passed.
If I run the stored procedure with param $b directly from sql management studio on #2, it returns the results perfectly fine (within 3-4s).
I've profiled the datawarehouse database on #2 and when param $b is used, the query from the reporting service to the database, never reaches #2.
The error message that I get upon timeout, when using param $b, when invoking the report directly from SSRS web interface is:
"An error has occurred during report processing.
Cannot read the next data row for the data set DataSet.
A severe error occurred on the current command. The results, if any, should be discarded. Operation cancelled by user."
The ExecutionLog for the SSRS does give me much information besides the error message rsProcessingAborted
I'm running out of ideas of how to nail this problem. So I would greatly appreciate any comments, suggestions or ideas.
Thanks in advance!
The first thing you need to do is to ensure your statistics are up to date.
(It sounds like a case of an incorrect query plan being used due to parameter sniffing, as described in this SO answer: Parameter Sniffing (or Spoofing) in SQL Server).
One way to fix this in SQL Server 2005, is using the OPTIMIZE FOR query hint. See also OPTIMIZE FOR query hint in SQL Server 2005
Also, do you have a regular scheduled index rebuild job for some or all of your indexes?