Transaction Not Working As Expected in Custom Procedure - neo4j

I'm having issues when trying to issue commits using a custom procedure. I am selecting data from an external datasource, and then transforming it into a graph DB. I am trying to make each row from the external datasource it's own transaction, but if any row fails, all the transactions fail. Please let me know where I've gone wrong. Here are snippets of my code:
#Context
public GraphDatabaseService db;
#SuppressWarnings("unchecked")
#Description("CALL myProc")
#Procedure(name = "myProc", mode = Mode.WRITE)
public void myProc() {
// Get the last time the procedure succeeded from the System node
try {
boolean processAgain = true;
int startRow = 1, endRow = 10000, batchSize = 10000;
int total = 0, succeeded = 0, failed = 0;
while (processAgain) {
result = db.execute(new StringBuilder("CALL apoc.load.jdbc(\"dbAlias\", \"SQL GOES HERE\")").toString());
while (result.hasNext()) {
total++;
Map<String, Object> row = result.next();
for (String key : result.columns()) {
Map<String, Object> currentRow = ((Map<String, Object>) row.get(key));
Transaction trans = db.beginTx();
log.info("Beginning transaction");
try {
// Do stuff here (i.e. db.execute...)
trans.success();
succeeded++;
} catch (Exception e) {
failed++;
log.error("Error message", e);
} finally {
trans.close();
}
}
} // End results while loop
if (total != 0) {
log.info(new StringBuilder("Processed ").append(total - startRow + 1).append(" rows").toString());
}
processAgain = total != 0 && total % endRow == 0;
startRow += batchSize;
endRow += batchSize;
} // End processAgain while loop
} catch (Exception e) {
log.error("Error message", e);
}
}
UPDATE: And here's the console output
ERROR (-v for expanded information):
TransactionFailureException: Transaction was marked as successful, but unable to commit transaction so rolled back.
org.neo4j.graphdb.TransactionFailureException: Transaction was marked as successful, but unable to commit transaction so rolled back.
at org.neo4j.kernel.impl.coreapi.TopLevelTransaction.close(TopLevelTransaction.java:100)
at org.neo4j.shell.kernel.apps.TransactionProvidingApp.execute(TransactionProvidingApp.java:250)
at org.neo4j.shell.kernel.apps.cypher.Start.execute(Start.java:82)
at org.neo4j.shell.impl.AbstractAppServer.interpretLine(AbstractAppServer.java:126)
at org.neo4j.shell.kernel.GraphDatabaseShellServer.interpretLine(GraphDatabaseShellServer.java:105)
at org.neo4j.shell.impl.RemotelyAvailableServer.interpretLine(RemotelyAvailableServer.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.neo4j.internal.kernel.api.exceptions.TransactionFailureException: Transaction rolled back even if marked as successful
at org.neo4j.kernel.impl.api.KernelTransactionImplementation.failOnNonExplicitRollbackIfNeeded(KernelTransactionImplementation.java:599)
at org.neo4j.kernel.impl.api.KernelTransactionImplementation.closeTransaction(KernelTransactionImplementation.java:541)
at org.neo4j.internal.kernel.api.Transaction.close(Transaction.java:189)
at org.neo4j.kernel.impl.coreapi.TopLevelTransaction.close(TopLevelTransaction.java:78)
... 22 more
Can someone please tell me what I'm doing wrong? Thanks.

Related

How to make two insertion all successful and failure?

In my recent project, for each user's payment, it needs to insert a Receipt and an Invoice into the SQL DB. I have 2 functions for it, InsertReceipt() and InsertInvoice(), so the code is like:
void DoPayment()
{
InsertReceipt();
InsertInvoice();
}
bool InsertReceipt()
{
// insert to SQL with a ReceiptId
// return true or false;
}
bool InsertInvoice()
{
// insert to SQL with an InvoiceId
// return true or false;
}
ReceiptId and InvoiceId have to be unique and consecutive here.
My question is, how can I do to make InsertReceipt() and InsertInvoice() all successful or all failure? Or I have to make a new function InsertReceiptAndInvoice(), and use SQL Transaction?
If you use a stored procedure, you absolutely can use transaction. You can read about transaction in Microsoft docs:
https://learn.microsoft.com/en-us/sql/t-sql/language-elements/begin-transaction-transact-sql?view=sql-server-ver15
With C#, you can use transaction with SqlConnection. Code example:
private static void ExecuteSqlTransaction(string connectionString)
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
SqlCommand command = connection.CreateCommand();
SqlTransaction transaction;
// Start a local transaction.
transaction = connection.BeginTransaction("SampleTransaction");
// Must assign both transaction object and connection
// to Command object for a pending local transaction
command.Connection = connection;
command.Transaction = transaction;
try
{
command.CommandText =
"Insert into Region (RegionID, RegionDescription) VALUES (100, 'Description')";
command.ExecuteNonQuery();
command.CommandText =
"Insert into Region (RegionID, RegionDescription) VALUES (101, 'Description')";
command.ExecuteNonQuery();
// Attempt to commit the transaction.
transaction.Commit();
Console.WriteLine("Both records are written to database.");
}
catch (Exception ex)
{
Console.WriteLine("Commit Exception Type: {0}", ex.GetType());
Console.WriteLine(" Message: {0}", ex.Message);
// Attempt to roll back the transaction.
try
{
transaction.Rollback();
}
catch (Exception ex2)
{
// This catch block will handle any errors that may have occurred
// on the server that would cause the rollback to fail, such as
// a closed connection.
Console.WriteLine("Rollback Exception Type: {0}", ex2.GetType());
Console.WriteLine(" Message: {0}", ex2.Message);
}
}
}
}
You can read docs: https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlconnection.begintransaction?view=dotnet-plat-ext-3.1

Problems detecting collision (Eyeshot) in an array of Tasks

I'm trying to check for each movement in a different task and after checking if there was a collision, in some iterations it generates an Exception "A source matrix was not long or sufficient. Check the index and length, as well as the lower limits of the matrix."
If you try to run sequentially in a "for" the error does not occur, I need to run in parallel to increase performance.
In debugging tests I notice that the error always occurs when trying to run cd.DoWork()
private void btn_Tasks_Click(object sender, EventArgs e)
{
// The source of your work items, create a sequence of Task instances.
Task[] tasks = Enumerable.Range(0,tabelaPosicao.Count).Select(i =>
// Create task here.
Task.Run(() =>
{
VerifiCollision(i);
})
// No signalling, no anything.
).ToArray();
// Wait on all the tasks.
Task.WaitAll(tasks);
}
private void VerifiCollision(object x)
{
int xx = (int)x;
int AuxIncrMorsa = Convert.ToInt32(tabelaPosicao[xx].Posicao) * -1;
bRef_BaseMorsa.Transformation = new Translation(0, AuxIncrMorsa, 0);
CollisionDetection cd = new CollisionDetection(new List<Entity>() { bRef_BaseMorsa }, new List<Entity>() { bRef_Matriz }, model1.Blocks, true, CollisionDetection2D.collisionCheckType.OBWithSubdivisionTree, maxTrianglesNumForOctreeNode: 5);
{
if (cd != null)
{
try
{
cd.DoWork();
}
catch (Exception e)
{
e.StackTrace;
}
catch (AggregateException ae)
{
var messege = ae.Message;
}
}
model1.Entities.ClearSelection();
if (cd3.Result != null && cd3.Result.Count > 0)
{
tabelaPosicao[xx].Tuple = new Tuple<string, string>(cd3.Result[0].Item1.ParentName,
cd3.Result[0].Item2.ParentName);
}
}
}
Before applying the transformation you need to clone the entity.
You can have a look at the "WORKFLOW" topic of this article.
I solved it by cloning the Bloks and BlockReference, so each iteration with its transformation performed separately, so there was no possibility that the transformation of one iteration would interfere with another. Grateful for the help.

Null Pointer Exception on testng datprovider

I am running in a strange problem. Let me explain:
I am passing set of input data from xml and then using JAXB to parse xml. This java object is then passed to my test method using testng dataprovider.
Here are some related code:
Testdata xml:
<TestData>
<TestDetails>
<testcasename>itemStatusTest</testcasename>
<testcasedetails>App in SUPPRESSED Status</testcasedetails>
<appid>28371</appid>
<status>SUPPRESSED</status>
<marketplace />
</TestDetails>
<TestDetails>
<testcasename>itemStatusTest</testcasename>
<testcasedetails>App in REVIEW Status</testcasedetails>
<appid>22559</appid>
<status>REVIEW</status>
<marketplace />
</TestDetails>
</TestData>
Method which returns object:
private static Object[][] generateTestData(String dataProvider,TestCaseName tcName) throws Exception {
Object[][] obj = null;
try {
JAXBContext jaxbContext = JAXBContext.newInstance(TestData.class);
Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
TestData testData = (TestData) jaxbUnmarshaller
.unmarshal(new FileInputStream(new File(dataProvider)
.getAbsoluteFile()));
List<TestDetails> testcaseList = testData.getTestDetails();
obj = new Object[testcaseList.size()][];
for (int i = 0; i < testcaseList.size(); i++) {
if (testcaseList
.get(i)
.getTestcasename()
.equalsIgnoreCase(tcName.testCaseName()))
obj[i] = new Object[] { testcaseList.get(i) };
}
} catch (JAXBException e) {
e.getMessage();
return null;
}
return obj;
}
and my dataprovider:
#DataProvider(parallel = true, name = "TestData")
public Object[][] TestData() {
try {
Object obj[][]= IngestionTestHelper
.generateTestDataForItemStatus(dataProvider);
Reporter.log("Size "+obj.length, true);
return obj;
} catch (Exception e) {
Reporter.log(
"Either XML input is in wrong format or XML is not parsed correctly",
true);
return null;
}
}
Till now everything works like a charm and I am not seeing any issue.
Now i am writing another test method for another test-case. For that I have added following in my exisitng xml like this:
<TestDetails>
<testcasename>itemWorkflowTest</testcasename>
<testcasedetails>Validate workflow for iap</testcasedetails>
<appid>26120</appid>
<status />
<marketplace />
</TestDetails>
Now once i have added this in my existing xml my existing test method is not working. When running I am getting following exception:
java.lang.NullPointerException
at org.testng.internal.Invoker.injectParameters(Invoker.java:1333)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1203)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1197)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1122)
at org.testng.TestNG.run(TestNG.java:1030)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
If i remove the newly added block in xml it starts working.
Please someone help!!!
Well, based on the code, and if I understood correctly :)
When you add the third item the name is different,
You have initialized the Object array with the size of the total number of elements,
obj = new Object[testcaseList.size()][];
But you are adding to the array selectively based on name, so though the init has been done for 3 objects, the data is available only for 2 - this may be causing the NPE..
List<TestDetails> testcaseList = testData.getTestDetails();
obj = new Object[testcaseList.size()][];
for (int i = 0; i < testcaseList.size(); i++) {
if (testcaseList
.get(i)
.getTestcasename()
.equalsIgnoreCase(tcName.testCaseName()))
obj[i] = new Object[] { testcaseList.get(i) };
}

Exception retrieving results in jena resultSet

I'm having some problems when querying DBpedia throught Jena. The exception is thrown when iterating over the resultSet in the nextSolution method. Here is the code:
ResultSet results = throwQuery(query);
ArrayList<Movies> movs = new ArrayList<Movies>();
//try {
while (results.hasNext()) {
try{
QuerySolution q = results.nextSolution();
Movies m = new Movies();
m.setUrl(q.get("film_url").toString());
RDFNode node = q.get("film_label");
// Set a default title
String title = "";
if (node != null) {
// We delete the "#en" part that indicates that the label is in
// english
title = node.toString();
int ind = title.indexOf("#en");
title = title.substring(0, ind);
}
m.setTitle(title);
node = q.get("image_url");
// Set a default image
String image = "http://4.bp.blogspot.com/_rY0CJheAaRM/SuYJcVOqKbI/AAAAAAAAA2Y/abClDm72TuY/s320/NoCoverAvailable.png";
if (node != null) {
// For some reason the image link retrieved from dbpedia is
// broken. Here we fix it
image = node.toString();
int ind = image.indexOf("common");
image = image.substring(0, ind) + "en" + image.substring(ind + 7);
}
m.setImageurl(image);
movs.add(m);
}
catch(Exception e){
System.err.println("Error catched: " + e.getMessage());
}
}
return movs;
Where throwQuery
private final static String SERVICE = "http://dbpedia.org/sparql";
private static ResultSet throwQuery(String q) {
Query qFactory = QueryFactory.create(q);
QueryExecution qe = QueryExecutionFactory.sparqlService(SERVICE, qFactory);
ResultSet results = null;
try {
results = qe.execSelect();
} catch (QueryExceptionHTTP e) {
System.out.println(e.getMessage());
System.out.println(SERVICE + " is DOWN");
} finally {
qe.close();
return results;
}
}
And the testing query
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT ?film_label ?image_url ?film_url
WHERE {
?film_url rdf:type <http://dbpedia.org/ontology/Film> .
OPTIONAL{
?film_url rdfs:label ?film_label
FILTER (LANG(?film_label) = 'en')
}
OPTIONAL{
?film_url foaf:depiction ?image_url
}
FILTER regex(str(?film_url), "hola","i")
}
ORDER BY ?film_url
When the program starts iterating, everything goes well until arrive to the value Nicholas Nickleby (2002 film) then I get this exception:
com.hp.hpl.jena.sparql.resultset.ResultSetException: XMLStreamException: Unexpected EOF in start tag
at [row,col {unknown-source}]: [67,116]
at com.hp.hpl.jena.sparql.resultset.XMLInputStAX$ResultSetStAX.staxError(XMLInputStAX.java:539 )
at com.hp.hpl.jena.sparql.resultset.XMLInputStAX$ResultSetStAX.hasNext(XMLInputStAX.java:236)
at client.DBPediaConnector.getMovie(DBPediaConnector.java:67)
at customServices.MoviesService.searchInsertMovie(MoviesService.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1052)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1124)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:5388)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:619)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:800)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:571)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doAround(SystemInterceptorProxy.java:162)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:144)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:861)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:800)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:370)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:5360)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:5348)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:214)
... 47 more
Caused by: com.ctc.wstx.exc.WstxEOFException: Unexpected EOF in start tag
at [row,col {unknown-source}]: [67,116]
at com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:677)
at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:1034)
at com.ctc.wstx.sr.StreamScanner.getNextChar(StreamScanner.java:785)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2790)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1065)
at com.hp.hpl.jena.sparql.resultset.XMLInputStAX$ResultSetStAX.getOneSolution(XMLInputStAX.java:435)
at com.hp.hpl.jena.sparql.resultset.XMLInputStAX$ResultSetStAX.hasNext(XMLInputStAX.java:232)
... 71 more
Seems for me like an internal error from Jena, but I have no idea. Am I doing something wrong? How can I solve this?
Please give a complete, minimal example. This is quite long.
DBpedia is returning broken XML for the results, possibly because the query is taking a long time to execute and the timeout is triggered. It seems to be a moderately slow query.
Try adding &timeout=60000 to query URL of 'http://dbpedia.org/sparql&timeout=60000', if your version of Jena is new enough. This may not be long enough. There is a hard internal limit on dbpedia which can not be overridden.
Executing at a different time of day may also help.
It may also be because corrupt XML is being returned. Execute the query at the DBpedia UI and get the XML results to check this.

BouncyCastle PGP Encryption / Decryption: Caused by: java.io.EOFException: premature end of stream in PartialInputStream

When attempting to encrypt then decrypt a file I run into the following exception:
com.example.common.crypto.CipherException: org.bouncycastle.openpgp.PGPException: Exception starting decryption
at com.example.common.crypto.PGPFileCipher.decrypt(PGPFileCipher.java:151)
at com.example.common.crypto.PGPFileCipherTest.testEncryptDecryptFile(PGPFileCipherTest.java:41)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:43)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127)
at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009)
Caused by: org.bouncycastle.openpgp.PGPException: Exception starting decryption
at org.bouncycastle.openpgp.PGPPublicKeyEncryptedData.getDataStream(Unknown Source)
at org.bouncycastle.openpgp.PGPPublicKeyEncryptedData.getDataStream(Unknown Source)
at com.example.common.crypto.PGPFileCipher.decrypt(PGPFileCipher.java:128)
... 28 more
Caused by: java.io.EOFException: premature end of stream in PartialInputStream
at org.bouncycastle.bcpg.BCPGInputStream$PartialInputStream.read(Unknown Source)
at org.bouncycastle.bcpg.BCPGInputStream.read(Unknown Source)
at java.io.InputStream.read(InputStream.java:82)
at javax.crypto.CipherInputStream.a(DashoA13*..)
at javax.crypto.CipherInputStream.read(DashoA13*..)
at org.bouncycastle.bcpg.BCPGInputStream.read(Unknown Source)
... 31 more
My PGPFileCipher class is:
package com.example.common.crypto;
public class PGPFileCipher implements FileCipher {
private static final Logger logger = LoggerFactory.getLogger(PGPFileCipher.class);
public File encryptFile(File inputFile, String encryptedFilePath, String publicKeyPath) throws CipherException {
FileInputStream publicKeyStream = null;
FileOutputStream encryptedFileOutputStream = null;
try {
publicKeyStream = new FileInputStream(new File(publicKeyPath));
File encryptedFile = new File(encryptedFilePath);
encryptedFileOutputStream = new FileOutputStream(encryptedFile);
encryptedFileOutputStream.write(encrypt(getBytesFromFile(inputFile), publicKeyStream, ""));
encryptedFileOutputStream.flush();
return encryptedFile;
} catch (Exception e) {
throw new CipherException(e);
} finally {
IOUtils.closeQuietly(encryptedFileOutputStream);
IOUtils.closeQuietly(publicKeyStream);
}
}
private PGPPrivateKey findSecretKey(PGPSecretKeyRingCollection pgpSec, long keyID, char[] pass) throws CipherException {
try {
PGPSecretKey pgpSecKey = pgpSec.getSecretKey(keyID);
if (pgpSecKey == null) {
return null;
}
return pgpSecKey.extractPrivateKey(pass, new BouncyCastleProvider());
} catch (Exception e) {
throw new CipherException(e);
}
}
/**
* decrypt the passed in message stream
*
* #param encrypted
* The message to be decrypted.
* #param keyIn
* InputStream of the key
*
* #return Clear text as a byte array. I18N considerations are not handled
* by this routine
* #exception CipherException
*/
public byte[] decrypt(byte[] encrypted, InputStream keyIn, char[] password) throws CipherException {
ByteArrayOutputStream out = null;
InputStream clear = null;
InputStream in = null;
InputStream unc = null;
try {
in = new ByteArrayInputStream(encrypted);
in = PGPUtil.getDecoderStream(in);
PGPObjectFactory pgpF = new PGPObjectFactory(in);
PGPEncryptedDataList enc = null;
Object o = pgpF.nextObject();
//
// the first object might be a PGP marker packet.
//
if (o instanceof PGPEncryptedDataList) {
enc = (PGPEncryptedDataList) o;
} else {
enc = (PGPEncryptedDataList) pgpF.nextObject();
}
//
// find the secret key
//
Iterator it = enc.getEncryptedDataObjects();
PGPPrivateKey sKey = null;
PGPPublicKeyEncryptedData pbe = null;
PGPSecretKeyRingCollection pgpSec = new PGPSecretKeyRingCollection(
PGPUtil.getDecoderStream(keyIn));
while (sKey == null && it.hasNext()) {
pbe = (PGPPublicKeyEncryptedData) it.next();
sKey = findSecretKey(pgpSec, pbe.getKeyID(), password);
}
if (sKey == null) {
throw new IllegalArgumentException(
"secret key for message not found.");
}
clear = pbe.getDataStream(sKey, new BouncyCastleProvider());
PGPObjectFactory pgpFact = new PGPObjectFactory(clear);
PGPCompressedData cData = (PGPCompressedData) pgpFact.nextObject();
pgpFact = new PGPObjectFactory(cData.getDataStream());
PGPLiteralData ld = (PGPLiteralData) pgpFact.nextObject();
unc = ld.getInputStream();
out = new ByteArrayOutputStream();
int ch;
while ((ch = unc.read()) >= 0) {
out.write(ch);
}
out.flush();
byte[] returnBytes = out.toByteArray();
return returnBytes;
} catch (Exception e) {
e.printStackTrace();
throw new CipherException(e);
} finally {
IOUtils.closeQuietly(out);
IOUtils.closeQuietly(in);
IOUtils.closeQuietly(clear);
IOUtils.closeQuietly(unc);
}
}
/**
* Simple PGP encryptor between byte[].
*
* #param clearData
* The test to be encrypted
* #param keyInputStream
* Input stream of the key
* #param fileName
* File name. This is used in the Literal Data Packet (tag 11)
* which is really inly important if the data is to be related to
* a file to be recovered later. Because this routine does not
* know the source of the information, the caller can set
* something here for file name use that will be carried. If this
* routine is being used to encrypt SOAP MIME bodies, for
* example, use the file name from the MIME type, if applicable.
* Or anything else appropriate.
*
* #return encrypted data.
* #exception CipherException
*/
public byte[] encrypt(byte[] clearData, InputStream keyInputStream, String fileName)
throws CipherException {
OutputStream out = null;
OutputStream cOut = null;
PGPCompressedDataGenerator comData = null;
PGPLiteralDataGenerator lData = null;
ByteArrayOutputStream encOut = null;
ByteArrayOutputStream bOut = null;
OutputStream cos = null;
OutputStream pOut = null;
try {
PGPPublicKey encKey = readPublicKey(keyInputStream);
if (fileName == null || fileName.trim().length() < 1) {
fileName = PGPLiteralData.CONSOLE;
}
encOut = new ByteArrayOutputStream();
out = encOut;
bOut = new ByteArrayOutputStream();
comData = new PGPCompressedDataGenerator(
PGPCompressedDataGenerator.ZIP);
cos = comData.open(bOut); // open it with the final
// destination
lData = new PGPLiteralDataGenerator();
// we want to generate compressed data. This might be a user option
// later,
// in which case we would pass in bOut.
pOut = lData.open(cos, // the compressed output stream
PGPLiteralData.BINARY, fileName, // "filename" to store
clearData.length, // length of clear data
new Date() // current time
);
pOut.write(clearData);
pOut.flush();
PGPEncryptedDataGenerator cPk = new PGPEncryptedDataGenerator(
PGPEncryptedData.CAST5, false, new SecureRandom(),
new BouncyCastleProvider());
cPk.addMethod(encKey);
byte[] bytes = bOut.toByteArray();
cOut = cPk.open(out, bytes.length);
cOut.write(bytes); // obtain the actual bytes from the compressed stream
cOut.flush();
encOut.flush();
return encOut.toByteArray();
} catch (Exception e) {
throw new CipherException(e);
} finally {
IOUtils.closeQuietly(out);
IOUtils.closeQuietly(cOut);
IOUtils.closeQuietly(encOut);
IOUtils.closeQuietly(bOut);
IOUtils.closeQuietly(cos);
IOUtils.closeQuietly(pOut);
try {
if (lData != null) {
lData.close();
}
if (comData != null) {
comData.close();
}
} catch (IOException ignored) {}
}
}
private PGPPublicKey readPublicKey(InputStream in)
throws CipherException {
try {
in = PGPUtil.getDecoderStream(in);
PGPPublicKeyRingCollection pgpPub = new PGPPublicKeyRingCollection(in);
//
// we just loop through the collection till we find a key suitable for
// encryption, in the real
// world you would probably want to be a bit smarter about this.
//
//
// iterate through the key rings.
//
Iterator rIt = pgpPub.getKeyRings();
while (rIt.hasNext()) {
PGPPublicKeyRing kRing = (PGPPublicKeyRing) rIt.next();
Iterator kIt = kRing.getPublicKeys();
while (kIt.hasNext()) {
PGPPublicKey k = (PGPPublicKey) kIt.next();
if (k.isEncryptionKey()) {
return k;
}
}
}
throw new IllegalArgumentException("Can't find encryption key in key ring.");
} catch (Exception e) {
throw new CipherException(e);
}
}
private byte[] getBytesFromFile(File file) throws CipherException {
InputStream is = null;
try {
is = new FileInputStream(file);
// Get the size of the file
long length = file.length();
if (length > Integer.MAX_VALUE) {
throw new CipherException("File is too large: " + file.getName());
}
return IOUtils.toByteArray(is);
} catch (IOException e) {
throw new CipherException(e);
} finally {
IOUtils.closeQuietly(is);
}
}
}
And the test I'm attempting to run is:
package com.example.common.crypto;
public class PGPFileCipherTest {
private static final Logger logger = LoggerFactory.getLogger(PGPFileCipherTest.class);
#Rule
public TemporaryFolder folder = new TemporaryFolder();
#Test
public void testEncryptDecryptFile() throws IOException, CipherException {
FileOutputStream decryptedFileOutputStream = null;
InputStream privateKeyStream = null;
try {
PGPFileCipher fileCipher = new PGPFileCipher();
Resource inputFile = new ClassPathResource("testInputFile.txt");
Resource publicKey = new ClassPathResource("PUBLIC.asc");
Resource privateKey = new ClassPathResource("PRIVATE.asc");
// Encrypt file
File encryptedFile = fileCipher.encryptFile(inputFile.getFile(), folder.newFile("testInputFile_enc.txt").getPath(), publicKey.getFile().getPath());
// Now decrypt the file
File decryptedFile = folder.newFile("testInputFile_dec.txt");
decryptedFileOutputStream = new FileOutputStream(decryptedFile);
privateKeyStream = new FileInputStream(privateKey.getFile());
decryptedFileOutputStream.write(fileCipher.decrypt(FileUtils.readFileToByteArray(encryptedFile), privateKeyStream, "".toCharArray()));
decryptedFileOutputStream.flush();
} finally {
IOUtils.closeQuietly(decryptedFileOutputStream);
IOUtils.closeQuietly(privateKeyStream);
}
}
}
Unfortunately I'm not all that familiar with the BouncyCastle and it would seem as if a stream is not properly being closed / flushed but I can't seem to track it down. This is a modified version of one of the BouncyCastle examples FYI. Thanks in advance for your help.
check the jdk version.. we were facing issue running encryption and both in jdk1.5. but jdk 1.6 it work fine. and yes the bc pro jars "bcpg-jdk16" and" bcprov-ext-jdk16" verison should be 1.46
Be careful when using new BouncyCastleProvider() as it will result in the provider being registered in JceSecurity each time you instantiate it. This will cause a memory leak.
Consider doing something like this instead
private Provider getProvider() {
Provider provider = Security.getProvider("BC");
if (provider==null){
provider = new BouncyCastleProvider();
Security.addProvider(provider);
}
return provider;
}

Resources