I am trying to get a list of files with no extension using org.apache.commons.io.FileUtils.listFiles() like here
http://www.avajava.com/tutorials/lessons/how-do-i-get-all-files-with-certain-extensions-in-a-directory-including-subdirectories.html
package test;
import java.io.File;
import java.io.IOException;
import java.util.List;
import org.apache.commons.io.FileUtils;
public class GetAllFilesInDirectoryBasedOnExtensions {
public static void main(String[] args) throws IOException {
File dir = new File("dir");
String[] extensions = new String[] { "txt", "jsp" };
System.out.println("Getting all .txt and .jsp files in " + dir.getCanonicalPath()
+ " including those in subdirectories");
List<File> files = (List<File>) FileUtils.listFiles(dir, extensions, true);
for (File file : files) {
System.out.println("file: " + file.getCanonicalPath());
}
}
}
but I need a list of files with no extension. I've tried {".", ""} but that didn't help. Is it possible at all?
As #Jens reported in his comment, null parameter does the trick:
Related
I have a app in which i need to scan the QR code. Taking the picture from the app is not feasible as i need to run the app in multiple devices at once and it require human presence. How can i provide the QR code image/data to the app without scanning? Is there any way possible to simulate taking of picture and give store image as input to app?
If you have scanned test "QR Code image" then you can push it to the device from where app can read it.
You can ask dev team about the path from where app is reading the scanned image, and at same path you can push the test image.
Below is the code for how to push image file to device and other methods to push/pull different file formats
import java.awt.image.BufferedImage;
import io.appium.java_client.android.AndroidDriver;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.net.URL;
import javax.imageio.ImageIO;
import org.apache.commons.codec.binary.Base64;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
#Test
public class pushFileTest {
public static AndroidDriver<WebElement> _driver;
#BeforeClass
public void setUpAppium() throws InterruptedException, IOException {
DesiredCapabilities cap = new DesiredCapabilities();
cap.setCapability("platformVersion","5.1");
cap.setCapability("platformName","Android");
cap.setCapability("deviceName","ZX12222D");
cap.setCapability("appPackage","io.appium.android.apis");
cap.setCapability("appActivity","ApiDemos");
//System.out.println("Before calling appium");
_driver = new AndroidDriver<WebElement>(new URL("http://127.0.0.1:4725/wd/hub"), cap);
//System.out.println("After calling appium");
}
#Test
public void pullImageFileFromMobileSDCardTest() throws IOException {
byte[] returnData = _driver.pullFile("/storage/sdcard1/IMG_20140828_072840.jpg");
//System.out.println("Base 64 Converted String received from mobile :: " + returnData);
BufferedImage image=ImageIO.read(new ByteArrayInputStream(returnData));
ImageIO.write(image, "jpg", new File("C:\\eclipse","snap.jpg"));
}
/* Test Case to pull log file from mobile device*/
#Test
public void pullTextFileFromMobileSDCardTest() throws IOException {
byte[] returnData = _driver.pullFile("/storage/sdcard1/mili_log.txt");
//System.out.println(" Printing Text of File received from mobile :: " + new String(Base64.decodeBase64(returnData)));
File fs = new File("C:\\eclipse\\MobileFile.txt");
FileOutputStream fos = new FileOutputStream(fs);
fos.write(returnData);
fos.flush();
fos.close();
}
#Test
public void pushImageFileToMobileTest() throws IOException {
File fi = new File("C:\\eclipse\\img1.jpg");
byte[] fileContent = Files.readAllBytes(fi.toPath());
_driver.pushFile("/storage/sdcard1", fileContent);
}
#Test
public void pushTextFileToMobileTest() throws IOException {
File fi = new File("C:\\eclipse\\MobileFile.txt");
byte[] data = Files.readAllBytes(fi.toPath());
System.out.println("Base 64 Converted String sent to mobile :: " + data);
_driver.pushFile("/storage/sdcard1/appium.txt",data);
}
public void pullVideoFileFromMobileSDCardTest() throws IOException {
byte[] returnData = _driver.pullFile("/storage/sdcard1/VideoIconfile.mp4");
//System.out.println(" Printing Text of File received from mobile :: " + new String(Base64.decodeBase64(returnData)));
//File fs = new File("C:\\eclipse\\video.mp4");
FileOutputStream fos = new FileOutputStream("C:\\eclipse\\video.mp4");
fos.write(returnData);
fos.flush();
fos.close();
}
#AfterTest(alwaysRun= true)
public void tearDown(){
if (_driver!= null )
_driver.quit();
System.out.println("tearDown() :: driver.quit() executed");
}
}
I am looking through the documentation for a sample of how to handle a submit from an Orbeon form that I gather some data in and then submitting to another application via REST. I am not seeing anything that shows how to do that. Does Orbeon provide functionality to do that or do I need to code some JSP or something else on the backside to handle that?
My understanding is, that you have to provide/implement the REST service yourself. You aren't restricted to do it in Java, but if this is your preferred language, here's how a very simple servlet would look like. In this case the REST service saves the form in a file in the temp directory.
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Enumeration;
import java.util.Optional;
import java.util.logging.Logger;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
public class FormDumpServlet extends HttpServlet {
private static final Logger logger = Logger.getLogger(FormDumpServlet.class.getName());
private static final SimpleDateFormat FORMAT = new SimpleDateFormat("yyyy-MM-dd-HH-mm-ss-SSS");
protected Optional<String> makeTempDir() {
final String dir = System.getProperty("java.io.tmpdir");
logger.info(String.format("java.io.tmpdir=%s", dir));
if (dir == null) {
logger.severe("java.io.tmpdir is null, can't create temp directory");
return Optional.empty();
}
final File f = new File(dir,"form-dumps");
if (f.exists() && f.isDirectory() && f.canWrite()) {
return Optional.of(f.getAbsolutePath());
}
if (f.mkdir()) {
return Optional.of(f.getAbsolutePath());
}
logger.severe(String.format("failed to create temp dir <%s>", f.getAbsolutePath()));
return Optional.empty();
}
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
String path = req.getPathInfo();
if (!path.equalsIgnoreCase("/accept-form")) {
resp.setStatus(HttpServletResponse.SC_NOT_FOUND);
return;
}
Enumeration<String> parameterNames = req.getParameterNames();
while(parameterNames.hasMoreElements()) {
final String name = parameterNames.nextElement();
final String value = req.getParameter(name);
logger.info(String.format("parameter: name=<%s>, value=<%s>", name, value));
}
Optional<String> tempPath = makeTempDir();
if (tempPath.isPresent()) {
String fn = String.format("%s.xml", FORMAT.format(new Date()));
File f = new File(new File(tempPath.get()), fn);
logger.info(String.format("saving form to file <%s>", f.getAbsolutePath()));
try(PrintWriter pw = new PrintWriter(new FileWriter(f))) {
req.getReader().lines().forEach((l) -> pw.println(l));
}
}
resp.setStatus(HttpServletResponse.SC_OK);
}
}
You also have to configure a property in properties-local.xml which connects the send action for your form (the form with the name my_form in your application my_application) to the REST endpoint. This property could look as follows:
<property
as="xs:string"
name="oxf.fr.detail.process.send.my_application.my_form"
>
require-valid
then save-final
then send(uri = "http://localhost:8080/my-form-dump-servlet/accept-form")
then success-message(message = "Success: the form was transferred to the REST service")
</property>
I am writing a class to recursively extract files from inside a zip file and produce them to a Kafka queue for further processing. My intent is to be able to extract files from multiple levels of zip. The code below is my implementation of the tika ContainerExtractor to do this.
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.Stack;
import org.apache.commons.lang.StringUtils;
import org.apache.tika.config.TikaConfig;
import org.apache.tika.detect.DefaultDetector;
import org.apache.tika.detect.Detector;
import org.apache.tika.exception.TikaException;
import org.apache.tika.extractor.ContainerExtractor;
import org.apache.tika.extractor.EmbeddedResourceHandler;
import org.apache.tika.io.TemporaryResources;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.AbstractParser;
import org.apache.tika.parser.AutoDetectParser;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
import org.apache.tika.parser.pkg.PackageParser;
import org.xml.sax.ContentHandler;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;
public class UberContainerExtractor implements ContainerExtractor {
/**
*
*/
private static final long serialVersionUID = -6636138154366178135L;
// statically populate SUPPORTED_TYPES
static {
Set<MediaType> supportedTypes = new HashSet<MediaType>();
ParseContext context = new ParseContext();
supportedTypes.addAll(new PackageParser().getSupportedTypes(context));
SUPPORTED_TYPES = Collections.unmodifiableSet(supportedTypes);
}
/**
* A stack that maintains the parent filenames for the recursion
*/
Stack<String> parentFileNames = new Stack<String>();
/**
* The default tika parser
*/
private final Parser parser;
/**
* Default tika detector
*/
private final Detector detector;
/**
* The supported container types into which we can recurse
*/
public final static Set<MediaType> SUPPORTED_TYPES;
/**
* The number of documents recursively extracted from the container and its
* children containers if present
*/
int extracted;
public UberContainerExtractor() {
this(TikaConfig.getDefaultConfig());
}
public UberContainerExtractor(TikaConfig config) {
this(new DefaultDetector(config.getMimeRepository()));
}
public UberContainerExtractor(Detector detector) {
this.parser = new AutoDetectParser(new PackageParser());
this.detector = detector;
}
public boolean isSupported(TikaInputStream input) throws IOException {
MediaType type = detector.detect(input, new Metadata());
return SUPPORTED_TYPES.contains(type);
}
#Override
public void extract(TikaInputStream stream, ContainerExtractor recurseExtractor, EmbeddedResourceHandler handler)
throws IOException, TikaException {
ParseContext context = new ParseContext();
context.set(Parser.class, new RecursiveParser(recurseExtractor, handler));
try {
Metadata metadata = new Metadata();
parser.parse(stream, new DefaultHandler(), metadata, context);
} catch (SAXException e) {
throw new TikaException("Unexpected SAX exception", e);
}
}
private class RecursiveParser extends AbstractParser {
/**
*
*/
private static final long serialVersionUID = -7260171956667273262L;
private final ContainerExtractor extractor;
private final EmbeddedResourceHandler handler;
private RecursiveParser(ContainerExtractor extractor, EmbeddedResourceHandler handler) {
this.extractor = extractor;
this.handler = handler;
}
public Set<MediaType> getSupportedTypes(ParseContext context) {
return parser.getSupportedTypes(context);
}
public void parse(InputStream stream, ContentHandler ignored, Metadata metadata, ParseContext context)
throws IOException, SAXException, TikaException {
TemporaryResources tmp = new TemporaryResources();
try {
TikaInputStream tis = TikaInputStream.get(stream, tmp);
// Figure out what we have to process
String filename = metadata.get(Metadata.RESOURCE_NAME_KEY);
MediaType type = detector.detect(tis, metadata);
if (extractor == null) {
// do nothing
} else {
// Use a temporary file to process the stream
File file = tis.getFile();
System.out.println("file is directory = " + file.isDirectory());
// Recurse and extract if the filetype is supported
if (SUPPORTED_TYPES.contains(type)) {
System.out.println("encountered a supported file:" + filename);
parentFileNames.push(filename);
extractor.extract(tis, extractor, handler);
parentFileNames.pop();
} else { // produce the file
List<String> parentFilenamesList = new ArrayList<String>(parentFileNames);
parentFilenamesList.add(filename);
String originalFilepath = StringUtils.join(parentFilenamesList, "/");
System.out.println("producing " + filename + " with originalFilepath:" + originalFilepath
+ " to kafka queue");
++extracted;
}
}
} finally {
tmp.dispose();
}
}
}
public int getExtracted() {
return extracted;
}
public static void main(String[] args) throws IOException, TikaException {
String filename = "/Users/rohit/Data/cd.zip";
File file = new File(filename);
TikaInputStream stream = TikaInputStream.get(file);
ContainerExtractor recursiveExtractor = new UberContainerExtractor();
EmbeddedResourceHandler resourceHandler = new EmbeddedResourceHandler() {
#Override
public void handle(String filename, MediaType mediaType, InputStream stream) {
// do nothing
}
};
recursiveExtractor.extract(stream, recursiveExtractor, resourceHandler);
stream.close();
System.out.println("extracted " + ((UberContainerExtractor) recursiveExtractor).getExtracted() + " files");
}
}
It works on multiple levels of zip as long as the files inside the zips are in a flat structure. for ex.
cd.zip
- c.txt
- d.txt
The code does not work if there the files in the zip are present inside a directory. for ex.
ab.zip
- ab/
- a.txt
- b.txt
While debugging I came across the following code snippet in the PackageParser
try {
ArchiveEntry entry = ais.getNextEntry();
while (entry != null) {
if (!entry.isDirectory()) {
parseEntry(ais, entry, extractor, xhtml);
}
entry = ais.getNextEntry();
}
} finally {
ais.close();
}
I tried to comment out the if condition but it did not work. Is there a reason why this is commented? Is there any way of getting around this?
I am using tika version 1.6
Tackling your question in reverse order:
Is there a reason why this is commented?
Entries in zip files are either directories or files. If files, they include the name of the directory they come from. As such, Tika doesn't need to do anything with the directories, all it needs to do is process the embedded files as and when they come up
The code does not work if there the files in the zip are present inside a directory. for ex. ab.zip - ab/ - a.txt - b.txt
You seem to be doing something wrong then. Tika's recursion and package parser handle zips with folders in them just fine!
To prove this, start with a zip file like this:
$ unzip -l ../tt.zip
Archive: ../tt.zip
Length Date Time Name
--------- ---------- ----- ----
0 2015-02-03 16:42 t/
0 2015-02-03 16:42 t/t2/
0 2015-02-03 16:42 t/t2/t3/
164404 2015-02-03 16:42 t/t2/t3/test.jpg
--------- -------
164404 4 files
Now, make us of the -z extraction flag of the Tika App, which causes Tika to extract out all of the embedded contents of a file. Run like that, and we get
$ java -jar tika-app-1.7.jar -z ../tt.zip
Extracting 't/t2/t3/test.jpg' (image/jpeg) to ./t/t2/t3/test.jpg
Then list the resulting directory, and we see
$ find . -type f
./t/t2/t3/Test.jpg
I can't see what's wrong with your code, but sadly for you we've shown that the problem is there, and not with Tika... You'd be best off reviewing the various examples of recursion that Tika provides, such as the Tika App tool and the Recursing Parser Wrapper, then re-write your code to be something simple based from those
package sample;
import java.io.InputStream;
import com.hp.hpl.jena.rdf.model.Model;
import com.hp.hpl.jena.rdf.model.ModelFactory;
import com.hp.hpl.jena.util.FileManager;
public class ReadRDF extends Object {
static final String fileName = "foaf-ijd.rdf";
public static void main(String[] args) {
Model model = ModelFactory.createDefaultModel();
InputStream in = FileManager.get().open(fileName);
if (in == null) {
throw new IllegalArgumentException("File: " + fileName
+ " not found");
}
model.read(in, "");
model.write(System.out);
}
}
Errors getting populated
Exception in thread "main" java.lang.NoSuchMethodError:
org.slf4j.Logger.isTraceEnabled()Z at
com.hp.hpl.jena.util.LocatorFile.open(LocatorFile.java:118) at
com.hp.hpl.jena.util.FileManager.openNoMapOrNull(FileManager.java:527)
at com.hp.hpl.jena.util.FileManager.openNoMap(FileManager.java:510)
at
com.hp.hpl.jena.util.LocationMapper.initFromPath(LocationMapper.java:132)
at com.hp.hpl.jena.util.LocationMapper.get(LocationMapper.java:61)
at com.hp.hpl.jena.util.FileManager.makeGlobal(FileManager.java:116)
at com.hp.hpl.jena.util.FileManager.get(FileManager.java:82) at
sample.ReadRDF.main(ReadRDF.java:17)
This error can apper, if you don't add to CLASSPATH all jar file from /lib dir in Jena ditrib.
Also, if version's slf4j, which you use, an d jena's slf4j is differenf, this error can appear.
I am trying to validate generated WSDL to be correct. I have tried WS-i test tool downloaded from http://www.ws-i.org/ but it's test tool require all input to go through a config xml and the output is again an output xml file. Is there other easier way of validating a WSDL?
The Woden library/jar provides adequate functionality to be able to do this. If your wsdl isn't valid, the last statement, reader.readWSDL(...), will throw an exception.
import static junit.framework.Assert.fail;
import java.net.URISyntaxException;
import org.apache.woden.WSDLException;
import org.apache.woden.WSDLFactory;
import org.apache.woden.WSDLReader;
import org.apache.woden.wsdl20.Description;
import org.junit.Test;
public class WSDLValidationTest {
String wsdlFileName = "/MyService.wsdl";
#Test
public void validateWSDL2() throws WSDLException {
String wsdlUri = null;
try {
wsdlUri = this.getClass().getResource(wsdlFileName).toURI().toString();
}
catch( URISyntaxException urise) {
urise.printStackTrace();
fail( "Unable to retrieve wsdl: " + urise.getMessage());
}
WSDLFactory factory = WSDLFactory.newInstance("org.apache.woden.internal.OMWSDLFactory");
WSDLReader reader = factory.newWSDLReader();
reader.setFeature(WSDLReader.FEATURE_VALIDATION, true);
reader.readWSDL(wsdlUri);
}
}
And should you need a unit test for WSDL 1.1, see the following:
import static junit.framework.Assert.assertNotNull;
import static junit.framework.Assert.fail;
import java.io.IOException;
import java.net.URISyntaxException;
import java.net.URL;
import javax.wsdl.Definition;
import javax.wsdl.WSDLException;
import javax.wsdl.factory.WSDLFactory;
import javax.wsdl.xml.WSDLReader;
import javax.xml.stream.XMLStreamException;
import org.junit.Before;
import org.junit.Test;
import org.xml.sax.SAXException;
import com.sun.xml.ws.api.model.wsdl.WSDLModel;
import com.sun.xml.ws.api.server.SDDocumentSource;
import com.sun.xml.ws.api.wsdl.parser.WSDLParserExtension;
import com.sun.xml.ws.api.wsdl.parser.XMLEntityResolver;
public class WSDLValidationTest {
String wsdlFileName = "/MyService.wsdl";
String wsdlUri = null;
URL wsdlUrl = null;
#Before
public void before()
{
try {
wsdlUrl = this.getClass().getResource(wsdlFileName);
wsdlUri = wsdlUrl.toURI().toString();
}
catch( URISyntaxException urise) {
urise.printStackTrace();
fail( "Unable to retrieve wsdl: " + urise.getMessage());
}
}
#Test
public void parseAndValidateWSDL1_1WithWSDL4J() throws WSDLException
{
WSDLReader wsdlReader = null;
try {
WSDLFactory factory = WSDLFactory.newInstance();
wsdlReader = factory.newWSDLReader();
}
catch( WSDLException wsdle) {
wsdle.printStackTrace();
fail( "Unable to instantiate wsdl reader: " + wsdle.getMessage());
}
// Read WSDL service interface document
Definition def = wsdlReader.readWSDL(null, wsdlUri);
assertNotNull(def);
}
#Test
public void parseAndValidateWSDL1_1WithJaxWS() throws IOException, XMLStreamException, SAXException
{
final SDDocumentSource doc = SDDocumentSource.create(wsdlUrl);
final XMLEntityResolver.Parser parser = new XMLEntityResolver.Parser(doc);
WSDLModel model = WSDLModel.WSDLParser.parse( parser, null, false, new WSDLParserExtension[] {} );
assertNotNull(model);
}
}