In my Hadoop Map Reduce application I have one input file.I want that when I execute the jar of my application, then the input file will automatically be called.To do this I code one class to specify the input,output and file itself but from where I am calling the file, there I want to specify the file path. To do that I have used this code:
QueriesTest.class.getResourceAsStream("/src/main/resources/test")
but it is not working (cannot read the input file from the generated jar)
so I have used this one
URL url = this.getClass().getResource("/src/main/resources/test") here I am getting the problem of URL. So please help me out. I am using Hadoop 0.21.
I'm not sure what you want to tell us with your resource loading, but the usual way to add an input file is this:
Configuration conf = new Configuration();
Job job = new Job(conf);
Path in = new Path("YOUR_PATH_IN_HDFS");
FileInputFormat.addInputPath(job, in);
job.setInputFormatClass(TextInputFormat.class); // could be a sequencefile also
// set the other stuff
job.waitForCompletion(true);
Make sure your file resides in HDFS then.
Related
I'm using Grails 2.5.6 to store uploaded images to folder on a server.
The following are my code to store the image
mpr.multiFileMap.file.each{fileData->
CommonsMultipartFile file = fileData
File convFile = new File(file.getOriginalFilename());
file.transferTo(convFile);
/** Processing File **/
File uploadedFile = new File("${directory}${generatedFileName}.${extension}")
convFile.renameTo(uploadedFile)
}
I have no problem running on development (MacOSX High Sierra)
But when i deployed on production (Ubuntu 14.04 server), i could not save the file outside CATALINA_HOME directory.
I have checked the permission and ownership of the destination directory, but still, the directory was created but the file was never stored.
For Example, i've tried to store the file on /home/tomcat/ directory (/home directory was in separate partition with tomcat which stored it /var), the directory was created, but the file was never stored.
When i put the destination directory within CATALINA_HOME folder, everything works fine. But this was not the scenario i want to do.
You say your destination directory is on another partition, so maybe another filesystem is used on this partition.
Or if you look on the javadoc of the renameTo method it is said :
Many aspects of the behavior of this method are inherently
platform-dependent: The rename operation might not be able to move a
file from one filesystem to another, it might not be atomic, and it
might not succeed if a file with the destination abstract pathname
already exists. The return value should always be checked to make
sure that the rename operation was successful.
...
#return true if and only if the renaming succeeded;
false otherwise
Thus I think the renameTo method is not able to move the file, don't know why but you can rewrite your code like this :
mpr.multiFileMap.file.each{fileData->
CommonsMultipartFile file = fileData
File uploadedFile = new File("${directory}${generatedFileName}.${extension}")
// String originalFilename = file.getOriginalFilename()
// you can store originalFilename in database for example
if(!uploadedFile.getParentFile().exists()) {
uploadedFile.getParentFile().mkdirs()
// You can set permissions on the target directory if you desired, using PosixFilePermission
}
file.transferTo(uploadedFile)
}
there is some way from inside the "main.js" electron to save a file out of the asar?
I'm fighting with this command to point the way out of the write-only area but I can not do it.
It would be nice that the path was inside /my-project/resources/ and would work even without the electron-package.
let configFilePath = `${__dirname}/../config.json`
db = new loki(configFilePath)
if(fs.existsSync(configFilePath))
db.loadDatabase()
Attempting to write a file within the application installation directory is a bad idea, often the user will not have the permission to do so. Instead you should write files to the location returned by app.getPath('userData').
I'm trying to save an image from an OpenLDAP server. It's in binary format and all my code appears to work, however, the image is corrupted.
I then attempted to do this in PHP and was successful, but I'd like to do it in a Grails project.
PHP Example (works)
<?php
$conn = ldap_connect('ldap.example.com') or die("Could not connect.\n");
ldap_set_option($conn, LDAP_OPT_PROTOCOL_VERSION, 3);
$dn = 'ou=People,o=Acme';
$ldap_rs = ldap_bind($conn) or die("Can't bind to LDAP");
$res = ldap_search($conn,$dn,"someID=123456789");
$info = ldap_get_entries($conn, $res);
$entry = ldap_first_entry($conn, $res);
$jpeg_data = ldap_get_values_len( $conn, $entry, "someimage-jpeg");
$jpeg_filename = '/tmp/' . basename( tempnam ('.', 'djp') );
$outjpeg = fopen($jpeg_filename, "wb");
fwrite($outjpeg, $jpeg_data[0]);
fclose ($outjpeg);
copy ($jpeg_filename, '/some/dir/test.jpg');
unlink($jpeg_filename);
?>
Groovy Example (does not work)
def ldap = org.apache.directory.groovyldap.LDAP.newInstance('ldap://ldap.example.com/ou=People,o=Acme')
ldap.eachEntry (filter: 'someID=123456789') { entry ->
new File('/Some/dir/123456789.jpg').withOutputStream {
it.write entry.get('someimage-jpeg').getBytes() // File is created, but image is corrupted (size also doesn't match the PHP version)
}
}
How would I tell the Apache LDAP library that "image-jpeg" is actually binary and not a String? Is there a better simple library available to read binary data from an LDAP server? From looking at the Apache mailing list, someone else had a similar issue, but I couldn't find a resolution in the thread.
Technology Stack
Grails 2.2.1
Apache LDAP API 1.0.0 M16
Have you checked whether the image attribute value is base-64 encoded?
I found the answer. The Apache Groovy LDAP library uses JNDI under the hood. When using JNDI certain entries are automatically read as binary, but if your LDAP server uses a custom name, the library will not know that it's binary.
For those people that come across this problem using Grails, here's the steps to set a specific entry to binary format.
Create a new properties file call "jndi.properties" and add it to your grails-app/conf directory (all property files in this folder are automatically included in the classpath)
Add a line in the properties file with the name of the image variable:
java.naming.ldap.attributes.binary=some_custom_image
Save the file and run the Grails application
Here is some sample code to save a binary entry to a file.
def ldap = LDAP.newInstance('ldap://some.server.com/ou=People,o=Acme')
ldap.eachEntry (filter: 'id=1234567') { entry ->
new File('/var/dir/something.jpg').withOutputStream {
it.write entry.image
}
}
I am trying to set up Weceem using the source from GitHub. It requires a physical path definition for the uploads directory, and for a directory for appears to be used for writing searchable indexes. The default setting for uploads is:
weceem.upload.dir = 'file:/var/www/weceem.org/uploads/'
I would like to define those using relative paths like WEB-INF/resources/uploads. I tried a methodology I have used previously for accessing directories with relative path like this:
File uploadDirectory = ApplicationHolder.application.parentContext.getResource("WEB-INF/resources/uploads").file
def absoluteUploadDirectory = uploadDirectory.absolutePath
weceem.upload.dir = 'file:'+absoluteUploadDirectory
However, 'parentContext' under ApplicationHolder.application is NULL. Can anyone offer a solution to this that would allow me to use relative paths?
look at your Config.groovy you should have (maybe it is commented)
// locations to search for config files that get merged into the main config
// config files can either be Java properties files or ConfigSlurper scripts
// "classpath:${appName}-config.properties", "classpath:${appName}-config.groovy",
grails.config.locations = [
"file:${userHome}/.grails/${appName}-config.properties",
"file:${userHome}/.grails/${appName}-config.groovy"
]
Create Conig file in deployment server
"${userHome}/.grails/${appName}-config.properties"
And define your prop (even not relative path) in that config file.
To add to Aram Arabyan's response, which is correct, but lacks an explanation:
Grails apps don't have a "local" directory, like a PHP app would have. They should be (for production) deployed in a servlet container. The location of that content is should not be considered writable, as it can get wiped out on the next deployment.
In short: think of your deployed application as a compiled binary.
Instead, choose a specific location somewhere on your server for the uploads to live, preferably outside the web server's path, so they can't be accessed directly. That's why Weceem defaults to a custom folder under /var/www/weceem.org/.
If you configure a path using the externalized configuration technique, you can then have a path specific to the server, and include a different path on your development machine.
In both cases, however, you should use absolute paths, or at least paths relative to known directories.
i.e.
String base = System.properties['base.dir']
println "config: ${base}/web-app/config/HookConfig.grooy"
String str = new File("${base}/web-app/config/HookConfig.groovy").text
return new ConfigSlurper().parse(str)
or
def grailsApplication
private getConfig() {
String str = grailsApplication.parentContext.getResource("config/HookConfig.groovy").file.text
return new ConfigSlurper().parse(str)
}
I have multimodule project
Project
|--src
|-JavaFile.java
Web-Project
|-Web-Content
|-images
| |-logo.PNG
|-pages
|-WEB-INF
regular java module - contains src with all java files
dynamic web project module - contains all web related stuff
eventually regular java module goes as a jar file in dynamic web module in lib folder
Problem
java file after compilation looks for an image file in c:\ibm\sdp\server completepath\logo.png rather in context. File is defined in java file as below for iText:
Image logo = Image.getInstance("/images/logo.PNG");
Please suggest how can I change my java file to refer to image. I am not allowed to change my project structure.
You need to use ServletContext#getResource() or, better, getResourceAsStream() for that. It returns an URL respectively an InputStream of the resource in the web content.
InputStream input = getServletContext().getResourceAsStream("/images/logo.PNG");
// ...
This way you're not dependent on where (and how!) the webapp is been deployed. Relying on absolute disk file system paths would only end up in portability headache.
See also:
getResourceAsStream() vs FileInputStream
Update: as per the comments, you seem to be using iText (you should have clarified that a bit more in the question, I edited it). You can then use the Image#getInstance() method which takes an URL:
URL url = getServletContext().getResource("/images/logo.PNG");
Image image = Image.getInstance(url);
// ...
Update 2: as per the comments, you turn out to be sitting in the JSF context (you should have clarified that as well in the question). You should use ExternalContext#getResource() instead to get the URL:
URL url = FacesContext.getCurrentInstance().getExternalContext().getResource("/images/logo.PNG");
Image image = Image.getInstance(url);
// ...