I have a Dart [client side only, polymer] web-app. So I cannot use dart.io. I would like to do this semi-pseudo (language/platform mix!) code:
..
String str = "a load of text";
File file = new File('C:\\folder\\test.txt');
file.write(str);
file.close();
or
window.clipboard.copy(str);
so the user can paste it. I have used a TextAreaElement to put the string on (using textarea.value=str), and then the user can copy and paste it himself, but that's a bit naff.
Thanks
Steve
Sorry, you cannot write to files on the file system from a web app. The restriction is from browsers and their security model.
Related
I am working on a Firefox add-on which among other stuff generates thumbnails of websites for use by the add-on. So far I've been storing them by their image data URL using simple-storage. Two problems with this: the storage space is limited and sending very long strings around doesn't seem optimal(I assume the browser has optimized ways of loading image files, but maybe not data URLs). I think it shouldn't be a problem to save the files to disk, the question is where though. I googled quite a bit and could not find anything. Is there a natural place for this? Are there any restrictions?
As of Firefox 32, the place to store data for your add-on is supposed to be: [profile]/extension-data/[add-on ID]. This was established by the resolution of "Bug 915838 - Provide add-ons a standard directory to store data, settings". There is a follow-on bug, "Bug 952304 - (JSONStore) JSON storage API for addons to use in storing data and settings" which is supposed to provide an API for easy access.
For the Addon-SDK, you can obtain the addon ID (which you define in package.json) with:
let self = require("sdk/self");
let addonID = self.id;
For XUL and restartless extensions, you should be able to get the ID of your addon (which you define in the install.rdf file) with:
Components.utils.import("resource://gre/modules/Services.jsm");
let addonID = Services.appInfo.ID
You can then do the following to generate a URI for a file in that directory:
userProfileDirectoryPath = Components.classes["#mozilla.org/file/directory_service;1"]
.getService( Components.interfaces.nsIProperties)
.get("ProfD", Components.interfaces.nsIFile).path,
/**
* Generate URI for a filename in the extension's data directory under the preferences
* directory.
*/
function generateURIForFileInPrefExtensionDataDirectory (fileName) {
//Account for the path separator being OS dependent
let toReturn = "file://" + userProfileDirectoryPath.replace(/\\/g,"/");
return toReturn +"/extension-data/" + addonID + "/" + fileName;
}
}
The object myExtension.addonData is a copy that I store of the Bootstrap data provided to entry points in bootstrap.js.
I would like to find out, how can I email a .txt file that is located in the application storage directory.
Thanks
Jared
Use applicationStorageDirectory and navigateToURL with mailto like so,
(Note: I am not going in depth with reading text file. I can help though if you need help. Putting it only to give some idea here.)
//Find the target file
var fileToFind:File = File.applicationStorageDirectory.resolvePath("filename.txt");
//Extract content from file
var fileStream:FileStream = new FileStream();
fileStream.openAsync(fileToFind, FileMode.READ);
//fileStream.addEventListener(Event.COMPLETE, showContent);
//fileStream.addEventListener(IOErrorEvent.IO_ERROR, alertError);
//To mail
navigateToURL(newURLRequest("mailto:someEmail#gmail.com"+"?subject=Subject"+"&body="+ textfileContents + "));
--
If you want to send attachment or HTML formatted email, see this link,
SMTPMailer
.
Best luck.
I'm new to search engines and web crawlers. Now I want to store all the original pages in a particular web site as html files, but with Apache Nutch I can only get the binary database files. How do I get the original html files with Nutch?
Does Nutch support it? If not, what other tools can I use to achieve my goal.(The tools that support distributed crawling are better.)
Well, nutch will write the crawled data in binary form so if if you want that to be saved in html format, you will have to modify the code. (this will be painful if you are new to nutch).
If you want quick and easy solution for getting html pages:
If the list of pages/urls that you intend to have is quite low, then better get it done with a script which invokes wget for each url.
OR use HTTrack tool.
EDIT:
Writing a your own nutch plugin will be great. Your problem will get solved plus you can contribute to nutch by submitting your work !!! If you are new to nutch (in terms of code & design), then you will have to invest lot of time building a new plugin ... else its easy to do.
Few pointers for helping your initiative:
Here is a page which talks about writing own nutch plugin.
Start with Fetcher.java. See lines 647-648. That is the place where you can get the fetched content on per url basis (for those pages which got fetched successfully).
pstatus = output(fit.url, fit.datum, content, status, CrawlDatum.STATUS_FETCH_SUCCESS);
updateStatus(content.getContent().length);
You should add code right after this to invoke your plugin. Pass content object to it. By now, you would have guessed that content.getContent() is the content for url you want. Inside the plugin code, write it to some file. Filename should be based on the url name else it will be difficult to work with that. Url can be obtained by fit.url.
You must do modifications in run Nutch in Eclipse.
When you are able to run, open Fetcher.java and add the lines between "content saver" command lines.
case ProtocolStatus.SUCCESS: // got a page
pstatus = output(fit.url, fit.datum, content, status, CrawlDatum.STATUS_FETCH_SUCCESS, fit.outlinkDepth);
updateStatus(content.getContent().length);'
//------------------------------------------- content saver ---------------------------------------------\\
String filename = "savedsites//" + content.getUrl().replace('/', '-');
File file = new File(filename);
file.getParentFile().mkdirs();
boolean exist = file.createNewFile();
if (!exist) {
System.out.println("File exists.");
} else {
FileWriter fstream = new FileWriter(file);
BufferedWriter out = new BufferedWriter(fstream);
out.write(content.toString().substring(content.toString().indexOf("<!DOCTYPE html")));
out.close();
System.out.println("File created successfully.");
}
//------------------------------------------- content saver ---------------------------------------------\\
To update this answer -
It is possible to post process the data from your crawldb segment folder, and read in the html (including other data nutch has stored) directly.
Configuration conf = NutchConfiguration.create();
FileSystem fs = FileSystem.get(conf);
Path file = new Path(segment, Content.DIR_NAME + "/part-00000/data");
SequenceFile.Reader reader = new SequenceFile.Reader(fs, file, conf);
try
{
Text key = new Text();
Content content = new Content();
while (reader.next(key, content))
{
System.out.println(new String(content.GetContent()));
}
}
catch (Exception e)
{
}
The answers here are obsolete. Now, it is simply possible to get the plain HTML-files with nutch dump. Please see this answer.
In apache Nutch 2.3.1
You can save the raw HTML by edit the Nutch code firstly run the nutch in eclipse by following https://wiki.apache.org/nutch/RunNutchInEclipse
After you finish ruunning nutch in eclipse edit file FetcherReducer.java , add this code to the output method, run ant eclipse again to rebuild the class
Finally the raw html will added to reportUrl column in your database
if (content != null) {
ByteBuffer raw = fit.page.getContent();
if (raw != null) {
ByteArrayInputStream arrayInputStream = new ByteArrayInputStream(raw.array(), raw.arrayOffset() + raw.position(), raw.remaining());
Scanner scanner = new Scanner(arrayInputStream);
scanner.useDelimiter("\\Z");//To read all scanner content in one String
String data = "";
if (scanner.hasNext()) {
data = scanner.next();
}
fit.page.setReprUrl(StringUtil.cleanField(data));
scanner.close();
}
Can someone recommend a load testing tool which allows you to either:
a. replay an IIS (7) log(s) to simulate a real live site daily run;
b. import a CSV or equivalent list of URLS so we can achieve a similar thing as above but at a URL level;
c. .net API so I can create simple tests easily from my list of URLS is also a good way to go.
I do not really want to record my tests.
I think I can do B) with WAPT but need to create an XML file manually, not too much grief, but wondering if any tools cover these scenarios out the box.
Visual Studio Test Edition would require some code to parse the file into a suitable test run.
It is a great load testing solution.
Our load testing service lets you write a very simple script using JavaScript to pull data out of a CSV file and then fetch those URLs. For example, the following code would pluck 10 random URLs from the CSV file and fetch them as part of a single session:
var c = browserMob.openHttpClient();
var csv = browserMob.getCSV("urls.csv");
browserMob.beginTransaction();
for (var i = 0; i < 10; i++) {
browserMob.beginStep("Step 1");
var url = csv.random().get("url");
c.get(url);
browserMob.endStep();
}
browserMob.endTransaction();
The CSV file itself needs to be a normal CSV file with the first row containing a header named "url". This script would be run repeatedly for each virtual user participating in a load test.
We have support for so called 'uri-format' in our open-source tool called Yandex.Tank You simply put all your uris to a file, one uri -- one line, then specify headers in your load.ini like this:
[phantom]
address=example.org
rps_schedule=line(1, 1600, 2m)
headers = [Host: mts-maps.yandex.ru]
[Connection: close] [Bloody: yes]
ammo_file = ammo.uri
ammo.uri:
/
/index.html
/1/example.html
/2/example.html
My files are referenced like so (it's all relative):
// WHERE YOU KEEP THE PAGE TITLE XML
public static string myPageTitleXML = "xml/pagetitles.xml";
and
using (StreamReader r = new StreamReader(myPageTitleXML))
{ //etc.. . .etc....etc..
}
I get system.io.directorynotfound, and "this problem needs to be shut down", when I double click the executable. But running it from the console works like a charm. What's wrong here?
I played around with attempting to set Environment.CurrentDirectory but couldn't get anything to work. Why should I have to do that anyway? It defeats the purpose of a relative path no?
responding.. .
"application" does not exist in the current context, i'll keep trying what people have mentioned, this is not a windows.form
testing
Path.GetDirectoryName(Reflection.Assembly.GetExecutingAssembly().GetName().CodeBase), myPageTitleXML); gives error URI formats are not supported, as does Path.GetFullPath(). Server.MapPath results in an error as well, this is currently offline
Well assuming this directory is somewhere under the directory in which your code is executing, it sounds like you can use ..
Application.ExecutablePath()
or
Application.StartUpPath()
.. to get an idea as to what your application is seeing when it goes in search of an 'xml' directory with the 'pagetitles.xml' file in it.
If the directory returned by one of these methods does not point where you thought it did, you'll need to move the location of your application or the location of this folder so that it is within the same directory as the app.
Hope this gets you on the right path.
So, when you run it from double clicking the executable, is there a file named pagetitles.xml in a folder named xml, where xml is a folder in the same location as the executable?
It's certainly possible to use relative paths like this, but I wouldn't really recommend it. Instead, maybe use something like:
string fileToOpen = System.IO.Path.Combine(System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().GetName().CodeBase), myPageTitleXML);
using (StreamReader r = new StreamReader(fileToOpen))
{
//etc.. . .etc....etc..
}
Is this ASP.NET code? If so then you probably need to do MapPath("xml/pagetitles.xml")