I am developing a Delphi XE4 application that crawls a web site for certain information. It reads web pages with TIdHTTP and then parses with MSHTML using the IHtmlDocument2 interface.
The code works, but on one web site (www.orlytaitzesq.com), I get a Microsoft security dialog popup:
"Windows Security Warning
To allow this website to provide information personalized for you, will you allow it to put a small file (called a cookie) on your computer?"
It doesn't matter how I set "Allow Cookies" on the TIdHTTP component or whether I include a cookie handler. This warning just pops up randomly (following an Application.ProcessMessages) and my program stops until I respond to the dialog. This happens over and over again at random intervals. I have the Windows 7 Internet Options set to allow all cookies from the Internet Zone. I have IE 11. The web site I'm having the problem with is crawling with advertising.
I really don't understand what's happening, nor how to stop it.
Related
I have an IIS/MVC.Net application that has recorded thousands of action-not-found exceptions. When I investigated these it appears that they are all HTTP OPTIONS requests to an MVC action that only supports GET.
This action allows caching and returns minified CSS or JS content. Within the application it's accessed by <link> and <script> tags in the <head>. The application is not making the requests and we haven't seen this in testing with any browser.
What application is making all these OPTIONS requests?
What is it expecting in return?
As stated here, an usual case triggering those Microsoft Office Protocol Discovery queries are mails including images hosted on your server and viewed with Outlook (MS Office Outlook, not Outlook Express).
That does trigger OPTIONS request, as if it was trying to check if the server has some webdav support. I speculate MS Office does that for enabling integration with Sharepoint, by example.
So I usually consider it is only some annoying noise.
If you host mail images on your MVC app IIS site, maybe could you consider to move them on a dedicated static IIS site. Of course, as you cannot change previously sent mails, you may have to maintain old images and you will continue to have those requests till users cease to open old mails. Otherwise you may have to tweak your logging logic to lower the log level of those noisy requests.
We are willing to implement a web application with edit functionalities of WORD documents using WebDav.
We are starting to have a better picture of the implementation required however, we have read on your site (http://www.webdavsystem.com/server/documentation/ms_office_read_only) that it is possible to avoid MS office Word login dialog to open when URL-authentication is implemented.
We can hardly find solutions on how to achieve this or on how to use URL-authentication with MS Office.
Would you be able to point us in the right direction with this process or even let us know how to get some examples on that?
Here are some options to avoid login dialog:
Implement URL authentication. You will need to include some session
ID in the URL. Note that you will need to include it in path. For
example:
https://server/[SessionID1234567890]/path/file.docx
Do not include your session ID in the query string, as MS Office will
truncate it when saving a document.
Implement cookies/Forms authentication. In this case you will need the IT Hit
WebDAV Ajax Library to open documents for editing. You will also need to install the protocol application (Edit Document Opener, supplied with the WebDAV Ajax Library) on a client machine.
Implement MS-OFBA authentication. You can generate sample MS-OFBA
code using 'Add WebDAV Server Implementation' wizard for Visual
Studio supplied with IT Hit WebDAV Server for .NET SDK.
Note that MS-OFBA will display an HTML login dialog inside MS Office UI, which is rendered by Internet Explorer. If you open a document from any other web browser the login dialog will be always displayed during first open.
To avoid login dialog during subsequent access you will need to set permanent
authentication cookie (check the 'Keep me logged-in' checkbox).
Use Kerberos authentication. By default Kerberos authentication will display login dialog during first document access only. This option can be configured in Active Directory policies.
My current Intraweb application is actually a DataSnap Client which connect to my DataSnap Server that connects and sits together with an Interbase Server on the same machine. It works correctly but quite slow and require constant Internet connetion to work. Each button clicked or any event triggered will require the browser to connect to the Web Server (Intraweb).
I am thinking of creating an offline web application using Intraweb in Delphi XE2, HTML5 cache manifest feature, and use the browser-based SQL storage (such as webSQL or IndexedDB) as local browser storage when the mobile device goes offline. It will only connect to real DataSnap server when Internet connection is available to do initialization or synchronization back to the DataSnap Server.
Is is possible?
My main problem is to get the webpages' url out from the intraweb web application, and I do not want to put all the browser's storage code inside the template files.
It is also quite tedious to move the JavaScript code generated by Intraweb to other js files, and by doing this I may break the Intraweb application codes and logic. Is there any workaround on this?
As you stated by yourself: "Each button clicked or any event triggered will require the browser to connect to the Web Server".
This is the design of IntraWeb: a Client-Server application, in which most code logic is executed on the server side. You can add some AJAX widgets to your applications, but IntraWeb, by itself, is a Server-Side framework.
In order to have a full HTML5 AJAX Client application able to run stand-alone, you'll need a pure JavaScript application. Even Sacha/ExtJS based AJAX frameworks (like ExtPascal or UniGUI) or Morfik require a server to run.
But creating a pure HTML5 JavaScript application is some difficult task - but it is possible, since you can consume DataSnap content from JavaScript (using XML or JSON). You can try http://www.appcelerator.com/ which is a great IDE and platform for creating JavaScript applications, which run as native apps.
In order to have a disconnected HTML5 application, you may have to wait for the following products to be released:
Smart aka OP4JS;
Elevate Web Builder.
Thanks to these two projects, you would be able to code in object pascal, then the JavaScript will be compiled from the pascal source, then use HTML local storage. See for instance this article about using storage with Smart/OP4JS - I've tested it (in Alpha), and it works very well: you have a pure stand-alone HTML file which is able to run without any server, and have local storage. SQLite3 storage is planned (not yet finished).
Embedded Chromium does not have the same protection as stand-alone Chrome does when it comes to warning users about "bad sites". Users of Chrome are familiar with the warning screen that comes up when you attempt to navigate to a web site that Google has detected malware or other nefarious web elements on. However, repeating this operation in embedded Chromium does not lead to the warning screen. Does anyone know how to reproduce this capability so I can add it to my embedded Chromimum project (in Delphi 6 Pro)? If someone has "traced out" the web calls necessary to recreate this capability I'd like to know.
You can check a URL against the Google lists of suspected phishing and malware pages using the Google Safe Browsing API. to see how access this API from delphi check this article Using the Google Safe Browsing API from Delphi from here you can integrate the code of the article inside of your own project.
At one of our customer sites, we have a Delphi 2007 application that makes a number of HTTPS requests using indy components. All requests are made using the proxy settings the client provides.
For this to work, in IE we have to put the URL's in the trusted zones section. After a month due to security settings the trusted zones are cleared. This means we have to re-add the URLs again to make our application work.
Is there a way of bypassing IE settings or using a client side HTTP stack so we do not go through the browser to make https requests?
JD
Indy is the client-side HTTP stack. It doesn't use Internet Explorer's proxy settings. It uses whatever proxy settings you've set in the TIdHTTP component's proxy-related properties. If you don't want to use those settings, then don't set them.
Furthermore, I'm pretty sure the "trusted zone" has nothing to do with this. It controls what Internet Explorer allows Web pages to do, such as run scripts and load ActiveX controls. Indy never loads, displays, or executes anything. All it does is download. Internet Explorer's settings should have no bearing on your program's ability to connect to whatever sites you wish.
IE proxy settings are handled by the Internet options -> Connection -> Lan settings -> Proxy server configuration. Here you can tell which addresses should bypass a proxy server, it's not the security tab controlling that, it controls what a site is allowed to do within IE, and zones are used by IE only (and applications using the Web Browser control). Anyway, Indy does not use them automatically, you have to set a proxy explicitly. Do you use the web browser control?