I have developed a Delphi prog which logs into a website, collects "today's number" and displays it on a screen. Having got the data, the only place I could access it was in the final 'download completed' event. This doesn't seem right, as there seems to be no way to get away from the string of 'completed' events. The program never returns from the WB.Navigate call. WB.Stop and WB.Quit seem to refer to browser activity. I want a WB.TERMINATE or something! It's OK for now, but I want to put the web access in a loop, and with the current technique it just burrows deeper into the stack etc. I assume it's running in a separate thread, but can find no info about this. Should I put the browser component on a separate form created at run-time? Would freeing the form tidy things up? Any advice would be appreciated.
EDIT: Thanks for early responses. I must have had a senior moment. In fact the WB.Navigate call does return. The IDE tells me that it launched a thread with the web navigation request. So it's down to me to do some internal synchronizing to check the data arrival. Yes, the data is on the web page not a download. There is no API. I could have done basic searching of the web pages but it's 3 levels deep. The component made it easier to select the wanted data. All good now. Sorry for wasting your time.
Related
I've been hunting around my issue for a while, probably the best I've come up with is another Stack Overflow question: How should I perform a long-running task in ASP.NET 4?
I'm in a similar place in that I'm wanting to understand what my options are, but I don't feel I know enough specifically about MVC to come to a view. I'm using MVC 5 but with the 4.8 framework, plus I note that technologies such as SignalR have become available since this question was asked. I was wondering if any experienced MVC'ers could give me a view?
I too have a long running process. More specifically, the user is importing a file. The file is delimited so the import happens line by line. The file might be thousands of lines long. Each line will be parsed and imported in a fraction of a second but the whole operation might take several minutes.
I don't particularly need behaviour to be asynchronous, but because of the length of the entire process I want to regularly update the user on progress. I'm wondering what options I have?
I've got a vague recollection that I might have looked at this problem 20-odd years ago (Classic ASP), and solved it by regular flushes, sending a bit more of the page to the client every few seconds, but I'm trying also to use a _Layout page now, so I've sent the page back already. So I don't think I have that option, even assuming such a mechanism still exists. A bit more recently, but still a while ago, I might have used javascript to poll but everything I'm reading now seems to point me to newer technologies which I'm not sure I fully understand yet.
I'm just wondering how would you solve this problem?
I would not be performing any of the file parsing on the web server, especially if it's thousands of rows long. I would delegate this to a background service of sorts, whether that be a Lambda service in the cloud or a Windows service or a scheduled task. You could then call your SignalR hub from the background task (whatever that might be) to update the progress of the import.
Updates
2016-02-18: Added process information
I have a Delphi program compiled using XE4. It is being used by a few hundred customers. A couple of weeks ago one of these customers reported that some areas of the executable was being erased (images bellow) randomly during the day. This client has 35 sites using this exe and the problem occurs on no more than 10 of these sites.
Investigation
1 - My first suspicion was an infinite loop. The exe keeps responding while the components are erased, nothing changed on the code so radically from the time this problem did'n happen and the logs don't show any loop (this exe has logs everywhere).
2 - Misbehaving threads. I have a separate thread that syncs data between this exe and our server in the cloud. Again, logs don't show that the thread is running when the problem occur and again, nothing was changed here.
3 - Some other program (antivirus?) is affecting my exe. Couldn't investigate this hipotesis properly yet, but until now couldn't find any installed program that raised my attention.
My question is: What could be causing this issue? How can I investigate further? I know this may be a wide question but this is all information I could gather and I can't imagine many more places to look at.
Images
1 - In the image bellow the red-stroked area should be a TToolBar
2 - In this second image there are three areas, from the top to the bottom the first one should be a TToolBar, the second one should be the title of the child form and the third one should be a TwwDBGrid
3 - The third example shows on the top the erased area where should be a TEdit, just bellow it there's what should be a line on a TwwDBGrid and on the side we can see an erased scrollbar from the TwwDBGrid
4 - This last example shows 5 erased areas: The title of the application, the main TToolBar, The title of the Form, a TButton and two TwwDBGrid
5 - This is an interesting example beacause beyond the erased components there are 4 TSpeedButtons that are not erased but they are without the images they have originally (the first red stroked areas). The other 3 red stroked areas are, in order, 2 TEdits, a TwwDBGrd and a TButton
Process Information
I got a screenshot by the momment the problem occurs. scgolr is my software.
There is really not enough detailed information to give you a definite answer. However, I can answer with some directions on your question:
How can I investigate further?
Because of what you have stated:
The program is in use by a few hundred customers
One (only) customer experiences the problem
First occurance of the problem was some weeks ago
the first thing to do, is get in contact with the customer, and get the information you say that you have asked for but not got. The questions that need to be answered are:
What has changed in the customers environment at the time the problem
started with respect to hardware, network, server, OS, other software
running in the PCs?
Has anything changed in the way your customer is using your software?
What do the customer have to do to get rid of the problem, once it occurs? Close the program? Restart the PC? Or maybe just minimize - restore the erroneous window?
With the above I do not suggest that the fault is with this one customer and their equipment or their way of using the software. It may just be that the combination at the site which is different from all your other customers, triggers the problem to show up.
Some specific things to check in your software and at the site when problem occurs and if the problem goes away with a minimize - restore of the application (which would suggest a painting interrupted problem:
Do you call Application.ProcessMessages at any time?
Does the background thread access same data as the GUI? If yes, are the data protection properly in place (locking, synchronisation).
Does the background thread access any GUI components without Synchronize?
Finally I suggest that you visit the customer onsite. You get much better and faster answers in a direct discussion.
Edit after process information received.
There is nothing alarming concerning GDI or User objects. But it is alarming when you say in the comments that you call Application.ProcessMessages in many places, obviously to 'fix' a non-responsive UI. For example, what happens if the user double clicks a button, but does it slowly enough that Windows detects it as to separate clicks? First click may start your long lasting procedure within which you call A.P. The second click is read from the message que which starts the same procedure. Now the second call to the procedure runs (with its own calls to A.P.) and eventually ends and execution returns to the first call. Depending on what you do in this procedure, you may well be messing up handles and device contexts etc. A strong recommendation said with a friendly intent: Get rid of those calls to A.P.
the problem is with the security plugin (Warsaw - Gas Tecnologia) bank's website that your client is accessing , update it and it will be solved , the problem happens in Brazil
As #SebastianZ and #AlekseyK pointed out you may experiment exaustin of some GDI resource (handles?).
If the system coukd be accessed some tools like Process explerer or process hacker could give you some hints. This utility may help too GDIView
I don't know if this may apply to your case, but sometimes database data corruption can lead to strange effect in running programs (i remember 'Data Bombs' causing out of memory exceptions ...
So if something cause a GDI allocation loop, the graphics of your app cauld be affected in 'strange' ways
Im in the need of changing a client's browser location on a certain time X. It has to be solid even if a user reloads page it should redirect to location X
Below is my current (imperfect) implementation of this:
Im in the need to a way to make this solution solid OR an alternative solution if that would fit better. ITs to complex to share code snippets so I would like to reframe from that and discuss the concept instead and discuss how to code wise implement the missing gaps to keep this question overseen able.
Current solution:
Im currently using redis and faye to send data to a subscribed channel (at the client) at time X
This means that a jquery event (redirect of url ) will be triggered when the redis background scheduler pushes/sends this data at a certain time X ( which is always in future)
The problem is this is not solid, since on a page reload the pushed/send data is never received and rendered useless.
Even if it would be possible to save the last 10 sends/pushes in lets call it a "history" one could read it but say a user drops connection the excecution of this data would be useless since it will be to late and not in time.
*How could I overcome this problem? I prefer to keep the load server side and not use a client polling the server every 1 seconds to check if there is data received. that would not scale at all and would become a big problem if usage of the app grows.*
What ideas on this concept to make it solid or use another method to get a solid implementation of this idea? any comments ideas suggestions welcome thanks in advanche!
I haven't done any coding of this kind and would like some pointers how to start. The service will eventually do several things and perhaps someone has already thought of it made it happen.
The big picture is this: Detect if a PowerPoint presentation has been updated on the server. If it has extract the slides and save them as individual jpegs then upload them to a specific image list in SharePoint. All this has to happen without human intervention.
I assume this would be a window service project, right? Then a file stream property that with some property that deals with changes in the file?
As far as dissecting a .pptx/.ppsx files and get the slides converted, it there a "api" or some dll class?
What about uploading files to a library list on SharePoint automatically?
Thanks,
Risho
I've done this in Topshelf http://topshelf-project.com/, a windows service host for .NET.
https://github.com/Topshelf/Topshelf/blob/master/src/Topshelf/FileSystem/FileSystemEventProducer.cs
Since Windows has an event pump issue if events take too long, we also implemented polling on top of this since the FileSystemWatcher gets disconnected during those times.
https://github.com/Topshelf/Topshelf/blob/master/src/Topshelf/FileSystem/PollingFileSystemEventProducer.cs
Now, these producers are supposed to be tied to actors so they might seem a bit overly complicated for just checking on file system events. It's up to use if that model is useful or just the core part. Remember that you can often receive events even if the file is locked or not done yet, so handle those exceptions.
SharePoint has what is called a timer service for just these types of situations. Andrew Connell has an article regarding creating your own timer jobs.
http://www.andrewconnell.com/blog/archive/2007/01/10/5704.aspx
I'm going to create a view counter for articles. I have some questions:
Should I ignore article's author
when he opens the article?
I don't want to update database each
time. I can store in a
Dictionary<int, int> (articleId, viewCount) how many times
each article was viewed. After 100
hits I can update the database.
I should only count the hit once per
hour for each user and article. (if
the user opens one article many
times during one hour the view count
should be incremented only once).
For each question I want to know your suggestions how to do it right.
I'm especially interested how to do #3. Should I store the time when the user opened the article in a cookie? Does it mean that I should create a new cookie for each page?
I think I know the answer - they are analyzing the IIS log as Ope suggested.
Hidden image src is set to
http://stackoverflow.com/posts/3590653/ivc/[Random code]
[Random code] is needed because many people may share the same IP (in a network, for example) and the code is used to distinguish users.
Sure - I think that is a good idea
and 3. are related: The issue is where would you actually store this dictionary and logic.
An ASP.NET application or session scope are of course the easiest choice, but there you really need to understand the logic of application pools. ASP.NET applications are recycled from time to time: when there is no action on the site for a certain period or in special situations - e.g. if the process starts to take too much memory the application is shut down and a new one is started in the next request. There are events for session and application shut-down, but at least some years ago they were not really reliable: In many special cases they did not always fire. Perhaps they are better now, but it is painful to test. And 1 hour is really a long time: Usually sessions are kept alive only like 20 minutes after last request.
A reliable way would be to have a separate Windows service (a lot of work to program) or always storing to database with double-view analyses (quite a lot of overhead for such a small feature).
Do you have access to IIS logs? How about analyzing IIS logs e.g. every 30 minutes with some kind of timer process and taking the count from there? Or then just store all the hits to the database with user information and calculate the unique hits with a similar timed process.
One final question: Are you really sure none of the thousands of counter applications/services in the Internet wouldn't do the job close enough to your requirements?
Good luck!
This is the screenshot of this page in Firebug. You can see that there is a request which returns 204 status code (No Content).
This is stackoverflow's view counter. They are using a hidden image which point to a controller's action.
I have many articles. How to track which articles the user visited already?
P.S. BTW, why is this request made two times?