watin dll: other équivalents solutions for parsing web page? - parsing

I using watin in a wpf c# project for firing clicks on links and parsing pages in web sites.
It works post part of time but it launchs sometimes exceptions without raisons ans it is not reallly intuitive .
Do you know équivalents more realiable , intuitive ans with a big community and a good documentation

There is one big contender to Watin and it is called Selenium. It is different from Watin and there is a learning curve (you can't just switch to Selenium) but it could suit your needs better (I have VERY stable systems made in Watin and never had an issue with them).
It could help, also, if you post more details about your issue, in the way you asked your question leaves a lot to be desired (what's making it unstable, what portions of the code, what exceptions are you getting, when, etc.)

Related

Which would be fastest and most secure among FastCGI, ASP.NET MVC and Web forms for Server Applications

I have to build an Automated Chat Application which will have it's front end as a widget on the website and back-end logic at server. It's main purpose will be to provide an interface to the customers to solve their queries with a chat personnel sitting at a location with the server as the communication point. If there is no person available then the application will parse the string and try to give a few answers by checking among a database.
I have a very good experience in C++ and a little experience in Java. The main concerns of the application are source code security and highest efficiency.
The restrictions which i am facing from the server side is that i have to run it on IIS and the only scripting languages supported are PHPv5, ASP, ASP.NET and FastCGI.
I am not going for PHP since i have found from googling that it's possible to reverse engineer it and it will take me a lot of time to learn a new language, so a strict no.
I prefer going for FastCGI cause it will serve exactly what my requirements are, but i am unable to find a way to compile on windows with C++ or any good tutorial. Most of the results i found on google related to FastCGI development were about Linux or in Perl for windows. I am also unable to find a way as to how to compile it on windows platform.
I am now being forced to go for ASP.NET, which will be my last option since it's an Interpreted Language and Interpreted languages have a tendency to be weak against reverse engineering.
So can anyone suggest me how to go about with it. If there is no way out for FastCGI then suggest me which will be good amongst MVC and WebForms for the given situations. I am new to both of them and i need to complete the application within two weeks.
First things first.
EVERYTHING can be reverse-engineered. It just depends on how smart the guy trying to reverse engineer your code is. Obfuscating is not a replacement for securing your code. If your server side PHP is safely stowed away on your server with all the proper security parameters configured, then you have nothing to worry about. But if you don't know PHP already then fair enough.
ASP.NET is only partly interpreted. The C# or visual basic that you use for the server side code is compiled. What IS interpreted is the code directly on the page. Remember though that most of your logic does not take place there... it takes place in the code pages which are compiled and the page code is mainly for displaying what you have already calculated. But don't forget that any server side tags that you put into your HTML on your pages get parsed and stripped out when the page is served to the user. If you have a tag <%: ViewData["value"] %>, the user will only see the value for "value" when they look at your source, and not the actual server side tag.
You are going to have a tough time learning ASP.NET AND completing your app in two weeks. I personally love ASP.NET MVC, but if you don't have much of a web programming background then it probably isn't the easiest thing to learn in comparison to Web Forms. That said however...
I'm not a fan of PHP, but it is widely used and a great language for setting up simple projects quickly. There are also tons of examples out there and it is of the first web languages that people learn (in part for those reasons). If you have two weeks to learn a language and set up a project, go with PHP. Even with a C background, there is a lot more to web programming than just knowing a server side language, so you'd be best off a quick and dirty language that gets what you want to get done, done, with minimal effort. That is to say, PHP.
FastCGI / C++ has a better performance. If you need a link on how-to: http://cgi.sourceforge.net/ and much more with google search.

Delphi Application over the web

Possible Duplicate:
What Web Application Framework for Delphi is recommended?
We have a Delphi 2007 desktop application which we have hosted using Citrix. Now we want to get rid of Citrix and somehow web-enable it.
I have done bit of research and found that it is possible by using the uniGUI.
http://www.unigui.com
Conclusion: Can be done, but would require a re-write and only a subset of components are supported. Serious questions remain are the monolithic application structure in a web environment.
There are two more options morfik and atozed and they also require a re-write.
I want to know if there is any other option which requires a very less re-write work and how fragile is it?
How fragile it is, is based on the quality of your code. If you have a good structured application, with business logic and data access fully separated from the GUI, it will be pretty safe, although you still have to rewrite mostly all your GUI.
If there's logic in your forms, and the code that talks to the GUI components is intwined with the code that checks your input and stores the data, then you have a big problem.
In that case, this is a great opportunity to refactor large portions of your app and do it better this time. ;)
Since there is no "silver bullet" here, it doesn't matter much which product you use. You have the same challenges with any of them. I would recommend spending a few days on a Proof-of-Concept (PoC) re-write of 2-3 typical screens. Implement the POC for each "finalist" product, and see how it works out. Keep track of how long it took for each one, things that were easier/harder, and how the end result appears to the end-user (performance, good/funny-looking, robustness, "feel").
As for the actual re-write, I would recommend the following:
Re-factor existing application to remove business logic from the UI.
Full Regression testing, and push that into production.
NOW proceed with conversion to one of the web tools.
Oops - I left out a step. Step 0: FREEZE all features/fixes. If fixes are needed to current production, they'll need to be done in a separate branch, and then rolled-up into this project later.
Note that this type of work lends itself nicely to outsourcing, as the work is straightforward and the requirements are simple. Especially if it can be delivered one form at a time, so progress, timelines, and $$$ can be measured in small chunks.
Another preliminary step is to develop a "cook book" for stripping the business logic from the existing GUI layer. It should identify naming conventions, common libraries (for code that should have been shared all along but wasn't), and should describe the conversion methodology.
AFAIK, there's not tool will convert your desktop application to web application without requiring rewrite for most of GUI Parts.
as Golez said, you will have to refactor your application, try to separate your business logic from the GUI, then you can use some tools like Intraweb to develop the GUI as web and reusing the existing business logic with it.
Another option by converting your application to n-tiers technology and warp your business logic as web services or any open technology and make your web part by any web languages such as ASP.Net or PHP.
Depending on how 'Web enabled' you want the App to be.. I use Cybele Software's (https://www.cybelesoft.com/) Thinfinity UI to extend Apps to the Web, including Database Apps.
It only requires the installation of their ThinFinity Server and one line of code added to the Proj source and you are in business.
The Apps all run on your PC.
Well perhaps I simplified it a little, but worth a look.
HTH.
Regards,
Ian

Vaadin vs Apache Click which one to choose for my webapp development

Vaadin and Apache Click seem to be equally good, which one should I choose for developing my web application. Or rather, what are the Pros and Cons of each framework.
I'm a committer of Apache Click but hopefully you will find my opinion objective.
I don't know Vaadin at all, just had a cursory look at their website and examples and mission statement.
Apache Click and Vaadin are meant for different problem spaces. Apache Click is targeted at traditional enterprise web applications while Vaadin targets the Rich Internet Application (RIA) space.
Apache Click is a traditional share nothing, stateless framework. Unlike traditional action based web frameworks Click provides a slightly higher level of abstraction by using Pages and Components. At the of the day Click isn't a revolution. Its just one of many approaches of doing web application development. In my opinion its a very good approach, both from a developer and maintainer point of view. (In enterprise environments the developer and maintainer are often not the same individual, so I differentiate between the two roles).
However developers sometimes have requirements that cannot be satisfied by traditional web applications, which is where RIA comes in. They provide a richer user experience (think desktop) but this does not come for free. RIA comes at a cost in terms of complexity, productivity and time-to-market. It makes sense though, if the requirements are for a richer experience, you need to do more in order to deliver that experience.
My advice here is: think carefully about your requirements. Don't simply assume that RIA is better, there is a price to pay, so make sure you get return for your investment.
Lastly, if your requirement is for RIA, then you should compare Vaadin with Flex. If your requirement is for a more traditional web application then compare Click with Struts, Stripes, Wicket, Tapestry etc.
Kind regards
Bob
I use both ;-)
Click is the best for classic web applications (pure html with no/ small/ handmade javascript).
It is very lightweight (small size and stateless), less verbose (you can do your html in html with velocity/freemarker power ;-).
A real php-killer.
Vaadin is the best for desktop like enterprise applications.
It is statefull, heavyweight, verbose, but amazing.
Grails, Play!, Spring MVC is a real step back after Click/Vaadin.
So my advice: use both!
For most of your pages you will use Click, for complicated tasks - Vaadin.
I've been using Click since 2006. I've built various web applications with it in the last 4 years. All apps (B2B) are still going strong and continue to be maintained/added to now. The best things about Click IMO include:
Very little magic and almost no plumbing. It's all just plain java (and jsp/velocity/freemarker take your pick; or alternatively use all 3 if you so choose; I have for some scenarios because it's so easy to).
It's super fast (It doesn't mean your app won't be slow, it just means Click will never be the reason why your app is slow)
It is so easy to integrate your favourite java lib eg. Ibatis SQLMAP, Spring, displaytag, DWR etc.etc.
You don't need to buy a book to understand/use it (although you may need books for all the other 3rd part libs you plan to use).
Most of my code written 4 years ago still works with the current release. That is just awesome.
Click fits in your head. The whole framework that is. It is so simple you don't have too many questions even as a beginner.
The guys that maintain Click are super responsive and very noob friendly.
It's simplicity helps my apps adapt to new/old tech. i.e I can easily do AJAXy stuff if I want or not.
The form features make handling html forms very easy/fast.
The table features make displaying data super easy/fast
Click's features help me write apps very quickly. Makes the need for scaffolding unnecessary imo.
If you want a light weight, fast, easy to understand framework which helps you build applications as fast as you or your team are personally able to, check out click, it will be worth your while.
Before making your choice you should evaluate all options by taking them out on a test drive.
I'm not familiar with Click, and someone should correct me if I'm wrong, but I understand Click basically a technique to link your server side code to web pages. The UI is page-based and you define the UI with components corresponding to HTML elements. Ajax is not (yet) supported, I gather.
Vaadin operates at a higher level of abstraction. With Vaadin you get to concentrate on your business logic, and build your UI naturally with views, layouts and other components (check out the Sampler). You don't need to care about RPC, page transitions, HTML or templates. Vaadin gives you a fully cross-browser compliant Ajax UI that looks and works great out of the box. It's very easy to get started, and the forum is very active (>700 messages last month compared to 96 on the Click mailing list).
Disclaimer: I'm a member of the Vaadin team and have not used Click.
I would choose, and I already did in a few projects Click (over all other frameworks).
The main selling point for me was that I was able to learn Click and be quite productive in one week-end (Vaadin - but other frameworks too took me much more).
The free and available documentation for Click is much better - you can compare the sites yourself (even if the Vaadin site looks cooler, the Click site is simply more useful - at least it was for me).
The live examples are much better - I always look at them as a snippet repository, and just copy what I need from there (since I'm a lazy programmer :) ) - there's a "Page Java" and "Page HTML" link for every example that shows the source code for every available example.
Regarding the message traffic, I don't think that should be a criteria, since having so good free online docs, and the framework being so much simpler, the users simply don't need to ask that much.
Disclaimer: I'm just a simple user that have used both frameworks.
Is Apache wicket an option as well ?
My experience is with jboss/ejb/jquery, but looking at technologies for starting a new project. I've done a quick bit of research myself abd Vaadin seems to be very well regarded.
Surely Apache Wicket is something you can not ignore if you are comparing UI frameworks. ( I am doing same.) I dropped Vaadin because of poor documentation ( maybe I am not a good Googler.)
Currently comparing Wicket, Click, and ZK.
Click and Wicket are ahead so far for the following reasons:
Generates pure HTML instead of rendering UI using JavaScript which happens in GWT, Vaadin, and ZK. You can use HTML5 features then.
Even with native HTML, server-side binding of UI forms is possible.
You don't need to worry about communication with the server. ZK also does this but with their own language ZUML.
I have found server-side memory footprint higher in ZK in the case of components like the ZK Grid. (Memory footprint for Click is not known to me yet.)

Web Design: Dreamweaver vs custom HTML

I've been having this discussion with my brother, and it looks like none of us is well qualified to answer the question.
n fact, since I've been learning ASP.Net MVC, I've the impression that I should create all my .aspx pages using HTML/CSS. He (my brother) sustains that - in the modern day - we should not spend to much time on HTML/CSS because there are software (like Dreamweaver) which do just that.
Although I do more of the server side coding, I think it's important to settle this matter for further orientation.
By the way, in case I should use Dreamweaver-designed pages, how would I integrate them in my ASPNET-MVC project?
Thanks for helping
This is the classic discussion pro/against html generation, and just by asking the question you risk the post being closed as "subjective" =)
Basically, you have to decide whether you want simplicity in your work creating the code, or in the code itself.
Dreamweaver (and every other WYSIWYG HTML tool I've ever encountered) will generate a lot of extra code, which will make it harder to incorporate your server side code - simply because there's a larger mess to navigate in.
On the other hand, creating the HTML page might be a lot faster than writing the code by hand.
If you're good at using a WYSIWYG editor, be it Dreamweaver, the built-in one in Visual Studio or some other one, you might benefit from drawing up the rough scetches in that. I would recommend, though, that you always do the finishing by hand, in the code. That's the only way you can retain complete control over your output.
From a purist perspective, unless you can already code your HTML, CSS and JavaScript by hand you shouldn't use a tool that does it for you.
Tools are very valuable at making your more productive, but the experience to know when they screw up is more valuable.
Well, the answer is pretty simple. Dreamweaver is just a glorified code editor with autocomplete, the ability to connect to FTP and upload your work within the program, and a unique "design view" which lets you move coded objects around with your mouse and also give you a visual representation of what your work looks like without opening a full browser. It doesn't matter if you use Notepad, Dreamweaver, or another code editor to create your code. For a fact, I know a lot of ASP/.NET programmers just use MS Visual Studio to do their work because ASP/.NET is simply a MS language and Visual Studio supports a lot of tools which ASP/.NET programmers find useful. That said, you can still code ASP/.NET with any other code editor, it just may be more manual or time consuming on your part to do so. You said "Dreamweaver-designed pages" and all this is is just a document coded with Dreamweaver being used as the code editor, there are no Dreamweaver specifics that would add to the document. Unfortunately, your bother does not seem to understand the differences or the complexities between programs or programming languages and vice-a-versa where they are similar. Whether you are programming for HTML or ASP/.NET, making a stylesheet with CSS, your project being server-side or client-side, it is always good practice to use standard markup and make sure all your work is cross-browser compatible with a strong influence on being compatible with IE. Just remember that most of all code is done by hand, and the reason for this is because the programmer needs to fact check everything he/she is doing to minimize errors and create an efficient document or program. Unfortunately again, a program just cannot do as good a job of realizing the ultimate coding goal than that of a human brain can achieve. Oh, and by the way, unless you are creating a simple document and utlizing only CSS to design that document, leave the design up to the designer.
Tomas is on the money with his comments. Your brother's notion that in the "modern day" you shouldn't have to mess with the code is sadly not true. The tools that we have for html/css/web development will get you only so far. You will likely be able to build a "functioning" site with any of them individually or in combination but to get the real fit and finish that you see in a excellent application you will be working in code as well.
Moreover, you have chosen ASP.NET MVC which for ASP is a pretty thin abstraction layer and works pretty close to the metal. It was designed as a framework for folks who do a lot of the code by hand and with minimal wizardry. The tooling at this point reflects that. The tooling will stub you out a basic application, but the rest you do either outside VS2008 or by hand in the VS2008 editor.
If you are "starting to learn" MVC as you said, your workflow will probably start off by using the framework code generated by VS, so now are working in VS2008. The html editor in VS2008 is limited to say the least, so if you want some more wizard driven development of css or html for your views you may work on a view page or a html snippet in Dreamweaver, Expression or your some other editor. You'll then be dropping that code back in VS2008 and doing more work there to link up the view to the controller.
As you get more of an understanding of the css and html you will use the wizards and dialog driven stuff in Dreamweaver less and less simply because it is faster to just open up the code and make the change.

Web development for a Computer Scientist [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have BS in Computer Science, and thus have experience developing software that runs at the command line or with a basic GUI. However, I have no experience making real, functional, websites. It has become apparent to me that I need to expand my skills to encompass web development. I have been using Ruby to develop applications, but I am aware that it is quite popular for web development. I want to use my skills as a programmer to assist me in developing a personal website for a band.
I have experience with HTML, but very little with CSS. I want to leverage my skills with programming languages to create a website containing pictures, audio clips, a dynamic calendar, a scheduling request tool, and other features common to band websites.
What kind of resources are available for a competent desktop programmer to learn the entire process for developing a website? Is it best to use free CSS templates and WordPress as a foundation for my site or make it from scratch? Should I use GUI tools or write it all in Vim/Emacs? Is Ruby on Rails sufficient for my personal website, or should I consider a more mature development platform?
My main goal for this project is to come up to speed on current web design technology, and actually understand the entire process for building a website.
I think one really important thing to understand in web development is HTTP. HTML and CSS are important, but I think it's more critical to understand the stateless nature of the web, and how each of the HTTP verbs work, and what they can/can't do.
http://www.freeprogrammingresources.com/http.html
A good tool for seeing how HTTP works is Fiddler.
If it's as much a learning exercise as anything then take an iterative approach. Build revise. Build revise. My (very) rough guideline below:
Client
Start with the structure of a website and concentrate on the client.
Use notepad and build a bunch of static pages for your band. i.e. Hand code initially. Try to build all your pages with CSS. No table markup. Then play around with some Javascript to bring things to life. (Navigational menu\ Calendar selections\etc). Learn about how to import and link to Javascript and CSS files.... and how these files are treated re:caching etc.
Try to learn up to the limits of what you can do on the client (generally). Factor in the nuances of 3-4 browsers (Firefox/IE6/IE8/Chrome) re:DOM and client side eventing.
Server
Then start looking for what you might want to change across pages/sessions. i.e. what needs to be manipulated server side. And pick a server side technology.
Start with basic post-back processing. Forget databases at this point. Learn how your framework of choice maintains state..... not just the name of the technology but the real nuts and bolts of it. One of your single greatest assets as a web developer is understanding the state model(s) of the technology you're using.
Then go for a deep dive on the web server technology of choice (and in general). Understand the full request pipeline from client to server and back. This will teach you forms, http and its verbs, web server, filters and modules, server to framework hand off, page and control life cycles, back to the client.
Now start working on dynamic content injection and the like. How to make and use reusable components in your web pages.
Databases, caching, performance and diagnostics.
Then get into into all the fun stuff like ajax, etc. Replace your javascript with jQuery, etc.
Then you got the whole Webservices\XML\JSON\etc side of things to discover.
Resources
Well the web obviously. For client side stuff, going to the sites of companies who make third party web controls can be quite interesting. Asking how the hell they did that? Viewsource is your friend. Look at how they structure and build their pages. Pick a couple of good web designer sites, and you find a plethora of rants about browser wars etc that will give you good (under the hood) info.
Once you hit server side, I'd go for white paper type learning from your vendor of choice for your technologies.i.e. webservers/frameworks/etc. Again find a 3rd party howto/evangelist site (I used to use a lot of "4 guys from Rolla" for example) that will demonstrate how to do various things. Language learning is ongoing. Basically just do the best you can till you find a better way.... and always be on the lookout for a better way.
You really need to understand html, forms and css to get anywhere. I say forms as this will give you the round trip needed to understanding the stateless nature of web dev.
To further labour the point, I have interviewed many people who think you can only have one form on a page and can only have one submit button per form. This is all down to a lack of foundation knowledge.
So for that I'd recommend starting with htmldog.com.
After that, a lot of web development is done with frameworks. Gone are the days where you do it yourself (well mostly) but my above point still stands. You do need to be able to peep under the hood with some confidence.
I've been doing web development for 12 years and started out with Perl on Solaris and Linux. Since then I've also done Java and more recently ASP.NET. However, I'm slowly falling for Django in my private projects.
What I've found over the years is that the inherent problems - cookies, javascript, presentation, state, authentication are all the same but just handled differently. So ultimately its down to you and your language preference. Plus a little of enlightened self interest when it comes to potential employment.
Programming aside, you should also become familiar with web servers (Apache and IIS spring to mind), Http codes and headers, Mime types and encoding and FTP. As well as Javascript (mentioned already), plugins, browser platforms and good development practises such as using Firebug, Fiddler and so on. It also wouldn't hurt to have a good idea of the image formats available, image optimisation, CSS sprites, content compression, caching and the like.
All depends on where you want to start!
For a newbie, I'd pick Django and (obviously) Python. Good, clean language with cheap startup options, low cost IDEs (ie free) and hosting your sites is very affordable.
But that's just a subjective opinion.
If your goal is to
My main goal for this project is to
come up to speed on current web design
technology, and actually understand
the entire process for building a
website.
Then start from scratch in Ruby, PHP, Java, ASP.NET, etc...
When you run into a design problem or just want to know how others have approached something, then look at the frameworks.
Once your up to speed, and your website is starting to grow, then segway into a framework, to get up to speed on the frameworks.
I agree with John on this one.
As you know from your own experience in pursuing your BSc, understanding the basics of any language is what makes you even more capable in expanding that knowledge or specializing.
With that in mind, it would be best to understand the basics of HTML and CSS.
Understanding the syntax and overall language will help in the future when you want to pursue large projects using frameworks like Django and Rails. The basics will also especially help with tweaking CMS' like Wordpress to be more customizable to your needs.
One thing in particular that I'd like to mention is that web programming, like many other forms of programming has its own special structure and "proper" way of doing things.http://www.w3.org is a great way to ensure that your work is passing general web design standards, most sites don't follow this because it is tedious, but from a learning perspective it ensures that you get a nice strong foundation.
www.w3schools.com is also a great resource for detailed help on web programming. Lastly, I like colourful code, so I like using basic text editors such as notepad++ or notepad2 or gedit to do my web coding. GUIs like dreamweaver may tend to fill up your code with extra junk and spaces, so I don't recommend them, but they are still great tools.
Don't bother with Rails yet -- write CGI scripts in Ruby. It will be very similar to what you have done for class.
After you have about thirty of those under your belt, you'll know what you want out of a web framework.
I'm a Computer Scientist and a web programmer and I would suggest you learn both HTTP and CGI:
CGI Made Really Easy
HTTP Made Really Easy
As the titles of the above tutorials claim, they made the concepts "really easy" for me.
Once you've got CGI and HTTP down pat, I'd suggest checking out following sites that provide a wealth of articles and references for web programming:
webmonkey
w3schools
Mozilla Developer Center
Assuming you want to concentrate on writing web apps, then Perl, PHP, Python and Ruby are all fine choices (I myself use Perl predominantly) and I'd suggest doing some research into the popular web frameworks available for each language.
Most importantly, pick something simple as your first web app, e.g. a form and a page that shows the results of submitting that form. Some good examples (using Perl's CGI module) can be found here:
CGI.pm - a Perl5 CGI Library -- see the first set of links on this page.
When you want to start writing web apps that use a database, read up on SQL and popular libraries/modules in your chosen language for database manipulation, especially ORM (Object-Relational-Mapping) interfaces that allow you to deal with records in an Object-Oriented fashion.
Good luck with it! Being a web programmer is fun because your audience is teh intarwebz! :)
If you are starting from scratch as per John MacIntyre's suggestion, you may lean towards PHP. With all of its shortcomings, it does have one really good user manual. It is also easy to get started with and is installed on pretty much every host and goes well with Apache.
Also, w3 schools is good to begin learning about CSS and XHTML but don't forget to check out the specs at W3C.
Also, please read this Stack Overflow question & answers.
For what you're describing, Rails or Django might be slight overkill but it wouldn't hurt to learn them. Django, in particular, might be good because of the notion of a project containing multiple apps (e.g. calendar).
Whether you use a framework or write everything yourself, though, you'll need to know HTML and CSS. CSS is extremely simple if you have a BS in CS...you could read a tutorial and know it in five minutes.

Resources