Practical example of architecture using EBC? [closed] - ruby-on-rails

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
I was intrigued by Robert Martin's talk about "Architecture: The Lost Years". In it he discusses the Entity, Boundary, Control design pattern on which MVC is based. I love the idea of deferring architectural decisions. He described deferring the decision about how to implement the DB layer in his own wiki app FitNesse. I have organically deferred decisions like this in my own coding, though there wasn't a preconceived modular design that brought this about.
I want to better understand this EBC architecture (which seems closely related to DCI) from a practical standpoint so that I can begin using in a small project. I want to capitalize on "deferring decisions" and the ability to swap out aspects of the design like the UI.
Rails, for example, uses a form of EBC (MVC) but it's so heavily baked in that one could not easily substitute an alternate UI thus converting a Rails app to a console app or a desktop app. The intriguing thing about design for me is this ability to transform applications by swapping one thing out and plugging another in. That is, I wonder at the idea of designing an architecture so that one can, in a manner of speaking, swap out the UI or the persistence layer. I feel that if the architecture is well designed, the coupling will be low, and such a feat will be within grasp.
I've ordered the book by Ivar Jacobson that Bob mentioned in his talk. I've search online quite a bit but all of the examples I've found show simple diagrams. I speak code. I would benefit more from looking over a few simple classes that demonstrate the concept and show how one might swap out one layer (UI, DB) for some other implementation through the use of boundary classes.
If someone can't point me to a good resource illustrating this, would this be hard to whip up? Maybe we could use the standby example used in lots of software books: a video rental store (almost a relic these days). Please demonstrate how the UI or DB layer could be swapped. One thing that's confusing me is views. I can't tell from the diagrams I've seen if the views are the boundary classes themselves or if they just communicate with them. Also, Bob mentioned that the original intent of EBC was that we'd have lots of micro-views not a single macro-view (as we do in typical MVC); I'm curious what this might look like. (I prefer Ruby or JavaScript but, as beggars can't be choosers, any example would be fine.)
Thank you.

As far as I understand the video by Uncle Bob using "EBI" (Entity, Boundary, and Interactor) you should completely decouple your business behavior/state from frameworks/OS and services.
So in case of an Rails app your business behavior/state is completly free of dependencies to the Rails framework and hence can be tested like with rspec without firing Rails!
So on the business side you have Boundary classes wich interact with the Rails side using request and response models (very simple dataholders, not to be exchanged with the usual models from Rails). Only the Boundary classes interact with the Interactor classes which implement the (business) use cases / scenarios. And only the Interactor classes interact with the Entity classes which encapsulate the business state.
On the Rails side you find Controller classes interacting with Boundary classes (using Request models) and backwards a Boundary class interacts with a Presenter (using a Response model). Only Presenters/Controllers interact with Views (with the help of models (again simple data-holders). Note that in the realm of Rails Presenters are most likely Controllers.
Where does this leave AR? Well AR just provides the persistant service. On the same level as the Presenter/Controller level you will find Service classes which provide their services to the Boundary classes. So they provide all the necessary services which are frameworks/OS/technology dependent like persistance, security, timing, notifaction, etc..
With this architecture you are really able to reuse your business logic and completely replace the UI or database technology. For example, porting to mobile (iOS, Android, Windows) should be pretty straight forward.
With Rails, your app folder could look like:
app/
controllers/ Only these interact with Boundary classes
models/ simple data-holders, no AR here! (see services)
views/
services/ AR-stuff
boundaries/ To be tested without Rails
models/ Request & Response
interactors/ use cases / scenarios, to be tested without Rails
entities/ "the real business model without technical dependencies"
With this architecture, you need to code a bit more but don't forget the benefits of a good architecture:
A good architecture allows major changes to be deferred
A good architecture maximizes (major) changes not made
Last note: compared to the MVC pattern, its more like the M is replaced by EBI, the C can be splitted in CP/resenter), and an S(ervice) is added. So this could be called: VCPS/EBI but that sounds ugly to me ;-) BEPVICS maybe?
#Seralize, thanks for your feedback.
Let me try to answer your questions, so far I understand them: the stuff in services are coupled to Rails. They provide the implementation for the logic in EBI side. In the usecase of security, you have to be clear what (quantified) requirements you have, so you know what logic you can implement on EBI side, for instance (business) rules about when a user(role) has access to what content(and needs to be authenticated).
This means to implement authentication will be implemented using Rails, this service will be used by EBI. This security related logic in EBI is then pretty easy to reuse in your Java GUI example. There you have only to reimplement the authentication service.
To be clear in the security example:
The EBI side has the logic: what stuff needs what kind of security and when and how. The Rails knows nothing about this, it request what to do from the EBI side and or the EBI side request the Rails side to act.
The Rails side only implements the way how to do security like asking the user to authenticate (when needed) and passing the result of this to EBI so the logic can decide what should be done next.
EBI demands both sides to be decoupled & independent. It were as you
are developing the EBI as a library with a defined API.

Ask and you shall receive. I kept my eyes open and discovered this resource by Avdi Grimm:
http://avdi.org/devblog/2011/11/15/early-access-beta-of-objects-on-rails-now-available-2/
In it, he covers some of the reason that Rails projects get so tightly coupled to both the framework and ActiveRecord. He uses TDD to assure loose coupling with techniques like
Dependency Injection
Presenters
Strategy Pattern
DCI
It provides a good start to answering this question in a practical way. (It costs $5 for the early beta but will eventually be free.) In fact, it's the first resource I've found that does. Please add any others you find.
Here's the real gem that elucidates the heart of the problem:
One day, after years of witnessing and addressing the technical debt incurred in various maturing Rails codebases as a result of ActiveRecord-inspired tight coupling, I had an epiphany. What if we stopped treating ActiveRecord as the backbone of our model classes, and instead, programmed as if ActiveRecord were merely a private implementation detail?
Corey Haines puts it another way:
I pull the behavior out of my models into other objects that wrap the models.
I prefer to make the AR objects simple wrappers around the db-access stuff in AR.
I have a fairly strict rule that controller actions cannot use AR finders or,
in fact, interact with AR at all. AR should be accessed within api methods
inside your model, not from the outside.

This should be of interest too. It's the other book, not referred to by name in "Architecture: The Lost Years"
"Agile Software Development: Principles, Patterns, and Practices", by "Uncle Bob" Martin.
Taken from this SE question & answer. Read the other answers too.

Related

Break apart a monolithic rails app

I have a large rails application with 3 separate 'components'.
One is a mostly static WWW site, one is a shopping cart based on Spree, and one is a reward-program based on Instagram's API.
Currently they are all one giant Rails 4.0 application. As this app has grown over time, I have it harder to make distinctions between components. I want to modularize the app to keep these 3 components separate.
Is there a preferred way to do this "SOA" sort of architecture? or would it be better to turn each 'component' of the app into their own mountable engine? Or is there an even better strategy?
I have been looking at Spree's core, and how they have each component as an engine, and load them in the top level, and I'm thinking this may be the best route.
I don't have any experience with Ruby or Rails, but based on my experience you will need to ask/answer the below question and then decide how you want to proceed forward.
Who's going to be developing the code base and who'll be maintaining it?
If it's just you who's wearing all the hats, you may not want to have the overhead of implementing SOA (web-services to be specific). That said, you should definitely have a 'Contract' between each of these components or modules (however you refer to them). That way your modules/components can evolve independently and changes made to one to make the logic better doesn't necessarily mandate changes to the other components.
If it's you and a couple of other developers, I'd still say that you may not want to take the WS route yet.
If it's different teams that are developing and maintaining these components, then you are taking about an application at enterprise level and then you will start seeing the benefits of SOA (based on WS).
Cheers,
K

Implementing cross-cutting concepts into LOB/DI application

I am currently creating a small personal windows (desktop) .NET LOB application and I want to use the opportunity to increase my knowledge and experience with DI. I've separated my application into model, DAO and GUI parts but I am wondering how to implement some cross-cutting concepts such as:
currently logged on user - used for:
asserting rights - in some parts of the application I check if the user has necessary rights
auditing - recording user actions into a separate database table
etc
current application parameters (loaded from configuration file or table) - used for:
defining business strategy
defining UI (theme for example)
etc
logging to file/database log - used for:
logging UI actions (clicking on buttons etc.)
logging business processes (results of calculations, strategy decisions etc.)
logging infrastructure stuff (SQL used to for CRUD operations)
etc
At the moment I can think of several ways to provide this information:
Using static properties - UserEntity.Current, Configuration.Current, Logger.Current, etc.
Pros:
Simple to implement
Simple to use
Cons:
Gets messy
It is unclear which part of the application uses what
Can not be used if you need finer granularity (for example if for some processes in the application you need to override current values)
Using DI - giving each class which needs this information a property/ctor parameter
Pros:
It is clear for each class what it needs
It is easy to unit test
Cons:
It just seems to explode constructors
Makes problems if class needs to have a default constructors
Difficult to setup when classes get instantiated by 3rd party (XAML)
Using ServiceLocator
Pros:
Easy to setup
Easy to use
Cons:
It is unclear which part of the application uses what
Difficult to setup finer granularity (but not impossible)
I am currently leaning towards ServiceLocator as I've worked with it before and it worked quite nice. However I am concerned about loss of control. It gets very easy to just reach for the service locator instead of trying to fix a design problem.
Can somebody provide their experiences/knowledge?
Sounds like perfect case to start with aspect-oriented approach. Your application's LOB will be designed according to business functional requirements that has been cross-cutted with different non-functional requirements: authentication, audit, logging, etc.
In same time, some of current application requirements could be solved by dependency-injection. To start, I recommend first to identify composition root. For example, in wpf applications it’s the Application.OnStartup method. In case if you are able to identify composition root it is better to avoid service-locator. Service locator will add unneeded complexity while maintaining and testing, because it could resolve literally anything, thus dependency management will be complicated.
Next step, to decide: should dependency injection and aspect-oriented approaches be separated or combined. Both approaches has benefits and drawbacks.
While choosing separated approach you could use postsharp with a lot of benefits: great samples and documentation, community and ready to use aspects. But nothing come for free, postsharp has only limited number of features in free version and complicated integration with continues-integration.
Another solution: combine dependency-injection with dynamic-proxy. As long as you follow conception: program to an interface, not an implementation — you will achieve all requirements. Benefits: one place to wire all components. There are two major drawbacks: first dynamic proxy is quite limited itself, second — integration between dependency injection container and dynamic proxy — for some container it already exists, for others not. Example: Ninject extension Interception, or StructureMap and Interception.
I recommend you to take a look at following resources to find more answers yourself:
* Book AOP in .NET: Practical Aspect-Oriented Programming by Matthew D. Groves: first chapter available for free
* Book Dependency Injection in .NET by Mark Seemann: well-written book about dependency injection, and chapter #9 dedicated to interception, approach that I found quite useful in cases that you had described in question. Author of book also has an excellent blog dedicated to dependency injection and video about aspect-oriented programming with dependency Injection.

Which pattern is appropriate for my project?

I've been seeing a lot of articles and references about how to use this patterns for my first (please keep this in mind) serious project with ASP.NET MVC 3 and EF.
The project is not so big (13 tables) and it's basically a CRUD maintenance: organisations that have some users; users that can belong to a group and can create some work areas and templates... Nothing complicated. The point is that this will be the first of a project series and I'd like to create a base to work with.
My questions are:
Which of the previous patterns is the best? Does it depend of the project size?
My models would be these:
Unit of work
Dependency Injection
Do you think they are good enough and appropriate for my project?
I appreciate all your comments.
Serious application doesn't mean to be complex at first sight.
Over engineering an application upfront can be a real disaster, especially if you don't grasp all the technologies involved.
My advice would be to keep it simple. Create a basic application that fulfill requirements (get the thing done and make your boss happy) and then add new concepts along your learning path.
That doesn't mean I promote bad code, no way! Keep your code clean, well organized, etc. But don't be killed by the fear of doing something wrong.
It's normal for a developer to look back to an application made a few weeks ago and then realize that he did some shitty stuff. That's how we progress!
Last but not least, have FUN!
ASP.NET website provides usefull resources to learn the framework and all related guidances. There are a few application samples created step-by-step.
ASP.NET MVC was built with Dependency Injection in mind.
If you want to give a chance to your code to be loosely coupled and easier to change in the future you have to follow the patterns like Dependency Injection, Repository (for presistance abstraction), and UoW (for transaction abstraction).
So my answer is, you should learn about them in the first place to decide after if you want or no to follow the best practices. Even for simple project it's good to apply these patterns because often it gets bigger and bigger. and it's easy to do it in MVC so why to avoid it ?
There is many resources around to learn about. You can just google it.
I would like to answer this question in more generic way. Creating something which can be used in future is difficult than what it seems. All the pattern above can provide you infrastructure pieces to come up with some base framework.
But I would strongly suggest you to look at S.O.L.I.D principals (DI being part of it) to understand some qualities of good code. These are applicable irrespective of the technology involved.
You cannot predict the future requirement of a product\framework, but following these principle you can be better prepare to handle any future modification to the software
You might want to check out S#arp Lite which has many good examples of how to implement the things you want and can serve as a very good base on which to build something quickly.
None of the mentioned patterns are mutually exclusive. You should use the patterns that make sense based on what you are trying to accomplish, not attempt to shoehorn your application design into someone elses idea of how it should work. Sometimes trying to bend your scenario to fit a particular design pattern / practice is the worst thing you can do.
Want to make sure good unit test coverage / do TDD / ensure loose coupling? Then Dependency injection is your friend, as is the Unit of Work pattern. This is definitely a good idea for creating a maintainable, scalable application, but you can go too far with it for a small-scale application.
Need a centralized caching strategy for your data source? Then use the repository pattern. Or an ORM like NHibernate, which might do it for you.

Which web framework for someone who wants a job?

I want to learn a framework that promotes good programming practices and is respected by the programming community.
However, I also want a framework that I can use for a day job.
Which one would you recommend?
This question comes from my experience of learning the basics of Django because it was highly acclaimed by developers on Stack Overflow and Hacker News. However.. there's hardly any jobs in my area (NYC) that are asking for Django developers.
As a long-time ASP.NET guy, I've recently gone through a similar decision process to figure out what other web frameworks I should try. Here's what I learned so far which may apply to your case too:
framework/platofrm choices (and hence job opportunities) are highly regional-- the Bay Area job market differs alot from what you'll find in NYC, Chicago, Montreal, or London. Look at local job listings (craigslist and indeed are good places to start) to get a good sense at what's in demand.
similarly, usage varies alot based on the size and type of company. if you want to get a job in a large company, Spring MVC and ASP.NET MVC may be your best bets. In small companies, DJango and (especially) Rails seem to be on the rise.
usage also sometimes varies by industry. for example, many HR apps seem be to .NET based, while financial/banking apps seem to favor Java. if you want to work in a particular industry, check out what up-and-coming companies in that industry are using.
when investing your scarce time in learning something new, favor technologies which are on the upswing of the adoption curve (e.g. Rails) rather than frameworks with wider adoption which may not be growing as fast. Also be wary of very early or niche frameworks which may not ever gain wide adoption.
the one common thread between most (or almost all) frameworks gaining in popularity is that they're MVC frameworks and rely heavily on a solid understanding of REST. Learning those concepts in depth is a good idea.
before deciding to invest a lot of time in one framework, gain a basic understanding of several of them, so you can get a reasonable sense of what you like and don't about each-- and so if you end up applying for a job using a framework you haven't learned, at least you'll be able to talk intelligently about it.
If you focus on what you enjoy, you'll be more motivated to learn it. For example, personally I found Rails (regarless of employment opportunities) more interesting than Spring or Django, so I decided to focus on Rails first. Others may have different impressions-- follow your programmer instincts. That said, there are often few jobs using technologies you find fascinating, so try to strike the right balance: technology you like that many companies are actually hiring people to use!
once you answer the basic "what framework" question, there are many more questions lurking, including picking a javascript framework, validation framework, an ORM, etc. Don't worry too much about those choices yet-- when starting, just pick the default implementation for your framework. But as you get more advanced, the same argument about frameworks also hold for those other things-- e.g. it's useful to know a few ORMs.
Personally, I decided on this approach:
continue building stuff in what I knew best (ASP.NET) but transition all work to ASP.NET MVC, where I can better understand MVC and REST concepts which apply cross-platform
learn JQuery (again, platform neutral)
blow off the ORM choice alltogether for now-- too many other things to worry about
build a few projects in Rails, which is the framework I see used most in the newer SF-Bay-Area startups I've been looking at
learn the basics (e.g. read a book or two, try a few samples) about Python/Django, Java/Spring, and Groovy/Grails.
I've encountered real projects at cool, small companies using Django, Ruby on Rails and (eiuw!) even Zope. .NET is for teletubbies - I've only ever heard of it being used by big corporations that don't know better.
I would say that knowing two or three is better than knowing one that is widely used because you will gain a better understanding of how it works as a concept. For instance if you've only used Java, there is something probably missing in your understanding of OOP, because you're pigeon-holed into thinking about it in one way. If you already know Django though you Spring would probably be a good compliment to that.
i'd probably say ASP.NET MVC. I always see lots of .NET jobs around and this seems to be a solid framework which i think in fact powers all the stackoverflow family. As a PHP developer i must also make a mention of Zend Framework which is used by a number of big sites including bbc.co.uk and is now frequently mentioned in advertisements for PHP jobs.
I want to learn a framework that promotes good programming practices and is respected by the programming community. However, I also want a framework that I can use for a day job.
Sorry to be the bearer of bad news here, but those two desires tend to conflict. IMHO most business managers tend to go for (ugly) rapid development on top of CRMs or other higher-level 3rd party codebases. Building elegant websites from the ground up mostly happens in startups, or true web companies where the website is the sole product. There are not that many of those companies; and many of those that seem to fit are actually a mess on the inside, i.e. due to time pressure, messy legacy code and many other reasons you often don't get to write according to "good programming practices" anyway.
I agree with Kaleb Brasee that Java and .NET are the two main platforms when job availability is a priority.
Every job market is unique, so look at job openings in your area, or call a handful of recruiters and ask what they see a need for / could easily place you in a junior position for. What I'm seeing is that Microsoft Sharepoint is in demand, and a few other regional CMS'es are in demand (in Denmark I see Sitecore regularly).
I think ASP.NET MVC 2.0 together with MVC Areas and ASP.NET Dynamic Data will have a good story, a good solution, for many of those bosses who want rapid development. And I think the resulting code could be quite okay, or at least not bad compared to many of the "CMS beaten into something else" sites that exist. But this is a brand new thing for the .NET platform, and it will need to be sold to the decision makers first...
Bottom line: If you want job security first and foremost, then look at large CMS's like Sharepoint, and work on other technologies in your spare time. Optionally you could take a job at a startup / a web company later; but look before you leap.
Have you tried Spring MVC? Many companies do use Java for web-apps (or .NET) and web service based applications.
Since you mentioned Ruby on Rails, you might want to learn Ruby on Rails. It has got some good programming practices in it and a very well thought architecture. The Ruby community itself have also (in my personal opinion) created very innovative frameworks and highly favor testing and quality. You can see this by the innovative testing framework like Cucumber, webrat, shoulda, coulda, rspec, test/spec. Many startups also uses Rails as their platform, so it should be easier for you to get a job. You can start looking at Working With Rails and 37signals job board. So there is a good ecosystem inside Rails and Ruby community.
But the downside of Rails compare to Django is mainly there are too much magic (less explicit) and the docs is not as good as Django. If you want to get a Django job, try looking at several news site because Django grew up from a newspaper site so it is adopted alot in news based sites.
I would recommend ASP.NET MVC, Ruby on Rails, or Python/Django, they all seem to be popular and successful, and based on the MVC paradigm which is definitely the right tool for the job when it comes to the web.
.NET and Java are by far the 2 largest platforms used by employers, and hence the most in-demand when searching for a job. Java has a few popular frameworks, with JSF, Spring MVC and Struts all seeming to be about equal in demand. I don't use .NET, but from what I've seen, ASP.NET and ASP.NET MVC are the major ones.
I would say that most of the frameworks mentioned here promotes good practices. But that doesn't neccesarily mean that the companies using those frameworks are actually following those good practices! In fact most probably aren't. So don't expect too much.
You see, places like Stack Overflow, Hacker News etc. are a great way to connect with people who really care about their craft. Sadly this is a minority. There are millions of programmers in the world. Most of them suck. The code they write sucks. They don't care. They are not interested in improving their skills. They just want to learn the bare minimum required to collect their paycheck, go home, feed the dog, spend some time with the family, watch some TV, go too bed and do it all over again the next day.
Okay that was a bit harsh :) What I'm getting at is that you are probably better off asking this question to some of the managers at the companies where you would like to work. My guess is that most of them will answer .NET or Java. If you are up for a laugh ask them why they chose that particular technology over something else, and see how many buzzwords they throw at you ;)

Best practices for refactoring classic ASP?

I've got to do some significant development in a large, old, spaghetti-ridden ASP system. I've been away from ASP for a long time, focusing my energies on Rails development.
One basic step I've taken is to refactor pages into subs and functions with meaningful names, so that at least it's easy to understand # the top of the file what's generally going on.
Is there a worthwhile MVC framework for ASP? Or a best practice at how to at least get business logic out of the views? (I remember doing a lot of includes back in the day -- is that still the way to do it?)
I'd love to get some unit testing going for business logic too, but maybe I'm asking too much?
Update:
There are over 200 ASP scripts in the project, some thousands of lines long ;) UGH!
We may opt for the "big rewrite" but until then, when I'm in changing a page, I want to spend a little extra time cleaning up the spaghetti.
Assumptions
The documentation for the Classic ASP system is rather light.
Management is not looking for a rewrite.
Since you have been doing ruby on rails, your (VB/C#) ASP.NET is passable at best.
My experience
I too inherited a classic ASP system that was slapped together willy-nilly by ex excel-vba types. There was a lot of this stuff <font size=3>crap</font> (and sometimes missing closing tags; Argggh!). Over the course of 2.5 years I added a security system, a common library, CSS+XHTML and was able to coerce the thing to validate xhtml1.1 (sans proper mime type, unfortunately) and built a fairly robust and ajaxy reporting system that's being used daily by 80 users.
I used jEdit, with cTags (as mentioned by jamting above), and a bunch of other plugins.
My Advice
Try to create a master include file from which to import all the stuff that's commonly used. Stuff like login/logout, database access, web services, javascript libs, etc.
Do use classes. They are ultra-primitive (no inheritance) but as jamting said, they can be convenient.
Indent the scripts properly.
Comment
Write an external architecture document. I personally use LyX, because it's brain-dead to produce a nicely formatted pdf, but you can use whatever you like. If you use a wiki, get the graphviz add-in installed and use it. It's super easy to make quick diagrams that can be easily modified.
Since I have no idea how substantial the enhancements need to be, I suggest having a good high-level to mid-level architecture document will be quite useful in planning the enhancements.
On the business logic unit tests, the only thing I found that works is setting up an xml-rpc listener in asp that imports the main library and exposes the functions (not subroutines though) in any of the main library's sub-includes, and then build, separately, a unit test system in a language with better support for the stuff that calls the ASP functions through xml-rpc. I use python, but I think Ruby should do the trick. (Does that make sense?). The cool thing is that the person writing the unit-test part of the software does not need to even look at the ASP code, as long as they have decent descriptions of the functions to call, so they can be someone beside you.
There is a project called aspunit at sourceforge but the last release was in 2004 and it's marked as inactive. Never used it but it's pure vbscript. A cursory look at the code tells me it looks like the authors knew what they were doing.
Finally, if you need help, I have some availability to do contract telecommuting work (maybe 8 hours/week max). Follow the link trail for contact info.
Good luck! HTH.
Since a complete rewrite of a working system can be very dangerous i can only give you a small tip: Set up exuberant tags, ctags, on your project. This way you can jump to the definition of a function and sub easy, which i think helps a lot.
On separating logic from "views". VBScript supports som kind of OO with classes. I tend to write classes which do the logic which I include on the asp-page which acts as a "view". Then i hook together the view with the class like Username: <%= MyAccount.UserName %>. The MyAccount class can also have methods like: MyAccount.Login() and so on.
Kind of primitive, but at least you can capsulate some code and hide it from the HTML.
My advice would be to carry on refactoring, classic ASP supports classes, so you should be able to move all everything but the display code into included ASP files which just contain classes.
See this article of details of moving from old fashioned asp towards ASP.NET
Refactoring ASP
Regarding a future direction, I wouldn't aim for ASP.NET web forms, instead I'd go for Microsoft's new MVC framework an add-on to of ASP.NET) It will be much simpler migrating to this from classic ASP.
I use ASPUnit for unit testing some of our classic ASP and find it to be helpful. It may be old, but so is ASP. It's simple, but it does work and you can customize or extend it if necessary.
I've also found Working Effectively with Legacy Code by Michael Feathers to be a helpful guide for finding ways to get some of that old code under test.
Include files can help as long as you keep it simple. At one point I tried creating an include for each class and that didn't work out too well. I like having a couple main includes with common business logic, and for complicated pages sometimes an include with logic for each of those pages. I suppose you could do MVC with a similar setup.
Is there any chance you could move from ASP to ASP.Net? Or are you looking at keeping it in classic ASP, but just cleaning it up. If at all possible, I would recommend moving as much as possible moving to .Net. It looks like you may be rewriting/reorganizing a lot of code anyway, so moving to .Net may not be a lot of extra effort.
Presumably someone else wrote most or all of the system that you're now maintaining. Look for the usual bad habits (repeated code, variables that are too widely scoped, nested if statements, etc.), and refactor as you would any other language. Keep an eye out for recurring things in the same file or different files and abstract them into functions.
If the code was written/maintained by various people, there might be some issues with inconsistent coding style. I find that bringing the code back into line makes it easier to see things that can be refactored.
"Thousands of lines long" makes me suspicious that there may also be situations where loosely-related things are being displayed on the same page. There again, you want to abstract them into separate subroutines.
Eventually you want to be writing objects to help encapsulate stuff like database connectivity, but it will be a while before you get there.
This is very old, but couldn't resist adding my two cents. If you must rewrite, and must continue to use classic ASP:
use JScript! much more powerful, you get inheritance, and there some good side benefits like using the same methods for server-side validation as you use for client-side
you can absolutely do MVC - I wrote an MVC framework, and it was not that many lines of code
you can also generate your model classes automatically with a bit of work. I have some code for this that worked quite well
make sure you are doing parameterized queries, and always returning disconnected recordsets
Software Development Project Management practices indicates that softwares like this are requiring to retire.
I know how hard it is to do the right thing, even more when the responsible manager knows sht and is scared of everything other than the wost way possible.
But still. It's necessary to start working on the development of a new software. It's simply impossible to maintain this one forever, and the loger they wait for retiring it the worse.
If you don't have proper specification/requirements documentation (I think no asp software in the world does, given the noobatry hability of those coders), you'll need both a group of users that know the software features and a manager to be responsible for validating the requirements. You'll need to review every feature and document its requirements.
During that process you'll go learning more about the software and its business. Once you have enough info, you can start developing a new one.

Resources