Looking at all the answers to the question: "Are the days of stored procedures numbered".
It appears as if some people are jumping in with strong opions based on what I perceive to be a narrow range of software development experiences.
I would like to re-submit the question, but first I need help putting together a list of qualifications required before answering.
Here is one version of the requirements to get you started.
Worked on one large database like
for ever (10+ years).
Starting some time in the 1990s went
to work for a startup with the goal
of creating a world beating ERP
system.
Wrote code in either VB6 or Cobol.
Released new versions of said system
once every 2+ years.
Came in as new developer of said
system and had good the fortune to
be allowed spelunk through thousands of lines of stored procedures so that some major changes to business logic could be implemented.
Worked deep in the bowels of a
Financial institution adding web
based front ends to various parts of
large mostly ossified database.
Wrote more SQL code than many .NET
developers have written C#.
Etc.
Have you seen the similar question Stored Procedures - End of days which has answers from some people I would consider to be some of the most experienced on SO in this area: Joel Coehoorn, Stephen A Lowe, and edits by Jeff Attwood (the OP refers to a podcast by Jeff where he says the days of SP are over).
Related
Pardon if this question is not appropriate. It is kind of specific and I am not asking for actual code but moreso guidance on whether or not this task is worth undertaking. If this is not the place, please close the question and kindly point me in the correct direction.
Short background: I have always been interested in tinkering. I used to play with partitions and OS X scripts when I was younger, eventually reaching basic-level "general programming" aptitude before my father prohibited my computer usage. I am now going to law school and working at a law firm but I love development and I want to implement more tech innovation in the field.
Main point: At our firm, we have a busy season every year from mid march to the first week of april (immigration + H1B deadline). We receive a lot of documents and scanned files that need to be verified, organized, and checked.
I added (very) simple lines of code to our online platform to help in organization; basically, I attached tags to all incoming documents, and once they were verified, the code would organize them by tag (like "identification doc", "work experience doc" etc.). This would my life much easier every year, as I end up working 100+ hour weeks this season.
I want to take this many steps further with an algorithm that can check for signatures and data mismatches between documents and ultimately organize the documents so they are ready to print. Eventually, I would like to maybe even implement machine learning and a very basic neural network to automate the whole mind-numbing and painful process...
Actual Question(s): I just wanted to know the best way for me to proceed or get started. I know a decent amount of python and java, and we have an online platform already with the documents. What other resources would you recommend in terms of books, videos, or even classes? Is there a name for this kind of basic categorization? Can I build something like this through my own effort without an advanced degree?
Stupid and over-dramatic epilogue: Truth be told, a part of me feels like I wasted my life thus far by not pursuing what I knew I loved at the age of 12. This is my way of making amends I guess, and if I can do this then maybe I can keep doing it in law and beyond...
You don't give many specifics about the task but if you have a finite number of forms in digital form as images, then this seems very possible.
I have personally used OpenCV with Python a lot and more complex machine learning tasks have become increasingly simple in the past 10 years.
Take for example object detection (e.g. 1, 2) to check whether there is anything in a signature field or try extracting the date from an image (e.g 1, 2).
I would suggest you start with the simplest thing that would improve your work. A small and easy task will let you build up your knowledge on how to do things.
Similar questions has been asked more than 10 years ago. I have tried to search the web for the last two days, and read numerous pages all over the place. Unfortunately, I am still not finding a good answer.
This question should not have been tied to any particular database, but my particular server is mysql 5.6.12. I tried mysql-workbench, and it could not generate the links between the entities even though my database has clear foreign key constraints defined.
In the past I have used a program from Sybase, and it worked fine and need to pay.
I have seen one from Oracle, not sure this one is good or not. It might be a time for people to share experiences here.
I have seen a pattern that usually people who has no problems won't visit this site. Just to make a simple answer to my own question.
Mysql-workbench can display relationships, but if you have one single error in the schema, then not a single one will show up. I found DbSchema a better alternative, and it is not expensive.
I've seen interesting projects on sourceforge:
schemacrawler --> http://schemacrawler.sourceforge.net/
schemaspy --> http://schemaspy.sourceforge.net/
I've never used them but they looks good with maven integration, programatically and command line modes, and many other stuff.
There is Ruby on Rails (1.8, 2.3.2) project. First version of project was made by some organisation. I will implement next versions of this project without any help from this organisation. I will be able to talk with developers from previous development team during meeting (1-3 hours).
Project statistics: ~10k LOC, 1.0/0.6 code to test ratio, rspec
What questions about project can you recommend to ask?
First review the entire project and to figure out as much as possible so you have context and can actually understand what they tell you.
Ask
If you can record the conversation
For an architectural overview
Why they made certain architectural decisions over another
A complete list of dependencies (if you can't figure that out on your own)
What the biggest problems are
Which parts of the projects are always / never being fixed
What the Achilles' Heel of the project is
What will cause the biggest headaches
What security issues are there and what the constraint is to fixing it
What would you do next if you were me?
What you should know that you didn't ask (most important question)
Also, don't be judgemental, you want them to reveal any problems they know about. There are probably tons of things wrong with the app that they are embarassed about, which you need to know sooner rather than later. They're not going to open up to you if they don't trust you.
I would ask for a code walkthrough. Not line-by-line, but more for the overall structure of the project, relationships between individual modules, etc.
Find out the Why's. How is easy enough to see in the codebase, but the why is sometimes impossible to figure out, and will bite you in the ass.
For instance...
Which parts of the application were the biggest performance issues? Which of those issues were resolved? Which are still issues?
Why did you opt for pattern / tool / library x? What other things did you consider? Why?
This will hopefully. (Hit some wood.) Help keep you from having to trudge through the same learning curve and mistakes that the first team had to deal with, and should give you good insight into where the first team actually made a poor choice, instead of making a choice based on factors you have not accounted for yet.
Ask if the new features will cause any major changes to the existing code (architecturally) and what the implication of that will be with other dependent parts of the application.
Also get their emails, as you will have more questions.
One of the most important things, in my opinion, is to get as much technical documentation as you can prior to meeting with them. You should try to go into the meeting as informed as possible, so that you not only know what areas you need to focus on the most, but also to have a preexisting knowledge of how some of the subsystems relate to each other.
Also, do not be afraid to ask what they would have done differently, if given the chance. Some of the best ideas come too late in the development process to be implemented - be it from library availability, change in requirements, change in team, etc.
Bring cookies (or pizza, beer, or wine as appropriate); you will want them to have positive memories of you for when you call with questions.
Edit: to put my answer in the form of a question: "May I offer you a home-baked cookie?"
Perhaps you have done this already, but I would make sure you can:
Checkout the latest version
Run all the migrations
Run all the tests
Deploy (even if to a staging server)
Run the application locally
Before you go to the meeting, so you can make sure you can by the time it is over.
Other things that might be useful
data model
UI wireframes
bug tracker data / issue tracker data
who are the customers / people representing customers
development environment configuration
source control locations, etc.
explanation of special configuration settings
Wow! All great answers, right down to the cookies.
My contribution assumes that this is your one and only chance to access the old dev team, therefore you need to kick it up a notch:
Agenda. Split the meeting into several parts, for instance:
A quick (15 min) introduction and arch overview
One on one with team members.
Design review as a group, etc.
Positive Energy. Especially if the relationship is inherently difficult, keep a positive focus by postulating: what improvements would you put into the next version - (rewrite is not an option, right Joel) - capture every nuance, and drill down past their comfort level only nearer the end.
Facilitator. Use a trained design meeting facilitator. They can help prep for the meeting, conduct pre-meeting interviews, design the agenda. During the meeting they can drive the intensity, and keep the focus. They can also suggest forms of capturing what can be a fair amount of information.
Also, I would try to id all design artifacts beyond the code, if any, and come to an understanding of how accurate it is. This may include doing design reviews of key elements of these documents vis a vis the as-built system.
Don
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I recently joined a new company with a large existing codebase. Most of my experience has been in developing small-medium sized apps which I was involved with since the beginning.
Does anyone have any useful tools or advice on how to begin working on a large app?
Do I start at the models and move to controllers? Are there scripts to visually map out model relationships? Should I jump in, start working on it and learn the apps structure as I go?
Railroad should help you understand the big picture. It generates diagrams for your models and controllers.
I found unit tests to be the most efficient, effective and powerful tool. So, before making any change, make sure your application has a minimum LOC so that you won't break any existing feature working on.
You should care about unit tests (of course I'm talking about unit/functional/integrational tests) because:
they ensure you won't break any existing feature
they describe the code so that you won't need tons of comments everywhere to understand why that code fragment acts in that way
having test you'll spent less time debugging and more time coding
When you have tests, you can start refactoring your app.
Here's a list of some tools I usually work with:
Rack::Bug
New Relic
You might want to view some of the wonderful Gregg's videos about Scaling Rails to get more powerful tools.
Also, don't forget to immediately start tracking how your application is performing and whether it is raising exceptions. You can use one of the following tools
Hoptoad
Exceptional
If you need to fix some bug, don't forget to reproduce the issue with a test first, then fix the bug.
Not specific on Rails, but I would start reading the requirements and architecture documentation. After that get familiar with the domain by sketching the models and their relationship on a big sheet of paper.
Then move on to the controllers (maybe look at the routes first).
The views should not contain that much information, I guess you can pretty much skip them.
If you still need to know more, the log of the versioning system (given they use one) is also a good place to get to know how the project evolved.
When I've been in this situation, I try one of three things:
Read all the code top to bottom. This lets you see what code is working, and you can report progress easily (I read through all the view code this week). This means you spend time on things that may not be helpful (unused code) but you get a taste of everything that is there. This is very boring.
Start at the beginning and go to the end. From the login page or splash screen, start looking at that code, then the next page, then the next page. Look at the view, controller, and database code. This takes some time, but it gives you the context for why you need that code or database table. And it allows you do see most often the ones that get used in the most places. This is more interesting.
Start fixing bugs. This has the benefit of showing progress on your new project (happy boss) taking work from other people (happy co-workers) and learning at the same time (happy developer). It provides the context of number 2, and you can skip rarely used code from number 1. This is the most interesting way for me.
Also, keep track of what you've learned. Get a cheap spiral-bound notebook and write down an outline of what you've learned. Imagine yourself giving a talk on the code you're learning about or bug you're fixing. Take enough notes to give that talk, and spice it up with a factoid or two to make it interesting. I give my notebooks dignity and purpose by calling them "Engineering Notebooks", put a title on the front (my name, company, date), and bringing them to every meeting. It looks very professional compared to the guys who don't show up with paper to take notes. For this, don't use a wiki. It can't be relied upon, and I spend a week playing with the wiki instead of learning.
As mentioned above, being the new guy is a good chance to do the things nobody ever got around to like unit tests, documenting processes, or automating running tests. Document the process to set up a new box and get all the software installed to be productive. Take an old box under someone's desk and put a continuous integration install on it, and have it email you when the tests fail. You can forward that email whenever someone else checks in code that breaks the tests. Start writing tests to figure out how things work together, esp. if there aren't any/very many tests.
Try to ask lots of questions in one-on-one situations. This shows you're engaged and learning, and it helps you find the experts in the different parts of the app. On a big project you may need to go to one person for one topic and a different person for other topics. It also helps identify who thinks they know more than they really do or who talks more than you really need.
In FogBugz 6, how do I represent the concepts of a "feature" versus a "task"? As defined by Joel Spolsky, the owner of Fog Creek Software (which makes FogBugz), a feature is essentially a user-visible capability. To estimate the time to implement a feature, the developer should break the implementation into short tasks (2 days max) to ensure they think about each step.
FogBugz has only cases. I can't tell whether they're supposed to correspond to features or tasks. Some FogBugz documentation indicates that each case is a task, which is fine except there is no way to group all the tasks for a given feature together. This is especially odd given that, before FogBugz 6, Joel advocated using a spreadsheet with that grouped all the tasks for each feature. But his own software doesn't appear to meaningfully support that grouping.
I realize that the Joel article I reference includes a disclaimer pointing to a later article. However, the later article does not settle this issue, in fact it doesn't discuss features versus tasks at all, which is surprising given how well Joel advocates for those concepts in the first article.
For FogBugz 6.0 and earlier:
Make a case for each work item (task). FogBugz calls them "Features," only to distinguish them from bugs, but you do want one case for each task.
The best way to group a bunch of tasks is to make a Release (Fix-For) and assign all of the tasks to that release.
Responding to AviD's comment/question to Joel:
So, if you have 10 new features coming
in the next version, with each feature
needing 5 tasks to implement, you
recommend creating 10 releases? And
how do I define that these are the
features/"releases" that are to be
included in the upcoming release?
Here is how we dealt with this specific problem in our development process:
First, we made a regular release schedule: monthly internal releases and quarterly external releases. This schedule never changes but task assignment / feature completion does. This is hugely important in terms of simplifying our inter-human communication: don't try to argue with the calendar.
Major features ("10 new features" in your example) are turned into cases (e.g., case 101 to case 110).
Each task that is a sub-component of a major feature also gets created as a sub-case with a description of what makes this chunk of work an important part of the larger picture. Previously, in Fogbugz 6, we used the "See also" feature by allowing it to search the text for us ("This is a sub-component of case 101" for example). This was effectively the same thing but less aesthetic.
Now that we've broken down the work to its finest level of usefulness, we bring the actual developers into the discussion. Each task and major feature is individually assigned to a particular developer.
The developer determines when they can get their assigned work done by picking the appropriate internal release date that they think they can commit to.
At this point, we have a rough sketch of what will get done for each release. Further refinements continue as the working people actually estimate the hours that they'll need to do the work, enabling evidence-based scheduling, etc.
For AviD's question, though, he would have the release-assignment problem solved by step 5 above.
However, I think point 6 is the most interesting as that's where you really get a solid schedule. For example, if developers are having trouble estimating a larger task, they break it down into sub-cases even further. Notice how my assessment of "finest level of usefulness" can differ (perhaps greatly) from the person who really needs to get the work done.
This is also a time when a developer can reach out to someone else and say "I can do most of this but it would really help if person X could help me with this little piece Y." This is actually where I get most of my development tasking: I personally sit in multiple places during this process, from large-scale planning meetings to little fiddly tasks that no-one else has time to do.
PS: Making it a personal goal to get this answer rated higher than Joel's.... ;-)
PPS: My original response is now overcome by events since Fogbugz 7 has lovely sub-tasks. Program managers love those reports.
You may have better luck asking your questions in the FogBugz Discussion Forum
We use a combination of projects in order to accomplish your grouping goals. We also commonly setup a project "parking" Wiki where links to development cases, technical documentation, systems requirements, user documentation, external links to resources etc. can all be placed. It provides a good "one-stop-shop" for everything related to that project.
As part of that Wiki, we would then setup two specific projects. One in relation to the large overall goals/outlines similar to what might correspond to your Project Management charts/whatnot. One in relation to the development tasks of each feature as they are broken down into the smaller and more manageable chunks. You can then, as was mentioned use case linking to both reference the "master" cases in the other project as well as reference the project Wiki itself so that you can quickly and easily get back to all of your project related information which is conveniently in one spot.
You can accomplish a pile of different organizational structures using FogBugz, you just have to approach things a little differently sometimes in order to hit each and every situation.
Hope that helps.
haha, that article has a disclaimer, but I understand what you are saying.
We use Fogbugz and the only 'Feature' that I am aware of is under category and I don't think you can associated it with sub-tasks.
You can type in 'Case N' is the feature for this task if you just wanted to reference it in the case text.
That kind of stuff sound like is lies more in the project management domain instead of software used to track bugs.
thats a good question, i have asked that myself, too..
we currently test-drive fogbugz for 45 days in a group of 5 developers, and we currently create a "release" for major features. in fact we do not release it, but multiple releases together when something is ready.
there should definately be some sort of advanced task grouping in fogbugz.