I'm releasing my first large app soon, and I've been thinking of writing a small app to manage the databases for my own app with master privileges, to clean up data and the like. This will just be a dev project shared with a few people within the company.
Is this something that people do?
Has anyone had any experience with this?
It feels like a good idea, but I'd also really love feedback.
I'm assuming when you say database you are not referring to coredata specifically but rather some remote database. There is certainly nothing wrong with you creating tools to help you do your job.
I think really this answer is going to come down to time management. If your database has complicated relationships,etc that a simple database editor is not going to be able to maintain or will be highly prone to human error mistakes, I'd say go for it. But, if you are just making trivial changes it might be a more efficient use of time to use a existing solution.
Related
What is the best way to get a ballpark idea of what kind of resources an application needs if it is running on Heroku (how many dynos and what db plan you should be running)?
My non-technical friend had his site built in rails and it is currently being hosted on Linode by the shop that built the app. They aren't providing the support he was hoping for, and is interested in having me move it over to Heroku, but we are trying to get an idea of how much more that is going to cost him. I tried contacting the dev shop directly to find out what the current instance is running on, but they aren't getting back to me.
I have access to their analytics, and the current repo, so from this can I glean what kind of initial setup we are probably going to need, and if so what should I be looking for? I realize that this is very situation dependent and not an easy question to answer, but any insight would be greatly appreciated.
dyno-blitzer will help you find out how many dynos you'll need to support 1,000 users, but there are a lot of questions you need to answer in order to try and ballpark what you'll need with Heroku:
How big is the database right now? How fast is it growing?
How many users do you need to support? If you only need 5, the numbers dyno-blitzer gives you probably won't be anywhere near what you need.
How much delayed processing work is being done?
What sort of things does the app need that have to be provided through add-ons with Heroku?
Once you know these, you should be able to get a rough ballpark, but it won't be laser accurate - this will heavily depend on the app and exactly what it's doing.
I am currently working on a large project. It is already lunched but the problem is users are already using the app and having very erroneous results. They don't complain much but each and every day I am receiving emails on errors and improvements.
I would appreciate some help in deciding weather to debug the app or just create another one. I'm working on a rails 2.5 app with old plugins.
And I'm the only one doing all this. T.T
Its really tempting on a project like you describe to toss out old code and rewrite, but its almost always a mistake (see http://en.wikipedia.org/wiki/Second-system_effect. links at end regarding rewrites are invaluable, especially http://chadfowler.com/2006/12/27/the-big-rewrite).
I'm guessing you don't have any real test suite in place, or it'd be easier to track down problems and you'd probably have a smaller project as well-tested projects tend to be well-factored (though not always). That's going to make it very difficult to reimplement and have any assurances that you've replicated all the functionality and that any dependencies work well with the "new and improved" code.
And if your users are getting erroneous results, I'd venture to say you don't actually know what the problem is, so a rewrite isn't going to fix that.
When I take over a project like this, step one is to write a suite of characterization tests, documenting the way I think the system is supposed to work AT THE PRESENT TIME. Often in doing so you'll uncover a piece of functionality that doesn't make sense or is inconsistent with the rest of the system—that may well be where your problem is. Once we're through that phase, we can start refactoring the ugly parts, cleaning up views, moving logic to someplace it belongs, removing dead code, etc. But those tests are really important to have if you want to keep the system functioning.
Finally, set reasonable expectations for yourself. Projects like this don't turn into messes overnight—you can't fix them overnight either.
Fujisan I am in the exact situation as you are. I think it is best to first debug the current rails app and then think about rebuilding it. If you are rebuilding the rails app it takes more time than expected. I started out like this:
Created a separate database and repository which has the same code and data that pulls from the live website. I use this website to work with the bugs and not on the live site.
First started off knowing what models are present and started with user model. It is best to use the rails console and dbconsole in this step if you havent used them before. It is easy to find the dependencies from the console with ruby.
Just tweaked around the models and found the relations between different models. After the models looked at the controllers. Now once I have an idea of what is happening in the application I started with a sketch of the purpose of each model and what it is doing and how and why the dependencies are made.
FInally went with working on the bugs and suggestions. The major hurdle was with gems. When the website was created by seniors the gems were very active and now they are all orphaned. This makes it hard to get answer to any of the bugs that are persisting because of those gems.
Railscasts are your best friends!
I hope that helped. Good luck!
I am looking to start developing a relatively simple web application that will pull data from various sources and normalizing it. A user can also enter the data directly into the site. I anticipate hitting scale, if successful. Is it worth putting in the time now to use scalable or distributed technologies or just start with a LAMP stack? Framework or not? Any thoughts, suggestions, or comments would help.
Disregard my vague description of the idea, I'd love to share once I get further along.
Later. I can't remember who said it (might have been SO's Jeff Atwood) but it rings true: your first problem is getting other people to care about your work. Worry about scale when they do.
Definitely go with a well structured framework for your own sanity though. Even if it doesn't end up with thousands of users, you'll want to add features as time goes on. Maintaining an expanding codebase without good structure quickly becomes fairly horrible (been there, done that, lost the client).
btw, if you're tempted to write your own framework, be aware that it is a lot of work. My company has an in-house one we're quite proud of, but it's taken 3-4 years to mature.
Is it worth putting in the time now to use scalable or distributed technologies or just start with a LAMP stack?
A LAMP stack is scalable. Apache provides many, many alternatives.
Framework or not?
Always use the highest-powered framework you can find. Write as little code as possible. Get something in front of people as soon as you can.
Focus on what's important: Get something to work.
If you don't have something that works, scalability doesn't matter, does it?
Then read up on optimization. http://c2.com/cgi/wiki?RulesOfOptimization is very helpful.
Rule 1. Don't.
Rule 2. Don't yet.
Rule 3. Profile before Optimizing.
Until you have a working application, you don't know what -- specific -- thing limits your scalability.
Don't assume. Measure.
That means build something that people actually use. Scale comes later.
Absolutely do it later. Scaling pains is a good problem to have, it means people like your project enough to stress the hardware it's running on.
The last company I worked at started fairly small with PHP and the very very first versions of CakePHP that came out (when it was still in beta). Some of the code was dirty, the admin tool was a mess (code-wise), and sure it could have been done better from the start. But do you know what? They got it out the door before their competitors did, and became extremely successful.
When I came on board they were starting to hit the limits of their current potential scalability, and that is when they decided to start looking at CDN's, lighttpd caching techniques, and other ways to clean up the code and make things run smoother when under heavy load. I don't work for them anymore but it was a good experience in growing an architecture beyond what it was originally scoped at.
I can tell you right now if they had tried to do the scalability and optimizations before selling content and getting a website live - they would never have grown to the size they are now. The company is www.beatport.com if you're interested in who I'm talking about (To re-iterate, I'm not trying to advertise them as I am no longer affiliated with them, but it stands as a good case study and it's easier for people to understand what I'm talking about when they see their website).
Personally, after working with Ruby and Rails (and understanding the separation!) for a couple of years, and having experience with PHP at Beatport - I can confidently say that I never want to work with PHP code again =p
Funny to ask "scale now or later?" and label it "ruby on rails".
Actually, Ruby on Rails was created by David Heinemeier Hansson, who has a whole chapter in his book labeled "Scale later" :))
http://gettingreal.37signals.com/ch04_Scale_Later.php
I agree with the earlier respondents -- make it useful, make it work and get people motivated to use it first. I also agree that you should pick off-the shelf components (of which there are many) rather than roll your own, as much as possible. At the same time, make sure that you choose components for your infrastructure that you know to be scalable so that you can go there when you need to, without having to re-write major chunks of your application.
As the Product Manager for Berkeley DB, I've seen countess cases of developers who decided "Oh, we'll just write that to a flat file" or "I can write my own simple B-tree function" or "Database XYZ is 'good enough', I don't have to worry about concurrency or scalability until later". The problem with that approach is that a) you're re-inventing the wheel (and forgoing what others have learned the hard way already) and b) you're ignoring the fact that you'll have to deal with scalability at some point and going with a 'good enough' solution.
Good luck in your implementation.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I work as a lone developer in a very small company. My work is quite chaotic and I'm looking for ways to make it more organized.
One problem is that my projects have practically no management. Rarely anyone asks me what I'm doing, or if I have any problems. At some point there was talk about weekly status meetings, but that's some time ago. Seems that if I'd want something like that, I would have to arrange those myself.. Sometimes I'm a bit lost on what I should do next because I don't have tasks or a clear schedule defined.
From books and articles I have found many things that might be helpful. Like having a good coding standard (there exists only a rough style guide which is somewhat outdated in my opinion), code inspections, TDD, unit testing, bug database... But in a small company it seems there are no resources or time for anything that's not essential. The fact that I work in the embedded domain seems to make things only more complicated.
I feel there's also a custom of cutting corners and doing quick hacks on short notice. This leads to unfinished and unprofessional products and bugs waiting to emerge at a later date. I would imagine they are also a pain to maintain. So, I'm about to inherit a challenging code base, doing new development that requires learning a lot of new things and I guess trying to build a process for it all at the same time. It might be rewarding in the end, but as not too experienced I'm not sure if I can pull it off.
In a small shop like this the environment is far from optimal for programming. There's many other things needed to be done occasionally like customer support, answering the phone, signing parcels, hardware testing, assembly and whatever miscellaneous tasks might appear. So you get the idea about the resources. It's not all bad (sometimes it's enlightening to solve some customer problems) and I believe it can be improved, but it's the other things that I'm really concerned.
Is it possible to have a development process in a place like this?
Would it help to have some sort of management? What kind of?
Is it possible to make quality products with small resources?
How do I convince myself and others that the company which has worked successfully for decades needs to change? What would be essential?
Maybe there's someone working in a similar shop?
Use Source Control for EVERYTHING
Develop specifications and get signoff before starting - there will be resistance, but explain it's for their own good.
Unit tests! It hurts because you just want to get it done, but this will save you in the long run.
Use bug tracking - Bugzilla or FogBugz if you can afford it.
My advice is not to be extreme. From my experience, pure agile or pure traditional will not work. Before you use any process, know what it mean to solve.
I personally use a variation of Agile RUP. I do some upfront effort such as investigate the actual needs, do high-level design with possible extension. And ask customer to sign-off some major high-level requirements.
If you work in small group, detail design or specification may not worth. Of course, if there is some libraries that are shared by many, it will be worth the trouble.
Deciding what to invest in up-front depending on its risk (likelihood and effect).
Moreover, many SW best practice is really 'best' like version control, automatic testing (to me I only used it way to early detect regression as I do not believe in TDD as driven force of the development). I suggest you read 'Pragmatic Programmer' it presents many of those techines.
As to answer you questions:
(1). Is it possible to have a development process in a place like this?
Yes, but as I say, trailer it to fit your organization.
(2). Would it help to have some sort of management? What kind of?
Management helps but no control freak. Plan what to do when: integrate, resolve conflict, dead line of some mile stone. And roughly keep them on schedule (I particularly like Scrum's sprint).
(3). Is it possible to make quality products with small resources?
Definitely as soon as the size of the work, the time to develop and the size of the team is balance. If you definition of Quality is the same with me. To me, Quality means: solve the problem it set out to in an efficient and reliable fashion.
(4). How do I convince myself and others that the company which has worked successfully for decades needs to change? What would be essential?
Point out the problems. If there are none, why change? If you want to change, you should be able to identify the problem OR potential problems. Point out the problem.
Some big one are:
Without any process, it is harder for new recruited to blend in as they must learn from observing other how to deal with things.
Without process, it is harder to work in stress.
Without schedule, it is hard to determine the progress.
Without automatic testing, it will takes more time to identify problems and regression.
Without version control, it will be harder to roll-back mistake and separation of work to each team members will be mess.
Just my though.
You need to work with the owner and setup short medium and long term goals. You will want to let them know progress even if only through email.
You will need to enforce some order on your workday or you will never get anything done (those long term goals).
Divide your day up into chunks when you can code, when you are working on hacks to keep it togther, when answering emails etc.
Definitely setup a bug tracker. This can help keep your email clean. You can even setup an email address to forward bugs to be categorized later. This is good because the bug reporters will eventually tire of the bug tracker and want to just email you the bugs anyway.
edit
And as lod3n said, source control, but you are using that already right???!!?!
Been there, done that.
The book Planning Extreme Programming helped a lot. I used 3x5 cards stuck on a wall. This kept my boss informed of my progress, helped with estimates and planning, and kept me on track. The Planning Game gives good ammo if your boss's expectations are unrealistic.
Unit testing, as others have stated, helps even if you're a sole developer. I find the TDD style valuable.
lod3n is absolutely right about Source Control.
I've gone with XP-style iterations before; you may want to establish a backlog of items (even something simple like a spreadsheet or 3x5 cards) as well.
Avoid deathmarches. Stick to 40 hours in the sense of not working overtime 2 weeks in a row. Spend extra time outside of work learning new skills - not just technologies, but principles and best practices.
Have a bug tracking system, both for defects and new features. Don't rely on your memory.
Continuous integration and an automated build can help even a single developer.
Can't emphasize the recommendation for source control enough.
I've decided I don't need unit tests because I can have automated functional/integration tests instead: because with incremental development there's no need to test before integration.
as crazy as this sounds I use scrum just because I like the concepts of sprints, and backlogs. It makes it easier to set realistic goals. Of course the idea of scrum master and team is all you but if you are working on outside projects where it is possible that you may pick up an extra team member with your backlogs it will be easy to distribute work. Maybe I just like backlogs. With scrum you will need to get someone to be the product manager to talk about the features of the product. Version control is a must at probably should be implemented b4 even worrying about a software development process. I have worked in a company that started from 2 developers and went to 12 in a year. We started with no version control and low coding standards. The changes you will need to make will be gradual so don't worry about rushing to do a 180. Set a goal to change one thing a month and find supporters of your changes to make things go smooth.
As well as the recommedations of others I'd say that if you are tight on resources but have more say over how things get done you should make good use of off the shelf and open source products and libraries. This leverages the efforts of others, saving you time, ensures your code base doesn't become too esoteric and adds to your skillset so you don't end up being an expert in something that's useless everywhere else.
First, lets make a distinction between a development process and best practices. Best practices like source control, defect tracking, unit testing, etc. are an given.
Then there is the actual development processes. I would always recommend having a process, no matter small or large the team is. The trick is finding the right process. You have a process now - it is just an ad-hoc process that doesn't seem to be working out too well for for you. Rarely can you take a textbook development process and directly apply it. What you need to do is tailor the process to your companies needs and culture. Look at as many development paradigms as you can and try to find something that is a good fit and them start molding it to your needs. You may have to try and fail with a number of different processes. Perhaps the Personal Software Process will be a good starting process, maybe an agile process, a variant of RUP? You have a lot of options, start trying them out.
You are also going to have to work with the rest of your organization - they need to be a part of the process. You may be the lone developer, but a development process involves more than the developer, it involves ever person that has a say or impact in the product.
This may not be a specific answer, but my point is that you will need some kind of process. So start researching them and trying them out and molding them to your needs until you have something that works.
When writing code for a new system I don't want to introduce unnecessary complexity in the design that I might never have any need for. So I'm following YAGNI here, and rather refactoring as I see the need for more flexibility or as responsibilities becomes more clear. This allows me to move faster.
But there is a problem here with junior devs, in that they will not recognize when to refactor or where build out the design. They just stuff more code into the existing design.
So, what are the best ways to tackle this? Should I more often build a more future-proof design so when adding to it they have a good example to follow, even if we might never have to add anything? Or should I just go ahead with more code reviews, education, etc? Or both?
Have any of you had any experience with this type of problem? How did you solve it?
I would recommend code reviews or pair programming. It gives you the chance to educate your fellow developers and increase the overall quality.
Perhaps you begin by recognizing explicitly that part of your job is to help develop the junior devs. If you're not the boss, management should sign off on this. Management needs to recognize that your choices are to develop them now or clean up after them later, and you need management's backing for the time this will take.
Code reviews and pair programming are fine ideas. They are especially good because they are not "just for junior people"–I do both with one of my close colleagues; together we are nearly 100 years old and have more than 70 years of programming experience :-)
But there's a larger problem here: the programming methodology that enables you to be most effective (YAGNI + refactor) is not effective for your junior partners. My experience is that it takes people years to learn the benefits of YAGNI, so if you expect them just to learn your way of doing things, you are setting yourself up for disappointment.
I would encourage you to identify some methodology that you think is going to be useful with your junior partners. The particular methodology probably doesn't matter (heresy!); I've had success with composite/structured design, object-based design, algebraic specification (!), and extreme programming. But
Do pick something that has a name and some literature devoted to it, that your juniors can take pride in learning, and that is a skill they can carry to future projects.
In order to show that it is tasty, you may need to eat the dog food yourself. Pick something you can live with and be productive in.
Observe your juniors carefully and teach them a decision procedure they can use to identify when they should ask you for guidance.
Good luck!
There is a reason they are junior and you are senior.
The ability to realise when a change in design is needed is one of them.
I would carry on as you are but encourage them to come to you when things are getting difficult. You can then work with them to alter the design if needed, this will be easier for you than refactoring it and will help you pass on knowledge to your junior developers.
A very good way to show how far to build out a design is to be specify about what the design will do when built out, then write tests to cover the new functionality. When the tests pass, development is done.
You might realize along the way that you forgot to test for something. That's fine, and is useful feedback to help you specify better next time. Write the missing test(s), and only enough code to make them pass.
Then refactor. Knowing what to look for when refactoring takes a bit of practice. Start with
Is there duplication in the code we've just written that we can eliminate?
Is there duplication between what we've just written and pre-existing code?
Does the code we've just written concern itself with too many things? (I.e., should we break out collaborators?)
Repeat this a few dozen times, and it'll get easier.
Another way of looking at YAGNI is that any changes to code need to be justified.
Might it be helpful to require any commit needs an associated unit test (or BDD user story, choose your poison)? It helps to communicate your intent (you want people to think about why they are adding this feature) and you get regression tests for free.
Also gets the noobs to start thinking about modularity (usually needed to make your code testable) and will help a lot if you do need to refactor later on.
I'm all for code reviews and teaching, but I think futureproof design is also important. Maybe you could think of it in terms of you designing an API and the junior developers using the API. In this way you are the one who does the hard work that they would screw up (identifying duplicated code and eliminating it) while they do all the grunt work that isn't a productive use of your time.
Of course this has to be balanced with a need to develop your junior developers skills. Neither side of the equation can be neglected.
It may help to map out what work they will do and then verify it to help build their sense of judgement which is really what you are asking, to my mind. Pairing is one option but if you can't spare that much time then having a sort of "check point" to see how they are doing and preventing them from going down the wrong path.