Software Estimation

Steve McConnell

Mentioned 25

Covers software estimation techniques with information on how to successfully estimate scheduling, cost, and project activities.

More on Amazon.com

Mentioned in questions and answers.

I am having a difficult time with management asking for estimates on programming tasks that are using third-party controls that I have no prior experience with.

I definitely understand why they would want the estimates, but I feel as though any estimate I give will either be a) too short and make me look bad or b) too long and make me look bad.

What estimate or reply could I give management to get them off of my back so that I can continue doing my work!

I'll let you in on a secret. Even if you were an expert with that technology, your estimate is likely to be highly inaccurate. It is the nature of the beast when doing something that is an inherently R&D task. Unfortunately management often tries to apply a manufacturing model and demand accurate estimates. To illustrate my point, consider the difficulty in accurately estimating the following two efforts:

A) Manufacture another 11K umbrellas that are the exact same as the 2K you churned out last month. B) Design a new kind of umbrella and build the first one.

Software development is B, but they are asking for an estimate assuming A.

The best you can do is break the task down into the smallest pieces possible (no more than 1/2 day each) and then triple the final number you come up with.(Spolsky Method)

Alternately, Steve McConnell has a whole book (arguably several) on this aspect of software engineering. http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351

Someone has approached me to build a website for their small business. What is the protocol for laying out an offer for building a website in terms of pricing?

Do I charge for number of pages? If they want advanced features (Flash, Ajax, etc.) is it just price per hour of development? What if I have no idea how long it will take? Are there any good ways of looking at a similar website to help determine if something is going to be a 20 hour job or a 100 hour job.

Are there any standard templates that are used when determining scope and pricing for a website development job?

This question is exactly the same as "How much does it cost to build a house?" In both cases, the answer depends on what the customer wants. You can build a "house" for less than $100, especially if the person living in the house has four legs and a tail.

Before you talk about cost, get specifics on what that business wants. Customers will have very different ideas of what a "page" is than you do. ("What do you mean that you're charging me extra for a shopping cart that automatically queries and updates my inventory, that contacts my suppliers when I get low, and that sends out a thank-you note with a survey? That should all be in one page!")

If the business doesn't know what they want (and that will happen often!), then charge them for one day to mock up many different ideas. (You will need to emphasize what a mock up means. Many people think that once it displays a page, all the work is done.)

Once you know enough information to reach a goal -- even if it's not the final goal of the web site -- you can then estimate your time and effort.

I highly recommend the book Software Estimation: Demystifying the Black Art by Steve McConnell. Good luck.

Being relatively new to the software industry I have come across a question of deadline enforcement:

Back in the idyllic age of academia, the deadline was the end of the semester and the penalty was a well defined 'F' (or local equivalent). Out here in the real world we need to make code our current and future peers can work with, I face the situation where deadline comes, deadline goes, and the project is still not finished.

Now what? On one extreme we could fire everyone involved, on the other we could richly reward everyone involved.

  1. What actions have you seen applied as 'penalty' for missed deadline, and which of these eventually resulted in more-good-code?

  2. What project-management responses caused the project to fail outright,

  3. What responses restored working order and resulted in code that could be maintained afterward?

  4. What responses resulted in more-bad-code?

"Who does what by when" is a question that each project team member must provide a professional commitment/response to in any profession. As far as when a deadline is missed use that evidence to improve the estimating process and ask the individual to make a new commitment. This assumes that their was in fact a commitment made to the previous deadline. A great series on 'Who does what by when' is available at manager-tools.

Also, I would recommend that you distinguish between Estimates, Targets and Commitments. And manage the 'gap' or the risk between the estimate <--- gap ---> commitment. Look at Software Estimation: Demystifying the Black Art.

In his wonderful book about project management - "Deadline" - Tom DeMarco tells us a story, about project manager from a western-world is managing a project in some fictional post-communist eastern European wild country (wild is a really good term, because the citizens are a bit.. uncivilized).
One day PM discovers, that something went wrong, some part of his project dramatically missed the unrealistic schedule. Previous PM established penalty for missing deadline simply by hanging responsible person on a butcher’s hook, but as schedules were unrealistic, one man already missed deadline.
So the story tells us about a day, when western-style PM is presented with a responsible person, and he should send him to be hanged on butcher’s hook. PM, as most people do, is terrified of vision of sentencing someone to cruel death simply because some was never able to finish his project in time. And – by all means – hanging this poor man does not advance the project. Since this is a fiction novel about project management, and not about tortures, our hero cancels the penalty.
But there is some big issue behind this story about hanging someone: if you set a deadline, and establish some kind of penalty for missing this deadline, the day will come, you will probably have to actually punish someone. And will you do it? No matter what the punishment will be: hanging, bonus loss, firing, breaking the deal or some fee – you may have to punish someone. Will this penalty do some good to your project? You have to answer it by yourself.
So: do not establish a penalty for missing the deadline, you will not want to execute…

  • What are your experiences regarding project planning and creating hour estimates for new projects?

  • What is the approach you are using, and why has or has it not worked for you?

  • Are there any best practices to take into account?

I highly recommend the book "Software Estimation: Demystifying the Black Art" by Steve McConnell. It really covers this question well.

As a lead developer I often get handed specifications for a new project, and get asked how long it'll take to complete the programming side of the work involved, in terms of hours.

I was just wondering how other developers calculate these times accurately?

Thanks!

Oh and I hope this isn't considered as a argumentitive question, I'm just interested in finding the best technique!

There is no general technique. You will have to rely on your (and your developers') experience. You will have to take into account all the environment and development process variables as well. And even if cope with this, there is a big chance you will miss something out.

I do not see any point in estimating the programming times only. The development process is so interconnected, that estimation of one side of it along won't produce any valuable result. The whole thing should be estimated, including programming, testing, deploying, developing architecture, writing doc (tech docs and user manual), creating and managing tickets in an issue tracker, meetings, vacations, sick leaves (sometime it is batter to wait for the guy, then assigning task to another one), planning sessions, coffee breaks.

Here is an example: it takes only 3 minutes for the egg to become roasted once you drop it on the frying pen. But if you say that it takes 3 minutes to make a fried egg, you are wrong. You missed out:

  • taking the frying pen (do you have one ready? Do you have to go and by one? Do you have to wait in the queue for this frying pen?)
  • making the fire (do you have an oven? will you have to get logs to build a fireplace?)
  • getting the oil (have any? have to buy some?)
  • getting an egg
  • frying it
  • serving it to the plate (have any ready? clean? wash it? buy it? wait for the dishwasher to finish?)
  • cleaning up after cooking (you won't the dirty frying pen, will you?)

Here is a good starting book on project estimation:

http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351

It has a good description on several estimation techniques. It get get you up in couple of hours of reading.

I think I've become quite good at the basics of programming (for a variety of languages). I can write a *good** line of code. I can write a good method. I can write a good class. I can write a good group of classes. I can write good small or medium application.

I do not however know how to build a good large application. Particularly in the case where multiple technologies are involved and more are likely to become involved with time. Say a project with a large web front-end, a large server back-end that connects to some other integration back-end and finally a large and complex database. Oh, I've been involved in a few of these applications and I could build one I'm sure. I'm not so sure however that it could qualify as "good".

My question is thus for a reference to a book or other good source of reading where I could learn how to distribute and organize code and data for general large projects. For example, would I want to layer things very strictly or would I want to encapsulate it independent units instead. Would I want to try to keep most of the logic in the same pool, or should it just be distributed as it seems most logical when adding whatever feature I'm adding.

I've seen lots of general principals on these issues (e.g. No spaghetti code, meatball code...) and read a few excellent articles that discuss the matter but I've never encountered a source which would lead me to concrete practical knowledge. I realize the difficultly of the question and so I'd be happy to just hear about the readings that others have found to help them in their quest for such knowledge.

As always, thank you for your replies.

****Given the debated nature of the definition of "good" code, the term "good" in this context won't be defined (it means whatever you think it ought to mean).

Here's a book that we have used to guide our coding standards and methods:

alt textLarge-Scale C++ Software Design

The program I'm working on has been in development for almost 10 years since it was first drawn up on the back of the proverbial napkin. And the project is still going strong today. It hasn't been perfect, and there are still problems with circular dependencies and some class interfaces not being very clean, but most classes aren't like that, the program works and our users are happy.

I would also recommend, as has been done before by many others, Code Complete and Software Estimation by Steve McConnell. I particularly like his metaphor of "growing" software rather than constructing or building. This way of viewing software lends itself better to something that will have a long life-cycle.

Everywhere I've ever worked I consistently run into the same problem with management, they have no idea what is going on in the depths of an engineer's mind. Understandably, they're not developers so a technical explanation just won't do. How do the rest of you relay the complexity of a project to management after giving an honest estimate and they look at you like the lazy overpaid punk you only wish you could be?

I've been stung by this several times. The best advice I can give you is to try to separate estimates (how long you, in your professional opinion, believe a task will take) from schedules (when your boss wants a unit of work finished). Give your estimate as a three part figure (worst case, best case, and most likely at the moment), and do not revise it due to management pressure. Having conviction in your professional ethics is the best way to convince others that your job is worthwhile, and will negate the "lazy punk" stereotype.

The best resource I've read on this topic is Software Estimation by Steve McConnell. It's very good at explaining how to represent your profession to management.

Suppose you are a project manager. You can estimate an effort in days for specific task for specific developer. After performing estimation you obtain some min and max values.

After this you delegate a task to developer. Actually you also set up deadline.
Which estimation is better to use when set up deadline: min or max?

As I see min estimation can result in stress for developer, max estimation can result in using all the time which is allocated to developer even if task can be complete faster (so called Student syndrome). Which other pros and cons of two approaches?

EDIT:

Small clarification: I speak about setting up deadlines for subordinates when delegating the task, NOT for reporting to my boss.

EDIT:

To add one more clarification: I can keep in mind my real estimation, provide to boss slightly larger estimation, to subordinates - slightly smaller. And this questions touches the following thing: is it good idea to provide to developer underestimation to make him working harder?

Use neither min nor max but something in between.

Erring on the side of overestimation is better. It has much nicer cost behavior in the long term.

  • To overcome the stress due to underestimation, people may take shortcuts that are not beneficial in the long term. For example, taking extra technical debt thast has to be paid back eventually, and it comes back with an interest. The costs grow exponentially.

  • The extra cost from inefficiency due to student's syndrome behaves linearly.

Estimates and targets are different. You (or your managers and customers) set the targets you need to achieve. Estimates tell you how likely you are to meet those targets. Deadline is one sort of target. The deadline you choose depends on what kind of confidence level (risk of not meeting the deadline) you are willing to accept. P50 (0.5 probability of meeting the deadline) is commonplace. Sometimes you may want to schedule with P80 or some other confidence level. Note that the probability curve is a long-tailed one and the more confidence you want, the longer you will need to allocate time for the project.

Overall, I wouldn't spend too much time tracking individual tasks. With P50 targets half of them will be late in any case. What matters most is how the aggregate behaves. When composing individual tasks estimates into an aggregate, neither min or max is sensible. It's extremely unlikely that either all tasks complete with minimum time (most likely something like P10 time) or maximum time (e.g. P90 time): for n P10/P90 tasks the probability is 0.1^n.

PERT has some techniques for coming up with reasonable task duration probability distributions and aggregating them to larger wholes. I won't go into the math here. Here's some pointer for further reading:

What is the least offensive way to confront a programmer about an estimate they've provided?

Even considering that some programmers consistently under or over estimate, sometimes an estimate is far off from what you would ever imagine it to be (one of those... you're kidding me moments). How do you confront them without putting them on the defense?

Get him Software Estimation: Demystifying the Black Art (Paperback) by Steve McConnell for his birthday, fathers day, Fourth of July, etc.

I am a PHP developer, and I often have no idea in terms of days--let alone hours--how long something will take me at work. I am often writing new stuff, merging it with old legacy crap. I can tell my boss what week I will likely have something done--and maybe what half of what week--but I how in the world am I to know specifically what day something will be done? Isn't that a bit unrealistic, considering that bugs and other unknowns often arise and eat time? I can only minimize these things so much...

I am thinking of saying the following:

"Look, i understand that me saying "tomorrow! tomorrow!" is not helpful. The best I can do for you on many things I'll work on is tell you what half of a given week I will likely have it finished. and if it looks like I can possibly have it done by Friday of the given week, then we better move to the next week."

Here's a link to a great book by Steve Mcconnell (aka Mr Code Complete): http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351/ref=ntt_at_ep_dpt_3 - Software Estimation Demystified

The basic steps are:

  • break down the pieces to understandable/workable chunks (look into workbreakdown structures)
  • if there's an item that you need more info on, note it and make sure you inform your manager ASAP
  • the piece's that you have should include items that are like others you've done before - which should make it easy to provide an est. on those, items you have some 'rough' idea about how long it SHOULD take (add a % over your gut reaction time based on your comfort level - say somewhere between 10% and 30%) and those items you just have to take a wild guess at (add a % range to those, which could be anywhere from 30% to 500%)
  • put you list of deliverables together in a spreadsheet, name each item, comfort level of completing (high to low), your time est. - which could be a range (min. 1 day to max 5 days) and any notes that could include questions and assumptions.
  • present it to your manager and discuss

A doc to talk to that includes the items needed to be completed, questions, assumptions and a range of possible delivery dates speaks a lot for your level of programming maturity - compared to someone who throws out a # of days/weeks (or just a snide remark) and never comes close. If your manager doesn't appreciate that - find a new manager.

So you've just been put on the spot by The Boss. You've got 15 minutes to come up with a back of the envelope estimate for the addition of some new feature. Your boss (fortunately) recognizes that you can't provide an accurate estimate in that time so expecting something that is in the right order of magnitude.

The question is how do you go about giving a estimate in the time frame that is accurate to an order of magnitude?


Note that this is meant to be a quick and dirty estimate, not something that might be expected from questions like this

I've recently been reading Agile Estimating and Planning, and can't recommend it enough.

If you really need very quick estimation, you can do work breakdown structure with every task for 1-2 days or smaller and after this estimate every task by providing min and max estimated values.

sum of min and max values specify interval for the whole task. This gives information abouts risks to your boss, which is always very useful.

You will obtain some interval, e.g. 12-15 day or 5-30 days - this is much more useful than 16 day instead of mentioned intervals.

It can be useful for you good book by Steve McConnel Software Estimation: Demystifying the Black Art.

I often work with sales and marketing types that cannot figure out how to use Excel, let alone understand the scope of their requests from a technical perspective. Of course, it would not be fair to expect them to, but that still leaves me with a problem.

What is the best way to show marketing and sales types that they have asked for something that requires a lot of complex programming and some patience?

Could you please share examples of problems and solutions?

Could you please recommend books on this subject?

Thanks!

Break the problem up into as many sub-divided tasks as possible. Provide a per-item estimate in hours beside each one.

When they think of a project as a whole, it seems simple. However, when they see each individual thing that must be done and the number of hours each item will require, it is putting it into terms business people can understand. Suddenly the software solution they want isn't a "black box" to them anymore and they now have some insight into the process.

If you are looking for books I would suggest Software Estimation - Demystifying the Black Art.

This one may be a good book for non-programmers to understand some of these issues and pitfalls of runaway requirements:

Dreaming in Code: Two Dozen Programmers, Three Years, 4,732 Bugs, and One Quest for Transcendent Software

I have used three point estimation for one of my project. Formula is

   Three Point Estimate = (O + 4M + L ) / 6

That means,

  Best Estimate + 4 x Most Likely Estimate + Worst Case Estimate divided by 6

Here

divided by 6 means, average 6

and there is less chance of the worst case or the best case happening. In good faith, most likely estimate (M), is what it will take to get the job done.

But I don't know why they use 4(M). Why they multiplied by 4 ???. Not use 5,6,7 etc... why most likely estimate is weighted four times as much as the other two values ?

Isn't it a well working thumb-number?

The cone of uncertainty uses the factor 4 for the beginning phase of the project.

enter image description here

The book "Software Estimation" by Steve McConnell is based around the "cone of uncertainty" model and gives many "thumb-rules". However every approximated number or a thumb-rule is based on statistics from COCOMO or similar solid researches, models or studies.

My situation: I recently migrated my company's old ordering system to a new one I developed for them. It had to be backwards compatible and work with the existing database structure until the migration was finally completed. Now that the migration is partially complete, I would like to redesign the database, but I find myself almost 100% of the time supporting the interface, and existing idiosyncrasies of the old database structure. I'm not given any time to document anything so updating anything requires me to rely on the comments I created when originally developing the application.

Not only do I now have to support it for every department, but it seems like even more projects are being thrown at me - which are all getting triaged, and some take weeks before I can even look at them. I hate to do sloppy work in order to get everything done, but it seems like that's where I'm headed :(

I was wondering (and hoping) if anyone is/was in a similar situation and share their insight and perhaps give me some idea's before I rip my hair out :)

Sounds like there are two sides to this:

1) You're having to work with lots of legacy code. I'll let other people talk about how to manage that, but suffice to say a "big bang rewrite of everything" is not the right way to do it.

2) It sounds like you're feeling pressured into rushing things and producing substandard work as a result. If this is true, here is some advice (taken from this awesome book and my personal experience.)

When someone comes to you and says "How long will this take", you need to give them an honest answer. Give them the "do it properly" answer, not the "quick hack" answer.

So, if you say "6 to 8 weeks", you'll most likely get back "that's way too long!". This is where the pressure comes in.

Here is where most programmers kind of just give up, and give in to the pressure to do a crappy job because they feel like if they rushed it they could do it in 3 weeks. Don't do this. The person asking you to do the work is trying to get their own problems solved, and you're not doing them any favors by delivering substandard work. Remember it's your job to work with them to reach the best conclusion.

Some examples of this in practice are discussing and saying stuff like "well, there are 5 parts to this, if we leave this one out we can get it done in time". If that doesn't work then you have other options such as "if we can get carl or john to do these 3 other bits then it will cut down the time by 2 weeks".

Remember, you're working with the other person to develop a plan to get things done. You're not fighting against them! If you keep this in mind in my experience about 80% of the "pressure to do crap work" goes away.

I work with a small team (4 developers) writing firmware and software for our custom hardware. I'm looking into better ways to organise the team and better define processes.

Our Current Setup

  • Developers are generally working on 2-3 projects at a time.

  • We have projects that work in an iterative sort of way, where a developer is in regular contact with the customer and features are slowly added and bugs fixed.

  • We also have projects with fixed delivery dates, and with long lead times, final hardware might appear only a few weeks before delivery. The fixed projects are usually small changes to an existing product or implementation and the work is somehow intermingled.

  • We are also moving from consulting to products, so we are occasionally adding features that we think will add value, at our own cost.

The Issues

We have a weekly meeting where proportions of time are allotted to each project. "Customer A wants to test feature X next week", so the required time is allotted. "Customer B is having issues with Y, could developer P drive down and take a look?", etc.

When we're busy, these plans are very loosely followed. Issues arise and lower priority stuff gets pushed back. Sometimes, priorities are not clear to developers so there is friction when priorities appear to change. The next week there will be a realisation that we're getting behind on project Z and we all pull-off some long days.

I'm told that this is all quite common for a small start-up in our industry, but I'm just looking for ways to limit the number of "pizza in the office" all-nighters.

Two thoughts: drive quality and improve estimates.

I work in a small software shop that produces a product. The most significant difference between us and other shops of a similar size I've worked in a is full time QA (now more than one). The value this person should bring on day one is not testing until the tests are written out. We use TestLink. There are several reasons for this approach:

  1. Repeating tests to find regression bugs. You change something, what did it break?
  2. Thinking through how to test functionality ahead of time - this is a cheek-by-jowl activity between developer and QA, and if it doesn't hurt, you're probably doing it wrong.
  3. Having someone else test and validate your code is a Good Idea.

Put some structure around you estimation activity. Reuse a format, be it Excel, MS Project or something else (at least do it digitally). Do so, and you'll start to see patterns repeating around what you do building software. Generally speaking include time in your estimates for thinking about it (a.k.a. design), building, testing (QA), fixing and deployment. Also, read McConnell's book Software Estimation, use anything you think is worthwhile from it, it's a great book.

Poor quality means longer development cycles. Most effective step is QA, short of that unit tests. If it were a web app I'd also suggest something like Selenium, but you're dealing with hardware so not sure what can be done. Improving estimates means the ability to attempt forecasting when things will suck, which may not sound like much, but knowing ahead of time can be cathartic.

I know this is quite a subjective question ( and is likely to be closed), but I think it would be useful if I ( and the rest of SO!) know how other developers are handling this situation, so I make it a community wiki and hope that it won't be closed.

As a software consultant, one must be able to accurately estimate the amount of time needed for a software project, because the clients demand it anyway. But we all know how complex software development is, and we all know the tendency of software developers to underestimate the amount of effort needed, and overestimate their abilities to deliver the solution just-in-time.

So, if you were a software consultant who is billing your client on complete project basis( but you have to come up with an estimate and then bill based on that), what would you do, if you found that the initial quote you delivered were way too optimistic, and that you needed twice the amount of time (or more) you estimated in order to deliver the solution?

If you are billing based upon a completed project basis, there really isn't much you can do if you underestimate how long the project will take. You might try discussing it with the client to see if they would be willing to renegotiate the cost, but they are under no obligation to actually pay you more and it might in fact hurt your future chances for business if they talk with others about it.

Your best bet for the future is to work on improving your project estimation skills that you use when estimating how long the project will take. While it applies a bit more to larger projects with teams, Software Estimation: Demystifying the Black Art might be a book worth picking up.

What are common empirical formulas that can produce a rough estimate of project duration for waterfall methodology ( up to 20% fluctuation is acceptable). If it helps in narrowing down the answer, you can assume that following is more or less known :

  1. Number of devs is known and fixed, most devs are above average in terms of know-how, however some learning about domain-specific issues might be required.

  2. Known and fixed max. number of app users.

  3. Technology stack to be used is reasonably diverse (up to 4 different languages and up to 6 various platforms).

  4. Interfacing to up to three legacy systems is expected.

Please feel free to provide estimate methods which cover a broader scope than the above points, they are just provided for basic guidance.

Do yourself a favor and pick up Steve McConnell's Software Estimation: Demystifying the Black Art. If you have access to past estimates and actuals this can greatly aid in producing a useful estimate. Otherwise I recommend this book and identifying a strategy from it most applicable to your situation.

Option A

We are working on a small project that requires a pricing wizard for custom tables. (yes, actual custom tables- the kind you eat at. From here out I'll call them kitchen tables so we don't get confused) I came up with a model where each kitchen table part was a database table. So the database looked like this:

TableLineItem
-------------
ID
TableSizeID
TableEdgeWoodID
TableBaseID
Quantity

TableEdgeWoodID
---------------
ID
Name
MaterialUnitCost
LaborSetupHours
LaborWorkHours

Each part has to be able to calculate its price. Most of the calculations are very similar. I liked this structure because I can drag it right into the linq-to-sql designer, and have all of my classes generated. (Less code writing means less to maintain...) I then implement a calculate cost interface which just takes in the size of the table. I have written some tests and this functions pretty well. I added also added a table to filter parts in the UI based on previous selections. (You can't have a particular wood with a particular finish.) There some other one off exceptions in the model, and I have them hard coded. This model is very rigid, and changing requirements would change the datamodel. (For example, if all the tables suddenly need umbrellas.)

Option B:

After various meetings with my colleagues (which probably took more time than it should considering the size of this project), my colleagues decided they would prefer a more generic approach. Something like this:

Spec
----
SpecID
SpecTypeID
TableType_LookupID
Name
MaterialUnitCost
LaborSetupHours
LaborWorkHours

SpecType 
--------
SpecTypeID
ParentSpecType_SpecTypeID
IsCustomerOption
IsRequiredCustomerOption

etc...

This is a much more generic approach that could be used to construct any product. (like, if they started selling chairs...) I think this would take longer time to implement, but would be more flexible in the future. (although I doubt we will revisit this.) Also you lose some referential integrity- you would need triggers to enforce that a table base cannot be set for a table wood.

Questions:

  1. Which database structure do you prefer? Feel free to suggest your own.
  2. What would be considered a best practice? If you have several similar database tables, do you create 1 database table with a type column, or several distinct tables? I suspect the answer begins with "It depends..."
  3. What would an estimated time difference be in the two approaches (1 week, 1 day, 150% longer, etc)

Thanks in advance. Let me know if you have any questions so I can update this.

Here's what my experience would tell me:

  1. Which database structure do you prefer? Without a doubt, I'd go for approach one. Go for the simplest setup that might work. If you add complexity, always ask yourself, what value will it have to the customer?
  2. What would be considered a best practice? That does indeed depend, among others on the size of the project and the expected rate of change. As a general rule, generic tables are worth it when you expect the customer to be adding new types. For example, if your customer wants to be able to add a new "color" entity to the table, you'd need generic tables. You can't predict beforehand what they will add.
  3. What would an estimated time difference be in the two approaches? Not knowing your business, skill, and environment, it's impossible to give a valid estimate. The approach that you are confident in coding will take the least time. Here, my guess would be approach #1 could be 5x-50x as fast. Generic tables are hard, both on the database and the client side.

Are there any online repositories of completed real-world projects with their timescales that I can use to callibrate my own development time estimates?

I don't think relating your project to a set of generic other projects to determine a time est. would be hugely beneficial. You can compare to other projects for defect rates - here's a good starting point for that: http://www.scribd.com/doc/7758538/Capers-Jones-Software-Quality-in-2008

My suggestion would be to look at prior projects in your company with the same relative technology and resources and develop a table (via function points) and then a continual resync. If there is no prior info and/or the technology is new and/or the resources are different - then it's best to use the team's prior experience from prior jobs.

Here's a good book: Software Estimation: Demystifying the Black Art

Assuming the global development effort is known (from a preliminary work structure breakdown), what percentage of that effort should I add in order to allocate enough manpower for requirement definition and refinement?

Read this: (Sorry for the short answer, but you asked a vague question)

Software Estimation: Demystifying the Black Art

alt text

Does anyone have any suggestions about how to estimate how long it would take to comment and reorganize the code of an entire project?

I just started doing contract work for a company along with another programmer: he's fixing bugs and I'm reorganizing the code. I'd like to make an estimate of how long it would take me to go through the code line-by-line and comment and/or reorganize the code is I go. It's a very large solution: it has two projects, each with multiple files.

It was suggested to me to count all the lines of code in each file, sum it all up, and double your hours to give you padding. It was also suggested that I square the size of the code before estimating. But I don't know how long it's supposed to take to comment, say, 100 lines of code.

Any suggestions would be more than welcome. If it makes a difference, the project is made in Windows Forms using Visual Studio 2012.

Thanks!

First, this is an estimate - estimates are not exact numbers, they are approximations and ranges. The estimate you have at the start may be wildly off once you start getting into it and the estimate would need to be refined. Take into consideration the code of uncertainty when giving an estimate.

There exist many well established models for software estimation. COCOMO II is one of these. I believe that this particular approach has added value to its techniques in that it can work from an already known amount.

A web tool for the COCOMO II can be found at usc. Count up the lines of code you have now. Make an approximation of how many lines of new comments you will need, how much will be able to be reused, and how much will need to be modified. Plug those numbers in. The definitions of how COCOMO II works and all the terms can be found in the usc ftp site

Lets say you have a 10k SLOC existing code of which 75% can be reused, 25% will need to be modified (75% design modification, 100% code modification, ...) and an additional 10% for commenting. There are arguments for and against tweaking the various cost drivers (changing things from 'Nominal' or leaving them as they are).

Plug this in and you get an equivalent size of 2825 SLOC which then translates to 9.2 person months of effort (remember, this isn't just going thorugh the code line by line - but also making sure you have the redesign correct and testing it). Nine months is approximately 1500 work hours.

Consider getting Software Estimation: Demystifying the Black Art which goes into more aspects of what an estimate is and other techniques to do an estimate (this is an estimate by proxy and just one of many techniques).

Remember to save the data on how long it took and your estimate - over time, historical data can help refine estimates.

Further reading on Wikipedia Cost estimation in software engineering

I have been a software developer for a while but was not interested in the above topics, currently I am put in the position of wanting to learn more about them but don't have a clue where to begin.

I have done task estimations and I can do decent ones, but have little/none experience in the field of budget/product pricing and would want to learn more.

Do you have any suggestions of good resources that I could look up in order to learn more (eg. books, blogs, ...)

This need arised while talking to a friend about management and he brought up management terms that I wasn't aware of (eg. KPI, ...)

Project Management with Respect to Time and Budget

There are a couple of books you should check out:

  • Exploring Requirements: Quality Before Design.
  • Software Project Survival Guide
  • Software Estimation : Demystifying the Black Art
  • Joel Spolsky posted a quick estimation technique.

    I found this software project management blog.

    I attended a Project Management class (as part of a Masters in Technical Management) and they used the Project Management: A Systems Approach to Planning, Scheduling, and Controlling book. It is a very dry book but has a ton of very useful information; luckily our professors were pretty good at their job (one topic that helped me a lot was dealing with conflict -- but that has little to do with estimation and budget which your question seems to focus on).

    Product Pricing

    I think that the book, Micro-ISV: From Vision to Reality, discusses that. Basically, you should already have established a relationship with some people that represent your customer base and who have cobbled together a crappy solution. If your software represents an elegant solution that fixes their needs, you ask them 1) would you use this if it was free? If they say yes, then you ask them 2) Would you pay a million dollars for it? At which point they'll say, no way, the most I would pay would be $500 (or whatever) -- that's one way to price your product. :)

    Management

    If you are going to be managing people, read these books:

  • Behind Closed Doors: Secrets of Great Management
  • Peopleware: Productive Projects and Teams
  • What actions do you recommend for improving the "Size Estimation Accuracy" in the context of the Personal Software Process/Team Software Process (PSP/TSP)?

    Traditionally one would recommend you to perform Function Point estimation.
    Which turns out to be simple math in practice.

    However typical literature recommending Function Point Analysis almost always concludes with couple of tips from the following list Source

    Tips for accurate estimation:

    1. Avoid off-the-cuff estimates or simple guess: take the time to estimate, never answer without having quietly worked on the estimation.

    2. Allow time for the estimate and plan it.

    3. Use data from previous project.

    4. Use developer-based estimates: use estimation by developers who will do the work

    5. Estimate by walk-through: have each team member estimate pieces of the project individually.

    6. Work until you reach consensus on the high and low ends of the estimation ranges.

    7. Estimate by categories: easy, medium, hard...

    8. Estimate at low level details:
      a 10% error on a big piece is 10% high or 10%low.
      but a 10% error on 50 small pieces when combined, will eliminate each other.

    9. Don't Omit Common tasks like: cutover, data-conversion, installation, customization, management of the beta test program, demonstrating the program to customers or users, attendance at change-control meeting, maintenance work on existing systems during the project, defect corrections, administration related to defect tracking,coordination with QA, support for user documentation, review of technical documents, integration, vacations, holidays, sick days,company and department meetings and training.

    10. Use several different estimation techniques and compare the results.

    11. Change estimation practices as the project progresses.

    12. Don't forget risk management in your estimate.

    In essence, which basically means,

    A. Do function point analysis.
    B. Throw it out the window or Lock it up in your safe.
    C. Now do the actual practical estimation scientifically.

    Seriously though, Function Point Analysis is perfect for projects that can be accurately modeled. This is often not possible in practice, unless one has historical data of the execution of an identical/similar project in the past.

    Based on my experience, the 2½ steps towards generating fairly accurate estimates are :


    Step1. Breakdown the activity into "bite-sized" tasks

    The estimation error (usually under-estimation) grows exponentially for larger estimates.

     Prevalence of Under-Estimation. Source:Software Estimation - Demystifying the Black Art (Developer Best Practices)

    For example, a 1 day task might extend by a couple of hours or a day at best i.e. 10-100% error. However an activity estimated to take 1week can extend for more than a month in the worst case i.e. ~400% error.

    This is due to the fact that unless backed by previous experiences of performing the exact same activity, estimates tend to be optimistic. Initial estimates often do not account for the additional challenges that are discovered as partial progress is made. The sum of such additional tasks can sometimes exceed the initial effort estimate i.e. the initial estimate turns out to be just the "tip of the iceberg".

    Attempt to break-down large activities into tasks and estimating the individual tasks and breaking them further down until each task can be performed by an individual in 1 sitting.

    This "loop" of breaking-down into smaller tasks and estimating them individually helps identify any such tasks that would otherwise be not accounted for in the initial estimates, thus make the initial estimates more accurate.

    The net effort estimate = sum total of all such individual "bite-sized" tasks.


    Step2. Increase confidence of estimates as you progress

    Cone of Uncertainity

    Another method to improving accuracy of estimates is to assert that the initial estimates are really poor quality. Accept this fact and assign a high degree of uncertainty to initial estimates.

    This degree of uncertainty can be regularly lowered as progress is made into the individual tasks.

    NOTE: The estimate itself might increase but the degree-of uncertainty will definitely be reduce in the latter phases.


    Step2½. Individual Discipline

    In addition to the above improvement to the process, a certain amount of discipline is essential in every individual involved in estimation.

    Estimates, Targets (schedule) and Commitments must be clearly defined and differentiated.

    For an in-depth study of the subject of Software Estimation, checkout this book.

    What are the points I must remember during the planning phase of the project to have a really firm foundation?

    Thanks

    Edit: I mean more specifically related to coding. (I don't mean the budgets etc etc).

    For example: Where can we use generics,reflection or concepts in C#

    Check out Steve McConnell's book on Software Estimation. It will help you consider all area's before getting started. For if you have to estimate it then you should know what has to be done.

    You should also consider reading Code Complete.

    Software Estimations, Code Complete