Results tagged “Software Project Management” from Cordinc Blog

I recently listened to a Software Engineering Radio podcast on Sustainable Architecture. It was an interesting listen despite much of it being given over to discussion of well known and generally good software architecture practices, such as loosely coupled design. It also spent a little too much time on defining software architecture - everyone defines terms to suit their own needs.

However, the podcast did raise a couple of good points that anecdotally feel right and I hadn't heard before. It talked about an option space when designing software systems. That is, that there are a number of ways to design and implement a system. Many developers come up with one way to build a system based on the requirements; then stop designing and start building. This podcast advises looking at multiple ways to solve the problem, thus allowing a determination of which parts of the system will be hard to change or replace at a later date. How costly a decision is to change should be a major determinant of which direction to take. The podcast suggests the result will be easier to change in response to unknown future demands while at the same time not requiring possibly unnecessary effort in the present (to handle these unknown future changes). Interesting.

This week I met up with some old colleagues. While talking about what we were all doing now, one person who still worked at the place we worked together mentioned that there had been a change in software architecture. Thus all my old code would be rewritten. This news got me thinking. I've been writing software commercially for nearly 12 years and most of my code is no longer used. Three years of work is gone due to my employers going bust. Another 3 and a half years worth was (or will be) rewritten after changes in platform architecture. Two more years of work was retired after completing the purpose for which it was written. One employer from years ago still uses 2 years of coding work - just last month I crossed paths with one of the users who did acceptance testing (he remembered me after 5 years, although neither of us could remember the other's name) and he said they were still very happy with it. So about 70% of my work, measured by time, is gone and no longer of any use. Is that normal?

I'm not sure what is the appropriate comparison. Engineers and architects build structures that last for many years. I have known people who assisted building houses, power stations, and gas pumping stations. All of which still stand and should for decades to come. However, these are physical things, so are obviously physically harder to construct. Creating software is a thought driven logical process. Perhaps a better comparison is with other professionals dealing with abstract concepts like accountants, lawyers, analysts, etc. Much of their output is transactional in nature: calculating this year's corporate accounts; writing or checking a contract; or, producing a report. While such work may exist for some time, its utility drops greatly once the reason for its creation has passed. I suppose this is analogous to code still existing in sourcecode repositories long after it no longer runs. In which case still having 30% of my work useful is probably a good record.

However, I don't find the comparison to other abstract workers wholly compelling. Software developers seem to fall in between them and engineers. We think abstractly for a living, but produce something, even if it is just logic, that has utility beyond the point of its creation. Some computer programs have been running for decades - otherwise there couldn't have been such an issue around Y2K. Even in the faster moving consumer software world code can hang around. Just today I used a DOS command window - probably some very old code hanging around there.

Perhaps the best way to think about my profession is that it is a discipline of engineering which works with logic and thus allows the perception that its artefacts are easily replaced (even if in reality that often is not the case). Also it is a fairly new profession and as such it is developing and changing rapidly, which also encourages code to be replaced with updated, more modern, versions. As for my record, when I mentioned the figures to my old colleagues they didn't think it was strange or exceptional in any way. That is probably just modern commercial software development.

Recently I got in a drunken conversation with another long time developer and consultant about what makes software projects succeed.

We both had many years of experience with many different projects, so it was particularly gratifying that we largely agreed with each other (or was that the alcohol?). Having seen many people write their prescriptions to project success, I thought I'd add my 2 cents in the form of my observations. I would consider project success to be that the software is delivered, the clients are satisfied with the result and there were no surprises on time or budget (time/budget could be higher than initially planned, but any slippage was identified and explained well in advance). As a technical person, I also include the subjective measure that the codebase would not cause embarrassment (no WTFs here).

Below are the some characteristics of successful projects I have seen, in no particular order. None of them is sufficient by themselves and I've seen projects succeed when some of these characteristics were glaringly absent. However, I can't remember seeing any project succeed without more than a few of these present.

• Good People. If not the most factor important for success, certainly the most important for enjoyment. I actually think that smart, motivated people who want to get things done can ameliorate many project difficulties. Thus forming a team of good developers should be the first goal of any project. Note here I haven't said "Smart People". While being smart certainly helps it's is not everything (I have a little disagreement with Google's "only the best" hiring practices here). On more than one occasion I have seen some very smart people get tied up in irrelevancies and doing things "right" resulting in nothing being done at all!
• Political Support or Cover. The senior management have to want the project to succeed (not always a given in some political firms!). Otherwise, the project will have problems getting the resources it needs. Similarly in larger organisations a project needs to have managerial support, both in terms of the problem being solved and the manner it will be solved. I was once at a place where the senior IT management were all old waterfall process COBOL coders and distrusted iterative development. There was so much friction and so many people just waiting to say "I told you so" (on both sides) that forward movement was difficult. At another place with similarly minded senior management, our immediate manager provided us with cover. To higher management he presented reports conforming to their waterfall style, but allowed us to run the project as we saw fit.
• Well Understood Problem. The more similar the problem being solved is to previous problems the team has solved the more effectively pitfalls will be avoided. This is one of the premises behind prototyping - solve the exact same problem twice and the second time should be much easier.
• Tight Iterations with End-user Interaction. I started commercial work just as iterative development was becoming popular (we were taught waterfall at university) and I'm a big believer. I have found iterations lasting just a week or so (or as short as possible while still getting some decent work in each iteration) and then deployed to an environment where an end-user can see it (and hopefully try it out) work best. The idea is to get regular and high quality feedback on whether the project is heading in the right direction as well as priorities for future work and changes required. Every project I have been on has significantly changed requirements from start to end. The chances for project success are greatly enhanced if change is accepted and handled. Indeed the best option is to proactively seek out the changes required and this is where short iterations with user feedback help immensely.
• Small Teams. I was once on a project with over 100 other technical people. The back and forth of meetings and coordination was inordinately complex. I think Brooks had this right in The Mythical Man Month, and teams should be kept no larger than around 9 people. That number seems to work well and if they are good people, a great deal can be achieved. On that 100 person project, I strongly believe that the 9 best people in a single team would have done much, much better than the other 91 as a team.
• Flexible Process. I wrote about this before, basically I think software projects work best where they are done by people who look to understand their environment and can adapt - not just retrying the same old thing that worked last time (for you or someone else).

Previously I wrote about big companies not using Ruby, Scala or similar new languages because they are not mature enough, adding that this probably wouldn't apply in small companies. This implies an underlying assumption - that big and small companies are different. This might be obvious to most readers, but based on my experience it isn't to many people I see working in software. Much of the software development advice around is only useful for one type of organisation, although people get hung up on trying to apply it to everyone.

To me, as far as a software team is concerned, a big company is one where there is a sizeable layer of bureaucracy. This can manifest as a large number of political players, passive stakeholders who can greatly change requirements, seemingly arbitrary requirements, or just filling in forms and otherwise jumping through hoops. This is not to say that all of this is unnecessary - some degree of bureaucracy is required to stop developers going off on complete flights of fancy. Also government regulation often requires development to follow a set path; a developer doing their own thing could unknowingly expose the company to sanctions or lawsuits. There seems to be a correlation between the traditional measures of a big firm and bureaucracy (employees and revenues), but they don't always align. I have worked for a very profitable 50K+ employee firm and been allowed to do largely as I wished. While at a 50 person startup in a highly regulated (and litigious!) domain, I was drowned in paperwork.

Each system has its own advantages & disadvantages. Personally I prefer working in smaller companies, or at least in parts of large companies that have more of a small company feel. However, far more developers work under big firm conditions, and I have met good developers who prefer the system. So when I read about avoiding corner cases I think "yeah, lets get things done", but at the same time realise this might not work in an IT department of a large commercial bank. I can also empathise with Eoin Woods about how a Software Architect needs to be able to find and involve all the stakeholders of a project, especially the hidden, passive ones that can stop a project right before it goes into production - a lesson I learnt the hard way. Although, most small firms wouldn't even need a Software Architect. Obviously the context of the advice is crucial.

Some of the best advice I ever received in software development was while working at a small Rational Unified Process consultancy in the late 90's. The RUP is a large and detailed beast often favoured by large companies, although the consultancy successfully applied it to many clients large and small. This was done by never straying from the spirit of the process, but very regularly deviating from the written word of the process. I remember one of the owners there telling me the first step of the RUP on a project was to work out which bits of it you should throw out as unnecessary (the implication being that most of it would be bypassed).

Basically, there is no silver bullet. Advice is just that, advice. Think about how the advice fits your situation - what are the premises, and don't just blindly follow it. I think software projects work best where they are done by people who look to understand their environment and can adapt - not just retrying the same old thing that worked last time (for you or someone else).

I was reminded recently that sometimes managers can wilfully ignore the reality that they work with humans rather than machines. Some examples from my past workplaces:

• At a large well-known consultancy a manager used Microsoft Project to estimate the elapsed time of a project at 6 weeks. I thought that was impossible given the estimates for individual tasks in the project (which I had a part in determining). The manager assured me it was all fine because he just put the tasks into Project and that was the date that came out. I asked to see the Project data. It turned out that he had assumed that all people in the project would work on the project for 5 days a week, no holidays or time off. Problem was the project started on the 1st of December! He had ignored Christmas, New Years and all the holiday time that people had already booked. The project was several weeks late according to the original finish date. However, after taking the holidays into account the project was on time, but the client thought we were hopeless anyway because even after pointing out his error the manager had still shown the client his original unrealistic date!

• At a different firm it was generally accepted that a rota was required to fairly apportion weekend work. The manager would assign each weekend to a team member in turn. So who would do the first weekend, just a few days away? Well, the person who was currently out on holiday for the next couple of weeks of course!
  

And just to show it is not always the managers. I was doing a short, tactical project for a couple of financial markets' traders (consider them very important users). One Monday I deliver the project and they are immediately unhappy - "unworkable and obviously wrong" they say. I return to my desk and work on their fixes. On Tuesday I give them the next version - "what are you thinking, worse than before" they say. I sit down with them and they give me a 10 minute lecture on how they see it working. I leave with a good understanding of what they want. For Wednesday's version they say, "perfect, why couldn't you do this the first time". Only thing was I just gave them Monday's version again!

Of course my lips are sealed on my stupid mistakes.