Agile Methologies, Unit Testing and the whole bussiness of software development

If you have been following my blog or know me from MSN etc you will have seen that I have gone a bit quiet. I thought I would spend some me time with my programming books and chill myself out a bit before studying for the MS certs that I have chosen to do. (P.S x:Lex IS finished it is just missing regular expressions, if you want to help out and are good at regex pm me.)

The to main books I have in my possession that deal with software development are Code Complete 2 and Agile Methodologies in C#. Now usually I prefer the hands on approach but I do like to read and you occasionally get some got stuff out of these tomes.

In software there seems to be two conflicting views, the all or nothing OO approach and the agile (in it various forms) approach. Now, I wont claim to know either in great detail but I am interested in both.

For instance I like how XP utilizes user stories to manage a project into definable tasks that are consistent and managable throughout the project.

What I would like to establish here is a bit of conversation on some different concepts that people find work for them.

Do you use unit testing?

Do you only develop required architecture even if you know a project will eventually need to be reworked?

Should everything be an object?

How much OO is too much?

Feel free to add more as I am on a fact finding mission that I hope will benefit more than just me...

Hmmm, I would like to say I am suprised this has received no responses but it only further compounds the fact that a lot of either don't bother with this stuff.

This is interesting though because it means you cut code some other way, what way is that?

I do need sleep occassionally. :p I am only going to tackle the question How much OO is too much? as I am learning more functional styles lately.

Like many programmers raised on OO, functional programming has been a big head scratcher to me. But as I really think and play around with it more, I can start to see it's advantage to use it along side OO. As your project grows and you refactor more and more code, it does start to get harder to navigate. I think this is when trying to move some code into functional style anonymous methods really helps your readability. Especially if they are methods you are not exposing to the outside world. I am now in the habit of assigning EventHandlers to anonymous methods when the code I need to call when the event fires is small and tidy. Because not only is it easier to follow, for me anyway, it also reduces a bit of code as in these cases you and the compiler really don't care about method names and defining their scope.

So IMHO there is a definite benefit in using both OO and functional together as straight can get a bit overwhelming.

The bulk of my professional experience is centered on the banking environment, so I would guess that my opinions are highly influenced by this fact.

Quote:

---

Originally Posted by DeanMc

Do you use unit testing?

---

I've been several years late but I eventually realized the importance of unit tests, so now I try to use them as often as possible. However, I've yet to find an easy (see time-saving) method of incorporating unit tests with anything else other that isolated classes that solve a particular problem. For example, I would definitely recommend implementing unit tests for a library that parses an IBAN but I haven't found an efficient way of introducing meaningful unit tests in a client application that performs cheque data entry.

Quote:

---

Originally Posted by DeanMc

Should everything be an object?

---

Not always but a reasonably architected solution using OO principles tends to be easier to maintain and ammend with new features.

Quote:

---

Originally Posted by DeanMc

How much OO is too much?

---

To me the KISS principle is the golden rule of implementation. Recently I handed out a programming task to one of my reports. I needed a small forms application to perform a tricky part of database housekeeping and I didn't want to deliver to my customer a simple CRUD application - but in terms of UI I needed two forms. My report took three weeks and came up with a working solution that incorporated 20 classes mainly aimed at having a configurable user interface. He had completely missed the point of the exercise and over-architected his solution to provide functionality that will never be used. Obviously, my personal mistake was that I didn't check what he intended to do and save two weeks of development effort by focusing his energies to solve the real problem.

As I've posted elsewhere, if I want to talk to an MS SQL server with a zero percent probability of that ever changing to Oracle, DB2 or MySQL, then there is no point in implementing a DAL that abstracts the database engine so I can choose something else in the future. I think that in any project one should weight how it will be used by the customer, how complex the task to solve is and how it will be maintained and/or enhanced in the future and come up with the simplest architecture that does the job.

Quote:

---

Originally Posted by DeanMc

Do you only develop required architecture even if you know a project will eventually need to be reworked?

---

I've yet to see a pure agile methodology being put to real use in a project. In my field, there are two main problems with trying to use an agile methodology.

The most important would be the daily cooperation with business people and the addition of a customer representative in the development cycle. This would never work in any of my projects - to begin with, the customer person I'd get would probably be one that won't be missed by the customer while I do my job. That would mean that either that person is incompetent or a complete arse to work with (or both), something that won't help me do my job at all. In addition, the idea of having constant interaction with business people is meant to deal with the lack of business knowledge of the implementors. Although interacting with the customer and understanding the business need that you're been called upon to cover is very important for the success of any project, in a banking environment you won't go very far if you're not already very familiar with the business perspective - as an integrator would. For example, it's vital to understand how the customer sees and understands his virtual VISA card product but I won't need constant daily interaction with their business guys for that. The other, often called upon, reason of having a business guy around is to make sure that they get what they wanted - in my experience the customer never really knows exactly what they want.

The other problem would be the lack or severe lack of documentation. That's something that would be totally unacceptable in the environment I work with. However, I can understand that in other fields, this may not be as important - for example, having a great and really cool-looking web site may be more important than documenting in detail the interaction it has with your ERP.

Having said that, I have been faithfully using the Waterfall model in the past but I started incorporating some ideas also found in Agile methodologies. The most important one, especially for projects that tend to be larger than six months, is to be flexible enough to incorporate new functionality into the project. If you find yourself in an eighteen-month project, it is unrealistic to expect that business needs won't change during that period - therefore, you should allow for some amount of scope creep. Another idea is to get some feedback from people in the customer organization regarding your design before you start implementing. Are the business people happy with your proposed representation of the business logic? How about the end-users - the lowly little guys that work for the customer and will be living and working with your solution on a daily basis?

To be fair, the waterfall model works best if you're not crossing unknown terittory. So, if you find yourself going into the same or similar types of projects over and over again, the waterfall model would be my preference. If you're going into uncharted waters in terms of technology or a business model then an agile methodology may be more suited to the task.

Now that's the sort of response I was hoping for, I have been reading more into agile and indeed the links posted by yourself....

It seems that one thing is clear there is an over architecture of designs and principles. Don't get me wrong, it is good to share what works for you (by this I mean anyone with something to say) but there is so much information on how to write code that the art of writing code itself seems to be taking a back seat.

Of course I am not a developer by trade (not yet anyway ;)). So my opinion may be off balance. There is some insightful nuggets to be gathered though. Before I delve into these remember this is from my, stand alone development, view.

The first is requirements are ever changing, even with yourself as the customer it is clear that by the time a project is half way done a slew of new features tend to creep in. For this I like the idea of breaking a project down into little cycles. Rather than saying I will be finished in x months you can aim for a functional build each cycle and let the software evolve naturally. The part of this I like the most is since each cycle is a functional build you could stop at any time and still have a working program. I'm sure many of use have have written applications lurking on our hard drive because of feature creep.

My second nugget that I have decided to take on board is how these cycles should be preformed. As an inexperienced programmer I have a lot to learn when it comes to implementation and because of this tasks do not get done sometimes. To combat this I have found it easiest to have a list of these user stories (Although, I will choose a better name). Each story in my view is an overall view of a certain aspect of the program in development, for instance in a bug tracker a user story could be the form that is used for the details of the bug. I can then decompose this into smaller tasks to be done. The idea of this is that I may not know all of the tasks but I can allocate time to them in the next cycle. But what if a task needs to be implemented for a feature to make the cycle. Well if a task is important then the user story must be removed from that cycle.

The third nugget is based on the above. I believe it makes sense to time the whole project by assigning a number of hours to each task. Like agile velocity. This means that I can work with each cycle in a timely manner and also facilitates the removal of a user story. If I finish all my stories before my allocated time I can work on a new story that fits in with the extra time. All of this may seem overboard for a lone programmer working on personal projects but I believe it is necessary for me to train myself to be a better programmer and produce better applications.

As for KISS I think that only comes with experience. I find it difficult to not over engineer some parts of my application. I feel this has to do with how I create my user stories rather than just ignorance. I believe as I get more projects under my belt I will be able to effectively design modules in a way that is extensible yet clean in their implementation.

Documentation is a whole different beast. I accept the fact that most people don't document or do but it quickly runs out of sync with the application. I have read that the best source of documentation is the code itself due to the fact that it is the final implementation of the original idea.

I wonder if I could create an application that tracks my user stories in a way that they are self documenting.

So how do you people document. What works, what doesn't? Feel free to add to this discussion.

Quote:

---

Originally Posted by DeanMc

I have read that the best source of documentation is the code itself due to the fact that it is the final implementation of the original idea.

---

If you look at my signature, you'll know that I partly agree. But there are two problems related to the lack of documentation. First of all, code itself may be poorly commented. For example:

Code:

---

using System;
using System.Collections.Generic;
using System.Text;
using System.Xml;

namespace framework.config
{
  [System.Serializable]
  public class NamespaceManager : XmlNamespaceManager
  {
      public NamespaceManager(XmlNameTable nameTable)
        : base(nameTable)
      {
      }
  }
}

---

I wouldn't know what the heck one would want to accomplish with this. With a simple comment, my entire view changes.

Code:

---

using System;
using System.Collections.Generic;
using System.Text;
using System.Xml;

namespace framework.config
{
  /// <summary>
  /// Extends the use of <see cref="XmlNamespaceManager"/> by making this class serializable.
  /// </summary>
  [System.Serializable]
  public class NamespaceManager : XmlNamespaceManager
  {
      public NamespaceManager(XmlNameTable nameTable)
        : base(nameTable)
      {
      }
  }
}

---

The other problem is that you may understand what some isolated piece of code does, but still miss the big picture. This is particularly true for solutions with large volumes of code. A functional document or an architecture document definitely helps everyone grasp what the underlying code is trying to accomplish and how. I'm not talking about endless volumes of information here, I'm totally against that. But (to take another example) if you're implementing a business workflow system wouldn't you want to write something that will explain how your workflow engine will work? Such a document would not go into code details but it would go as far as defining the problem to solve and explain the solution from a high level. Since solutions evolve, I would fully expect any such document to evolve - nothing is worse than having a spec that is out of date.

Hmm, I see what you mean, my take on this would be to put a large comment into the file itself identifying its usage. That works for me because I would update it, this of course falls down when a team is involved as they may not update it.


I agree with an overall document but would that not be more of a mission statement on a smaller scale? I mean once it is looked at it is seldom used, of course there are documents that need to be made to ensure a certain way of thinking is followed regarding business logic but I would hold these as an exception to the rule.

I often then wonder about code itself. I have often written code that makes sense only to come back to it and find it doesn't. I have combated this with better variable names and proper use of the language. I mean we can all write cryptic loops but a bit of whitespace never hurt anyone.

One thing that calls me is statements like:

Code:

---

if(p = f)
{
    g.subvert;
    f.implode();
}

---

This is quite prevalent even in code books, there is no good reason to use this sort of variable naming convention. Granted you may have to shorten a variable name to fit print but this sort of naming is crazy.

Originally I would not comment either, then I did for fear of getting lost, now I find a short abstraction contained within a region at the top of the code helps me memorize the best without ruining my clean source.