Software Development At Heart

Software development, as currently practiced, is slow, expensive and error prone, often yielding products with large numbers of defects, causing serious problems of usability, reliability, performance, security and other qualities of service.

According to the Standish Group, businesses in the United States spend around $250 billion on software development each year on approximately 175,000 projects, and only 16% of these projects finish on schedule and within budget. Another 31% are cancelled, mainly due to quality problems, for losses of about $81 billion. Another 53% exceed their budgets by an average of 189%, for losses of about $59 billion.

Projects reaching completion deliver an average of only 42% of the originally planned features.

These numbers confirm objectively what we all know by experience, which is that software development is very expensive, derived from the fact that it is labor intensive.

Only modest productivity gains have been accomplished over the last 10-15 years, the most important perhaps being patterns and agile methods. Apart from these advances, we still develop software the way we did ten years ago: hand-coded and file-oriented. Our methods and practices have not really changed much, and neither have its associated costs and risks.

Economies of scope can be realized in the context of a product family, whose members vary, while sharing many common features. A product family may contain either end products, such as portfolio management , ERP or CRM applications, or components, such as account management frameworks used by portfolio management and ERP/CRM applications.

A product family provides a context in which many problems common to family members can be solved in one place. Building on software product line concepts, software factories can provide family wide solutions, while managing variation among the family members.

A software factory systematically captures knowledge of how to produce the members of a specific product family, makes it available in the form of assets, like patterns, frameworks, models and tools, and then systematically applies those assets to automate the development of further family members, reducing cost and time to market, and improving product quality.

With software factories, you can develop in one place, and reuse that feature in multiple applications, without the associated increased maintenance costs. You only have to maintain the original code at the factory repository!

At Isomeris we are developing AtomWeaver, a software factory that allows you to capture knowledge and code and reuse them in multiple projects, without the burden of maintaining a large base of one-shot, uncategorized code.

Literate Programming (LP) is a computer programming methodology based on the principle that a computer program should be written like a book, newspaper or report, with human readability as its primary goal. Donald Knuth coined the term Literate Programming and created the original LP tool/language, WEB, which he used to write TeX and MetaFont.

While I personally don’t agree with all Literate Programming’s principles, I do agree that extensively documenting an application leads to long term gains.

Is literate programming a good thing? I believe it is. I guess it depends on how you feel about programming. If you are up against a deadline for getting code finished and working, trying to make a literate program instead of a working program might seem foolish. However, in long-term projects (all that code you plan to maintain in the near future), Literate Programming actually does seem to pay off.

One of the best examples of Literate Programming support is a programmer’s outlining editor called Leo, available at http://leo.sourceforge.net. With Leo you can organize your code in an intuitive manner, and at the same time organize and embed the description and specification of that very same code. Leo supports noweb  and CWEB markup. You should definitively try it if you want to write more than just code.

I’ve been applying LP’s dogma for a long time in my development efforts. I do not use the WEB-derived tools but I make sure all code, classes and methods are all well documented at coding time. That’s why my average comment line percentage in my projects goes around 40%. And it sure does pay off!

My microISV, Isomeris, is developing a Code Generator IDE that allows painless integration of Literate Programming in software projects. Stay tuned to find out more.

I will today focus on another trend that I am particularly interested on: Component-Based Development, also known as Component-Based Software Engineering (CBSE). Conventional wisdom says:

“Component-Based Development (CBD) claims to offer a radically new approach to the design, construction, implementation and evolution of software applications. Software applications are assembled from components from a variety of sources; the components themselves may be written in several different programming languages and run on several different platforms.”

Although CBD is normally associated with heterogeneous development (different sources and/or programming languages), I personally think that CBD is also useful within a closed environment.

For instance, CBD enforces a better organization of your application features. Does your application manipulate files? So, create a file-manipulation component! Obvious? Not quite.

It takes additional effort to write a software component that is truly reusable. The component needs:

- additional testing
- robust input validity checking
- to return useful error messages as necessary
- to be fully documented

Szyperski and Messerschmitt claim that a component should:

- Have multiple uses
- Not be context-specific
- Be composable with other components
- Be encapsulated : Only its interface is exposed
- Be independently deployed and versioned

I am yet to find a software component technology that fulfils my own requirements: A lean and mean arquitecture that allows any developer to better organize and reuse his/her own development efforts.

That’s why I am developing my own lightweight component arquitecture! I plan to incrementally describe a specify it on this blog on the following weeks. Stay tuned.

The concept of Model-Driven Development (MDD) is really starting to catch on because of its promise to increase the productivity of those charged with the task of developing and maintaining application systems.

A major issue in software engineering today is software productivity improvement. Likewise, model engineering is a growing trend for building systems that will be used and maintained for many years and sometimes for many decades, as in the defense and aerospace domains

Programming in today’s mainstream programming languages such as Java is theoretically still possible using a notepad editor and command line operations for compiling and executing code. In practice, most programmers will use a combination of programming IDE’s, wizards and code generation tools to make his/her development more productive. The next step in abstraction, using model-driven techniques, is not widespread yet.

One of the highest priorities in MDD is to produce working software that can be validated by end users and stakeholders as early as possible. This is consistent with the major shift towards agile software development methodologies in the industry. MDD provides the scalability that is not inherent in popular agile methodologies such as Extreme Programming.

The first paradigm shift in modeling is to develop productive models rather than contemplative models that only draw informal sets of concepts about the system to be developed. The paradigm shift with model-driven software factories is to automate as far as possible the software production from models. This approach implies tools built in this purpose but also adapted methods of work and software factory awareness from the development teams.

As a newbie to MDD, you should focus on finding tools that have an open architecture, especially in the generation and implementation of code artifacts. For instance, you should be able to generate unit tests and you should be able to generate not only source code but also documentation. The generation aspect is a huge win. You can use MDD to remove the monkey-coding that would otherwise be sent offshore. And I think a lot of people who are looking at offshore strategies as a cost savings reduction should also be looking at MDD and seeing the big cost and efficiency savings that can be made there.

Finally I am writing my first post, that I hope will be on a regular basis. I’ve been intending to get into the blogging crowd, but somehow development activities have always had priority over it. Let’s change that from now on.

I will certainly focus on software development on its different scenarios, as a solo developer, as a microISV, or as an enterprise.

Let’s see how it goes. I wish a profitable reading to everyone.