Are we willing to cross the crevasse?

posted by cjh, 20 January 2011

Dan North’s post on software craftsmanship, and Martin Fowler’s response to it, are right on the money regarding what’s required for programming to become a proper profession.

Ultimately, developers are mostly rather good at putting software together, even those who don’t use the most fashionable methods. Good software can be made using many different methods, not just the ones espoused by the current golden children of the software development fashion cliques, whether it be extreme, lean, agile, or whatever.

Where we continue to fail is in not even attempting to build the bridge that Dan and Martin talked about in The Yawning Crevasse of Doom. We mutter and sneer about how business folk don’t know what they want, and how they’re incapable of applying rational thought to that problem. They feed us endless waffle, and our job is to make it all make sense, to figure out what they need and to meet those needs.

I call BULLDUST! The business knows what it needs, mostly, or they wouldn’t open the dialog. The problem is with developers. We don’t want the business to read, understand or speak our languages, because that makes our speciality less special. Face it, we’ve spent all this time learning out how to describe the structure and function of a computer program in a formal way, and we don’t want someone else telling us why it doesn’t meet their needs. Especially when they don’t even seem to know what they need. It’s bad enough that we often build software that doesn’t meet the business needs, but we certainly don’t want to be told why.

The level of dialog engendered by these attitudes devolves to the business throwing their waffle words at us, and us throwing software back. Agile methods tell us how we can do that really quickly, so that a single interchange in this dialog might take as little as just one week.


And all because all we can speak is software, and all we can hear is waffle. Let me tell you something - the business actually does know what it wants. All we have to do is help them know how to say it. A way to capture the meanings that are in their heads.

If we can find a way to talk the same language, we’ll have dialog interchanges that take only seconds or minutes. Interchanges that don’t require building anything. Interchanges that carry meaning, in each direction, until we can agree on the nature of the things we’re talking about. Only then should we apply all those craftsmanship techniques to build the right thing in the right way.

That’s the kind of dialog a semantic modeling language can facilitate. You might like my one or you might not, but to advance, we must start trying to build such things. I have to say however, the dismissive responses I see from software craftsmen doesn’t improve my confidence that we have the will to fix this problem. Yes, I know that a semantic modeling language doesn’t do anything you can’t already do… except communicate back to your business users before you build the software. I can only fall back on the incredible enthusiasm I get from business folk and regular computer users when I show them what I’m attempting. They understand the nature of the problem that developers are so unable or unwilling to see.

Why software tools fail and what's needed to succeed

posted by cjh, 13 January 2009

Some of my perceptions of the social dynamic of the use of software tools.

Designers of software are motivated mainly by kudos - if their success might be put down to a tool, they aren’t as motivated to use it. The thing that motivates them is to produce the nice software (that’s already in their head) in the shortest possible time. They like tools when they lessen the work without reducing the quality. Tool output is often seen as a compromise and suboptimal, so there really has to be a big time saving to impress these people. Existing ORM and CASE design tools often haven’t produced the right artifacts to shorten schedules - they’ve been targeted at doing things better and getting them right more often. The developer’s hubris doesn’t allow them to see this as an advantage, since they think if left alone, they could produce perfect software without such tools, and that’s what they want to be admired for.

People who want to be “top dog” in a development team, wielding control in excess of their work output, like to use tools - because their knowledge of the tool gives them a special place, they pull the strings. But such people produce so little of the final artifacts of a project they often contribute little to the success of projects anyhow. Instead they create turmoil by insisting that things be done their way, and redone even if a solution is already working.

These last two paragraphs explain why the CASE tool movement of the 1980’s failed so miserably, not because the tools didn’t work.

Organisations will not invest in software development using tools from companies that may fail, or where the tool is seen as risky or dead-end. A technology has to be established in order to provide an escape route… but these days, a viable escape route is if the tool is open source. Nothing is invested to get started, and no vendor can take you down with them.

The IT function as a whole, is often viewed by the business as having far too much control. In part this is a natural reality, as the business can’t move forward without the IT changes, and they don’t have enough understanding of the challenges of succeeding in software development to trust IT. But on the other part, IT uses its power to gain some control over the business direction, sometimes with legitimacy but not always. So the IT function is further distrusted. In addition, IT often fails to deliver adequate functionality in a timely way, and so are seen as less competent than other areas of the business.

On the other side, the business isn’t often much good at writing specifications. The language used is too vague, and doesn’t reflect an understanding of how the IT systems will support business changes; because the business is concerned with what and why - as it should be. So they employ business analysts, who are meant to bridge the gap, but often fall too much on one side or the other. When IT try to explain that a feature cannot be implemented, or is incompletely specified, they have great trouble explaining why there’s a problem. In part, that’s because they think in terms of how, since that’s the natural tendency of the engineering mind. The failure of the business to understand the problem is seen as legitimizing the degree of control that IT asserts.

All this is down to communication and language. A semantic modelling language must make it easier for the business and IT to work together, not as opponents, engaging in paper warfare, but really collaborating. The best way to do that is to create a single language that both groups can read and write, that can provide both with what they need - precision and consistency for the IT folk, and verifiability against the business rules and process for the business folk.

In the process, the language can also be used to generate the artifacts that both groups need (schemas, code, documentation of business rules) - but it’s main attraction to the business is the way it changes the communication process. The generated artifacts are about reducing the project schedules while ensuring continuous compliance with the specification - but they must be of high quality, and preferably, the generators must be tweakable (open source).

That’s the language I hope CQL will become.

There are no attributes

posted by cjh, 23 April 2008

Things don’t have attributes, they have relationships to other things.

Programmers get taught to sort things into objects and their attributes, but that isn’t always helpful. We tend to treat anything we can write down as a value (like a name, a number, or a date) as an attribute of something. But sometimes, a value identifies a thing, and that thing might have other attributes. So the distinction breaks down, and we have to rearrange. In a database, that can mean a lot of extra work.

In a semantic model things don’t have attributes, they have relationships. A relationship might be to a value, and perhaps a given thing may allow only one value in that relationship. That makes it seem like an attribute, but we need to keep those ideas separate.

Is your birth date an attribute? No, it’s just a date to which you have a special relationship - the birth date relationship. Other people have other relationships to that date, and so do other things. Somebody registered their car that same day. The same date plays roles in other relationships, and those roles might even carry meaning in relation to your birth date.

Because you have only one birthdate, it seems obvious to store it as an attribute. That means it’s yours, and not intrinsically related to other things. But what if you made the wrong decision about which concept the birthdate belongs to?

Consider your birth place. Places have their own identity, just as dates do. You only have one birth place, so that could be an attribute too. But there were other people present at your birth… your mother for instance. Your doctor, and nurses. If these things matter in a database you’re designing, you might need to model the birth event. Birth place and birth date are now seen to be attributes of your birth event, not attributes of you at all. The other people involved in your birth also have a relationship to that birth event.

Consider your given name… or is that names? Do the separate names matter, or just the string of names joined up with spaces separating them? That depends on what purpose you have in using the names. You might want to be able to quickly find all the people who have “John” as a middle name - then it might make sense to store the names separately, not as an attribute of the person object.

The same reasoning follows for every kind of attribute. If you start out by thinking about objects (entities) and their attributes, you make assumptions about the way your data will be used.

Instead, start by thinking about how entities are related to values (and to other entities), and make sure you have that clear in your mind. Describe each relationship, using expressions such as “Person was born in Birth” and “Birth is of Person”, “Birth occured on birth-Date”, “Birth occured at birth-Place”, “Person (as Mother) gave Birth”. A hyphen after each adjective will help keep the idea of Place and Date separate from birth date and birth place, while making it clear that birth date is a special role of a Date.

Continue this process of semantic modeling until you have described most of the entities and values that matter, and the relationships (fact types) that join them up.

As you go, you can also record the cardinality of each relationship: “Birth was at exactly one birth- Place”. The “exactly one” isn’t part of the relationship, it’s just a constraint over it. It limits the cardinality of the relationship. Other constraint expressions you might use are “at least one”, or “at most one”. Don’t forget that when you say “exactly one”, or “at least one”, you will need to always know the answer. If you ever need to store information about a Birth but you might not know the birth date, say “at most one”.

When you’re done, or nearly done, you’ll know whether you need a separate Birth table in addition to the Person table, or a separate GivenNames table. Of all the things that you’ve decided matter to you, if Birth is only relevant to one of them (the Person) then since there’s only one Birth per person, then you may be able to absorb the birth date and place as columns of the Person table. It’s a little more complicated than that, but not much.

This is one reason why semantic modeling works better than traditional ER or UML modeling. It’s possible to make a complete model before making decisions about which things are attributes and which aren’t. It still won’t be your final model, but you’ve postponed some bad decisions so you can make good ones instead.

Because of these rules about when you can absorb things, even small changes or additions to your semantic model will cause changes in which tables you need. The shape of your relational database will always be more prone to change than the original semantic structure. For example, a new requirement might be to link up the Birth to the hospital records management system, to be used in paying the medical staff who assisted. Suddenly the Birth details don’t look very Personal any more - even though they haven’t changed!

But because your semantic model hasn’t changed much, you should be able to get away with making only small changes in your code - assuming you followed through properly! As long as the details of the changed tables are hidden under a semantic layer. But that’s a problem for another time…

In the meantime, restrain yourself making from early assignments concerning attributes, and you’ll find you discover new meanings in the information that embodies the rules of your business.