Table of Contents
Introduction
Structure
Status
NORMA
Metamodel
Download
Keep in touch
Future

ActiveFacts is a semantic modeling toolkit that revolutionises the processes of software specification, design, and implementation. The programmers and database people share joint control with the business expert, who becomes the central player in a way never before possible.

Software is not mainly about how, but about what and why. Software projects fail most commonly because they build the wrong thing, not because they build it wrongly. Other forms of modeling, such as object modeling and entity relationship modeling, attempt to capture what, but the models are shaped from the start by the needs of implementation (how). That shuts out the non-technical business expert, and bends the model out of shape, losing track of why each feature is needed, what it means and under what terms, the roles it plays in the overall system. Semantic modeling incorporates and transcends both approaches, and works in either Agile or traditional environments.

ActiveFacts uses a fact-oriented approach to constructing models in elementary form. Every concept, whether a business entity or a relationship, is represented in the simplest possible form, isolated from all other concerns. Every role that each concept plays in the system is incorporated in simple business rules that can be effectively verbalised in plain language, displayed on diagrams, and managed in a system ontology (vocabulary). The elementary form is the most direct mind-map of the business domain that can be constructed, is based on pure logic, and is unsullied by implementation concerns. That makes it much easier to understand and validate by all parties, especially the business expert. Purely for its benefits in specification, semantic modeling is revolutionary, but that's not all...

The true power of the semantic approach lies in automatic composition of the elementary form into convenient straightforward object hierarchies and database designs in efficient third normal form. There are many known ways to design good databases, but efficient physical storage necessitates composition (aggregation) from the elementary conceptual form. The process of aggregation causes the database to be much more sensitive to small changes than the elementary model; the physical database design is unstable. Reliance on SQL fails to hide most of this instability, so evolution is costly and the database design gets left behind. Existing tools for Object/Relational Mapping (such as Hibernate) help somewhat, but ActiveFacts automates the mapping by incorporating a conceptual query language and a semantic runtime that completely isolates the implementation from the flow-on effects of aggregation.

Many of these techniques have been studied for decades, but never before have they been combined. It's possible to be finished forever with

  • Systematic disenfranchisement of business users,
  • The perennial wars between DBAs and object modelers,
  • Database designs that are hopelessly out of date with the applications.

ActiveFacts Structure

ActiveFacts consists of the following main toolsets:

Base API
The base API is the switching-yard for all the tools and converters. It implements all parts of the semantic meta-model.
Reverse engineering tools
Extract the schema and analyse the data from your existing databases into a rudimentary semantic model that you can then complete. ActiveFacts can also import fact-based ORM models from the Neumont Object Role Modeling Architect, NORMA.
CQL
The Constellation Query Language combines the power of a formal language with the intuitive feel of natural language. It can be used both to define and query semantic models.
APRIMO
APRIMO is a semantic modeling scrapbook and IDE, written to run as a one-page web application using the powerful Ruby on Rails web framework.
Semantic Model Storage
This relational database adapter provides a way to store your models while you're developing them with APRIMO.
Relational compositor
The Relational Compositor turns your semantic model into an efficient normalized database schema, writing SQL for a range of commercial and open source DBMS products. Future work will automatically migrate your database between revisions.
Semantic Runtime
The semantic runtime allows you to query and update your relational database, without being dependent on the physical database structure. It translates your CQL queries into SQL, runs them, and translates the results back into conceptual form.

Status

The Base API (in Ruby) is complete, representing static fact-oriented information models. The Query API is under construction at present; future additions will include events, actions and agents (roles).

The Semantic Model Storage schema (congruent to the Base API), allows any model to be stored, retrieved, and manipulated in a relational database. The Semantic Model Storage module is not yet integrated, however, with the Base API.

Constellation Query Language is available as an export format from the base API, and as a language recogniser (parser). The parser back-end is incomplete at present - complex constraints are not yet handled.

Queries in CQL span tables transparently, crossing conceptual relationships (fact types) without explicit database joins, and return structured data in a nested-relational form (called constellation form). For every user action in an application, a single query can be defined that provides all the data required to respond to that action. There's no need to fetch this, that, and the other. All required results are fetched at once, which improves application structure, performance, and even concurrency.

ActiveFacts can currently read models from NORMA, which are in elementary form, and it can reverse engineer the schemas of existing databases in SQL Server, DB2, Oracle, and open-source databases not yet tested including MySQL, PostgreSQL, etc. Reverse-engineered models are still in composite form - the mapping to elementary form is incomplete. Reverse engineering does not yet analyse actual data.

ActiveFacts relies on the database adapter infrastructure of ActiveRecord, but needs additional fixes in the form of DRYSql (with some further patches from me if they are not yet incorporated into the released version), and the Composite Primary Keys gem (again, with further patches if not already incorporated).

APRIMO allows the definition of vocabulary aspects as well as the addition of entity and value objects and the fact types that link them. Subtyping, data types and units, and complex constraints are still to be implemented.

The Relational Compositor has not been started, and neither has the Semantic Runtime.

NORMA

NORMA is the latest in a line of products stretching back two decades which implement the Object Role Modeling (now version 2) fact-based modeling notation. NORMA is a design tool that is open-source, but runs as an extension to Microsoft Visual Studio 2005 - Professional Edition (the Express Edition doesn't have the ability to load plug-ins). You can download the latest version of NORMA (and source code, if you wish) from SourceForge.

ActiveFacts is loading all the NORMA models I can find and a few I built myself, with the exception of sample fact data populations.

Metamodel

ActiveFacts also contains a Metamodels directory which contains a diagram and SQL DDL of the relational schema of the Semantic Model Storage schema. The schema also exists in a form modified to be suitable for Ruby on Rails, which is the form APRIMO uses. The Base API can't yet save or load models into APRIMO database.

My meta-model merges mandatory, uniqueness, and frequency constraints into one "Presence Constraint" object. This is because all three constraint types have to do with the number of times a role or role sequence may be played.

For object types (Entity, Value and Objectified Facts), I have separate classes in the Ruby API, but these are absorbed into one ObjectType table in the SQL. This allows proper referential integrity from the Role table. A consequence is that in sample populations, the entity instances and value instances are absorbed into one Instance table as well, with a nullable Value field.

Finally, my API supports sub-models (known as Aspects or as Vocabularies), and more than one named Population, so that you can have a sample population used for model validation, another as a reference population, and even another for the production database itself.

Download and Examples

To look at ActiveFacts, install Subversion (either from the home site or one of the bundles like Tortoise SVN, cygwin, etc), and issue the checkout instruction on the RubyForge web-page listed above. You can also browse the files online at the Rubyforge source code repository browser.

My NORMA models are available here. You can read about them in the tutorial.

Most of the models are trivial demonstrations of some interesting feature of fact-based modeling. The exceptions are the Orienteering model and the Oil Supply model. The Orienteering model covers the orienteering system currently hosted at both street.orienteering.com.au and the Victorian Orienteering Association. I compete in these events, and have even built scoring hardware for it using low-power microprocessors (not yet in use). The point being that this is a real life model, which will continue to be my main demonstration model. The OilSupply model is extracted from an actual schema from a major oil company. It shows regional demand predictions against production commitments, taking transport availability into account.

Though the provided models are mostly simple, ActiveFacts has successfully reverse-engineered some large commercial database schemas, including that of ManageSoft, with all sub-products installed (265 tables).

If you have Ruby installed you can load a NORMA model using the command:

      ruby -I lib bin/read_norma.rb -C MyModel.orm
read_norma reads a NORMA file and emits CQL.

With the Polyglot gem and the edge version of Treetop (downloaded using Git from the repository), you can also parse CQL - but not yet into any useful form.

If you also have ActiveRecord, DRYSql (patched with the patches provided), and CPK (composite primary keys), you can reverse engineer your own schema:

      ruby -I lib bin/reflect.rb --dsn=Orienteering \
	  --adapter=sqlserver --mode=ODBC --user=sa \
	  --password=whatever
using whatever ActiveRecord connection options are appropriate to connect to your database.

Where can I get more information?

The ActiveFacts project lives on RubyForge, with access details here.

Status messages go to the Yahoo information_modeling group, which is also available as an NNTP feed on Gmane.

If you want pictures first, go straight the quick ORM2 tutorial. This document is part of the ActiveFacts project; if you aren't viewing this from a checkout of the released files, the local links in the remainder of this document won't work. Remote links are fine, of course.