Technical Articles by Sony Mathew, Software Architect, Twin Cities, MN

Monday, January 30, 2012

State Driven Design (SDD)

State Driven Design

Object-Oriented Design has been a powerful problem solving paradigm for many years now and it still continues to be a force, but as practiced traditionally it has also been limited in many ways. Any Architecture attempting to distribute objects across multiple nodes and processes while still trying to maintain object semantics soon encounters a scalability nightmare. The reasons for these limitations are a lack of constraints on object design around encapsulation and composition. Traditional Domain Driven Design builds on the principles of Object-Oriented Design and inherits some of these same limitations.

Traditional Object-Oriented Design principles work best when confined to the boundary of the local application context. Objects like String, Number, Lists, Maps, Trees and Graphics are represented well as fully encapsulated objects for consumption within a local application context. These objects are not meant to be distributed and scaled beyond one application process or even one user at a time in some cases. Scalability across many servers spanning many concurrent users requires a different design paradigm altogether.

Scalable object design can be achieved by softening some of these traditional object design constraints that impair scalability and replacing them with new constraints that harden scalability. We can achieve this in part by applying the constraints of REST atop a loose form of object design - REST being a proven scalable architecture of the World Wide Web. I call this design approach State Driven Design (SDD).

Let us start with some traditional object design and apply a few of these key constraints to see where we end up. We'll start with a truly old school object design around an Account.  We don’t see this type of traditional design much in practice any more due to its impracticality.

[Constraint] Separate Caller-Visible State

Applying this first constraint requires that all state a caller of an operation may inspect or modify should be separated into its own state object, while all operations that fulfill the needs of the caller be separated into a service. Caller-visible state is state which the caller must see, understand, and modify to interact with the system correctly. Its important to make a distinction between caller-visible state and internal implementation-specific state which the services still encapsulate and is of no interest to the caller.

Applying this constraint:

State objects should be lightweight, transportable and reconstitutable across the network if necessary without changing its value. Operations that formerly acted on caller-visible properties get separated into service objects that are now parametrized with state objects when necessary. Once a caller obtains a state object from the service, the caller should be able to inspect or modify the state independent of the service, additionally the caller must be able to construct a state object independent of the service.

Callers of operations are naturally other operations that should also be refactored in the same fashion, reverse-recurse outward until reaching the outermost operations acting on behalf of the external actor(s).

[Constraint] Caller-neutral Services

Services should not be caller-specific, they should be caller-neutral and be accessible to any number of concurrent callers, conversely a caller should be able to call any instance of a specific type of service at anytime. If caller-visible state has been externalized as dictated by the previous constraint then it can now be provided to the service as parameters instead of the service maintaining state on behalf of a specific caller. In the case of the AccountService example, it now expects an Account as a parameter and no longer assumes a specific Account or Customer.

This caller-neutral nature of a service is an important scalability measure. Services can now be replicated across nodes as necessary to accommodate varying caller loads.

[Constraint] Uniform Service Operations

We have already achieved a reasonable amount of scalability with just the previous constraints and could stop there. The Uniform Service Operations constraint takes it a step further and provides the framework for cross-cutting concerns such as security, and improved scalability through proxies and caches and in general the ability to nest and chain services to separate concerns.

This constraint requires that all services conform to a uniform set of operations. This is generally accomplished in two parts, first generalize the methods to the universal set of operations, then generalize the parameters and return values to a basic request-response model. Operations that no longer fit the service need to be organized into new states and services as shown in the example.

Applying constraints thus far (and others not yet described):

As you can see, AccountService offers a generalized set of operations which for simplicity's sake were kept to just the basic CRUD operations (create, read, update, delete) but this set can be be expanded to include other types of generalized operations. Also notice as a consequence, use-cases such as withdraw, deposit, and transfer no longer fit and needed to be refactored out to new states and accompanying services.

These generalized operations can now be monitored, proxied and secured in a generalized way e.g. an actor might be able to read AccountTransactions within the system but not update or delete them. Additionally, a generalized request-response model allows for states to be cached for requests that allow as such.

[Constraint] Hold References by ID

If a state object has an accompanying service to manage it then objects should reference this state by its ID and retrieve the state as needed on demand via the service rather than maintaining long-lived object references to the state object. These IDs can be database IDs or other forms of persistence IDs, or they could be UUIDs or GUIDs, or they could even just be plain timestamps or process scoped IDs for short-lived non-persistent state objects. Dependent state objects that don't require services to manage them may be embedded directly within their parent states as object references to form aggregates and managed by the aggregates' services.

This constraint goes a long way towards facilitating distribution and replication of services across many nodes. It allows state to be retrieved on demand in the most efficient and scalable way, be it remote retrieval or a local cache look-up.

[Concern] Business Logic

In all this CRUD one may be confused as to where Business logic would live. They live in the CRUD operations of course! Each state object represents a domain concept and its accompanying service represents how that domain will be managed and processed, hence it must also execute the business rules associated with this specific domain.

For example business rules associated with transferring funds would reside in the update operation of the AccountTransferService since the AccountTransfer state represents the domain around the transfer use-case. These rules may further be distributed across finer services that manage finer aspects of this domain, for example AccountTransferService will likely call on the AccountTransactionService to apply withdraw and deposit transaction rules who in turn will likely call the AccountService to apply the rules around updating an Account.

Additionally, uniform service operations allows for separating concerns by decorating and nesting services of the same type. For example, one could have 2 implementations of an AccountTransferService nested within each other, where one is solely responsible for business rules around transfers while the other solely responsible for its persistence. This nesting would be encapsulated from the caller.

[Concern] I/O optimization

You can choose to decompose state objects into finer and finer states each with their own services to manage them, where the outermost state and its accompanying service might represent a top-level use-case while the smallest state and service might be handling the persistence of a line-item. Now such a fine decomposition may not be optimal for persistence or other I/O concerns depending on your system, and batching or coarse-grained I/O operations may be required, If this is the case you should embed the fine-grained state objects directly as object references into their parent states to form coarse-grained aggregates and scrap the fine-grained services altogether, then optimize I/O directly within the aggregates' services managing aggregate states with batched I/O operations.

Additional I/O concerns may also exist around searches and bulk updates. Though not shown here explicitly, one could extend the universal operations of CRUD with overloaded variations of read and update to accommodate search and bulk updates. For example one could add operations like read(AccountFilter) or update(AccountFilter).

[Conclusion] Applying State Driven Design

For a given actor and use-case the designer of a State Driven Design (SDD) model looks to capture the state the actor would be interested in examining prior to execution of the use-case followed by the capture of the transformed state after execution of the use-case. It is important to capture the pre and post states of the use-case execution purely from the perspective of the actor who will examine these states.

Next create a CRUD service to manage this state. How this service manages or transforms this state internally should be encapsulated from the actor who only cares about the final state that represents the completion of the use-case. Now, apply this rule recursively, the newly created service now becomes the new actor (or caller) requiring the use of finer-grained states and services.

In essence, SDD is about modeling states and their transformations by the System.

[Conclusion] Applying Object-Oriented Design

Finally, apply traditional Object Oriented Design rules of generalization and composition to state and service objects independently to eliminate repetition of properties and operations respectively. Pull-up interfaces and abstract base-classes to generalize commonality between states and between services. Decompose complex states and services into finer-grained states and services then manage dependencies between services through Inversion of Control (IoC). As a final note, once in a while when the problem dictates as such, don't be shy about creating fully encapsulated old-school objects for local application consumption ;-)