Maintaining model integrity in service orchestrating applications

Cow and Fiddle
Cow and Fiddle, 1913 - Kazimir Malevich

Domain Driven Design (DDD) is a very useful approach to software development. The idea is that developers should not begin modeling their applications in terms of how the application will work technically, instead they should model the problem domain itself. Here I will give a quick introduction to DDD and demonstrate the importance of a rich domain model, then I will show a problem posed by service orchestration that forces many developers to compromise their models, and propose a way to solve it while maintaining model integrity.

These might be considered the main propositions of Domain Driven Design:

  1. For most software projects, the primary focus should be on the domain and domain logic. This is because in most projects, no matter if you’re developing a banking system, where the complexity is in the rules, exceptions to those rules and exceptions to those exceptions, or an HTTP client, where the complexity is in the protocol itself, the domain will usually be more complex than the technology stack. You might be an excellent Spring expert, but if you don’t know what you are building and for what purpose, you can’t make a good design.

  2. Complex domain designs should be based on a model. To understand something, you must have a model, at least in your head. Your understanding is the model. When you grasp an idea, you abstract from the non-essential details of the real world and focus on the essential ones - that is the model. At that stage it’s implicit. You need to make it explicit and make sure everybody working on your application share the same model.

  3. The code is the implementation of the model. How to make it explicit? You could draw some diagrams. You could write some documentation. Or you could implement the model in code, such that your code mirrors the domain - the real world. If it’s coupled not to the specific implementation you chose but to the domain in the real world, once some aspect of the real world changes, the corresponding aspect in the model, and so in the code, can be seamlessly changed as well.

  4. The model is the backbone of a ubiquitous language. Developers and business experts should be able to easily understand each other. If they speak in different languages (business language v.s. technical language), some information will get lost in translation. That’s why they need what Eric Evans calls a ubiquitous language, which is based on the model. This also gives a valuable opportunity to continuously assess the validity of the model. Due to our natural linguistic capabilities as human beings, we are able to detect when trying to communicate an idea is cumbersome and awkward given a language we use. That indicates the underlying model does not reflect the domain well enough.

  5. The model is distilled knowledge. Since the code we produce is the main purpose of development, the focus should be on it and it should be declared as the single source of truth for the model. If you want to know, say, what rules bank uses to open an account for a customer, you should be able to open an Account class or a Customer class and see the rules listed there. Or, even better, open unit tests for Account and User classes and read the rules there.

If you are interested in a more in-depth overview of DDD, including strategic design and tactical patterns, the article by Dan Haywood is pretty good.

Example

As we have seen, the domain model is of paramount importance in DDD. Let’s take a simple example to see how to implement it. First we need a domain. Since Open Banking seems to be the current hot thing in Fintech, let’s use that. We want to write an application that will fetch a list of transactions for a user from one of his accounts. Let’s review the domain and make a model.

User1 is a bank customer who owns one or more accounts. Each account has an ID and zero or more transactions associated with it. A diagram of this model (diagram is just a view, not a model in itself) might look like this:

user-account-transaction

Naive implementation

So we have three entities: User, Account and Transaction. Users have Accounts. And Accounts have Transactions. A common way to implement this model would be something like this:

public final class User {
    private final Map<String, Account> accounts;
    public Collection<Account> accounts() {
        return new ArrayList<>(accounts.values());
    }
    public Account account(String id) {
        return accounts.get(id);
    }
}

public final class Account {
    private final Collection<Transaction> transactions;
    public Collection<Transaction> transactions() {
        return new ArrayList<>(transactions);
    }
}

Here a User having Accounts is represented by the object User having a Map of Account objects. And Account having Transactions is represented by Account object having a Collection of Transaction objects. This is how it could be used:

Collection<Transaction> transactions = user.account(id).transactions();
// Do something with them.

As you can see, this implementation of the model is quite data-centric - class fields are used to model relationships between entities. It looks nice and elegant when your technical scope and infrastructure is simple enough to fit the whole domain in a single application. In such cases you usually have a database containing those transactions and some ORM which knows how to fetch them from the database and populate the collection in your object, either on object construction or later. But what if the transactions are provided in a different service and we have to retrieve them from there? We can no longer use ORM. Technical details like this are not part of the domain and therefore should not cause you to change the model. And yet what developers quite often do in such cases is they move the logic of retrieving transactions from entities to “service layer”. That involves creating a purely technical TransactionsService class, a thing that does not exist in the domain:

public final class TransactionsService {
    private final URI uri;
    private final HttpClient client;
    public Collection<Transaction> getTransactions(String accountId) {
        // This is simplified. You might also need headers, etc.
        return client.get(uri + accountId, Transaction.class);
    }
}

It would be used this way:

Collection<Transaction> transactions = service.getTransactions(accountId);

Since we have this service thing now, we have lost our Account object. Our application just got a lot more anemic. And our model? Well…

model-destroyed

The model is basically gone.

A better way

A better way is to introduce another level of abstraction and implement the model via interfaces, not concrete classes. This way we will be focused on behaviour, not data, and will not tie our model to one concrete implementation, as previously.

public interface User {
    Collection<Account> accounts();
    Account account(String id);
}

public interface Account {
    Collection<Transaction> transactions();
}

Here User having Accounts is represented by User object being able to give us a Collection of Accounts, and Account having Transactions is represented by Account object being able to give us Transactions. And the usage:

Collection<Transaction> transactions = user.account(id).transactions();
// Do something with them.

Now we need to implement the interfaces, and we have the freedom to do it in whatever way we need to in order to handle the constraints imposed by our infrastructure. If we need to use an ORM, fine. If we need to make HTTP calls, it’s fine too. These details don’t affect (or destroy) the model anymore.

So let’s say we need to retrieve the transactions via HTTP. Surely we need to put that in a “service layer”? No. Just use plain old constructor injection:

public final class RemoteAccount implements Account {
    private final String id;
    private final String uri;
    private final HttpClient client;
    public RemoteAccount(String id, String uri, HttpClient client) {
        this.id = id;
        this.uri = uri;
        this.client = client;
    }
    @Override
    public Collection<Transaction> transactions() {
        return client.get(uri + id, Transaction.class); // Simplified.
    }
}

Databases

What about the Account itself? Let’s say it’s stored in local database. The common way to instantiate it, as discussed above, is by delegating it to an ORM. The ORM, however, can’t inject the required dependencies into Account, and even if it could, it would most likely do it in a very cumbersome and restrictive way.2 Our goal is highly composable objects, so we want constructors that we can call ourselves. I see two options.

1. Abandon ORM completely.

Yegor Bugayenko has an entire article on this. Here I will just give a simple example using a generic imaginary SQL library. See Yegor’s article for more detailed examples with real libraries.

public final class SqlUser implements User {
    private final String id;
    private final Connection connection;
    private final String transactionsUri;
    private final HttpClient client;
    public SqlUser(
        String id, Connection connection, String transactionsUri, HttpClient client
    ) {
        this.id = id;
        this.connection = connection;
        this.transactionsUri = transactionsUri;
        this.client = client;
    }
    @Override
    public Collection<Account> accounts() {
        return connection.execute(
            "select AccountId from UserAccount where UserId = {}", id
        ).stream()
            .map(Row::readString)
            .map(this::account)
            .collect(Collectors.toList());
    }
    @Override
    public Account account(String id) {
        return new RemoteAccount(id, transactionsUri, client);
    }
}

2. Encapsulate repository / DAO.

If you are not ready to abandon your ORM yet (as I am at the time of writing this post), you could encapsulate repository or DAO in your objects. This way they would be tools for making database calls, just as Spring’s RestTemplate is a tool for making HTTP calls, and you could delegate establishing connections, managing transactions, etc. to the ORM. They work with DTOs, however, so we’ll have to create one. DTOs are a whole different topic, and generally we should be very cautious when introducing them in our code, but as long as we use them only as implementation details of our real objects, we should be fine. If your framework allows, you could even hide the DTO and repository inside the User object.

public final class SqlUser implements User {
    private final String id;
    private final Repository repo;
    private final String transactionsUri;
    private final HttpClient client;
    public SqlUser(
        String id, Repository repo, String transactionsUri, HttpClient client
    ) {
        this.id = id;
        this.repo = repo;
        this.transactionsUri = transactionsUri;
        this.client = client;
    }
    @Override
    public Collection<Account> accounts() {
        return repo.findByUserId(id).stream().map(
            dto -> account(dto.accountId)
        ).collect(Collectors.toList());
    }
    @Override
    public Account account(String id) {
        return new RemoteAccount(id, transactionsUri, client);
    }
    interface Repository extends CrudRepository<UserAccount, String> {
        UserAccount findByAccountId(String account);
        List<UserAccount> findByUserId(String user);
    }
    // We don't even pretend this is a real class by making fields private.
    static class UserAccount {
        public String userId;
        public String accountId;
    }
}

Summary

The common way of modeling domain entities and their relationships is in terms of classes and their fields. While this may work when you have an ORM to query the database and create those entities and their fields for you, it doesn’t when the data you need is in a different service. However, this doesn’t mean you have to move your logic to the service layer and revert to procedural programming. Just create your objects yourself and give them everything they need to do their job via constructor injection.

  1. In the domain they are actually called Payment Service Users or PSUs, and this term should be used in the model - not “customer”, not “owner”, etc. - but we will just call them “users” here for simplicity. 

  2. Spring, however, can inject dependencies into any object via annotation-driven aspects, but it’s tricky, and it’s an open discussion whether it’s worth the trouble. For the purposes of this post, we assume it’s not. 

Written on July 24, 2018