Docs‎ > ‎Architecture‎ > ‎

Logic Execution

This document explains how the Espresso Logic executes your business logic in response to REST post/put/delete requests.  This is not an internals document, rather, it is intended to help you understand how the server operates so you can debug problems.

Before reading this, you should be familiar with the Overview and the Architecture.  In particular, you will need to understand the Object Model - row objects that provide attribute/object accessors, persistence, and event publishing.   You will find the JavaScript Entry Points to be a handy reference.

Row Events are passed row, oldRow and logicContext.

Rule Operation

Update business logic is a large part of most database applications.  Like many other systems, Espresso enables you to provide logic in familiar update events, here in server-side JavaScript.

What makes Espresso unique is Reactive Programming: declarative, spreadsheet-like rules.  These enable you to deliver update business logic 10X faster.

While rules hold enormous promise, they operate differently than conventional programming.  Shown conceptually in this video, this page shows how to use them, how they work, and how they integrate with events.

Usage Overview

You declare rules, which the system then uses to process RESTful updates as outlined below.

Connect and Declare Rules

When you create a project and connect to your database, Espresso creates a JavaScript Object Model.  You can think of these as Object-Relational Mapping (ORM) objects: they provide access to attributes, related objects and persistence.  They also expose update events you can handle in JavaScript.

Many ORMs require tedious synchronization between your schema and objects.  The use of JavaScript means these objects are dynamic - no code generation, so they automatically stay in sync with your model.

But they are much more - they are logic aware. That is, they encapsulate both declarative rules, and conventional JavaScript events. You might think of these are Declarative Persistence Objects.

The declarative rules are expressions for deriving / validating data.  We'll look into more detail later, but here are two rules.  The first Validation rule ensures that the balance does not exceed the credit limit (else the transaction is rolled back), and the second rule computes the balance by Summing the related Orders.

Validation: return row.CreditLimit >= row.Balance
Derive Balance as Sum(Orders.AmountTotal where ShippedDate === null)

The expression operands are derived from the schema: tables, columns and foreign key relationships.  The rule expressions are entered with a combination of Forms and server-side JavaScript functions (providing if/else logic, access to libraries, etc).  You can see the details in the Tour. 

These rules are fully executable.  Once entered, you don't need to worry with the details of calling them, ordering them, reading/writing data - that is all addressed with system RESTful Update Processing.

RESTful Update Processing

The Sequence Diagram summarizes how your rules are automatically applied to incoming RESTful update requests (Put, Post, Delete):
  1. A database transaction is started.  Each Request is a Transaction.

  2. Request Pre Events are raised.  The Request Event is raised once for the entire update request.  Your JavaScript can inspect / alter the JSON request body.  

  3. For each JSON Object (recall you can update multiple objects in a single update request), the system

    1. Creates a Declarative Persistence Object row.  This combines the JSON data with current disk data (JSON rows might be a subset of columns), and performs Optimistic Locking checks based on a Hash.  The oldRow is also made available to your logic.

    2. Initiates Row Logic:

      1. Early Row Event are called, supplied with row, oldRow and logicContext.  Your event can inspect / alter the row before rules fire.

      2. Executes (only) rules whose dependent row data has changed (based on comparing row vs. oldRow).  Rule execution order is computed based on rule dependencies, discovered by automatic rule parsing.  The rule updates the row (state), so it's visible to ensuing rules.

      3. Row Events are executed, enabling you to do whatever is not expressed in declarative logic (send email, post data to other systems, credit card checks, etc etc).  You are passed row, oldRow and logicContext.  The effects of rule changes are visible.  You can alter the row, but must resave it.

      4. Row Chaining.  If the logic has altered data referenced by rules in related objects, the system instantiates rows for them, and invokes their logic.  For example, altering an OrderTotal would therefore update the related Customer's Balance, whose rules would verify the CreditLimit.  Note this is an efficient 1-row update - not an aggregate query.

  4. Rows are saved to disk from Transaction Cache.  Updates are buffered into a Transaction Cache so that only a single update is required for each row.

  5. Commit Row Events and Validations.  You can handle events that run after all rows are processed, so you can see the results of rule execution on all rows (e.g., the OrderDetails of an Order are reflected in the Orders AmountTotal).  Your commit validations are invoked after logic processing is complete for all rows.  This is useful when your validation needs to see the results of aggregations, such as the count of children as explained in the link.

  6. The Transaction is committed.

  7. Request Post Events are raised.  You can alter the response message, or perform other functions such as logging.

RESTful Update Processing Notes

In addition to the overview above, please take note of the special topics below.

Resource/Object Mapping

Recall that you can define multiple custom Resources on the same underlying Base Table, and that these can represent projects and aliases of table columns.  Integrity demands that the Base Table logic is enforced regardless of the Resource used.

Resource/Object Mapping is therefore required to map the Resource objects onto their respective Row Objects, including column name de-aliasing and materialization of columns not projected (sometimes called "hydrated").  This means that the declarative and event logic is always dealing with a full row, and that logic is shared over all Custom Resources.  This of course requires a database read, with concurrency control supplied by the DBMS.

So that you can define logic based on data changes, the system builds 2 row objects:
      • row: reflects client changes from JSON, and all logic processing
      • oldRow: reflects the row before the transaction started (this is from the database read noted above)
Optimistic locking checks are performed to ensure that updated rows will not overlay the updates of other users.   This check is on a time-stamp field if provided, otherwise it is the hash of all attributes in the Resource row.

Generated Primary Key Handling

Databases support system-generated primary keys.  There are special requirements when processing JSON POSTs that include child rows for such tables, such as the items for an order - the system needs to "stamp" the generated order# into each item.

This is done prior to logic execution (described here, and below), and is also when Managed Parent processing is performed.

Row Commit Logic Cycle

The write-cache is flushed to the database at the end of the transaction, after all rows have been processed.  In addition to the flush, there is a very important logic provision.

Your logic specifications for commit and action rules can stipulate that they run during normal per-row execution, or be deferred until commit.  If you elect that option, such logic is executed just prior to transaction flush.

To see what this is important, consider the example where we wish to ensure Purchase Orders have at least 1 Line Item.  A good approach would be to define a Purchaseorder.item_count, along with a Purchaseorder validation that item_count > 0.

While a good approach, this would fail.  Why?  The system will process the Purchaseorder insert first, before Line Items.  At this point, the count will be 0, so the validation will fail.

We therefore provide the commit option for validations and events that need to operate on the end-result of logic processing for all the rows in the entire transaction.

Forward Chaining

Forward Chaining is an often-used term for Dependency Management.  It simply means that when referenced values are changed, all the derived referencing attributes are recomputed.  The term chaining correctly infers that a derived attribute (e.g., Purchaseorder.amount_total ) is itself referenced in another derivation (Customer.balance).  It is of course the systems' responsibility to track these references and perform the forward chaining, automatically.

For formulas (e.g, price * quantity), this simply entails evaluating the expression (though see ordering, below).  It is much more complicated for dependencies and multi-table derivations, as discussed in the sub-sections below.

Columns dependent on changed columns may themselves have interdependencies.  For example:

    • a = b + x 
    • b = x + 2
It is clear that a depends on b, so if x is altered, we must recompute b before we recompute a.  Again, this is the systems' responsibility - you are not required to state these rules in any particular order.  This means you can change the rules during maintenance, without concern for ordering.
Parent Adjustment
Continuing our Customer.balance example, imagine a simple update where a Purchaseorder is marked paid.  We need to recompute the new balance.

A dreadful approach would be to issue a SQL Sum query to add all the existing orders.  In general, there could be thousands!  And worse, this could be chained, where the summed attributes depend on further summed attributes.  That is, in fact, just the case here: the Purchaseorder.amount_total is itself a sum of the Lineitem.amount.  This is a very significant performance factor; ORM (Object Relational Mapping) products are often blamed for poor performance due to excessive use of chained aggregate queries.

So, the system adjusts the parent sum by the change in the child.  The result is a one-row update (unless it was pruned per the discussion above).

Child Cascade
There are analogous considerations where the client alters a parent attribute referred to in child logic (e.g., a Formula).  When this occurs, the system visits each related child to run the dependent logic.  This may update the child, and trigger further adjustment / cascade processing to other related data.

Multi-table chaining
Adjustment and Cascade processing both make updates to related data.  So, you will observe that the system will often issue SQL updates for data beyond that originally sent from the client.  This is a natural consequence of your logic, and exactly what business logic is supposed to do.

Such triggered updates are subjected to the full logic analysis / chaining process, so will often result in still other updates.  For example: consider a simple update to a Lineitem.quantity:
    • Lineitem.amount is derived as price*quantity, so is recomputed
    • Purchaseorder.amount_total is derived as Sum(Lineitem.amount), so it is recomputed (adjusted)
    • Customer.balance is derived as Sum(Purchaseorder.amount_total where Paid = false), so is is adjusted
    • The customer logic re-evaluates the credit limit check - if not passed, the entire transaction is rolled back, and an exception is returned to the client.

Logic Re-execution
Observe that chaining means your logic may be executed more than once on the same row multiple times within a transaction.  Consider a transaction comprised of a Purchase Order with multiple Line Items.  So the Purchase Order logic is clearly executed on insertion.  Now consider that each Line Item would adjust the Purchase Order's amount_total.  This re-executes the Purchase Order logic, now as an update.  Your logic can determine nestLevel and initialVerb via the LogicContext.

Key Observations

We can now make some key observation about some fundamental characteristics that distinguish Reactive Programming from conventional Procedural (Imperative) programming:
  •  No Control flow:  you do not invoke your rules - they are invoked by the system, and only in reaction to actual changes.  You do not order their execution.

  • Elimination of Boiler Plate code: in a conventional approach, the bulk of your code is Change Detection, Change Propagation, and Persistence handling (SQL commands).  Reactive Programming automates all of this, so the logic shown above is fully executable.

Simple Example: Check Credit

So, let's continue our example, to devise a solution of Check Credit.  Building on the two rules above, we have:

Validation: return row.CreditLimit >= row.Balance
Derive Customer.Balance as Sum(Orders.AmountTotal where ShippedDate === null)
Derive Orders.AmountTotal as Sum(OrderDetails.Amount)
Derive OrderDetails.Amount as row.Price * row.Quantity  // JavaScript Expression
Derive OrderDetails.Price as copy(Product.Price)

And that represents the complete, executable solution; note:
  1. Ordering: The rules above can be entered in any order, since they are automatically ordered per dependency management, above
  2. Re-use: The rules are applied to all incoming transactions, automatically invoking the (relevant) logic above 
  3. Automatic Persistence: The system provides all the SQL to process incoming transactions.  So, adjusting a Quantity automatically reads / adjusts the Orders and Customer rows, and it does so efficiently (a 1 row update, not an expensive  select sum query)

These are common Logic Patterns

This simple "Balance < CreditLimit" example illustrates one of the most common Logic Patterns:

Constrain derived Result

Other examples of the pattern:
  • Rollup employee salaries to department, constrain to budget
  • Rollup departments, constrain to budget
  • Rollup Student Course Credit, constrain to max for student, max for course

A similar pattern is:

Existence Checks: validations on [qualified] counts

Such as
  • Order must have items
  • Department must have employees

Scaling to Complexity

Basing rule definition on JavaScript enables you to identify and automate your own patterns.  Espresso pre-supplies several, useful both directly and extensibility examples:
  • Copy - invoke copy, including deep copy, for patterns like auditing, or cloning.  It even automates a Bill of Materials explosion.

  • Allocation - allocate an amount to a set of recipient - a bonus to department employees, a payment to outstanding orders, etc.
You can explore these examples, which range for simple to quite complex - all handled with a few rules.

Business Perspective: agility, transparency, quality

Declarative logic is remarkably more expressive than imperative code.  The 5 lines of logic above equates to over 200 lines of triggers, or 500 lines of Java.  It's also far more readable - in fact, understandable to Business Users.  

In an industry where we walk over hot coals for a 30% gain, this is a remarkable 40X improvement in expression factor.  That's what delivers the 10X reduction in delivery.

So what, exactly, drives this compression factor?  It stems from 2 key factors: removing boilerplate code, and automatic re-use.

Boilerplate Elimination

We noted above:
  • ORM creation - considerable code is saved in the automatic creation of the Object Model

  • change detection - most of the alternative code noted above is detecting changes to determine when to propagate updates.  This is eliminated in the Declarative Reactive approach

  • sql (caching) - we're all painfully aware that sql handling is tedious; rules automate the sql, including the critical underlying services for caching.

Automatic Re-use

Rules are bound to the data, not a specific Use Case, so they apply to all in-coming transactions.  In other words, the logic above automatically processes all of these transactions:

  1. Order inserted - balance increased
  2. Order deleted - balance decreased (if not paid)
  3. Order unshipped - balance decreased
  4. Order shipped - balance decreased
  5. Order amountTotal changed - balance adjusted
  6. Order reassigned to different customer - balance increased for new customer, decreased for old
  7. OrderDetail inserted - obtain price, adjust Order and Customer (and check credit)
  8. OrderDetail Deleted - reduce Order and Customer totals
  9. OrderDetail Quantity increased - adjust Order and Customer (and check credit)
  10. OrderDetail Product Changed - obtain price, adjust Order and Customer (and check credit)
  11. OrderDetail Quantity and Product Changed - obtain price, adjust Order and Customer (and check credit)
  12. Customer CreditLimit changed - check credit
This results in a meaningful improvement in quality.  Reactive Programming eliminates of an entire class of programming error (e.g., balance checked when adding an order, but overlooked on unshipping an order).

JavaScript when you need it

As noted above, the Object Model provides for Row Events which you can handle in JavaScript.  Here is an example from the Add Payment example: