Results 1 to 9 of 9

Thread: Event source backed entities

  1. #1

    Thread Starter
    PowerPoster
    Join Date
    Jul 2002
    Location
    Dublin, Ireland
    Posts
    2,148

    Event source backed entities

    (looking for thoughts on this)

    Event sourcing is an architecture/idea in which the history of events occuring to any entity is used create the entity as represented in the read model. It is typically used in the CQRS architecture but this is not mandatory - any architecture which allows for the identification of discrete data impacting events can do this and in fact at the lowest level most actually do.

    However in most examples the event history applies to the entire entity. I think a more powerful way of looking at this is that an entity can comprise an associated collection of histories each of which is analoguous to a property.

    For example, a "Company" entity might have an event stream representing the company legal trading name, an event stream representing the employee count and so on. If each event stream has an "as-of" function that returns its value as at a given point in time it is possible to create the state of the company entity as at any effective date/time.

  2. #2

    Thread Starter
    PowerPoster
    Join Date
    Jul 2002
    Location
    Dublin, Ireland
    Posts
    2,148

    Re: Event source backed entities

    So - starting from the lowest level, every property is a history of change events:-

    Code:
    ''' <summary>
    ''' An interface to be implemented by any property change event record
    ''' </summary>
    ''' <remarks>
    ''' An event source property comprises a change history of change events
    ''' </remarks>
    Public Interface IPropertyChangeEvent(Of TProperty As Structure)
    
        Enum ChangeType
            NewValue = 0
            Changed = 1
            Deleted = 2
        End Enum
    
    
        ReadOnly Property Change As ChangeType
    
        ''' <summary>
        ''' Is this change delta (as opposed to an absolute) value
        ''' </summary>
        ReadOnly Property Delta As Boolean
    
        ReadOnly Property Value() As TProperty
    
    End Interface
    Last edited by Merrion; Nov 24th, 2013 at 12:55 PM. Reason: Nullable causes heartbreak further down the line

  3. #3

    Thread Starter
    PowerPoster
    Join Date
    Jul 2002
    Location
    Dublin, Ireland
    Posts
    2,148

    Re: Event source backed entities

    And a history is a date/time keyed set of changes:

    Code:
    ''' <summary>
    ''' A class that implements a 
    ''' </summary>
    ''' <typeparam name="TProperty">
    ''' The underlying type of the property at rest
    ''' </typeparam>
    ''' <remarks>
    ''' 
    ''' </remarks>
    Public Interface IChangeHistory(Of TProperty)
        Inherits System.Collections.Generic.IDictionary(Of DateTime , IPropertyChangeEvent(Of TProperty)  )
    
        ''' <summary>
        ''' Get the effective value of the property as at a given date
        ''' </summary>
        ''' <param name="effectiveDate">
        ''' The date for which we are getting the property value
        ''' </param>
        ''' <remarks>
        ''' This is the value at a given point in history
        ''' </remarks>
        Function GetEffectiveValue(ByVal effectiveDate As DateTime) As TProperty
    
        ''' <summary>
        ''' Write a checkpoint value with the current date/time
        ''' </summary>
        ''' <remarks>
        ''' Appends a change event (that is not a delta) taht holds the current value 
        ''' into the history 
        ''' </remarks>
        Sub Checkpoint()
    
    
    End Interface
    Last edited by Merrion; Nov 7th, 2013 at 10:13 AM. Reason: c# hangover

  4. #4

    Thread Starter
    PowerPoster
    Join Date
    Jul 2002
    Location
    Dublin, Ireland
    Posts
    2,148

    Re: Event source backed entities

    The most common criticism of event sourcing is that it is slow to search as you have to "play forward" the event stream in order to evaluate it.
    Therefore any non-academic implementation needs to maintain a cached "now" state against which to run queries.

  5. #5

    Thread Starter
    PowerPoster
    Join Date
    Jul 2002
    Location
    Dublin, Ireland
    Posts
    2,148

    Re: Event source backed entities

    Further thoughts:-
    1) Events on the stream need to be immutable otherwise you need to track for changes of every event in the stream.
    2) Events cannot be deleted from the event stream
    3) Events inserted in the stream trigger a recalculation from the most recent save point prior to insertion
    4) Combination properties can also exist which are derived from a point-in-time function over one or more other properties - for example in a financial application a "Market Value" property could be a function of a "Quantity" stream and a "Price" stream.

  6. #6

    Thread Starter
    PowerPoster
    Join Date
    Jul 2002
    Location
    Dublin, Ireland
    Posts
    2,148

    Re: Event source backed entities

    For many event streams the value you get at any given event is not the absolute value but is the delta (change). For very simple cases a delta can be performed using simple addition/subtraction but for more complicated objects (for example, vectors) an interface that allows the class to say how a delta is calculated is introduced:-

    Code:
    ''' <summary>
    ''' Interface to allow for delta-calculations on complex objects
    ''' </summary>
    ''' <typeparam name="TProperty">
    ''' The underlying type we are performing differentials on
    ''' </typeparam>
    ''' <remarks>
    ''' </remarks>
    Public Interface IDelta(Of TProperty)
    
        ''' <summary>
        ''' Add a difference to a the value
        ''' </summary>
        Function AddDelta(ByVal delta As TProperty) As TProperty
    
        ''' <summary>
        ''' Reduce a value by a differential
        ''' </summary>
        Function ReduceDelta(ByVal delta As TProperty) As TProperty
    
        ''' <summary>
        ''' Calculate the difference between two values
        ''' </summary>
        ''' <param name="newValue">
        ''' The value we are testing against
        ''' </param>
        ''' <returns></returns>
        ''' <remarks></remarks>
        Function CalculateDelta(ByVal newValue As TProperty) As TProperty
    
    
    End Interface
    Last edited by Merrion; Nov 24th, 2013 at 01:44 PM. Reason: Making TProperty As Class is unnecesarily restrictive

  7. #7

    Thread Starter
    PowerPoster
    Join Date
    Jul 2002
    Location
    Dublin, Ireland
    Posts
    2,148

    Re: Event source backed entities

    If anyone is following this I have to suggest that this is a whole lot simpler if you restrict your event source types to the basic structures and don't allow classes at all - if I get it working I'll post the results in the codebank...

    There is a good article on storing events here that probably goes into this same idea a bit.
    Last edited by Merrion; Jan 21st, 2014 at 05:37 AM.

  8. #8

    Thread Starter
    PowerPoster
    Join Date
    Jul 2002
    Location
    Dublin, Ireland
    Posts
    2,148

    Re: Event source backed entities

    I suppose the first step is to define the classes to support event sourcing (this is the idea of storing the history of events that have occurred to an object rather than the current state - the idea being that you can play these events forward to get to the current state)

    Defining an event
    An event can be just about anything so what is usually done is to create an empty interface that a developer uses to mark their class(es) as "This is an event" :-
    vb.net Code:
    1. ''' <summary>
    2. ''' The base interface for anything that is defined as an event
    3. ''' </summary>
    4. ''' <remarks>
    5. ''' An event is a message that is sent to indicate that a thing has happened.  This typically occurs when
    6. ''' a command has been executed and listeners require to know about this
    7. ''' </remarks>
    8. Public Interface IEvent
    9.  
    10.  
    11. End Interface

    Then, in order to store events and make meaning out of them we add two attributes - the "Aggregation Identifier" which is the unique identifier of the thing that it is (for example an ISBN for a book, a social security number for a person or a GUID if you can't make a unique identifier any other way) and a "Version" which is the incremental number of the event.

    In order to provide this aggregation identifier I implement an interface specifically for it:-
    vb.net Code:
    1. Public Interface IAggregateIdentity
    2.  
    3.     Function GetAggregateIdentity() As String
    4.  
    5. End Interface

    vb.net Code:
    1. ''' <summary>
    2. ''' Additional properties that uniquely identify an event
    3. ''' </summary>
    4. Public Interface IEventIdentity
    5.  
    6.     ''' <summary>
    7.     ''' Get the identifier by which this events aggregate is uniquely known
    8.     ''' </summary>
    9.     ''' <remarks>
    10.     ''' Most implementations use a GUID for this but if you have a known unique identifier
    11.     ''' then that can be used instead - e.g. ISBN, CUSIP, VIN etc.
    12.     ''' </remarks>
    13.     Function GetAggregateIdentifier() As String
    14.  
    15.     ''' <summary>
    16.     ''' The event version
    17.     ''' </summary>
    18.     ReadOnly Property Version As UInteger
    19.  
    20.     ''' <summary>
    21.     ''' The event that is identified by this event identity
    22.     ''' </summary>
    23.     ReadOnly Property EventInstance As IEvent
    24.  
    25. End Interface

    Most companies / situations require additional audit information to be stored along with each event (for the blamestorming) so another interface provides this bit
    vb.net Code:
    1. ''' <summary>
    2. ''' Additional properties used to identify the event in context
    3. ''' </summary>
    4. ''' <remarks>
    5. ''' This is used to record information about an event that does not map to the thing that the event
    6. ''' occured for - this can include when the event occured, who trigegred it, any commentary, a
    7. ''' sequence number etc.
    8. ''' Because different event sourcing implementations may wish top record different information about an
    9. ''' event this interface is sperate to IEvent
    10. ''' </remarks>
    11. Public Interface IEventContext
    12.     Inherits IEventIdentity
    13.  
    14.     ''' <summary>
    15.     ''' When the event occured
    16.     ''' </summary>
    17.     ''' <remarks>
    18.     ''' This should be stored as UTC (or with time zone information) as it may be set by a
    19.     ''' server running in any locale
    20.     ''' </remarks>
    21.     ReadOnly Property Timestamp As Date
    22.  
    23.     ''' <summary>
    24.     ''' The event sequence in whatever it is stored in
    25.     ''' </summary>
    26.     ''' <remarks>
    27.     ''' This has meaning to the storage system but not to the event itself
    28.     ''' </remarks>
    29.     ReadOnly Property SequenceNumber As UInteger
    30.  
    31.     ''' <summary>
    32.     ''' Information about where the event came from (ip address or process for example)
    33.     ''' </summary>
    34.     ReadOnly Property Source As String
    35.  
    36.     ''' <summary>
    37.     ''' User details of who made this happen
    38.     ''' </summary>
    39.     ReadOnly Property Who As String
    40.  
    41.     ''' <summary>
    42.     ''' Additional notes tagged to this event
    43.     ''' </summary>
    44.     ReadOnly Property Commentary As String
    45.  
    46. End Interface

    So now we want to store the events... again following the usual practice I have created a couple of interfaces to define how we read/write events to an event store to allow the actual implementation to be taken care of by concrete classes (that can be swapped out for unit testing for example)
    vb.net Code:
    1. ''' <summary>
    2. ''' Interface for anything that reads events from an event store
    3. ''' </summary>
    4. Public Interface IEventStoreReader
    5.  
    6.     ''' <summary>
    7.     ''' Get the event stream for a given aggregate
    8.     ''' </summary>
    9.     ''' <param name="AggregateIdentifier">
    10.     ''' The unique identifier of the aggregate to get the event stream for
    11.     ''' </param>
    12.     Function GetEvents(ByVal AggregateIdentifier As String) As IEnumerable(Of IEvent)
    13.  
    14.     ''' <summary>
    15.     ''' Gets the event stream for a given aggregate from a given starting version
    16.     ''' </summary>
    17.     ''' <param name="AggregateIdentifier">
    18.     ''' The unique identifier of the aggregate to get the event stream for
    19.     ''' </param>
    20.     ''' <param name="StartingVersion">
    21.     ''' The starting version number for our snapshot
    22.     ''' </param>
    23.     ''' <remarks>
    24.     ''' This is used in scenario where we are starting from a given snapshot version
    25.     ''' </remarks>
    26.     Function GetEvents(ByVal AggregateIdentifier As String, ByVal StartingVersion As UInteger) As IEnumerable(Of IEvent)
    27.  
    28.     ''' <summary>
    29.     ''' Gets the event stream and the context information recorded for each event
    30.     ''' </summary>
    31.     ''' <param name="Aggregateidentifier">
    32.     ''' The unique identifier of the aggregate to get the event stream for
    33.     ''' </param>
    34.     ''' <remarks>
    35.     ''' This is typically only used for audit trails as functionality should depend on the event alone
    36.     ''' </remarks>
    37.     Function GetEventsWithContext(ByVal Aggregateidentifier As String) As IEnumerable(Of IEventContext)
    38.  
    39.  
    40. End Interface
    41.  
    42. ''' <summary>
    43. ''' Interface for anything that writes events to an event store
    44. ''' </summary>
    45. Public Interface IEventStoreWriter
    46.  
    47.     ''' <summary>
    48.     ''' Save a set of events into the store
    49.     ''' </summary>
    50.     ''' <param name="AggregateIdentifier">
    51.     ''' The identifier of the aggregate to save the events against
    52.     ''' </param>
    53.     ''' <param name="StartingVersion">
    54.     ''' The initial version number to start numbering the events from
    55.     ''' </param>
    56.     ''' <param name="Events">
    57.     ''' The set of events to record agains this aggregate
    58.     ''' </param>
    59.     ''' <remarks>
    60.     ''' The events store must be both immutable and forward-only so to cater for the concept of "delete" a
    61.     ''' reversal event needs to exist
    62.     ''' </remarks>
    63.     Sub SaveEvents(ByVal AggregateIdentifier As String, ByVal StartingVersion As UInteger, ByVal Events As IEnumerable(Of IEvent))
    64.  
    65. End Interface
    66.  
    67. ''' <summary>
    68. ''' Interface for anything that stores an event stream for an aggregate and reads them back again
    69. ''' </summary>
    70. Public Interface IEventStore
    71.     Inherits IEventStoreWriter, IEventStoreReader
    72.  
    73.  
    74. End Interface

    And then you can decide how you are going to persist these events. To show the principle I have an in-memory based store that relies on a couple of generic dictionaries:-
    vb.net Code:
    1. ''' <summary>
    2. ''' A class to persist events to memory
    3. ''' </summary>
    4. ''' <typeparam name="TAggregate">
    5. ''' The class that provides the way of identifying the objects that this file
    6. ''' backed event store will store
    7. ''' </typeparam>
    8. ''' <remarks>
    9. ''' This is not optimised as there is no aggregate indexing going on so should only
    10. ''' be used in unit testing or demonstration code
    11. ''' </remarks>
    12. Public Class MemoryBackedEventStore(Of TAggregate As IAggregateIdentity)
    13.     Implements IEventStore
    14.  
    15.  
    16.     Private m_eventsStore As New Dictionary(Of String, List(Of IEventContext))
    17.     Private m_eventVersion As New Dictionary(Of String, UInteger)
    18.     Private m_sequence As UInteger = 0
    19.  
    20.     Public Function GetEvents(AggregateIdentifier As String) As IEnumerable(Of IEvent) Implements IEventStoreReader.GetEvents
    21.         Return GetEvents(AggregateIdentifier, 0)
    22.     End Function
    23.  
    24.     Public Function GetEvents(AggregateIdentifier As String, StartingVersion As UInteger) As IEnumerable(Of IEvent) Implements IEventStoreReader.GetEvents
    25.         If (m_eventsStore.ContainsKey(AggregateIdentifier)) Then
    26.             Return m_eventsStore(AggregateIdentifier).Where(Function(o) o.Version >= StartingVersion).Select(Function(e) e.EventInstance)
    27.         Else
    28.             Return New List(Of IEvent)
    29.         End If
    30.     End Function
    31.  
    32.     Public Function GetEventsWithContext(Aggregateidentifier As String) As IEnumerable(Of IEventContext) Implements IEventStoreReader.GetEventsWithContext
    33.         If (m_eventsStore.ContainsKey(Aggregateidentifier)) Then
    34.             Return m_eventsStore(Aggregateidentifier)
    35.         Else
    36.             Return New List(Of IEventContext)
    37.         End If
    38.     End Function
    39.  
    40.     Public Sub SaveEvents(AggregateIdentifier As String, StartingVersion As UInteger, Events As IEnumerable(Of IEvent)) Implements IEventStoreWriter.SaveEvents
    41.  
    42.         Dim currentVersion As UInteger = GetCurrentversionNumber(AggregateIdentifier)
    43.         If (currentVersion < StartingVersion) Then
    44.             'A gap will be left but this does no harm as the store is is immutable and forward only
    45.             currentVersion = StartingVersion
    46.         End If
    47.  
    48.         If Not (m_eventsStore.ContainsKey(AggregateIdentifier)) Then
    49.             m_eventsStore.Add(AggregateIdentifier, New List(Of IEventContext))
    50.         End If
    51.  
    52.         For Each thisEvent As IEvent In Events
    53.             'add the event
    54.             m_eventsStore(AggregateIdentifier).Add(AddContext(AggregateIdentifier, currentVersion, thisEvent))
    55.  
    56.             'increment the version
    57.             currentVersion = currentVersion + 1
    58.         Next
    59.         'Set the max version number
    60.         m_eventVersion(AggregateIdentifier) = currentVersion
    61.  
    62.     End Sub
    63.  
    64.     Private Function GetCurrentversionNumber(ByVal Aggregateidentifier As String) As UInteger
    65.  
    66.         If (Not m_eventVersion.ContainsKey(Aggregateidentifier)) Then
    67.             m_eventVersion.Add(Aggregateidentifier, 0)
    68.         End If
    69.         Return m_eventVersion(Aggregateidentifier)
    70.  
    71.     End Function
    72.  
    73.     ''' <summary>
    74.     ''' Add a context wrapper around this event
    75.     ''' </summary>
    76.     ''' <param name="AggregateIdentifier">
    77.     ''' The unique key against which an event history is aggregated
    78.     ''' </param>
    79.     ''' <param name="Version">
    80.     ''' The version of the event to save
    81.     ''' </param>
    82.     ''' <param name="eventToContextualise">
    83.     ''' The actual event being written
    84.     ''' </param>
    85.     Private Function AddContext(ByVal AggregateIdentifier As String, ByVal Version As UInteger, ByVal eventToContextualise As IEvent) As IEventContext
    86.  
    87.         m_sequence = m_sequence + 1
    88.         Return ContextualisedEvent.Create(AggregateIdentifier, Version, m_sequence, eventToContextualise)
    89.  
    90.     End Function
    91.  
    92.     Private Class ContextualisedEvent
    93.         Implements IEventContext
    94.  
    95.  
    96.         ReadOnly m_aggregateidentifier As String
    97.         ReadOnly m_version As UInteger
    98.         ReadOnly m_sequence As UInteger
    99.  
    100.         ReadOnly m_event As IEvent
    101.  
    102.         Private m_timestamp As Date = Date.UtcNow
    103.  
    104.         Private Sub New(ByVal AggregateIdentifierInit As String,
    105.                        ByVal VersionInit As UInteger,
    106.                        ByVal SequenceInit As UInteger,
    107.                        ByVal EventInit As IEvent)
    108.             m_aggregateidentifier = AggregateIdentifierInit
    109.             m_version = VersionInit
    110.             m_sequence = SequenceInit
    111.             m_event = EventInit
    112.         End Sub
    113.  
    114.         Public ReadOnly Property Commentary As String Implements IEventContext.Commentary
    115.             Get
    116.                 'This context does not support comments yet
    117.                 Return ""
    118.             End Get
    119.         End Property
    120.  
    121.         Public ReadOnly Property SequenceNumber As UInteger Implements IEventContext.SequenceNumber
    122.             Get
    123.                 Return m_sequence
    124.             End Get
    125.         End Property
    126.  
    127.         Public ReadOnly Property Source As String Implements IEventContext.Source
    128.             Get
    129.                 Return ""
    130.             End Get
    131.         End Property
    132.  
    133.         Public ReadOnly Property Timestamp As Date Implements IEventContext.Timestamp
    134.             Get
    135.                 Return m_timestamp
    136.             End Get
    137.         End Property
    138.  
    139.         Public ReadOnly Property Who As String Implements IEventContext.Who
    140.             Get
    141.                 Return ""
    142.             End Get
    143.         End Property
    144.  
    145.         Public Function GetAggregateIdentifier() As String Implements IEventIdentity.GetAggregateIdentifier
    146.             Return m_aggregateidentifier
    147.         End Function
    148.  
    149.         Public ReadOnly Property Version As UInteger Implements IEventIdentity.Version
    150.             Get
    151.                 Return m_version
    152.             End Get
    153.         End Property
    154.  
    155.         Public ReadOnly Property EventInstance As IEvent Implements IEventIdentity.EventInstance
    156.             Get
    157.                 Return m_event
    158.             End Get
    159.         End Property
    160.  
    161.         Public Shared Function Create(ByVal AggregateIdentifierInit As String,
    162.                        ByVal VersionInit As UInteger,
    163.                        ByVal SequenceInit As UInteger,
    164.                        ByVal EventInit As IEvent) As ContextualisedEvent
    165.             Return New ContextualisedEvent(AggregateIdentifierInit,
    166.                                                         VersionInit,
    167.                                                         SequenceInit,
    168.                                                         EventInit)
    169.         End Function
    170.  
    171.     End Class
    172.  
    173. End Class

    Using this you could have a number of event types that apply to a single "thing" and store them in an event store. For example suppose our thing is a bank account that has an account number, we can use the account number for the aggregate key and end up with something like:-

    Aggregate Version Event Type
    DEJ001 1 Account Created
    DEJ002 1 Account Created
    DEJ001 2 Deposit - $100.00
    DEJ001 3 Deposit - $150.00
    DEJ002 2 Bank Charge - $5.00
    DEJ001 4 Bank Charge - $5.00

    Then if you play the events for either account forward you can get a current view of the balance of each account.
    Last edited by Merrion; Jan 21st, 2014 at 03:23 PM. Reason: code tags wrong

  9. #9

    Thread Starter
    PowerPoster
    Join Date
    Jul 2002
    Location
    Dublin, Ireland
    Posts
    2,148

    Re: Event source backed entities

    OK - on consideration, I think that there needs to be a "something" that turns an event stream or multiple event streams into an object - I'm calling this a snapshot. This is in charge of applying the effect of each event on the aggregate and you can save them as a change history.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width