Class ThreadAffinityConfig

java.lang.Object
org.efs.dispatcher.config.ThreadAffinityConfig
public final class ThreadAffinityConfig extends Object
A typesafe configuration bean class containing necessary information needed to create an affinity between a thread and a core using OpenHFT Thread Affinity Library. User is assumed to understand thread affinity and the necessary operating system configuration needed to support it. Please see the above link for more information on how to use thread affinity and its correct use.

ThreadAffinityConfig consist of the following typesafe properties:

  • affinityType: Required. Defines how core is acquired for the thread. There are five acquisition types as defined by ThreadAffinityConfig.AffinityType:
    1. ANY_CORE: Use AffinityLock.acquireCore() to assign any free core to thread.
    2. ANY_CPU: Use AffinityLock.acquireLock() to assign any free CPU to thread.
    3. CPU_LAST_MINUS: Use AffinityLock.acquireLock(int cpuId to assign a CPU with specified identifier to thread. Requires property CPU_OFFSET_KEY be set.
    4. CPU_ID: Use AffinityLock.acquireLock(int cpuId to assign a CPU with specified identifier to thread. Requires property CPU_ID_KEY be set.
    5. CPU_STRATEGIES: Use AffinityLock.acquireLock(AffinityStrategies...) to assign a CPU for thread affinity. Selects a CPU for thread affinity based on the given selection strategies. Requires property STRATEGIES_KEY be set.

      Please note that this type may not be used by itself or as an initial CPU acquisition type. Rather there must be previous CPU allocation to this (for example a previous dispatcher configuration using thread affinity) which the strategy then uses to allocate the next CPU. Attempts to use this acquisition type either by itself or as the first strategy will result in an error and no CPU allocated for the thread.

  • bind: Optional, default value is false. If true, then bind current thread to allocated AffinityLock.
  • wholeCore: Optional, default value is false. If true, then bind current thread to allocated AffinityLock reserving the whole core. This property is used only when bind property is true.
  • cpuId: Required when affinityType is set to CPU_ID. Specifies the allocated CPU by its identifier.
  • cpuStrategies: Required when affinityType is set to CPU_STRATEGIES. Values are restricted to enum net.openhft.affinity.AffinityStrategies.

    Note: strategy ordering is important. AffinityStrategies.ANY must appear as the last listed strategy. This allows any CPU to be selected in case none of the other strategies found an acceptable CPU.

Users should be familiar with the OpenHFT Java Thread Affinity library and how it works before using efs thread affinity configuration. This includes configuring the operating system to isolate acquired CPUs from the operating system. This prevents the OS from preempting the thread from its assigned CPU which means the thread does not entirely own the CPU. That said, isolating too many CPUs from the OS can lead to a kernel panic. So using thread affinity is definitely an advanced software technique, requiring good understanding of how an OS functions.

The following example shows a typesafe HOCON file defining a how to use thread affinity for efs dispatcher threads and especially the CPU_STRATEGIES acquisition type. 

dispatchers  : [
    {
        dispatcherName = "mdDispatcher"
        threadType = SPINNING
        numThreads = 1
        priority = 9
        eventQueueCapacity = 65536
        maxEvents = 65536
        runQueueCapacity = 1 // only one market data agent.
        affinity {       // optional, selector thread core affinity
            affinityType = CPU_ID // required, core selection type.
            cpuId = 7             // required for CPU_ID affinity type
            bindFlag = true       // optional, defaults to false
            wholeCoreFlag = true  // optional, defaults to false
        }
    },
    {
        dispatcherName = "orderDispatcher"
        threadType = SPINNING
        numThreads = 1
        priority = 9
        eventQueueCapacity = 8196
        maxEvents = 128
        runQueueCapacity = 16 // equals number of order processing agents.
        affinity {       // optional, selector thread core affinity
            affinityType = CPU_STRATEGIES // required, core selection type.
            cpuStrategies : [             // required for CPU_STRATEGIES affinity type
              SAME_CORE, SAME_SOCKET, ANY // Note: ANY *must* be last strategy.
            ]
            bindFlag = true       // optional, defaults to false
            wholeCoreFlag = true  // optional, defaults to false
        }
    },
    {
        dispatcherName = "defaultDispatcher"
        threadType = BLOCKING
        numThreads = 8
        priority = 4
        eventQueueCapacity = 512
        maxEvents = 64
        runQueueCapacity = 128
    }
]

efs uses this configuration to pin EfsDispatcherThread instances to a core or cores.

Author:
Charles W. Rapp

  • Field Details

  • Constructor Details

    • ThreadAffinityConfig

      public ThreadAffinityConfig()
      Default constructor sets fields to invalid values.

  • Method Details

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getAffinityType

      public ThreadAffinityConfig.AffinityType getAffinityType()
      Returns affinity type used for creating thread affinity to selected CPU_ID.
      Returns:
      CPU_ID selection type.

    • getBindFlag

      public boolean getBindFlag()
      Returns the bind-thread-to-affinity lock setting. If true then thread is bound to the lock.
      Returns:
      true if thread is bound to the affinity lock.
      See Also:

    • getWholeCoreFlag

      public boolean getWholeCoreFlag()
      Returns whole core reservation bind settings. If true then thread reserves entire core and does not allow hyper-threading. This value is used only if getBindFlag() returns true; otherwise ignored.
      Returns:
      whole core reservation flag.
      See Also:

    • getCpuId

      public int getCpuId()
      Returns CPU_ID identifier used for ThreadAffinityConfig.AffinityType.CPU_ID affinity type. Set to a negative number for any other affinity type.
      Returns:
      CPU_ID identifier or < zero if no identifier specified.

    • getLastMinusOffset

      public int getLastMinusOffset()
      Returns CPU offset used for ThreadAffinityConfig.AffinityType.CPU_LAST_MINUS affinity type. Set to zero for any other affinity type.
      Returns:
      CPU offset or zero if no offset specified.

    • getStrategies

      public List<net.openhft.affinity.AffinityStrategies> getStrategies()
      Returns immutable list of CPU_ID selection strategies used for ThreadAffinityConfig.AffinityType.CPU_STRATEGIES affinity type. Set to null for any other affinity type.
      Returns:
      CPU_ID selection strategies.

    • setAffinityType

      public void setAffinityType(ThreadAffinityConfig.AffinityType type)
      Sets affinity type to the given value.
      Parameters:
      type - desired thread affinity type.
      Throws:
      com.typesafe.config.ConfigException - if type is null.

    • setBindFlag

      public void setBindFlag(boolean flag)
      Sets flag for binding thread to affinity lock.
      Parameters:
      flag - if true then bind thread to affinity lock.

    • setWholeCoreFlag

      public void setWholeCoreFlag(boolean flag)
      Sets flag reserving the entire core and not allowing hyper-threading on that core. This flag is used only when setBindFlag(boolean) is set to true; otherwise it is ignored.
      Parameters:
      flag - if true then binding reserves whole core.

    • setCpuId

      public void setCpuId(int id)
      Sets CPU_ID identifier used by ThreadAffinityConfig.AffinityType.CPU_ID affinity type. This value must be set when using CPU_ID affinity type and is ignored for all others.

      Note: id is not checked to see if it is a valid CPU_ID identifier beyond making sure it is ≥ zero.

      Parameters:
      id - CPU_ID identifier.
      Throws:
      com.typesafe.config.ConfigException - if id is < zero.

    • setLastMinusOffset

      public void setLastMinusOffset(int offset)
      Sets positive integer offset used by ThreadAffinityConfig.AffinityType.CPU_LAST_MINUS affinity type. This value must be set when using CPU_LAST_MINUS affinity type and is ignored for all others.

      Note: n is not checked to see if it is a valid CPU offset beyond making sure it is > zero.

      Parameters:
      offset - offset from end used to select CPU.
      Throws:
      com.typesafe.config.ConfigException - if n is ≤ zero.

    • setStrategies

      public void setStrategies(List<net.openhft.affinity.AffinityStrategies> strategies)
      Sets CPU selection strategies used by ThreadAffinityConfig.AffinityType.CPU_STRATEGIES affinity type. This list must be set when using CPU_STRATEGIES affinity type and is ignored for all others.

      Note: strategy ordering is important. The ANY strategy must appear as the last listed strategy. This allows any CPU to be selected in case none of the other strategies found an acceptable CPU.

      Parameters:
      strategies - CPU selection strategies list.
      Throws:
      com.typesafe.config.ConfigException - if strategies is either null. and empty list, or the final element is not AffinityStrategies.ANY.