FIX Session Layer Online
Technical Specification
Table of Contents
Scope
The Financial Information eXchange session layer is used to provide reliable and recoverable messaging for electronic trading. The protocol is intended for use by asset managers, trading firms, brokerages, trading venues, clearing houses, custodians, depositories, asset servicers, among others involved in the trading life cycle activities of a wide range of financial instruments. The FIX session layer functionality is a realization of the ISO/IEC 7498-1:1994 Open System Interconnection basic reference model level 5 session layer.
Normative references
The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.
— IETF RFC 2119 – Key words for use in RFCs to Indicate Requirement Levels March 1997
— Financial Information eXchange – FIX TagValue Encoding Technical Specification
— FIXS Technical Specification – Draft Standard https://staging.fixtrading.org/standards/fixs/
— EP113 CME Trading System Identification https://staging.fixtrading.org/packages/ep113/
— EP124 NGM Logon Extension https://staging.fixtrading.org/packages/ep124/
— EP167 AttachmentGrp Proposal https://staging.fixtrading.org/packages/ep167/
Terms and definitions
General terms and definitions
session layer message
A message carried over the FIX session that is integral to the operation of the FIX session.
application message
A message that is carried over a FIX session to accomplish some business purpose, such as an order to buy or sell a financial instrument, reporting market data, reporting an execution of a trade.
message type
An identifier (code) that defines the type of message being sent. The message type for the FIX session is a case sensitive string encoded in the MsgType(35) field.
valid FIX message
A session or application message that is a tagvalue encoded string of octets that is properly formed according to the FIX tagvalue encoding specification.
FIX session processor
A combination of computer hardware, firmware, and software that implements the FIX session layer. Commonly referred to as a FIX Engine.
initiator
The FIX session processor that establishes the transport layer connection and initiates the session via transmission of the initial Logon(35=A) message.
acceptor
The FIX session processor that is able to establish a transport layer connection and receive Logon(35=A) requests from FIX session initiators to start or resume a FIX session.
rules of engagement
A specification, usually provided in document form, that describes a specific use of FIX. Often referred to as a FIX service offering. FIX Orchestra is a standard that may be used to specify a machine readable rules of engagement.
peer
A peer is one of the FIX session processors being communicated with over a FIX session. The peer of the initiator is the acceptor. The peer of the acceptor is the initiator. The initiator and acceptor are peers.
counterparty
The firms or legal entities or individuals agreeing to use the FIX session layer to conduct some form of business endeavor.
NextNumIn
Each FIX session processor must keep track of the next message sequence number it is expecting to receive from its peer to guarantee ordered delivery of messages over the life of a FIX session that may span multiple FIX connections. The next expected incoming sequence number (NextNumIn) is compared to the value in the MsgSeqNum(34) field in each message received from the peer.
NextNumOut
Each FIX session processor must keep track of the next outbound sequence number (NextNumOut) it will send to its peer over a FIX connection.
retransmission
To retransmit a message that was previously sent in order to resynchronize the FIX session. A retransmitted message uses the original MsgSeqNum(34) value with PossDupFlag(43)=Y.
resend
The process of an application layer resending an application message because it has not received an application layer acknowledgement for the message. A resend may only be initiated by the application layer, not the session layer. The determination if the message was previously received is the responsibility of the application layer, not the session layer.
gap fill
The process to resolve gaps in message sequence numbers within a FIX session.
application version
The FIX session layer provides fields to communicate versions of the application layer messages. The FIX Trading Community defines standard application versions.
extension pack
An extension pack is an approved addition to the FIX standard. The granularity of an extension pack may vary widely from a single enumeration value addition to the definition of entire new categories of messages. Extension packs are identified by a sequential integer number and must be applied in order. An Extension pack is considered available for use if it has been approved and published by the FIX Global Technical Committee. Extension packs are created on an as needed basis and are generally driven by community requests. An extension pack is cumulative, in that the artefacts (Orchestra file, FIXimate) include all previous extension packs. When an extension pack is published, it becomes the FIX Latest version of FIX.
session profile
Session profiles are extensions or restrictions on the use of the standard session layer messages that can be used to represent context of usage. FIX4, FIXT, and LFIXT are the current FIX session profiles.
CompID
An alphanumeric identifier for the entity associated with a FIX session. As this is likely a financial markets participant company, the name was viewed colloquially as a company identifier abbreviated as a CompID.
SubID
A subidentifier optionally available to identify a subentity within a CompID. The SubID may be used to identify a specific trader or a subunit of a business entity. The use of SubIDs is at the discretion of the counterparties. Certain regulatory regimes globally require the use of SubID to identify specific traders.
LocationID
A location identifier providing additional location information either geographical or within a trading desk on a trading floor. The use of LocationIDs is at the discretion of the counterparties.
transport layer connection
A FIX session relies on a transport layer to provide for ordered delivery of messages and message recovery during the life of the transport layer connection. The FIX session does not require a specific transport layer, although TCP/IP is widely used and is a de facto standard transport layer for FIX sessions. TCP/IP provides an ordered reliable delivery of a stream of bytes during the life of the TCP connection. The FIX session processor reads this stream identifying FIX message boundaries.
in-band communication
Transmitting control information or metadata about the application layer or session layer in application and session FIX message types over the FIX session. Sending the HeartBtInt(108) field in the Logon(35=A) message is an example of in-band transmission of control information. Providing version information about a FIX service in the CstmApplVerID(1129) on the Logon(35=A) message is another example.
out-of-band communication
Exchanging control information or metadata about a FIX session via a separate communication mechanism than the FIX session. Providing the TestRequestThreshold in a rules of engagement document is an example of exchanging information out-of-band. Providing version information about a FIX service on a website or in a rules of engagement document is another example of out-of-band communication. Providing access to a specification online via a website or web service would be another example of out-of-band communication. Even though this is electronic communication, it is not being done over the FIX session to which that information applies.
TestRequestThreshold
The amount of time expressed as a multiplier of the heart beat interval before a TestRequest(35=1) message is sent to the peer when the heartbeat interval has been exceeded without receiving a message from the peer. This value may be specified out-of-band in a rules of engagement.
SendingTimeThreshold
The amount of time expressed in seconds in which the SendingTime(52) value in an inbound message differs from the system time available to the receiving FIX session processor. This value may be specified out-of-band in a rules of engagement.
LogoutAckThreshold
The amount of time expressed in seconds that a FIX session processor that has transmitted a Logout(35=5) request will wait for the Logout(35=5) acknowledgement before terminating the transport layer connection. This value may be specified out-of-band in a rules of engagement.
FIX session
A FIX session is a bidirectional stream of ordered messages between two peers within a continuous sequence number series beginning with 1. A single FIX session can exist across multiple sequential (not concurrent) FIX connections, which means that peers may intentionally or unintentionally connect and disconnect multiple times while maintaining a single FIX session. The FIX session can be thought of as a bi-directional durable session sharing characteristics of the guaranteed delivery and durable subscriber enterprise integration architecture patterns.
A FIX connection consists of three parts: logon process, message exchange (inclusive of resynchronization of state), and possible logout process over a transport layer connection. A FIX connection may be concluded by the unrecoverable loss of the transport layer, a system failure, or an application failure.
Connecting parties shall bilaterally agree upon the time the FIX session is started and the duration of a FIX session.1 A FIX session is often configured to correspond to a certain period of time, such as a trading day, calendar day, or a trading session. A FIX session may extend beyond multiple periods by counterparty agreement.
The FIX session is based on an optimistic model. Normal delivery of data is assumed (i.e. no session layer acknowledgement of individual messages) with errors in delivery identified by message sequence number gaps.
Sequence numbers
All messages sent over a FIX session shall be identified by a unique sequence number in the MsgSeqNum(34) field.
A FIX session shall start with a next expected outgoing sequence number (NextNumOut) of 1 and a next expected incoming sequence number (NextNumIn) of 1.
A FIX session processor must maintain the NextNumOut and NextNumIn for the entire FIX session.
A FIX session processor shall persist the NextNumOut and NextNumIn in order to support FIX session recovery across multiple FIX connections.
Resetting NextNumOut to 1 and NextNumIn to 1, for whatever reason, shall constitute the beginning of a new FIX session.
FIX session layer and application layer messages shall share the same sequence number space.
Each FIX session layer and application layer message sent consumes the next outbound sequence number, incrementing NextNumOut by 1.
Each FIX session layer and application layer message received consumes the next inbound sequence number, incrementing NextNumIn by 1.
A FIX session must always have peer1.NextNumIn <= peer2.NextNumOut and peer2.NextNumIn <= peer1.NextNumOut to be considered in a valid and recoverable state.
Identifying the FIX session
A FIX session is identified by the unique combination of BeginString(8) + initiator CompID + acceptor CompID.
The FIX session profile
BeginString(8) shall be used to identify the FIX session profile or version of FIX.
These FIX session profiles are defined within this specification.
FIX session profile | BeginString(8) | Description |
---|---|---|
FIX.4.2 | FIX.4.2 | The FIX session profile for use with the FIX.4.2 application layer. |
FIX4 | FIX.4.4 | The FIX session profile backward compatible with FIX.4.4 recommended when counterparties will only be using a single application version during the FIX session, such as FIX Latest. |
FIXT | FIXT.1.1 | The FIX session profile that must be used when mixing multiple application versions over the same FIX session. May be used with a single application version of FIX such as FIX Latest. |
LFIXT | FIXT.1.1 | Lightweight FIXT restricted session layer message recovery2 to simplify the protocol while maintaining compatibility with FIXT when using LFIXT compatible model of operation. |
FIX session acceptor may be configured to support multiple FIX session profiles (or FIX versions prior to FIX.4.4) over the same transport layer with only one FIX session profile (or FIX version prior to FIX.4.4) being used per FIX session.
The FIX session acceptor should use the incoming Logon(35=A) request BeginString(8) to identify the FIX session profile or FIX version being requested by the initiator.
Identification of FIX session peers
FIX relies on alphanumeric strings known as CompIDs to identify the initiator and the acceptor of FIX messages.
Counterparties must agree to their respective CompID values, which act as an identifier for the peer.
Each message sent over a FIX session layer must contain the sending entity in SenderCompID(49) and the receiving (destination) entity in TargetCompID(56). This requirement applies to both session layer and application layer messages.
FIX service processors must validate the SenderCompID(49) and TargetCompID(56) of each message and ensure that it is identical to the values present in the FIX connection Logon(35=A) message. A discrepancy in the SenderCompID(49)+TargetCompID(56) pair should result in the termination of the FIX connection by sending a Logout(35=A) message using the Text(58) field to indicate the error, followed by termination of the transport layer connection.
Validation of SendingTime(52)
SendingTime(52) must be set to the time the message is transmitted by the FIX session processor, not the time the message was queued for transmission.
SendingTime(52) must be in UTC (Universal Time Coordinated).
SendingTime(52) should be within the reasonable time of a synchronized time source of the receiving FIX session processor. This SendingTimeThreshold is highly dependent upon the type of application using the FIX session layer. The SendingTimeThreshold may be as low as seconds for high volume, low latency applications up to several minutes for certain order routing applications.
The SendingTimeThreshold may be defined and communicated to counterparties out-of-band within the rules of engagement.
The receiving peer shall transmit a Reject(35=3) message with SessionRejectReason(373) set to 10 (SendingTime Accuracy Problem) followed by a Logout(35=5) request when the SendingTime(52) is not within a specified tolerance of a synchronized clock.
Additional fields available for peer identification
The FIX session layer provides additional fields which may be used to further identify recipients within the respective counterparties.
Each counterparty may provide a subidentifier (SenderSubID(50) and TargetSubID(57)) known as SubID.
Each counterparty may provide a location identifier (SenderLocationID(142) and TargetLocationID(143)) known as LocationID.
The SubID and LocationID may vary across messages across the same FIX session.
The counterparties must agree upon the context and semantics of the SubID and LocationID if used.
Establishing a FIX connection
Establishing a FIX connection involves three distinct operations: creation of a transport layer connection, acceptance with optional authentication of the initiator by the acceptor and message synchronization (initialization).
The initiator shall establish a transport layer connection with the acceptor to establish a FIX connection.
Transport layer requirements
FIX session layer requires that the transport layer provides ordered and lossless message delivery and full duplex operation for the life of the transport layer connection.
FIX implementations that use the TCP/IP protocol as the transport layer must use the FIX-over-TLS (FIXS) specification which specifies the use of Transport Layer Security (TLS) with the FIX session layer to provide transport layer encryption.
The initiator shall send a Logon(35=A) request message to initiate a FIX connection.
The Logon(35=A) request message must always be the first message transmitted over a FIX connection. If the acceptor receives anything other than a valid Logon(35=A) request message, an error should be logged and the transport layer connection terminated without Logout(35=5) processing.
Using the TestMessageIndicator(464) to explicitly identify testing
Counterparties should populate the TestMessageIndicator(464) field with “Y” in the Logon(35=A) request and acknowledgement when testing.
FIX counterparties should validate that the TestMessageIndicator(464) field corresponds to the environment.
If the TestMessageIndicator(464) with value “Y” is present in a Logon(35=A) message indicating a test session and the environment is not a test environment, the peer should transmit a Logout(35=5) message with the Text(58) indicating that the TestMessageIndicator(464) was set to “Y” and the environment is a production environment. Certain venues support sending test messages within the production environment. Counterparties should communicate how testing is conducted within the production environment to prevent test messages from being accepted as production messages.
If the TestMessageIndicator(464) with value “N” is present in a Logon(35=A) message indicating a production session and the environment is a test environment, the peer should transmit a Logout(35=5) message with the Text(58) indicating that the TestMessageIndicator(464) was set to “N” and the environment is a test environment.
Failure to validate the TestMessageIndicator(464) against the environment may potentially lead to financial losses.
Application layer encryption
The use of application layer message encryption has been discouraged since the FIX.4.3 version of the FIX protocol.
EncyptionMethod(98) must be present in the Logon(35=A) message and set to “0” when application layer encryption is not in use.
Firms may implement application layer encryption by counterparty agreement.
Heartbeat interval
The HeartBtInt(108) field must be present and specify the interval in seconds for generating heartbeat messages during periods of inactivity.
The HeartBtInt(108) field must be present in the Logon(35=A) message.
The HeartBtInt(108) value must be agreed upon by the initiator and the acceptor.
The initiator and acceptor must use the same HeartBtInt(108) value within a FIX connection.
The HeartBtInt(108) value is the maximum time allowed between receipt of messages from the peer.
Each FIX session processor should implement a heartbeat interval timer. The timer should be reset upon receipt of each message from the peer.
Heartbeat interval determination
The acceptor should convey the rules placed on the expected heartbeat interval via out-of-band rules of engagement3 when such rules are required by the acceptor.
There are three methods for determining the heartbeat interval:
- Acceptor requires a specific heartbeat interval
- Acceptor requires initiator to specify a value within a heartbeat interval range
- Acceptor accepts the initiator specified heartbeat interval
Acceptor requires a specific heartbeat interval
The acceptor may refuse to establish a FIX connection if the HeartBtInt(108) value received on the Logon(35=A) message sent by the initiator does not match the heartbeat interval required by the acceptor.
The acceptor must terminate the FIX session using a Logout(35=5) message when the acceptor refuses to establish a FIX connection due to the inbound HeartBtInt(108) value not matching the expected value.
The acceptor should provide the reason for terminating the FIX connection in the Text(58) field of the Logout(35=5) message by setting it to “Invalid HeartBtInt(108), expected value N seconds”, where N is the number of seconds expected in the HeartBtInt(108) field on the initiator Logon(35=A) message. N shall be larger than zero.
Acceptor requires initiator specify a value within a heartbeat interval range
The acceptor may refuse to establish a FIX connection if the HeartBtInt(108) value received on the Logon(35=A) message sent by the initiator does not fall within a heartbeat interval range required by the acceptor.
The acceptor must terminate the FIX connection using a Logout(35=5) message when the acceptor refuses to establish a FIX session due to the inbound HeartBtInt(108) value not falling within the required heartbeat interval range.
The acceptor should specify the reason for terminating the connection in the Text(58) field of the Logout(35=5) message by setting it to “Invalid HeartBtInt(108), expected value between N and M seconds”, where N and M are the start and end values of the range of the heartbeat interval in seconds permitted by the acceptor. N shall be larger than zero and M shall be larger than N.
Acceptor accepts the initiator specified heartbeat interval
The acceptor may accept the heartbeat interval specified by the initiator in the HeartBtInt(108) field of the initiator Logon(35=A) message by echoing back the heartbeat interval of the initiator in the Logon(35=0) acknowledgement.
Maximum message size
The maximum message length in octets that the FIX session processor (initiator) is able to receive may be specified in the MaxMessageSize(383) field in the Logon(35=A) message sent to the peer (acceptor). The MaxMessageSize(383) value may be used to control message fragmentation at the application layer for those application messages that support fragmentation.
The acceptor may terminate the FIX session if the MaxMessageSize(383) received from the initiator is not sufficient. Termination should occur by sending a Logout(35=5) message and setting the Text(58) field to “MaxMessageSize(383) = InboundValue < required message size M” (where InboundValue is the value received from the initiator and M is the value required by the acceptor).
The values for MaxMessageSize(383) may be different between peers. If the value received from the initiator is sufficient for the acceptor then the acceptor responds with a Logon(35=A) message and can use the MaxMessageSize(383) field to specify the maximum message length in octets that the acceptor is able to receive. This value may be different from the value previously sent by the initiator. The initiator may then terminate the FIX session as described above if the MaxMessageSize(383) received from the acceptor is not sufficient.
Specifying application version
Three fields are provided to identify the application version to counterparties in the Logon(35=A) message. The use of these fields varies across FIX session profiles. For instance, these fields are not applicable to the FIX.4.2 session profile as the FIX.4.2 session profile may only be used with the FIX 4.2 application version.
Field | Definition |
---|---|
DefaultApplVerID(1137) | The FIX application version upon which the application layer is derived. |
DefaultApplExtID(1407) | An extension pack that is used with the application layer. |
DefaultCstmApplVerID(1408) | User defined custom application version information. DefaultCstmAppVerID(1400) may be used to identify the FIX Orchestra machine readable rules of engagement. |
Specifying supported message types
The initiator and acceptor may specify the types of FIX messages by using the repeating group MsgTypeGrp in the Logon(35=A) message.
Identification of application system and FIX session processor4
By counterparty agreement, peers may identify the application system as well as the FIX session processor (“FIX Engine”) in use by their respective name, version, and vendor as part of the Logon(35=A) message in the ApplicationSystemName(1603), ApplicationSystemVersion(1604), ApplicationSystemVendor(1605), FIXEngineName(1600), FIXEngineVersion(1601), and the FIXEngineVendor(1602) fields.
Responding to a request to establish a FIX session
The acceptor should authenticate the identity of the initiator by examining the incoming Logon(35=A) message, referred to as Logon(35=A) request.
The Logon(35=A) message should contain the data necessary to support the authentication method chosen by counterparty agreement.
The following fields are available for exchanging credentials for authentication:
Field | Use |
---|---|
Username(553) | Clear text userid |
Password(554) | Clear text password – use is not recommended |
NewPassword(925) | Clear text new password – use is not recommended |
EncryptedPasswordMethod(1400) | Method of encryption used for password field |
EncryptedPasswordLen(1401) | Length in octets of EncryptedPassword(1402) |
EncryptedPassword(1402) | Encrypted password |
EncryptedNewPasswordLen(1403) | Length in octets of EncryptedNewPassword(1403) |
EncryptedNewPassword(1404) | Encrypted new password permits changing password during logon |
RawDataLength(95) | Length in octets of RawData(96) |
RawData(96) | Binary octet stream that can be used for authentication |
SessionStatus(1409) may be used by counterparty agreement to communicate session status regarding authentication on both Logon(35=A) and Logout(35=5) messages.5
The acceptor shall respond with a Logon(35=A) message to accept the FIX connection initiation when the request is accepted, referred to as the Logon(35=A) acknowledgement.
The session acceptor should send a Logout(35=5) message to reject FIX connection initiation and provide a reason for the rejection in the Text(58) field and then immediately terminate the transport layer connection.
The initiator should not transmit any application message until the Logon(35=A) acknowledgement has been received from the acceptor (see the next section for further recommendations).
Initial synchronization of messages in a FIX connection
The acceptor shall compare MsgSeqNum(34) in the Logon(35=A) request received from the initiator with NextNumIn maintained by the acceptor.
The initiator shall compare MsgSeqNum(34) in the Logon(35=A) acknowledgement received from the acceptor with NextNumIn maintained by the initiator.
If MsgSeqNum(34) is present in the incoming Logon(35=A) message and equal to NextNumIn, then normal processing may commence over the FIX connection.
If MsgSeqNum(34) is present in the incoming Logon(35=A) message and greater than NextNumIn, then message recovery must be performed.
If MsgSeqNum(34) is present in the incoming Logon(35=A) message and is less than NextNumIn, then a Logout(35=5) message should be sent assuming that an error exists in the state of either the initiator or acceptor FIX session processor, followed by a termination of the transport layer connection.
The initiator and acceptor should wait a short period of time following receipt of the Logon(35=A) message from the counterparty before transmitting queued or new application messages to permit both sides to synchronize the FIX session.
Alternatively, the initiator and acceptor may send a TestRequest(35=1) message and wait for the HeartBeat(35=0) message sent in response to the TestRequest(35=1) message, before sending queued or new messages, as this practice forces both sides to perform message synchronization.
Synchronization after successful logon
The following sequence diagrams show three scenarios.
- acceptor requires retransmission of messages from initiator
- initiator requires retransmission of messages from acceptor
- both acceptor and initiator require retransmission of messages
Extended features for FIX session and FIX connection initiation
Using NextExpectedMsgSeqNum(789) to synchronize a FIX session
The FIX session processors may use the NextExpectedMsgSeqNum(789) field in the Logon(35=A) message to synchronize a FIX session by counterparty agreement defined in the FIX session rules of engagement6.
The time required to resume a FIX session over a new FIX connection can be reduced by alerting the peer to the next message that is expected using NextExpectedMsgSeqNum(789) in the Logon(35=A) message. Any missing messages can be retransmitted by the peer without requiring to use ResendRequest(35=2) messages.
A FIX session may implement next expected sequence number processing to reduce the amount of message retransmission needed to re-establish a FIX session by counterparty agreement specified in the rules of engagement.
Each FIX session processor may populate the NextExpectedMsgSeqNum(789) with the next message sequence number expected from its peer on the Logon(35=A) when establishing a FIX connection.
Peers should not generate a ResendRequest(35=2) message based on MsgSeqNum(34) of the incoming Logon(35=A) message but should expect any gaps to be filled automatically via the following process.
The receiving FIX session processor receiving a Logon(35=A) message with NextExpectedMsgSeqNum(789) present shall compare the value of NextExpectedMsgSeqNum(789) with NextNumOut.
- If the received NextExpectedMsgSeqNum(789) is equal to NextNumOut, proceed sending new messages beginning with that number.
- If the received NextExpectedMsgSeqNum(789) is less than NextNumOut, perform message recovery for messages starting with the message with MsgSeqNum(34) equal to the NextExpectedMsgSeqNum(789) through to the message with MsgSeqNum(34) equal to NextNumOut – 1.
- If the received NextExpectedMsgSeqNum(789) is greater than NextNumOut, send a Logout(35=5) message to end the session with the Text(58) field populated explaining that “NextExpectedMsgSeqNum(789) > than last message sent”.
Using ResetSeqNumFlag(141) to reset FIX session for 24 hour connectivity
The FIX session may be reset to NextNumIn=1 and NextNumOut=1 over an active FIX session. This capability is provided for continuously operating markets that need to periodically reset the FIX session. When using the ResetSeqNumFlag(141) to maintain 24 hour connectivity and establish a new set of sequence numbers, the process should be as follows.
Counterparties must agree to the time of day when the FIX session reset will be initiated.
Counterparties must agree which peer will initiate the FIX session reset.
The peer that initiates the FIX session reset may send a TestRequest(35=1) message and wait for the HeartBeat(35=0) response to ensure that there are no message gaps prior to sending a Logon(35=A) message to reset the FIX session.
The peer initiating the FIX session reset should set its NextNumIn to 1 and NextNumOut to 1, then send a Logon(35=A) message with ResetSeqNumFlag(141) set to “Y” and MsgSeqNum(34) set to 1.
The peer receiving the Logon(35=A) message should set its NextNumIn to 2 and NextNumOut to 1, then send a Logon(35=A) acknowledgement with ResetSeqNumFlag(141) set to “Y” and with MsgSeqNum(24) set of 1.
Upon completion of the session reset, both peers must have NextNumIn = 2 and NextNumOut = 2.
If the peer is not configured to accept resetting of an inbound session the peer should send a Logout(35=5) with Text(58) indicating that resetting the sequence number is not supported and then terminate the transport layer connection.
Using ResetSeqNumFlag(141) to reset FIX session during FIX connection establishment7
The initiator of a FIX session may reset the FIX session upon initial logon via sending the Logon(35=A) with ResetSeqNumFlag(141) set to “Y”.
Counterparties must agree to support the reset of a FIX session during FIX connection establishment.
Operational risks, such as order overfills, could occur if both parties are not configured to support resetting the FIX session upon initial logon. Users of the FIX session layer are warned to take special care when permitting the FIX sessions to be reset upon FIX connection establishment.
If the acceptor is not configured to accept resetting of a FIX session during FIX connection establishment, the acceptor should send a Logout(35=5) with Text(58) indicating that resetting the sequence number upon FIX connection establishment is not supported then terminate the transport layer connection.
Using initiator state to restore acceptor session state8
An acceptor may re-establish its state via the exchange of NextExpectedMsgSeqNum(789).
This should only be used for FIX service offerings where application message recovery is performed by the application layer instead of the FIX session layer.
The state of the FIX session is assumed to be:
- the acceptor FIX session processor NextNumOut is unknown and NextNumIn is unknown.
- the initiator FIX session processor has maintained and knows its state (the values of NextNumIn and NextNumOut) between FIX connections.
Acceptor state may be recovered when NextExpectedMsgSqNum(789) is present in the Logon(35=A) request by:
- The acceptor may reconstruct its session state solely based upon the Logon(35=A) request from an initiator if NextExpectedMsgSeqNum(789) is present in the Logon(35=A) request.
- The acceptor should set its NextNumIn to the MsgSeqNum(34) + 1 from the Logon(35=A) request and set its NextNumOut to the NextExpectedMsgSeqNum(789) from the Logon(35=A) request.
- The acceptor should then send a SequenceReset(35=4) message with GapFillFlag(123) set to “Y” and NewSeqNo(36) set to NextNumOut+1 and AppLevelIndicator(1744) set to “1” (application layer recovery is needed) to inform the initiator that application messages will need to be recovered.
Acceptor state may be recovered when NextExpectedMsgSeqNum(789) is not present in the Logon(35=A) request by:
- The acceptor may reconstruct its session state based upon the Logon(35=A) request from an initiator by setting its NextNumIn to the MsgSeqNum(34) + 1 from the Logon(35=A) request and setting its NextNumOut to 1 when the NextExpectedMsgSeqNum(789) is not present in the Logon(35=A) request.
- The initiator should respond to the Logon(35=A) acknowledgement with a Logout(35=5) with SessionStatus(1409) set to 9 (received MsgSeqNum(34) is too low) and NextExpectedMsgSeqNum(789) set to the initiator’s NextNumIn, followed by terminating the transport layer connection.
- The acceptor should set its NextNumOut to the NextExpectedSeqNum(789) value from the Logout(35=5) received from the initiator.
- The initiator should then transmit a new Logon(35=A) request with MsgSeqNum(34) set to its NextNumOut value.
- The acceptor should respond with Logon(35=A) acknowledgement with the MsgSeqNum(34) set to the NextNumOut value obtained from the previous initiator Logout(35=5).
- The acceptor should then send a SequenceReset(35=4) message with GapFillFlag(123) set to “Y” and NewSeqNo(36) set to NextNumOut+1 and AppLevelIndicator(1744) set to “1” (application layer recovery is needed) to inform the initiator that application messages may need to be recovered.
Message exchange during a FIX connection
After establishing a FIX connection within a FIX session and synchronizing messages, normal message exchange begins.
The initiator and acceptor may send a TestRequest(35=1) message and wait for the HeartBeat(35=0) message sent in response to the TestRequest(35=1) message, before sending queued or new messages, to further confirm that the FIX connection is established.
The initiator and acceptor shall validate incoming FIX messages to ensure proper tagvalue encoding.
The integrity of message data content shall be verified for message length and the tagvalue encoding checksum according to the FIX tagvalue encoding specification.
The initiator and acceptor shall synchronize their messages and detect gaps by comparing the MsgSeqNum(34) field in each message received from its peer with the NextNumIn it maintains and increments after successful receipt of each message.
A FIX session processor that encounters a sequence number gap shall perform message recovery.
FIX connection keep alive (heartbeat)
The FIX session processor must send a HeartBeat(35=0) message to the peer during periods of inactivity, ensuring that the HeartBeat(35=0) message will be received by the peer within the heartbeat interval.
If a FIX session processor does not receive a message within the heartbeat interval, test request processing should be initiated.
Garbled message processing
A message shall be considered garbled if any of the following occur as a result of encoding, decoding, or transmission errors:
- BeginString(8) is not the first tag in a message or is not one of the defined FIX session profile identifiers (“8=FIX.4.4”, “8=FIXT.1.1”).
- BodyLength(9) is not the second tag in a message or does not contain the correct byte count.
- MsgType(35) is not the third tag in a message.
- Checksum(10) is not the last tag or contains an incorrect value.
The FIX session layer presumes that a garbled message is received due to a transmission error rather than a FIX session processor defect and should be recoverable from the peer.
The receiving FIX session processor must disregard the garbled message and not increment NextNumIn.
The FIX session processor should log each encountered garbled message to assist in problem detection and diagnosis.
Receipt of the next valid FIX message after a garbled message was ignored will result in the detection of a sequence gap as the MsgSeqNum(34) will be greater than NextNumIn which should then cause the receiving FIX session processor to initiate message recovery.
It is recommended that FIX session processors implement logic to recognize the possible infinite retransmission loop, which may be encountered when a garbled message is repeatedly retransmitted due to a fault in the peer FIX session processor.
Missing sequence number
If MsgSeqNum(34) is missing, a Logout(35=5) message should be sent, terminating the FIX connection with the Text(58) field describing the missing field, as this likely indicates a serious application error that is likely only circumvented by software modification.
Rejecting invalid messages
Incoming messages should be rejected using the Reject(35=3) message when:
- an incoming message is not garbled but does not contain the appropriate required fields may be rejected by sending a Reject(35=3) message to the peer.
- MsgType(35) of the incoming message is unknown or not supported.
As a rule, messages should be forwarded to the trading application for business level rejections whenever possible instead of being rejected at the session layer.
Rejected messages must be logged and NextNumIn incremented by 1.
Generation and receipt of a Reject(35=3) message indicates a serious error that could be the result of faulty logic or noncompliance with the rules of engagement in either peer.
A diagnostic message describing the error should be provided in the Text(58) field of the Reject(35=3) message and the SessionRejectReason(373) field set to the appropriate reject reason.
The MsgSeqNum(34) from the message being rejected should be returned in RefSeqNum(45).
The RefTagID(371) should be set to the tag of the field associated with the reject reason.
The RefMsgType(372) field should be the MsgType(35) of the rejected message.
If the sending application chooses to retransmit the rejected message, the message being resent must be assigned a new MsgSeqNum(34) and sent with PossResend(97) set to “Y”.
Test Request Processing
A TestRequest(35=1) is sent to determine the status of the peer FIX session processor.
The TestReqID(112) field must be present in the TestRequest(35=1) message.
The recipient of a TestRequest(35=1) message must respond by sending a HeartBeat(35=0) message with TestReqID(112) present and set to the TestReqID(112) value from the TestRequest(35=1) message.
Note: Depending upon the implementation of the FIX session processor it may be possible to receive HeartBeat(35=0) messages from the peer even though the peer FIX session processor is not functioning properly. A TestRequest(35=1) may be sent to determine if the peer FIX session processor is still responsive. Not receiving a valid response to the TestRequest(35=1), which is a HeartBeat(35=0) message containing the value of the TestReqID(112) from the TestRequest(35=1) message, provides some evidence that the peer is not functioning properly.
Failure to receive a HeartBeat(35=0) response with TestReqID(112) present and set to the value from the TestRequest(35=1) message is considered an error. The FIX connection should be terminated by sending a Logout(35=5) message with the Text(58) field stating that the session was terminated due to lack of a response to a TestRequest(35=1) message, followed by terminating the transport layer connection.
The period of time that the FIX session processor should wait for the HeartBeat(35=0) response should be defined as a TestRequestThreshold multiplied by the heartbeat interval.
A reasonable TestRequestThreshold is anywhere from 1.2 to 2.0 depending on latency sensitivity of the application using the FIX session layer.
The TestRequestThreshold may be specified by counterparty agreement or specified in the rules of engagement9.
The operator of the FIX session processor may send a TestRequest(35=1) message at any time during a FIX connection. The following figure shows an example of a TestRequest(35=1) message issued to determine if the peer is still responsive.
FIX connection termination
Either the initiator or the acceptor may send a Logout(35=5) message to request termination of a FIX connection. Normal termination of the message exchange session shall be completed via the exchange of Logout(35=5) messages. Termination by other means should be considered an abnormal condition and dealt with as an error. FIX connection termination without receiving a Logout(35=5) message should treat the peer as logged out.
A FIX session processor may send a TestRequest(35=1) message before sending the Logout(35=5) message to force a heartbeat from the other side. This helps to ensure that there are no sequence number gaps, which indicate that a ResendRequest(35=2) message is necessary before completing the session.
The Logout(35=5) initiator must wait for the peer to respond with a Logout(35=5) acknowledgement message before terminating the FIX connection. This gives the peer a chance to perform any gap fill operations that may be necessary. Once the messages from the ResendRequest(35=2) message have been received, the peer should issue the Logout(35=5) message.
The Logout(35=5) initiator should terminate the FIX connection if the peer does not respond within a reasonable time frame. Twice the value specified in the HeartBtInt(108) field of the Logon(35=A) message is recommended.
Normal logout processing
Logout without acknowledgement from peer
Logout with retransmission of missed messages
The impact of a FIX connection termination on the application layer is left to the application layer; it is undefined in the FIX session layer.
When to terminate a FIX connection by terminating the transport layer connection instead of sending a Logout(35=5)
In general, a Logout(35=5) message should always be sent prior to shutting down a connection. If it is being sent due to an error condition, the Text(58) field of the Logout(35=5) should provide a descriptive reason, so that operational support of the remote FIX system can diagnose the problem. There are exceptions, when it is recommended that a Logout(35=5) message not be sent, these include:
- The FIX session should be immediately terminated by terminating the transport layer connection without sending a Logout(35=5) message when the BeginString(8), SenderCompID(49), TargetCompID(56), or IP address in the Logon(35=A) request of the session initiator is invalid. This logon attempt could be an unauthorized access attempt. Terminating the connection without transmitting a Logout(35=5) message avoids divulging information about the SenderCompID(49)/TargetCompID(56) and the version of FIX being used over the transport layer.
- If, during a logon, one receives a second connection attempt while a valid FIX session is already underway for that same SenderCompID(49), the session acceptor should immediately terminate the second connection attempt and not send a Logout(35=5) message. Sending a Logout(35=5) message runs the risk of interfering with and possibly adversely affecting the current active FIX connection. For example, sending a Logout(35=5) message should consume a sequence number that would cause an out of sequence condition for the established FIX session.
In all other cases, if sending a Logout(35=5) message does not create risk or violate security, it should be sent with a descriptive text message in the Text(58) field.
Extended features for FIX connection Termination
Using NextExpectedMsgSeqNum(789) when terminating FIX connection due to invalid MsgSeqNum(34)10
When a FIX session processor receives a Logon(35=A) message with a MsgSeqNum(34) that is less than the NextNumIn from a peer, a Logout(35=5) message may be sent that includes the NextExpectedMsgSeqNum(789) set to NextNumIn and the SessionStatus(1409) set to 9 (received MsgSeqNum(34) is too low). This approach permits the peer to reset their NextNumOut to reestablish the FIX session.
When a FIX session processor receives a Logon(35=A) message with a NextExpectedMsgSeqNum(789) value that is greater than NextNumOut from a peer, a Logout(35=5) message may be sent that includes the SessionStatus(1409) set to 10 (received NextExpectedMsgSeqNum(789) is too high), then terminate the transport layer connection. This approach allows the peer to reset their NextNumIn to the MsgSeqNum(34) present in the Logout(35=A) message + 1.
When a FIX session processor receives a Logon(35=A) message with a MsgSeqNum(34) that is less than the NextNumIn and a NextExpectedMsgSeqNum(789) value that is greater than NextNumOut from a peer, a Logout(35=5) message may be sent that includes SessionStatus(1409) equal 9 (received MsgSeqNum(34) is too low) and the NextExpectedMsgSeqNum(789) set to NextNumIn, then terminate the transport layer connection. This approach allows the peer to reset their NextNumIn to the MsgSeqNum(34) present in the Logout(35=5) message + 1 and the NextNumOut reset to the NextExpectedMsgSeqNum(789) value.
Message recovery
A message gap occurs when the MsgSeqNum(34) of an incoming message is greater than the NextNumIn maintained by the FIX session processor.
During initialization of a FIX connection or at any time during the FIX connection, a message gap may occur.
When the MsgSeqNum(34) on an inbound message is greater than NextNumIn, the FIX session processor must perform message recovery processing as this is an indication that there were messages sent by the peer that were not received or may have been garbled during transmission.
Ordered message processing
The FIX protocol assumes complete ordered delivery of messages processed in MsgSeqNum(34) order.
FIX session processors must detect gaps in the sequence of MsgSeqNum(34) values by maintaining the next expected sequence number NextNumIn.
If the MsgSeqNum(34) of an incoming message is greater than NextNumIn, then the receiving FIX session processor should request the peer to retransmit messages following the Request retransmission of messages specifications.
The FIX session should be terminated if the incoming message MsgSeqNum(34) is less than NextNumIn and PossDupFlag(43) is not set to “Y”, except if the message is the SequenceReset(35=4) with the GapFillFlag(123) set to “N” (Sequence Reset). A Logout(35=5) message should be sent with the Text(58) field set to indicate that the value in MsgSeqNum(34) is less than NextNumIn when PossDupFlag(43) is not set to “Y” and SessionStatus(1409) set to 9 (received MsgSeqNum(34) is too low.), followed by terminating the transport layer connection.
Request retransmission of messages
There are two alternative approaches to resolving message sequence number gaps.
The two approaches are:
- Request all messages subsequent to the last message received.For example, if the receiver misses the second of five messages, the application could ignore messages 3 through 5 and generate a resend request for messages 2 through 5, or, preferably 2 through 0 (where 0 represents all messages after message 2).
- Request only the specific messages missed while maintaining an ordered list of all newer messages.Using the above example, messages 3 through 5 would be saved for later processing and a ResendRequest(35=2) message would be sent only for message 2.
In both cases, messages 3 through 5 should not be processed before message 2.
The ResendRequest(35=2) message with BeginSeqNo(7) and EndSeqNo(16) present should be sent to the peer FIX session processor to request missing messages when a gap is detected.
The ResendRequest(35=2) may be used to request retransmission of a single message, a range of messages or all messages subsequent to a particular message.
The receiving FIX session processor must process incoming messages in MsgSeqNum(34) order.
The ResendRequest(35=2) message may be used to specify which messages require retransmission in the following ways:
- To request a single message: Both BeginSeqNo(7) and EndSeqNo(16) are set the MsgSeqNum(34) of the missing message.
- To request a range of messages: BeginSeqNo(7) is set to the MsgSeqNum(34) of the first message of the range of missing messages and EndSeqNo(16) is set to the MsgSeqNum(34) of the last message of the range of missing messages.
- To request all messages subsequent to a particular missing message: BeginSeqNo(7) is set to the MsgSeqNum(34) of the first message of range and EndSeqNo(16) is set to 0. Zero is used to specify all messages after the message with MsgSeqNum(34)=BeginSeqNo(7)11.
Responding to a ResendRequest(35=2) message
Upon receipt of a ResendRequest(35=2) message, the resender shall respond by either initiating a normal gap fill process or, in exceptional circumstances, by forcing a reset of the sequence number. In either case, the resender assumes responsibility for the results of the decision to retransmit or not retransmit messages.
The peer receiving the ResendRequest(35=2) may choose not to retransmit certain application messages. For instance, if orders or quotes require retransmission, the sender may choose not to retransmit them because too much time has elapsed, causing the orders to become stale or no longer acceptable due to a change in market state.
The SequenceReset(35=4) message with GapFillFlag(123) set to “Y” (gap fill) must be used to skip messages that a sender does not wish to retransmit in order to eliminate all gaps.
Possible duplicates
When a FIX session processor is unable to determine if a message was successfully received at its intended destination or when responding to a ResendRequest(35=2) message, a possible duplicate message is generated. The message will be a retransmission (with the same sequence number) of the application data in question with the PossDupFlag(43) included and set to “Y”. It is the responsibility of the receiving firm to properly respond to the message with PossDupFlag(43) set to “Y” by determining if the message was previously processed.
Any message resent in response ResendRequest(35=2) message must contain the PossDupFlag(43) field set to “Y”.
Messages lacking the PossDupFlag(43) field or with the PossDupFlag(43) field set to “N” shall be treated as an original transmission.
The FIX session processor retransmitting a message with the PossDupFlag(43) set to “Y” must modify the following fields:
- SendingTime(52) set to the current sending time
- OrigSendingTime(122) set to the SendingTime(52) from the original message
- Recalculate the BodyLength(9)
- Recalculate the CheckSum(10)
If the message is encrypted, SecureDataLen(90) and SecureData(91) may also require re-encryption and re-encoding12.
Gap fill process
The peer responding to a ResendRequest(35=2) message shall retransmit messages requested by the peer in message sequence number order, with the original sequence numbers and PossDupFlag(43) set to “Y”.
The resender shall not retransmit the following session layer message types:
- Logon(35=A)
- Logout(35=5)
- ResendRequest(35=2)
- HeartBeat(35=0)
- TestRequest(35=1)
- SequenceReset(35=4)
The responding peer shall send a SequenceReset(35=4) message with GapFillFlag(123) set to “Y” (gap fill) for each session layer or application layer message not retransmitted.
Reject(35=3) and XMLnonFIX(35=n) are the only session messages which may be retransmitted.
The responding peer may choose, by counterparty agreement, not to retransmit all application messages. The definition of which application messages are retransmitted is defined in the application layer.
A continuous sequence of messages not being retransmitted should be skipped over using a single SequenceReset(35=4) message with GapFillFlag(123) set to “Y” and MsgSeqNum(34) set to the sequence number of the first skipped message and NewSeqNo(36) set to the next sequence number after the sequence number of the last message to be skipped (or “gap filled over”).
NewSeqNo(36) must always be set to the value of the next sequence number to be expected by the peer immediately following the messages being skipped.
The SequenceReset(35=4) message with GapFillFlag(123) set to “Y” (gap fill) may also be used to skip application messages that the sender chooses not to retransmit (for example, aged orders).
Example using SequenceReset(35=4) to gap fill over multiple messages
The following example is provided to show how SequenceReset(35=4) should be used to gap fill over messages in response to a ResendRequest(35=2).
In this example a ResendRequest(35=2) is received with BeginSeqNo(7) set to 5 and EndSeqNo(16) set to 0 (meaning all subsequent messages following 5). Messages with MsgSeqNum(34) equal to 8, 10, 11 are application messages. Messages 5-7 and 9 are session layer messages. Only messages 8, 10, and 11 should be retransmitted.
Sequence reset
A FIX session processor may send a SequenceReset(35=4) message with GapFillFlag(123) set to “N” (Reset) and PossDupFlag(43) set to “Y” to force sequence number synchronization in the peer.
The firm sending the SequenceReset(35=4) with GapFillFlag(123) set to “N” is responsible for the results that occur at the application layer caused by resetting the sequence number and not retransmitting messages requested to be resent by the peer.
Processing inbound possible duplicate messages (PossDup(43) set to “Y”)
All FIX implementations must monitor incoming messages to detect inadvertently retransmitted session layer messages (PossDupFlag(43) set to “Y”, indicating a retransmission).
A FIX session processor, upon receipt of an inadvertently (or improperly) retransmitted session layer message as identified by the PossDupFlag(43) set to “Y”, should perform sequence number processing (increment NextNumIn) only and avoid processing the session layer message.
A message received with the PossDupFlag(43) set to “Y” should be ignored when a message with the same MsgSeqNum(34) was previously processed.
Processing gaps when receiving FIX session layer messages
Gaps detected in message sequence numbers must be processed differently for certain FIX session layer messages. The table below lists the actions to be taken when the incoming FIX session layer message has a MsgSeqNum(34) that is greater than NextNumIn.
Message type received | Action to be taken on sequence number mismatch MsgSeqNum(34) > NextNumIn |
---|---|
Logon(35=A) | Must always be the first message transmitted. Authenticate and accept the connection. After sending a Logon(35=A) acknowledgement, send a ResendRequest(35=2) message. |
Logout(35=5) request | Send ResendRequest(35=2) message requesting missing messages, receive and process retransmission of missing messages and gap fills, then send the Logout(35=5) acknowledgement message which serves as a confirmation of the Logout request. DO NOT terminate the session. The Logout(35=5) requestor should the terminate the transport layer connection. The only exception to the “do not terminate the session” rule is for an invalid Logon(35=5) request. The session acceptor has the right to send a Logout(35=5) message and terminate the transport layer connection immediately. This minimizes the threat of unauthorized connection attempts. |
ResendRequest(35=2) | Perform the requested retransmission then send a ResendRequest(35=2) message requesting missing messages, receive and process the retransmission of missing messages and gap fills. |
SequenceReset(35=4) (GapFillFlag(123)=N) | Reset: Ignore the incoming sequence number. The NewSeqNo(36) field of this message will contain the sequence number of the next message to be transmitted. |
SequenceReset(35=4) (GapFillFlag(123)=Y) | Gap fill: Send a ResendRequest(35=2). These messages behave similarly to SequenceReset(35=4) messages with GapFillFlag(123) set to “N”. However, it is important to ensure that no messages have been inadvertently skipped over. This means that these messages must be received in sequence. An out of sequence SequenceReset(35=4) message with GapFillFlag(123) set to “Y” is an abnormal condition. |
All other messages | Perform gap fill operations. |
Resending an unacknowledged application message
The application layer is responsible for the detection and proper handling of duplicate application messages.
The FIX session layer does not initiate a resend of an application message nor does the FIX session layer detect duplication application messages.
The application layer may resend a message if it has not received an application layer acknowledgement for that message.
The amount of time allowed to elapse before resending the message may be specified by counterparty agreement in a rules of engagement.
The application layer may use the PossResend(97) set to “Y” field in the StandardHeader component along with other fields in an application message to identify duplication application messages.
The use or non-use of the PossResend(97) field when resending application messages should be documented in the rules of engagement.
The resent application layer message shall consume the next value of NextNumOut resulting in a new MsgSeqNum(34) for the application message.
The FIX session processor shall treat the resent message as a new message.
The FIX session processor must pass the PossResend(97) field to the application layer when it is present in an application message.
The use of a globally unique persistent identifier that enforces idempotent behavior at the application layer may be used as an alternative to sending a PossResend(97) field by counterparty agreement.
The difference between application layer resend and session layer retransmission
The purpose of the following diagram is to visually show the difference between an application using the PossResend(97) value of a previously unacknowledged application message and a FIX session processor retransmitting messages that may not have been received by its peer as a response to a ResendRequest(35=2) message from that peer.
FIX session state matrix
Precedence | State | Initiator | Acceptor | Description |
---|---|---|---|---|
1 | Disconnected – No Connection Today | Y | Y | Currently disconnected, have not attempted to establish a connection “today”, and no MsgSeqNum(34) has been consumed (next connection “today” will start at MsgSeqNum(34) of 1). |
2 | Disconnected – Connection Today | Y | Y | Currently disconnected, have attempted to establish a connection “today” and thus MsgSeqNum(34)has been consumed (next connection “today” will start at MsgSeqNum(34) of (last + 1)). |
3 | Detect Broken Network Connection | Y | Y | While connected, detect a broken network connection (e.g. TCP socket closed). Disconnect the network connection and “shutdown” configuration for this session. |
4 | Awaiting Connection | N | Y | Session acceptor Logon awaiting network connection from counterparty. |
5 | Initiate Connection | Y | N | Session initiator Logon establishing network connection with counterparty. |
6 | Network Connection Established | Y | Y | Network connection established between both parties. |
7 | Initiation Logon(35=A) Sent | Y | N | Session initiator Logon send Logon(35=A) message.
*** Exception: 24hr sessions. |
8 | Initiation Logon(35=A) Received | N | Y | Session acceptor Logon receive counterparty’s Logon(35=A) message.
*** Exception: 24hr sessions. |
9 | Initiation Logon(35=A) Response | N | Y | Session acceptor Logon respond to counterparty’s Logon(35=A) message with Logon(35=A) message to handshake. |
10 | Handle ResendRequest(35=2) | Y | Y | Receive and respond to counterparty’s ResendRequest(35=2) sending requested messages and/or SequenceReset(35=4) gap fill messages for the range of MsgSeqNum(34) requested. Updated to include rejecting ResendRequest(35=2) received with MsgSeqNum(34) that is <= LastSeqNum processed. |
11 | Receive MsgSeqNum(34) Too High | Y | Y | Receive too high of MsgSeqNum(34) from counterparty, queue message, and send ResendRequest(35=2). |
12 | Awaiting/ Processing Response to ResendRequest(35=2) | Y | Y | Process requested MsgSeqNum(34) with PossDupFlag(43)=Y resent messages and/or SequenceReset(35=4) gap fill messages from counterparty. Queue incoming messages with MsgSeqNum(34) too high. |
13 | No messages received in Interval | Y | Y | No inbound messages (non-garbled) received in (HeartBtInt(108) + “reasonable period of time”), send TestRequest(35=1). |
14 | Awaiting/ Processing Response to TestRequest(35=1) | Y | Y | Process inbound messages. Reset heartbeat interval-related timer when ANY inbound message (non-garbled) is received. |
15 | Receive Logout(35=5) message | Y | Y | Receive Logout(35=5) message from counterparty initiating logout/disconnect. If MsgSeqNum(34) too high, send ResendRequest(35=2). If sent, wait a reasonable period of time for complete response to ResendRequest(35=2). Note that depending upon the reason for the Logout(35=5), the counterparty may be unable to fulfill the request. Send Logout(35=5) message as response and wait a reasonable period of time for counterparty to disconnect the network connection. Note that counterparty may send a ResendRequest(35=2) message if Logout(35=5) message response has MsgSeqNum(34) too high and then re-initiate the Logout(35=5) process. |
16 | Initiate Logout(35=5) Process | Y | Y | Identify condition or reason to gracefully disconnect (e.g. end of “day”, no response after multiple TestRequest(35=1) messages, too low MsgSeqNum(34), etc.). Send Logout(35=5) message to counterparty. Wait a reasonable period of time for Logout(35=5) response. During this time, handle “new” inbound messages and/or ResendRequest(35=2) messages if possible. Note that some logout/termination conditions (e.g. loss of database/message safe-store) may require immediate termination of the network connection following the initial send of the Logout(35=5) message. Disconnect the network connection and “shutdown” configuration for this session. |
17 | Active/Normal Session | Y | Y | Network connection established, Logon(35=A) message exchange successfully completed, inbound and outbound MsgSeqNum(34) are in sequence as expected, and HeartBeat(35=0) or other messages are received within (HeartBtInt(108) + “reasonable period of time”). |
18 | Waiting for a Logon(35=A) | Y | N | Session initiator waiting for session acceptor to send back a Logon(35=A) acknowledgement. |
FIX logon process state transition diagram
Session Initiator (e.g. buyside)Action |
Session Acceptor (e.g. sellside)Action |
Session Initiator (e.g. buyside)State |
Session Acceptor (e.g. sellside)State |
---|---|---|---|
Start | Disconnected – No Connection Today | Awaiting Connection | |
Disconnected – Connection Today | |||
Connect | Initiate Connection | Awaiting Connection | |
(Possible) Detect Broken Network Connection | |||
Accept Connection | Network Connection Established | Network Connection Established | |
Initiate Logon(35=A) | Initiation Logon(35=A) Sent | Network Connection Established | |
Receive Initiation Logon(35=A) | Initiation Logon(35=A) Sent | Initiation Logon(35=A) Received | |
Send Initiation Logon(35=A) Response | Initiation Logon(35=A) Sent | Initiation Logon(35=A) Response
(Possible) Initiate Logout(35=5) Process (e.g. if MsgSeqNum(34) too low) (Possible) Receive MsgSeqNum(34) Too High |
|
(Possible) Send ResendRequest(35=2) | Initiation Logon(35=A) Response
(Possible) Receive MsgSeqNum(34) Too High |
||
Receive Initiation Logon(35=A) Response | (Possible) Active/Normal Session
(Possible) Initiate Logout(35=5) Process (e.g. if MsgSeqNum(34) too low) |
Initiation Logon(35=A) Response | |
(Possible) Send ResendRequest(35=2) | (Possible) Active/Normal Session
(Possible) Receive MsgSeqNum(34) Too High |
(Possible) Active/Normal Session
(Possible) Handle ResendRequest(35=2) |
|
Active/Normal Session | Active/Normal Session |
FIX logout process state transition diagram
Logout Initiator Action | Logout Acceptor Action | Logout Initiator State |
Logout Acceptor State |
---|---|---|---|
Start | Active/ Normal Session | Active/Normal Session | |
No Messages Received in Interval | No Messages Received in Interval | ||
Awaiting/ Processing Response to TestRequest(35=1) | – Initiation Logon(35=A) Sent – Awaiting/Processing Response to TestRequest(35=1) – Awaiting validation of logon – Receive MsgSeqNum(34) Too High – Awaiting/Processing Response to ResendRequest(35=2) – Initiate Logout(35=5) Process – Waiting for a Logon(35=A) acknowledgement |
||
Send Logout(35=5) Request | Logout Pending | ||
Receive Logout(35=5) Request | Logout Pending | Logout Pending
(Possible) Receive MsgSeqNum(34) Too High |
|
Send Logout(35=5) Response | Logout Pending | Awaiting Disconnect | |
(Possible) Send ResendRequest(35=2) | Logout Pending | (Possible) Awaiting / Processing Response to ResendRequest(35=2) | |
(Possible) Receive ResendRequest(35=2) | (Possible) Awaiting / Processing Response to ResendRequest(35=2) | (Possible) Awaiting Response to ResendRequest(35=2) | |
Receive Logout(35=5) Response | Disconnected – Connection Today | Awaiting Disconnect | |
Disconnect | Disconnected – Connection Today | Disconnected – Connection Today |
FIX session profiles
The following session profiles are defined within the standard. Each session profile defines a standardized use of the FIX session layer for a specific context primarily related to application version usage.
FIX.4.2 session profile
FIX.4.2 is the FIX session profile that is used with the FIX 4.2 application layer. The FIX 4.2 application layer may use fields from subsequent versions of FIX.
The FIX.4.2 session profile may use extended features from the FIX session standard by counterparty agreement.
Profile identification
The value of BeginString(8) must be “FIX.4.2”
Application version identification
Field | Definition |
---|---|
DefaultApplVerID(1137) | Not used with FIX.4.2 session profile as FIX 4.2 is the mandatory application layer. |
DefaultApplExtID(1407) | May be used by counterparty agreement in-band. Recommendation is to communicate this information out-of-band via a rules of engagement, possibly by using a FIX Orchestra rules of engagement. |
DefaultCstmApplVerID(1408) | May be used by counterparty agreement in-band on the Logon(35=A) message to identify the version of the custom application or FIX Orchestra rules of engagement. |
FIX4 session profile
FIX4 is fully backward compatible with the FIX 4.4 session layer as defined in Volume 2 of the FIX 4.4 specification.
FIX4 adopters may use extended capabilities beyond the basic FIX 4.4 functionality by counterparty agreement.
The FIX4 session profile supports the communication of a single version of the FIX application layer.
Profile identification
The value of BeginString(8) must be “FIX.4.4”
Application version identification
The default FIX application version is FIX Latest.
Field | Definition |
---|---|
DefaultApplVerID(1137) | DefaultApplVerID(1137) may be used to specify the application version in the Logon(35=A) message. If the DefaultApplVerID(1137) is not specified the default will be assumed to be FIX Latest. |
DefaultApplExtID(1407) | May be used by counterparty agreement in-band. Recommendation is to communicate this information out-of-band via a rules of engagement possibly by using a FIX Orchestra rules of engagement. |
DefaultCstmApplVerID(1408) | May be used by counterparty agreement in-band on the Logon(35=A) message to identify the custom application or FIX Orchestra rules of engagement. |
FIXT session profile
FIXT supports transmitting messages from multiple application versions over the same session layer. This capability is the primary differentiator between the FIX4 session profile and FIXT session profile. FIXT may be used with a single application version, such as FIX Latest. The benefit in using FIXT for a single application version is it provides an option to include additional application versions at the individual message level at a future time.
Profile identification
The value of BeginString(8) must be “FIXT.1.1”.
Multiple application version support over a single FIXT session
The FIXT session profile (along with the LFIXT session profile based upon FIXT) permit application messages from different application versions of FIX to be used over a single FIX session. The benefit is to broaden service offerings, perform incremental enhancements at the individual message or message category level.
Session default application version identification
The session default application version must be present in DefaultApplVerID(1137) in the Logon(35=A) message.
The application version may be overridden by specifying the application version explicitly by specifying the ApplVerID(1128) in the StandardHeader component of any application message by counterparty agreement.
Field | Definition |
---|---|
DefaultApplVerID(1137) | DefaultApplVerID(1137) must be specified on the Logon(35=A) message. |
DefaultApplExtID(1407) | May be used by counterparty agreement in-band. Recommendation is to communicate this information out-of-band via a rules of engagement possibly by using a FIX Orchestra rules of engagement. |
DefaultCstmApplVerID(1408) | May be used by counterparty agreement in-band on the Logon(35=A) message to identify the custom application or FIX Orchestra rules of engagement. |
Message type default application version
FIXT expands upon the ability to specify an application version on the Logon(35=A) message by allowing different application versions to be specified by message type in the MsgTypeGrp component.
The MsgTypeGrp component may be used to specify what application messages are supported over the FIX session being initiated.
Within the MsgTypeGrp component, a FIX session may specify a message type default application version on the Logon(35=A) message. For each message type where the default application version is different from the session default application version, the message type default application version is specified using the RefMsgType(372) field to specify the message type, the RefApplVerID(1130) field to specify the application version, and the DefaultVerIndicator(1410) field to indicate if RefApplVerID(1130) is the default.
When present, the RefApplVerID(1130) in the RefMsgTypeGrp component has precedence over the session default application version specified in the DefaultApplVerID(1137) field of the Logon(35=A) message.
Explicit application version per message
The application version may be specified on a message instance using the ApplVerID(1128) field from the standard header.
The explicit application version only applies to the message instance in which it is sent, default application versions remain unaltered.
The explicit application version has precedence over the message type default application version and the session default application version.
FIX session processors must maintain state information regarding the default application version used during the FIX session.
Application version | Fields | Precedence |
---|---|---|
Session default | DefaultApplVerID(1137) | Lowest level of precedence. |
Message type default | RefMsgType(372), RefApplVerID(1130), DefaultVerIndicator(1410) | Supersedes the session default application version. |
Explicit | ApplVerID(1128) | Highest precedence over default application versions. Only applies to the message instance in which ApplVerID(1128) is present. |
Use of extension packs
A FIX session may specify a default extension pack as part of the session default application version by inclusion of a valid extension pack number in the DefaultApplExtID(1407).
An extension pack can be specified for a specific message type using the RefApplExtID(1406) field within the MsgTypeGrp component. If provided, RefApplExtID(1406) becomes part of the message type default application version.
A message instance can explicitly include an extension pack in the ApplExtID(1156) field. If ApplExtID(1156) is specified it becomes part of the explicit application version.
An extension pack is considered incompatible with the application version if the extension pack number specified is less than the last extension pack that was used to create the application version.
EP version | Fields | Precedence |
---|---|---|
Session default | DefaultApplExtID(1407) | Lowest level of precedence. |
Message type default | RefMsgType(372), RefApplExtID(1406), DefaultVerIndicator(1410) | Supersedes the session default extension pack. |
Explicit | ApplExtID(1156) | Highest precedence over default extension pack. Only applies to the message instance in which ApplExtID(1156) is present. |
Use of a custom application version
A default custom application version may be specified in-band over a FIX session by inclusion of a user defined value in DefaultCstmApplVerID(1408) on the Logon(35=A) message.
A custom application version can be specified for a specific message type using the RefCStmApplVerID(1131) field within the MsgTypeGrp component. If provided, RefCStmApplVerID(1131) becomes part of the message type default application version.
A message instance can explicitly include a custom application version in the CstmApplVerID(1129) field. If CstmApplVerID(1129) is specified it becomes part of the explicit application version.
EP version | Fields | Precedence |
---|---|---|
Session default | DefaultCstmApplVerID(1408) | Lowest level of precedence. |
Message type default | RefMsgType(372), RefCStmApplVerID(1131), DefaultVerIndicator(1410) | Supersedes the session default custom application version. |
Explicit | CstmApplVerID(1129) | Highest precedence over default custom application versions. Only applies to the message instance in which the CstmApplVerID(1129) is present. |
Lightweight FIXT (LFIXT session profile)
The Lightweight FIXT session profile (LFIXT) is a restriction of the FIXT session profile. LFIXT eliminates the use of session layer retransmission of messages to simplify the session layer protocol.
LFIXT defines two different modes of operation. Under LFIXT compatible mode, LFIXT peer is interoperable with existing FIX session processors that support the FIXT session profile. Under LFIXT succinct mode, LFIXT peer is no longer interoperable with FIXT peer, but the complexity of the session layer protocol is cut down further.
Profile identification
The value of BeginString(8) must be “FIXT.1.1”.
LFIXT shares the BeginString(8) value with the FIXT session profile in order to support its communication with the FIXT peer under the LFIXT compatible mode.
Application version identification
The LFIXT session profile follows the same application version identification rules as the FIXT session profile.
LFIXT transport layer requirements
LFIXT does not support retransmission of messages at the session layer and relies on the application layer to provide retransmission services during the FIX session when needed.
For LFIXT a FIX session exists over one FIX connection which exists over one transport layer. A loss of either the FIX connection or the transport layer connection constitutes the end of a FIX session.
Each establishment of a FIX connection creates a new FIX session, which means that NextNumIn is set to 1 and the NextNumOut is set to 1 in the FIX session processor.
LFIXT compatible mode
When operating under LFIXT compatible mode, a FIXT peer shall handle all possible types of FIXT inbound session layer messages, but only send out session layer messages of limited types to enable a seamless communication between the LFIXT peer and the FIXT peer. An LFIXT peer is compatible with its FIXT peer in this sense.
Similarly, two LFIXT peers have the ability to communicate with each other seamlessly when both operating under the LFIXT compatible mode.
Session Layer Message | Received from peer | Sent to peer |
---|---|---|
HeartBeat(35=0) | Yes | Yes |
Logon(35=A) | Yes | Yes |
TestRequest(35=1) | Yes | No |
ResendRequest(35=4) | Yes | No |
Reject(35=3) | Yes | Yes |
SequenceReset(35=4) GapFillFlag(123)=N |
Yes | Yes, but only passively |
SequenceResest(35=4) GapFillFlag(123)=Y |
Yes | No |
Logout(35=5) | Yes | Yes |
LFIXT succinct mode
When operating in LFIXT succinct mode, an LFIXT peer shall only accept a subset of inbound session layer message types and shall only transmit a subset of session layer message types. The complexity of the session layer protocol is reduced to a minimum in this way, while sacrificing the compatibility with FIXT peers.
Upon receiving an unexpected type of session layer message, an LFIXT peer operating under the LFIXT succinct mode shall terminate the connection by sending a Logout(35=5) with a description of the error in Text(58) and then terminate the transport layer connection.
Succinct mode shall only be used by an LFIXT peer when its counterparty is already known to be using the LFIXT session profile, e.g. by prior out-of-band exchange of the rules of engagement13. Its counterparty can operate under LFIXT compatible mode or under LFIXT succinct mode, but cannot be a FIXT peer (using the FIXT session profile) since the two peers are not compatible in this case.
Session Layer Message | Received from peer | Sent to peer |
---|---|---|
HeartBeat(35=0) | Yes | Yes |
Logon(35=A) | Yes | Yes |
Reject(35=3) | Yes | Yes |
Logout(35=5) | Yes | Yes |
LFIXT and FIXT operating mode interoperability
The following table shows the interoperability between LFIXT and FIXT peers for each operating mode of LFIXT.
Operating Mode | LFIXT acceptor using succinct mode | LFIXT acceptor using compatible mode | FIXT acceptor |
---|---|---|---|
LFIXT initiator using succinct mode | interoperable | interoperable | not interoperable |
LFIXT initiator using compatible mode | interoperable | interoperable | interoperable |
FIXT initiator | not interoperable | interoperable | interoperable |
Validation of message sequence numbers
If a received message has a MsgSeqNum(34) less than NextNumIn, a serious error exists. The FIX session and transport layer connection should be terminated unless the message received is:
- a SequenceReset(35=4) message with PossDupFlag(43) present with a value of “Y”.
- any message of a message type that supports retransmission with a PossDupFlag(43) present with a value of “Y”.
- a Logon(35=A) message with ResetSeqNumFlag(141) present with a value of “Y”.
If a received message has a MsgSeqNum(34) greater than NextNumIn, a serious error exists. The FIX session and transport layer connection should be terminated.
Application layer recovery
The application layer shall be responsible for message recovery.
An LFIXT session processor must not perform message recovery.
LFIXT initiator connects to LFIXT acceptor
An LFIXT initiator shall send a Logon(35=A) message with the following fields present
Field | Value |
---|---|
MsgSeqNum(34) | 1 |
ResetSeqNumFlag(141) | Y |
NextExpectedSeqNum(789) | 1 |
In the following diagram shows the standard LFIXT logon process.
FIXT initiator connects to LFIXT acceptor (compatible mode)
A FIXT initiator that is able to persist message history shall send a Logon(35=A) message with the following fields present. Upon receiving the Logon(35=A) request, an LFIXT acceptor shall adjust its own NextNumOut to the NextExpectedMsgSeqNum(789) contained in the Logon(35=A) request.
Field | Value |
---|---|
MsgSeqNum(34) | 1 |
ResetSeqNumFlag(141) | N |
NextExpectedSeqNum(789) | NextNumIn |
In the following example, a FIXT initiator is establishing a new FIX connection to the LFIXT session. This example shows how the LFIXT acceptor adjusts its NextNumIn and NextNumOut values to the FIXT session state being maintained by a FIXT initiator.
In the following example, an LFIXT initiator is establishing a new FIX connection to the FIXT session. This example shows how the FIXT acceptor resets its NextNumIn and NextNumOut values when the LFIXT initiator sends the Logon(35=A) request.
Receipt of ResendRequest(35=2) message from a FIXT peer
LFIXT peers are not expected to save outbound message history for retransmission.
If a ResendRequest(35=2) message is received, the LFIXT peer shall respond with a SequenceReset(35=4) with the GapFillFlag(123) set to “N” and a NewSeqNo(36) = NextNumOut.
The following example shows a FIXT initiator connected to an LFIXT acceptor. The FIXT initiator does not process messages with MsgSeqNum(34)=98 and MsgSeqNum(34)=99 for some reason. As a result, the FIXT initiator transmits a ResendRequest(35=2) asking for messages 98 and 99. The LFIXT acceptor responds with a SequenceReset(35=4) message with GapFillFlag(123) set to “N” and the NewSeqNo(36) set to the maximum of the EndSeqNo(16) from the ResendRequest(35=2) message and the LFIXT acceptor NextNumOut. This example demonstrates that an LFIXT session does not provide session layer message recovery.
Processing invalid messages
If LFIXT encounters a garbled message or other data parsing issues, the LFIXT session processor should terminate the connection by sending a Logout(35=5) with a description of the error in Text(58) and terminate the transport layer connection.
FIX message routing
Message routing details – one firm-to-one firm (point-to-point)
The following table provides examples regarding the use of SenderCompID(49), TargetCompID(56), DeliverToCompID(128), and OnBehalfOfCompID(115) when using a single point-to-point FIX session between two firms. Assumption (A=sell-side, B =buy-side):
SenderCompID | OnBehalfOfCompID | TargetCompID | DeliverToCompID | |
---|---|---|---|---|
A to B directly | A | B | ||
B to A directly | B | A |
Message routing details – third party message routing
The FIX session protocol supports the ability for a single FIX session to represent multiple counterparties. This can be in a 1-to-many, many-to-1, or 1-to-1 fashion. In addition, some third parties may be connected to other third parties effectively forming a “chain” of “hops” between the original message initiator and the final message receiver. The SenderCompID(49), OnBehalfOfCompID(115), TargetCompID(56), and DeliverToCompID(128) fields are used for routing purposes.
When a message travels over intermediary FIX sessions or an intermediary sends a message on behalf of another firm (using OnBehalfOfCompID(115)), that intermediary may add their details to the HopGrp component. The HopGrp may be used to build a “history” of FIX sessions through which the original message was transmitted. The HopGrp is not used to facilitate routing, rather it provides an audit trail of intermediary FIX sessions involved in transmitting the message to its final destination. An audit trail of intermediary involvement may be a requirement of some regulatory bodies or counterparties. When an intermediary FIX session forwards a message on to the next hop (may be the end point or another intermediary), that intermediary may add its hop details to the HopGrp (i.e. its SenderCompID(49) as HopCompID(628), its SendingTime(52) as HopSendingTime(629), and the received message’s MsgSeqNum(34) or some other reference as HopRefID(630)).
Note that if OnBehalfOfCompID(115) or DeliverToCompID(128) message source identification/routing is used for a FIX session, then it must be used on all application messages transmitted via that session accordingly (reject the message if not).
The following figure provides examples regarding the use of SenderCompID(49), TargetCompID(56), DeliverToCompID(128), and OnBehalfOfCompID(115) when using a single FIX session to represent multiple firms. Assumptions are that A=sell-side, B=buy-side, and Hub=third party:
Note that some fields—for example, ClOrdID(11) on a NewOrderSingle(35=D) message—must be unique for all orders on a given FIX session. Thus, when using OnBehalfOfCompID(115) or DeliverToCompID(128) addressing, a recommended approach is to prepend these fields to the original value. For example, if A sends the hub a ClOrdID(11) value of “A001”, then the hub could specify a ClOrdID(11) of “A-A001” when sending the message to B to ensure uniqueness.
Transmitting alternatively encoded messages over a FIX session
XML content may be transmitted over a FIX session layer. FIXML or other XML-based content (such as ISO 20022 XML messages, FpML XML messages) within the XmlData(213) field in the StandardHeader of the XMLnonFIX(35=n) message.
The application message type is assumed to be contained in the XML document sent in XmlData(213) and is not processed by the FIX session layer.
Use of Attachment group
Other encodings, such as SBE, JSON, Google Protocol Buffers, PDF, jpeg should be transmitted using the AttachmentGrp component within the XMLnonFIX(35=n) message.14
The AttachmentGrp component provides the ability to attach other media type documents to a FIX message for transmission. The media type can be any of the media types (previously referred to as MIME types) that are listed by IANA RFC2046.
- Firms may document the session layer configuration using the FIX Orchestra interface specification.↩︎
- The LFIXT session profile uses the same BeginString(8) value as the FIXT session profile. The use of LFIXT and its mode of operation must be agreed upon out-of-band by counterparty agreement.↩︎
- FIX Orchestra may be used to communicate which FIX session profile and which operating mode is being used for a FIX enabled service.↩︎
- Functionality added by EP113.↩︎
- Field added in EP56↩︎
- FIX Orchestra may be used to communicate which FIX session profile and which operating mode is being used for a FIX enabled service.↩︎
- Functionality added by EP124.↩︎
- Functionality added by EP124.↩︎
- The FIX Orchestra Interface specification may be used to specify the TestRequestThreshold.↩︎
- Functionality added by EP124.↩︎
- Setting EndSeqNo(16) to 0 has been the recommended approach since FIX.4.2.↩︎
- The use of SecureData(91) and its associated length field SecureDataLen(90) were deprecated with the introduction of the FIXT.1.1 Session Layer specification. The use of application layer encryption is still supported by counterparty agreement.↩︎
- FIX Orchestra may be used to communicate which FIX session profile and which operating mode is being used for a FIX enabled service.↩︎
- AttachmentGrp component added in EP167↩︎