Configuration parameter ValidateFieldsOutOfOrder not working when header fields found in body part

[jacques0803]

Hi FIX Community,

I'm attempting to exchange messages with a party that does properly order fields. When ValidateFieldsOutOfOrder=N, simpler messages such as Logon and TradeExecutionReportRequestAck message works fine, as all message header fields are at the start of the message, albeit not in order.

The problem I'm facing now is that the TradeExecutionReport fails to process (Rejected with "Received message without MsgSeqNum") as TAG34 is present in the message body, and not the header.

ValidateFieldsOutOfOrder: If set to N, fields that are out of order (i.e. body fields in the header, or header fields in the body) will not be rejected. Useful for connecting to systems which do not properly order fields.

So, my assumption is that, even if the Header field is provided within the message body, QuickFix/J should still be able to extract the relevant fields and populate the Message.Header object correctly.

QuickFix/J Version 2.3.1
JDK 1.8
FIX5.0SP2

TradeExecutionReport Test Message 1 - All Header fields at start of message, and reading the message works perfect:

8=FIXT.1.1|9=504|35=AE|49=ASX|56=MSA01_3383_TEST|34=545|52=20220210-02:44:00.820|1128=9|487=0|1125=|1003=1120000338|75=20220210|1015=0|64=20220214|60=20220210-02:43:27.796|55=ANZ|48=AU000000ANZ3|22=4|461=Exxxxx|381=135000|31=27|32=5000.000000000000|15=AUD|1301=XASX|20003=XT|20007=XD|552=2|54=1|11=7533509260093758035|1=1040445|576=1|577=0|453=1|448=338-3|447=D|452=1|54=2|11=7533509260093757876|1=1040445|576=1|577=0|453=1|448=338-3|447=D|452=1|880=7533509260093686098:0#NORMAL#1644451200000000000|167=CS|762=1|106=4075|10=087|

TradeExecutionReport Test Message 2 - Only TAG8,9 and 35 at start of message, all other header tags mixed into body of message:

8=FIXT.1.1|9=504|35=AE|487=0|1125=|1003=1120000338|75=20220210|1015=0|64=20220214|60=20220210-02:43:27.796|55=ANZ|48=AU000000ANZ3|22=4|461=Exxxxx|381=135000|31=27|32=5000.000000000000|15=AUD|1301=XASX|20003=XT|20007=XD|552=2|54=1|11=7533509260093758035|1=1040445|576=1|577=0|453=1|448=338-3|447=D|452=1|54=2|11=7533509260093757876|1=1040445|576=1|577=0|453=1|448=338-3|447=D|452=1|49=ASX|56=MSA01_3383_TEST|34=545|52=20220210-02:44:00.820|1128=9|880=7533509260093686098:0#NORMAL#1644451200000000000|167=CS|762=1|106=4075|10=087|

Configuration values I do set to prevent certain validations:
ValidateUserDefinedFields: N
ValidateFieldsOutOfOrder: N
ValidateFieldsHaveValues: N
ValidateUnorderedGroupFields: N
AllowUnknownMsgFields: Y

The test I perform (test Messages loaded from a resource file):

  public void testSetup() {
      quickfix.fix50sp2.MessageFactory fix50SP2MessageFactory = new quickfix.fix50sp2.MessageFactory();
      ApplVerID applVerID = new ApplVerID(ApplVerID.FIX50SP2);
      DataDictionary applicationDD = new DefaultDataDictionaryProvider(true).getApplicationDataDictionary(applVerID);
      applicationDD.setCheckFieldsOutOfOrder(false);
      applicationDD.setCheckUnorderedGroupFields(false);
      applicationDD.setCheckFieldsHaveValues(false);
      applicationDD.setAllowUnknownMessageFields(true);
      applicationDD.setCheckUserDefinedFields(false);

      final String beginString = "FIXT.1.1";

      new BufferedReader(new InputStreamReader(resource.getInputStream())).lines().forEach(msgString -> {
          TradeCaptureReport message = (TradeCaptureReport) fix50SP2MessageFactory.create(beginString, applVerID, MsgType.TRADE_CAPTURE_REPORT);
          try {
              message.fromString(msgString, applicationDD, true);
              messages.add(message);
          } catch (InvalidMessage e) {
              e.printStackTrace();
          }

      });
  }
  public void testMessages() {
      messages.forEach(message -> {
          if (!Objects.isNull(message.getException())) {
              log.debug("Field Exception {}", message.getException());
          }
          // print Header Fields
          printField(message.getHeader(), 8);
          printField(message.getHeader(), 9);
          printField(message.getHeader(), 35);
          printField(message.getHeader(), 49);
          printField(message.getHeader(), 56);
          printField(message.getHeader(), 34);
          printField(message.getHeader(), 52);
          printField(message.getHeader(), 1128);
      });
  }
  private void printField(quickfix.Message.Header header, int fieldNo) {
      try {
          log.debug("TAG: {}={}", fieldNo, header.getString(fieldNo));
      } catch (FieldNotFound e) {
          log.error("TAG {} not found", fieldNo);
      }
  }

The output of running the test:

-------- Testing message --------
TAG: 8=FIXT.1.1
TAG: 9=504
TAG: 35=AE
TAG: 49=ASX
TAG: 56=MSA01_3383_TEST
TAG: 34=545
TAG: 52=20220210-02:44:00.820
TAG: 1128=9
-------- Testing message --------
TAG: 8=FIXT.1.1
TAG: 9=504
TAG: 35=AE
TAG 49 not found
TAG 56 not found
TAG 34 not found
TAG 52 not found
TAG 1128 not found

Any help/comments would be greatly appreciated.

Kind regards

[chrjohn]

see #468