Uploaded image for project: 'Corda'
  1. CORDA-1976

FailNodeOnNotMigratedAttachmentContractsTableNameTests is sometimes failing with db constraint "Notary" is null


    • Type: Story
    • Status: Done (View workflow)
    • Priority: Medium
    • Resolution: Done
    • Affects versions: None
    • Fix versions: None
    • Components: None
    • Labels:
    • Feature Team:
      Corda Core
    • Sprint:


      The test is a fairly standard integration test with the difference that it only starts 1 node (besides the notary).
      The flow it's running is just a normal "issue" type flow.

      What is happening is that, during the finality flow, when the "notary" property is being converted to the X500 name, the code tries to read from the "keyToParties" cache to resolve the notary.
      The error happens because the notary is not present in that cache.

      The only place that writes to that cache is `IdentityService.verifyAndRegisterIdentity`, and the notary should be added by the NetworkMapCache, when the node reads the notaries node-info for the first time.

      The DriverDsl synchronously writes the node-info of the notary ( which was started previously) before the actual node is started, so the file should be there.

      There's a couple of layers of observers in between the NodeInfoWatcher ( responsible for scanning the folder), and the 'verifyAndRegisterIdentity' function. One of these observers runs on a Thread created by a custom TestThreadFactory.testThreadFactory function.

      There is also relevant logic in the PersistentNetworkMapCache that uses openFutures that sets the node to ready after a node has been added.

      Not sure how well this works, as the first thing a node does is call addNode on itself.
      Maybe a solution is to mark a node as ready only after the notary node was added to the caches.

      Given the above, there are a couple of options that are possible ( I couldn't replicate them though):
      1) The node was marked as ready before the notary was actually added to the caches, so the flow was started and failed as the notary was not present. ( A solution would be the one mentioned above. Node ready only after notary added)
      2) There is a bug in the AppendOnlyPersistentMap/Guava cache that is exposed by this.
      On master, both the AppendOnlyPersistentMap was changed and Guava was remove, so probably not worth investing in exploring this route.

      The "fix" I'm going to make for now is to just add a second node to the test, to make the flakyness less likely.




            • Assignee:
              Tudor Malene
              Tudor Malene
            • Votes:
              0 Vote for this issue
              2 Start watching this issue


              • Created:

                PagerDuty Incident