Node: IdentityService.partiesFromName() can cause excessive memory usage

Description

The following API call can cause excessive memory usage in 4.7:

It's a regression from certificate rotation.

The reason is that SQL query result of selecting the entire network map cache will be passed to the List and only then filtered:

Instead, it should use Stream, which allows to avoid keeping unwanted data in memory after processing. Something like:

Activity

Show:
Maxim Shadrin
January 14, 2021, 9:14 PM

Test results in CE-4.7.1-RC01: Passed

Scenario tested:

  • Making API calls to IdentityService.partiesFromName () from internal and external RPC client.

  • Launch of standard corda finance flows.

  • Comparison of results between CE-4.7.1-RC01 and CE-4.7.

 

Denis Rekalov
January 12, 2021, 2:56 PM
Edited

Only functional tests please, no memory checks needed.
We don't have unit tests for memory allocations, but we're using the same approach as in 4.6, so it should be fine.

partiesFromName() can be either invoked from flows (via IdentityService) or via RPC, so you will need to write something to test it.

Maxim Shadrin
January 12, 2021, 2:33 PM

Could you please confirm that from the QA side there is no need to look at the amount of allocated memory (for example, it was covered by unit tests) and we only need to check that the behavior of the node has not changed when this functionality is called? And if so we need to understand the scenario when this part of the code is involved. Please advise whether it is enough to register several nodes, get a network map on one of them and start a stream with a counterparty?

Denis Rekalov
January 8, 2021, 2:11 PM

Ported to OS and ENT.

Sachin Gokhale
January 8, 2021, 8:54 AM

- Kindly merge the changes in ENT and OS 4.7 by end of day today, we would be cutting CE 4.7.1 on Monday morning to start the QA activities on this.

Done

Assignee

Maxim Shadrin

Reporter

Denis Rekalov

Labels

Sprint

None

Epic Link

None

Priority

Medium

Engineering Teams

Core - Identity & Discovery Team

Fix versions

Affects versions

Ported to...

Corda 4.7
Corda Enterprise 4.7.1

Story Points / Dev Days

None

Build cut

None

Squad

None