We're updating the issue view to help you get more done. 

Database connection pools leaking memory on every checkpoint

Description

Due to the use of ThreadLocals inside the database connection pooling and Quasar, the flow state machine leaks around 300 bytes every flow suspension/context switch.

In discussion with we concluded this is because we take the database connection from the pool in the Fiber and return it in the Thread. The connection pool has an optimisation to recycle thread local connections first. The list of returned connections in the Thread builds up (leading to the memory leak) and the Fiber always experiences a miss (of the optimisation) and has to resort to the global pool.

The fix is to somehow wormhole the Thread's ThreadLocal into the Fiber so that experiences a hit and the Thread does not experience returns only. The underlying data structures would then be shared.

We think this is/must be possible via reflection.

This bug could be present in 3.x. No idea why it wouldn't be, but it didn't seem to be visible when we tested it. Maybe it was not visible due to lower transaction volumes in Azure vs. OVH. Or maybe the HikariCP dependency did not experience the same problem.

Status

Assignee

Rick Parker

Reporter

Rick Parker

Priority

Highest

Labels

None

Severity

High

Fix versions

Ported to...

None

Feature Team

Performance and Platform Sustainability

Affects versions

Corda 4
Corda Enterprise 4