I want to clarify the design here so we're all on the same page. The Delay control on the CW panel is designed to be used with the internal keyer (i.e. when Break In and Iambic are selected). The design intent to have this delay take place after the final element sent. This effectively allows the user to operate non-QSK and only drop out to RX as quickly as the user wants. This is not how the software behaves right now in v0.15.13. We're aware of the problem and have a fix coming to a release near you soon.
When using an external keyer (i.e. when Break In and Iambic are NOT selected), it is our understanding that the keyer itself should be in charge of when to assert the PTT and this effectively controls the "delay". This appears to work properly in v0.15.13 in my testing.