Denoising was setting session parameters for every frame, which was detected as a change and therefore caused a resync. Since the parameter modification change is only needed for viewport rendering (which doesn't support denoising anyways) and resyncing after a frame change (which isn't affected by denoising settings), an easy fix is to just ignore the denoising parameters like it's currently done with the samples.