OK wow, I stand corrected, in my case it was indeed a headphone incompatibility. The headphones I'm using have a TRRS (4-pole) connector, presumably because they have a mic for use with smartphone gadgets. I couldn't actually find a pair of headphones in my household with a standard TRS connector, but I did find a TRRS > TRS adaptor and it completely solves the problem. A picture might help. The one on the left, no good, the one on the right, works perfectly:
Does this apply to your situation, flbo? It's maybe obvious in retrospect, but the symptoms are super-unexpected, so if anyone can reproduce THIS problem that might be a bug worth reporting. Or they could at least put a warning in the manual.