Does it really work or does it diminish the responsibility?

If you look at Spotify, the PM has end to end responsibility. On the contrary of what you say, in start-ups I noticed more end-to-end responsibility than shared ones.

Product Owner is an accountability (previously a role in Scrum) while Product Manager is a job. You can read Marty Cagan or Melissa Perri, and that's what you will find.

The combination of two drivers is just limiting and slowing down, and often confusing. The more handovers you add, the less accountability each person will hold.