After all, if we let this slide, what next?
General election results are not actually constitutionally binding. How about there's a general election tomorrow, Labour win, but the Tories say "we are absolutely committed to handing over power to the Labour Party, but first we have to negotiate a proper power withdrawal agreement". And then three years later the Tories are still in power.
It is a small step from what's happening now, to what I suggested above, and it MUST be resisted.