So a 1st level friendly fighter could "mark" you to end the "mark" of a 20th level enemy fighter?

Why not have simply the greater penalty apply? The "last mark used" resolution to apply only between marks of equal level.