A new tube on this side may fix your problem. It may be a trick of the light reflection, but I f you look closely at the scratch pattern on top of the tube, at the location of the pointer in the first photo below you see it fade a way for a short section. Also, if you look at the fence bracket in the second photo, on the bottom edge of the left hand strengthener you see wear while there is less wear in that location for the next two strengtheners to the right, and none on the next three.. Also there is no wear on the top edge of the left three, but the wear on each of the right three increases as you move to the right.
This makes me think there may be a nonconformity in the tube at about that point where the arrow is in the first photo, which is where you hit with the trailing edge of the fence bracket as you slide it to the left (toward the headstock). That would coincide with the wear patterns seen on the inside of that bracket. It looks like there is more scratching on the top of the tube, the further to the left you look in the first photo, which also coincides with much more wear seen at the top of the strengthener on the right in the second photo.
Looking at the first photo, This wear pattern could happen if the tube was slightly malformed such that as you moved the fence toward the left (toward the headstock) the bottom of the back of the fence bracket would hit the bottom of the tube while the left side of the bracket hit the top of the tube. Can you see if the tube may be slightly bowed or has a small bulge at the bottom side below where the arrow is? Or maybe on top of the tube where the left side of the bracket would be when the right side was at the location of the arrow.
.


.
But then, I may just be seeing things! 😃