Mousewheel-Click removes selection

When clicking the mousewheel to move over a selected area, the pixel the mouse hovers over is unselected. It gets annoying when having a greater selection and moving to select more.