If DragScroll is enabled, raise or lower (lower for quicker context menus) the "watch for drag(millsecs)" value.
Settings>Editor>Mouse Drag Scrolling>Unix Context Menu Watch for Drag
It's adjustable for different systems and/or users so that DragScroll waits to see if the user moves the mouse first, or is invoking the context menu.
You can also change the drag key from RightMouse to MiddleMouse, or disable it altogether.