PMTeam@ZWO
This problem has been reported several times over the past couple of years by Southern Hemisphere users.
The 'drift' occurs even if you don't adjust the mount RA/DEC – just keep hitting "Refresh" and the error increases.
But ZWO has not addressed the problem in all that time.
The only work around is to adjust the mount and then restart the alignment from scratch; and, of course, repeat that process until the target is reached.
Tedious, but you eventually get there!