Users ought to be able to be banned by user ID even if they aren't and haven't in the past been on the server.
This is probably possible as if the user has in the past been on the server, they can be banned by an administrator through the user context menu by searching them. The !ban command would accept the bare user ID (not a reference, since that isn't possible).
if len(message.mentions) >= 1: member = message.mentions else: member = message.guild.get_member(int(args))
This is already the case.