Using the standard ssh key on every host is most likely enough. IRC does not need to be that 'secure'.