Though the delete button is availed on the Manage Roles Page, its more likely that a role can not be deleted via the page. Deletion of a role should be effective when done via System Administration → Advanced Administration →
Roles are defined in Reference Metadata Module which you cannot remove as part of reference application, However, you can delete a new added Role from legacyui admin section which i would reccommend to use that part as your work.
There was a conversation here which is still worth looking into to figure out whether we might need this functionality working again in ReferenceApplication.