Don't error out when cpu accelerator doesn't have torch (as default for whl building)

Don't error out when cpu accelerator doesn't have torch (as default for whl building) #12535