Can’t the phone just grab the private key and use it later?

No, the private key never leaves the ChipBase security chip. If you worry about the safety, you can regenerate new private key and just immediately use the fresh new one for transaction.