Michael, an ethical hacker at a New York-based e-commerce company, is evaluating the security of...

The algorithm described most accurately matches RSA. CEH cryptography coverage explains that RSA is a widely used public-key cryptosystem that supports both encryption and digital signatures. In a typical RSA-based signature process, the sender generates a hash of the message and applies their private key using modular exponentiation to create a digital signature. The recipient then uses the sender’s public key to verify the signature, also through modular exponentiation operations involving large prime numbers and a modulus. The scenario specifically references modular exponentiation for generating digital signatures, which is a defining mathematical operation in RSA.

Diffie-Hellman is primarily a key exchange algorithm and does not provide digital signatures by itself. It enables two parties to establish a shared secret over an insecure channel but does not authenticate messages directly through signing. ElGamal does support encryption and can be adapted for signatures, but in standard CEH-oriented contexts, RSA is the most commonly referenced algorithm for public-key encryption and digital signatures in payment systems. DSA is strictly a digital signature algorithm and does not provide encryption functionality; the prompt describes a broader public-key cryptosystem used for authenticity of payment confirmations, which aligns more directly with RSA’s dual capability.

Additionally, RSA’s reliance on key pairs stored and retrieved from directories fits the description of public-key verification in enterprise payment infrastructures. Therefore, RSA is the correct identification.