HMAC stands for Hash-based Message Authentication Code. It is a mechanism for verifying the integrity and authenticity of a message by using a cryptographic hash function and a shared secret key. HMAC is one way to ensure that the message has not been altered during transmission and that it comes from a trusted source (one that has the shared secret key and also knows exactly how to construct the HMAC).
In this post I'll discuss what HMAC is, how it works to ensure message integrity and authenticity, and provide a guide on implementing it securely using python. I'll also examine some attack scenarios to better understand its strengths and limitations.
Understanding the Basics of HMAC
At its core, HMAC combines a secret key with a cryptographic hash function (like SHA-256), to generate a unique code or "MAC" that will be used for verifying whether a message has been altered in transit and ensuring that it comes from a trusted source. This makes it useful in applications where secure message exchange is crucial, such as APIs, webhooks, and certain communication protocols.
How Does HMAC Work?
In general, HMAC involves the following steps:
- Both the sender and receiver share a secret key. The sender generates an HMAC from the message using this secret key and a hash function (e.g. SHA-256). The generated hash value (HMAC) serves as a "signature" for that message.
- The sender sends both the message and the HMAC to the receiver.
- Upon receiving the message, the receiver uses the same secret key to generate an HMAC from the message.
- If the receiver's HMAC is equal to the sender's HMAC, the message is verified to be unaltered and authentic.
When Should You Use HMAC?
Here are some common use cases:
- API Authentication: Many APIs use HMAC to validate the requests from clients.
- Webhooks: HMAC can be used in webhooks to verify the authenticity of incoming data from external sources.
- Data Integrity: HMAC can also simply be used to identify if the content of a file or message has changed, even outside the context of security.
Implementing HMAC
In this section, we'll implement HMAC once in a straightforward, general way and with additional security measures like a nonce and a timestamp to protect against replay attacks.
1. Basic HMAC Implementation
Let's start with a basic implementation in Python. We'll use hmac and hashlib packages:
import hmac
import hashlib
def generate_hmac(message, key):
# Ensure message and key are bytes
if isinstance(message, str):
message = message.encode()
if isinstance(key, str):
key = key.encode()
h = hmac.new(key, message, hashlib.sha256)
return h.hexdigest()
# Example usage
if __name__ == '__main__':
key = 'your-shared-secret-key'
message = 'This is a secure message.'
mac = generate_hmac(message, key)
print("Message:", message)
print("HMAC:", mac)
This approach is effective for basic integrity verification, ensuring that the message hasn't been tampered with. However, this approach doesn't protect against replay attacks where an attacker could intercept the message and resend it. To account for this, we'll add a nonce and a timestamp, which I'll explain in the second approach.
2. Enhanced HMAC with Nonce and Timestamp
For improved security, we'll implement HMAC with a nonce (a unique one-time value) and a timestamp (usually the timestamp of the request) to ensure each message is 1) unique and 2) recent. Here's how we can implement it:
Sender Side (Client)
import hmac
import hashlib
import time
import os
def generate_hmac_with_nonce_and_timestamp(message, key):
# Ensure message and key are bytes
if isinstance(message, str):
message = message.encode()
if isinstance(key, str):
key = key.encode()
# Generate a timestamp and nonce
timestamp = str(int(time.time()))
nonce = os.urandom(16).hex() # 16-byte random nonce for uniqueness
# Construct the payload
payload = b'|'.join([timestamp.encode(), nonce.encode(), message])
# Generate HMAC
h = hmac.new(key, payload, hashlib.sha256)
mac = h.hexdigest()
return {
'timestamp': timestamp,
'nonce': nonce,
'hmac': mac
}
# Example usage (sender side)
if __name__ == '__main__':
key = 'your-shared-secret-key'
message = 'This is a secure message.'
auth_data = generate_hmac_with_nonce_and_timestamp(message, key)
print("Message:", message)
print("Timestamp:", auth_data['timestamp'])
print("Nonce:", auth_data['nonce'])
print("HMAC:", auth_data['hmac'])
# Send message, timestamp, nonce, and HMAC to receiver
Receiver Side (Server)
import hmac
import hashlib
import time
def verify_hmac_with_nonce_and_timestamp(message, key, timestamp, nonce, received_hmac, nonce_store):
# Ensure message and key are bytes
if isinstance(message, str):
message = message.encode()
if isinstance(key, str):
key = key.encode()
# Construct the payload for HMAC calculation
payload = b'|'.join([timestamp.encode(), nonce.encode(), message])
# Recalculate HMAC
h = hmac.new(key, payload, hashlib.sha256)
calculated_hmac = h.hexdigest()
# Check if HMACs match
hmac_is_valid = hmac.compare_digest(calculated_hmac, received_hmac)
# Verify timestamp (e.g., within 5 minutes of current time)
current_time = int(time.time())
timestamp_is_valid = abs(current_time - int(timestamp)) <= 300 # 300 seconds = 5 minutes
# Check if nonce has been used before
nonce_is_valid = nonce not in nonce_store
# If nonce is valid, store it to prevent future replays
if nonce_is_valid:
nonce_store.add(nonce)
# Overall verification result
is_valid = hmac_is_valid and timestamp_is_valid and nonce_is_valid
return is_valid
# Example usage (receiver side)
if __name__ == '__main__':
key = 'your-shared-secret-key'
nonce_store = set() # Keep track of used nonces
# Let's assume these are received from the sender
message = 'This is a secure message.'
received_timestamp = '1638387200' # Example timestamp from sender
received_nonce = 'd1f2e3c4b5a69788d9e0f1a2b3c4d5e6' # Example nonce from sender
received_hmac = 'the-hmac-received' # Replace with actual HMAC received
# Validate
is_valid = verify_hmac_with_nonce_and_timestamp(
message,
key,
received_timestamp,
received_nonce,
received_hmac,
nonce_store
)
if is_valid:
print("Message is authenticated and valid.")
else:
print("Message authentication failed or message is stale/replayed!")
In the enhanced version, the sender includes a nonce and a timestamp both in their HMAC calculation process and in the request header. The receiver can then use this information to re-generate the HMAC and also ensures that the request is in an acceptable time window (using timestamp) and that the nonce hasn't been used before. This implementation not only ensures integrity and authenticity, but also prevents replay attacks by ensuring that each message is unique and recent.
There is, however, one problem with this code. It has a memory leak, since the nonce store can grow larger and larger. To solve this issue, you can implement a time-based expiration storage of nonces to remove nonces that are no longer relevant. In a distributed application, you can use an external cache solution (such as Redis) with automatic expiry rather than storing them in memory.
Attack Scenarios: How HMAC Protects Against Threats
Let's go through some attack scenarios to gain a better understanding of how HMAC works, where it shines, and where it falls short.
1. Passive Interception
In a passive interception scenario, an attacker intercepts the message and reads it to gain information, but doesn't modify it. In this scenario, if sensitive data is transmitted without encryption, it can lead to privacy breaches. While HMAC alone doesn't provide encryption and doesn't hide the message content, it can be paired with HTTPS (TLS encryption) to ensure that the content is hidden and unreadable.
2. Message Tampering
In a message tampering scenario, an attacker intercepts the message sent from the sender, modifies its content, and sends it on to the receiver and hopes for the receiver to accept it as legitimate. Lucky for us, since the receiver is using the content along with the secret key to re-generate the received HMAC, if the content got changed, the resulting HMAC would also change, therefore the receiver would render the request invalid and hence will not accept it. However, it's important to keep the shared secret keys safe and private. If the secret key gets compromised, the attacker can potentially sign each request and force it to be valid.
3. Replay Attack
In a replay attack scenario, an attacker intercepts a valid request and re-sends it to the receiver in an attempt to duplicate a previous transaction or action. This is where the enhanced implementation that we discussed earlier shines, as the nonce and timestamp are there to ensure that replay attacks are not possible.
4. Key Compromise
If an attacker gains access to the shared secret key, they can potentially generate valid HMACs, and trick the receiver into thinking their requests are legit. HMAC alone can't provide any solutions for this scenario, but it's recommended to use key management strategies such as Key Rotation to mitigate this risk. There are many ways to do this, but generally speaking, keeping your keys in environment variables would be a good start since it lets you update the values without the need to change the codebase. Additionally, you can use AWS Secrets Manager, HashiCorp Vault, or Azure Key Vault. These tools allow for secure storage and automatic rotation of secret keys in production.
Conclusion
In this post, we explored the fundamentals of HMAC, its implementation, and possible attack scenarios. When implemented correctly and combined with an encryption technique (such as HTTPS), HMAC is an effective method and can significantly increase your application's security by preventing message tampering and repaly attacks.
If you like to dive deeper and learn more about HMAC, I recommend the following links from well-known services like AWS and Stripe for secure API and Webhooks communication: