The NTLM Authentication Protocol and Security Support Provider

Abstract

This article seeks to describe the NTLM authentication protocol and related security support provider functionality at an intermediate to advanced level of detail, suitable as a reference for implementors. It is hoped that this document will evolve into a comprehensive description of NTLM; at this time there are omissions, both in the author's knowledge and in his documentation, and almost certainly inaccuracies. However, this document should at least be able to provide a solid foundation for further research. The information presented herein was used as the basis for the implementation of NTLM authentication in the open-source jCIFS library, available at http://jcifs.samba.org. This documentation is based on independent research by the author and analysis of functionality implemented in the Samba software suite.

Contents

What is NTLM?

NTLM is a suite of authentication and session security protocols used in various Microsoft network protocol implementations and supported by the NTLM Security Support Provider ("NTLMSSP"). Originally used for authentication and negotiation of secure DCE/RPC, NTLM is also used throughout Microsoft's systems as an integrated single sign-on mechanism. It is probably best recognized as part of the "Integrated Windows Authentication" stack for HTTP authentication; however, it is also used in Microsoft implementations of SMTP, POP3, IMAP (all part of Exchange), CIFS/SMB, Telnet, SIP, and possibly others.

The NTLM Security Support Provider provides authentication, integrity, and confidentiality services within the Window Security Support Provider Interface (SSPI) framework. SSPI specifies a core set of security functionality that is implemented by supporting providers; the NTLMSSP is such a provider. The SSPI specifies, and the NTLMSSP implements, the following core operations:

  1. Authentication -- NTLM provides a challenge-response authentication mechanism, in which clients are able to prove their identities without sending a password to the server.
  2. Signing -- The NTLMSSP provides a means of applying a digital "signature" to a message. This ensures that the signed message has not been modified (either accidentally or intentionally) and that that signing party has knowledge of a shared secret. NTLM implements a symmetric signature scheme (Message Authentication Code, or MAC); that is, a valid signature can only be generated and verified by parties that possess the common shared key.
  3. Sealing -- The NTLMSSP implements a symmetric-key encryption mechanism, which provides message confidentiality. In the case of NTLM, sealing also implies signing (a signed message is not necessarily sealed, but all sealed messages are signed).

NTLM has been largely supplanted by Kerberos as the authentication protocol of choice for domain-based scenarios. However, Kerberos is a trusted-third-party scheme, and cannot be used in situations where no trusted third party exists; for example, member servers (servers that are not part of a domain), local accounts, and authentication to resources in an untrusted domain. In such scenarios, NTLM continues to be the primary authentication mechanism (and likely will be for a long time).

NTLM Terminology

Before we start digging in any further, we will need to define a few terms used in the various protocols.

NTLM authentication is a challenge-response scheme, consisting of three messages, commonly referred to as Type 1 (negotiation), Type 2 (challenge) and Type 3 (authentication). It basically works like this:

  1. The client sends a Type 1 message to the server. This primarily contains a list of features supported by the client and requested of the server.
  2. The server responds with a Type 2 message. This contains a list of features supported and agreed upon by the server. Most importantly, however, it contains a challenge generated by the server.
  3. The client replies to the challenge with a Type 3 message. This contains several pieces of information about the client, including the domain and username of the client user. It also contains one or more responses to the Type 2 challenge.

The responses in the Type 3 message are the most critical piece, as they prove to the server that the client user has knowledge of the account password.

The process of authentication establishes a shared context between the two involved parties; this includes a shared session key, used for subsequent signing and sealing operations.

In this document, to avoid confusion (as much as possible, anyway) the following convention will be observed:

This should keep things fairly clear, except for the possibly awkward case of "NTLM2 Session Response" authentication (a variant of NTLMv1 authentication that is used in conjunction with NTLM2 session security). Hopefully by the time we get there this will all make much more sense.

For our purposes, a "short" is a little-endian, 16-bit unsigned value. For example, the decimal value "1234" represented as a short would be physically laid out as "0xd204" in hexadecimal.

A "long" is a little-endian, 32-bit unsigned value. The decimal value "1234" represented as a long in hexidecimal would be "0xd2040000".

A Unicode string is a string in which each character is represented as a 16-bit little-endian value (16-bit UCS-2 Transformation Format, little-endian byte order, with no Byte Order Mark and no null-terminator). The string "hello" in Unicode would be represented hexidecimally as "0x680065006c006c006f00".

An OEM string is a string in which each character is represented as an 8-bit value from the local machine's native character set (DOS codepage). There is no null-terminator. In NTLM messages, OEM strings are typically presented in uppercase. The string "HELLO" in OEM would be represented hexidecimally as "0x48454c4c4f".

A "security buffer" is a structure used to point to a buffer of binary data. It consists of:

  1. A short containing the length of the buffer in bytes.
  2. A short containing the allocated space for the buffer in bytes (typically, though not necessarily, the same as the length).
  3. A long containing the offset to the start of the buffer in bytes (from the beginning of the NTLM message).

So the security buffer "0xd204d204e1100000" would be read as:

Length: 0xd204 (1234 bytes)
Allocated Space: 0xd204 (1234 bytes)
Offset: 0xe1100000 (4321 bytes)

If you started at the first byte in the message, and skipped ahead 4321 bytes, you would be at the start of the data buffer. You would read 1234 bytes (which is the length of the buffer). Since the allocated space for the buffer is also 1234 bytes, you would then be at the end of the buffer.

The NTLM Message Header Layout

Now we're ready to look at the physical layout of NTLM authentication message headers.

All messages start with the NTLMSSP signature, which is (aptly enough) the null-terminated ASCII string "NTLMSSP" (hexadecimal "0x4e544c4d53535000").

Next is a long containing the message type (1, 2, or 3). A Type 1 message, for example, has type "0x01000000" in hex.

This is followed by message-specific information, typically consisting of security buffers and the message flags.

The NTLM Flags

The message flags are contained in a bitfield within the header. This is a long, in which each bit represents a specific flag. Most of these will make more sense later, but we'll go ahead and present them here to establish a frame of reference for the rest of the discussion. Flags marked as "unidentified" or "unknown" in the table below are outside the realm of the author's knowledge (which is not by any means absolute).

FlagNameDescription
0x00000001Negotiate Unicode Indicates that Unicode strings are supported for use in security buffer data.
0x00000002Negotiate OEM Indicates that OEM strings are supported for use in security buffer data.
0x00000004Request Target Requests that the server's authentication realm be included in the Type 2 message.
0x00000008unknown This flag's usage has not been identified.
0x00000010Negotiate Sign Specifies that authenticated communication between the client and server should carry a digital signature (message integrity).
0x00000020Negotiate Seal Specifies that authenticated communication between the client and server should be encrypted (message confidentiality).
0x00000040Negotiate Datagram Style Indicates that datagram authentication is being used.
0x00000080Negotiate Lan Manager Key Indicates that the Lan Manager Session Key should be used for signing and sealing authenticated communications.
0x00000100Negotiate Netware This flag's usage has not been identified.
0x00000200Negotiate NTLM Indicates that NTLM authentication is being used.
0x00000400unknown This flag's usage has not been identified.
0x00000800Negotiate Anonymous Sent by the client in the Type 3 message to indicate that an anonymous context has been established. This also affects the response fields (as detailed in the "Anonymous Response" section).
0x00001000Negotiate Domain Supplied Sent by the client in the Type 1 message to indicate that the name of the domain in which the client workstation has membership is included in the message. This is used by the server to determine whether the client is eligible for local authentication.
0x00002000Negotiate Workstation Supplied Sent by the client in the Type 1 message to indicate that the client workstation's name is included in the message. This is used by the server to determine whether the client is eligible for local authentication.
0x00004000Negotiate Local Call Sent by the server to indicate that the server and client are on the same machine. Implies that the client may use the established local credentials for authentication instead of calculating a response to the challenge.
0x00008000Negotiate Always Sign Indicates that authenticated communication between the client and server should be signed with a "dummy" signature.
0x00010000Target Type Domain Sent by the server in the Type 2 message to indicate that the target authentication realm is a domain.
0x00020000Target Type Server Sent by the server in the Type 2 message to indicate that the target authentication realm is a server.
0x00040000Target Type Share Sent by the server in the Type 2 message to indicate that the target authentication realm is a share. Presumably, this is for share-level authentication. Usage is unclear.
0x00080000Negotiate NTLM2 Key Indicates that the NTLM2 signing and sealing scheme should be used for protecting authenticated communications. Note that this refers to a particular session security scheme, and is not related to the use of NTLMv2 authentication. This flag can, however, have an effect on the response calculations (as detailed in the "NTLM2 Session Response" section).
0x00100000Request Init Response This flag's usage has not been identified.
0x00200000Request Accept Response This flag's usage has not been identified.
0x00400000Request Non-NT Session Key This flag's usage has not been identified.
0x00800000Negotiate Target Info Sent by the server in the Type 2 message to indicate that it is including a Target Information block in the message. The Target Information block is used in the calculation of the NTLMv2 response.
0x01000000unknown This flag's usage has not been identified.
0x02000000unknown This flag's usage has not been identified.
0x04000000unknown This flag's usage has not been identified.
0x08000000unknown This flag's usage has not been identified.
0x10000000unknown This flag's usage has not been identified.
0x20000000Negotiate 128 Indicates that 128-bit encryption is supported.
0x40000000Negotiate Key Exchange Indicates that the client will provide an encrypted master key in the "Session Key" field of the Type 3 message.
0x80000000Negotiate 56 Indicates that 56-bit encryption is supported.

As an example, consider a message specifying:

Negotiate Unicode (0x00000001)
Request Target (0x00000004)
Negotiate NTLM (0x00000200)
Negotiate Always Sign (0x00008000)

Combining the above gives "0x00008205". This would be physically laid out as "0x05820000" (since it is represented in little-endian byte order).

The Type 1 Message

Let's jump in and take a look at the Type 1 message:

Description Content
0NTLMSSP Signature Null-terminated ASCII "NTLMSSP" (0x4e544c4d53535000)
8NTLM Message Type long (0x01000000)
12Flagslong
(16)Supplied Domain (Optional)security buffer
(24)Supplied Workstation (Optional) security buffer
(32)start of data block (if required)

The Type 1 message is sent from the client to the server to initiate NTLM authentication. Its primary purpose is to establish the "ground rules" for authentication by indicating supported options via the flags. Optionally, it can also provide the server with the client's workstation name and the domain in which the client workstation has membership; this information is used by the server to determine whether the client is eligible for local authentication.

Typically, the Type 1 message contains flags from the following set:

Negotiate Unicode (0x00000001) The client sets this flag to indicate that it supports Unicode strings.
Negotiate OEM (0x00000002) This is set to indicate that the client supports OEM strings.
Request Target (0x00000004) This requests that the server send the authentication target with the Type 2 reply.
Negotiate NTLM (0x00000200) Indicates that NTLM authentication is supported.
Negotiate Domain Supplied (0x00001000) When set, the client will send with the message the name of the domain in which the workstation has membership.
Negotiate Workstation Supplied (0x00002000) Indicates that the client is sending its workstation name with the message.
Negotiate Always Sign (0x00008000) Indicates that communication between the client and server after authentication should carry a "dummy" signature.
Negotiate NTLM2 Key (0x00080000) Indicates that this client supports the NTLM2 signing and sealing scheme; if negotiated, this can also affect the response calculations.
Negotiate 128 (0x20000000) Indicates that this client supports strong (128-bit) encryption.
Negotiate 56 (0x80000000) Indicates that this client supports medium (56-bit) encryption.

The supplied domain is a security buffer containing the domain in which the client workstation has membership. This is always in OEM format, even if Unicode is supported by the client.

The supplied workstation is a security buffer containing the client workstation's name. This, too, is in OEM rather than Unicode.

Note that the supplied domain and workstation are optional fields; they may be empty (security buffer indicating a length of zero), or may not be sent at all (security buffer omitted altogether). If the supplied domain and workstation are omitted, the Type 1 message carries no data block (the message ends after the flags field, and is a fixed-length 16-byte structure). The "most-minimal" well-formed Type 1 message, therefore, would be: 

    4e544c4d535350000100000002020000

This message contains only the NTLMSSP signature, the NTLM message type, and the minimal set of flags (Negotiate NTLM and Negotiate OEM).

Type 1 Message Example

Consider the following hexadecimal Type 1 Message: 

    4e544c4d535350000100000007320000060006002b0000000b000b0020000000
    574f524b53544154494f4e444f4d41494e

We break this up as follows:

0 0x4e544c4d53535000NTLMSSP Signature
8 0x01000000Type 1 Indicator
12 0x07320000 Flags:

Negotiate Unicode (0x00000001)
Negotiate OEM (0x00000002)
Request Target (0x00000004)
Negotiate NTLM (0x00000200)
Negotiate Domain Supplied (0x00001000)
Negotiate Workstation Supplied (0x00002000)

16 0x060006002b000000 Supplied Domain Security Buffer:

Length: 6 bytes (0x0600)
Allocated Space: 6 bytes (0x0600)
Offset: 43 bytes (0x2b000000)

24 0x0b000b0020000000 Supplied Workstation Security Buffer:

Length: 11 bytes (0x0b00)
Allocated Space: 11 bytes (0x0b00)
Offset: 32 bytes (0x20000000)

32 0x574f524b53544154494f4e Supplied Workstation Data ("WORKSTATION")
43 0x444f4d41494e Supplied Domain Data ("DOMAIN")

Analyzing this information, we can see:

Note that the supplied workstation and domain are in OEM format. Additionally, the order in which the security buffer data blocks are laid out is unimportant; in the example, the workstation data is placed before the domain data.

After creating the Type 1 message, the client sends it to the server. The server analyzes the message, much as we have just done, and creates a reply. This brings us to our next topic, the Type 2 message.

The Type 2 Message

Description Content
0NTLMSSP Signature Null-terminated ASCII "NTLMSSP" (0x4e544c4d53535000)
8NTLM Message Type long (0x02000000)
12Target Namesecurity buffer
20Flagslong
24Challenge8 bytes
(32)Context (optional) 8 bytes (two consecutive longs)
(40)Target Information (optional) security buffer
32 (48) start of data block

The Type 2 message is sent by the server to the client in response to the client's Type 1 message. It serves to complete the negotiation of options with the client, and also provides a challenge to the client. It may optionally contain information about the authentication target.

Typical Type 2 message flags include:

Negotiate Unicode (0x00000001) The server sets this flag to indicate that it will be using Unicode strings. This should only be set if the client indicates (in the Type 1 message) that it supports Unicode. Either this flag or Negotiate OEM should be set, but not both.
Negotiate OEM (0x00000002) This flag is set to indicate that the server will be using OEM strings. This should only be set if the client indicates (in the Type 1 message) that it will support OEM strings. Either this flag or Negotiate Unicode should be set, but not both.
Request Target (0x00000004) This flag is often set in the Type 2 message; while it has a well-defined meaning within the Type 1 message, its semantics here are unclear.
Negotiate NTLM (0x00000200) Indicates that NTLM authentication is supported.
Negotiate Local Call (0x00004000) The server sets this flag to inform the client that the server and client are on the same machine. The server provides a local security context handle with the message.
Negotiate Always Sign (0x00008000) Indicates that communication between the client and server after authentication should carry a "dummy" signature.
Target Type Domain (0x00010000) The server sets this flag to indicate that the authentication target is being sent with the message and represents a domain.
Target Type Server (0x00020000) The server sets this flag to indicate that the authentication target is being sent with the message and represents a server.
Target Type Share (0x00040000) The server apparently sets this flag to indicate that the authentication target is being sent with the message and represents a network share. This has not been confirmed.
Negotiate NTLM2 Key (0x00080000) Indicates that this server supports the NTLM2 signing and sealing scheme; if negotiated, this can also affect the client's response calculations.
Negotiate Target Info (0x00800000) The server sets this flag to indicate that a Target Information block is being sent with the message.
Negotiate 128 (0x20000000) Indicates that this server supports strong (128-bit) encryption.
Negotiate 56 (0x80000000) Indicates that this server supports medium (56-bit) encryption.

The target name is a security buffer containing the name of the authentication target. This is typically sent in response to a client requesting the target (via the Request Target flag in the Type 1 message). This can contain a domain, server, or (apparently) a network share. The target type is indicated via the Target Type Domain, Target Type Server, and Target Type Share flags. The target name can be either Unicode or OEM, as indicated by the presence of the appropriate flag in the Type 2 message.

The challenge is an 8-byte block of random data. The client will use this to formulate a response.

The context field is typically populated when Negotiate Local Call is set. It contains an SSPI context handle, which allows the client to "short-circuit" authentication and effectively circumvent responding to the challenge. Physically, the context is two long values. This is covered in greater detail later, in the "Local Authentication" section.

The target information is a security buffer containing a Target Information block, which is used in calculating the NTLMv2 response (discussed later). This is composed of a sequence of subblocks, each consisting of:

FieldContentDescription
Typeshort Indicates the type of data in this subblock:
1 (0x0100):Server name
2 (0x0200):Domain name
3 (0x0300): Fully-qualified DNS host name (i.e., server.domain.com)
4 (0x0400): DNS domain name (i.e., domain.com)
Lengthshort Length in bytes of this subblock's content field
ContentUnicode string Content as indicated by the type field. Always sent in Unicode, even when OEM is indicated by the message flags.

The sequence is terminated by a terminator subblock; this is a subblock of type "0", of zero length. Subblocks of type "5" have also been encountered, apparently containing the "parent" DNS domain for servers in subdomains; it may be that there are other as-yet-unidentified subblock types as well.

The context and target information may be omitted, in which case the data block begins at offset 32 (immediately following the challenge). A minimal Type 2 message would look something like this: 

    4e544c4d53535000020000000000000000000000020200000123456789abcdef

This message contains the NTLMSSP signature, the NTLM message type, an empty target name, minimal flags (Negotiate NTLM and Negotiate OEM), and the challenge.

Type 2 Message Example

Let's look at the following hexadecimal Type 2 Message: 

    4e544c4d53535000020000000c000c003000000001028100
    0123456789abcdef0000000000000000620062003c000000
    44004f004d00410049004e0002000c0044004f004d004100
    49004e0001000c0053004500520056004500520004001400
    64006f006d00610069006e002e0063006f006d0003002200
    7300650072007600650072002e0064006f006d0061006900
    6e002e0063006f006d0000000000

Breaking this into its constituent fields gives:

0 0x4e544c4d53535000NTLMSSP Signature
8 0x02000000Type 2 Indicator
12 0x0c000c0030000000 Target Name Security Buffer:

Length: 12 bytes (0x0c00)
Allocated Space: 12 bytes (0x0c00)
Offset: 48 bytes (0x30000000)

20 0x01028100 Flags:

Negotiate Unicode (0x00000001)
Negotiate NTLM (0x00000200)
Target Type Domain (0x00010000)
Negotiate Target Info (0x00800000)

24 0x0123456789abcdefChallenge
32 0x0000000000000000Context
40 0x620062003c000000 Target Information Security Buffer:

Length: 98 bytes (0x6200)
Allocated Space: 98 bytes (0x6200)
Offset: 60 bytes (0x3c000000)

48 
0x44004f004d004100
  49004e00
 Target Name Data ("DOMAIN")
60 
0x02000c0044004f00
  4d00410049004e00
  01000c0053004500
  5200560045005200
  0400140064006f00
  6d00610069006e00
  2e0063006f006d00
  0300220073006500
  7200760065007200
  2e0064006f006d00
  610069006e002e00
  63006f006d000000
  0000
 Target Information Data: 

0x02000c0044004f00
  4d00410049004e00
 Domain name subblock:

Type: 2 (Domain name, 0x0200)
Length: 12 bytes (0x0c00)
Data: "DOMAIN" 

0x01000c0053004500
  5200560045005200
 Server name subblock:

Type: 1 (Server name, 0x0100)
Length: 12 bytes (0x0c00)
Data: "SERVER" 

0x0400140064006f00
  6d00610069006e00
  2e0063006f006d00
 DNS domain name subblock:

Type: 4 (DNS domain name, 0x0400)
Length: 20 bytes (0x1400)
Data: "domain.com" 

0x0300220073006500
  7200760065007200
  2e0064006f006d00
  610069006e002e00
  63006f006d00
 DNS server name subblock:

Type: 3 (DNS server name, 0x0300)
Length: 34 bytes (0x2200)
Data: "server.domain.com"

0x00000000 Terminator subblock:

Type: 0 (terminator, 0x0000)
Length: 0 bytes (0x0000)

An analysis of this message shows:

Note that the target name is in Unicode format (as specified by the Negotiate Unicode flag).

After the server creates the Type 2 message, it is sent to the client. The response to the server's challenge is provided in the client's Type 3 message.

The Type 3 Message

Description Content
0NTLMSSP Signature Null-terminated ASCII "NTLMSSP" (0x4e544c4d53535000)
8NTLM Message Type long (0x03000000)
12LM/LMv2 Response security buffer
20NTLM/NTLMv2 Response security buffer
28Domain Name security buffer
36User Name security buffer
44Workstation Name security buffer
(52)Session Key (optional) security buffer
(60)Flags (optional) long
52 (64) start of data block

The Type 3 message is the final step in authentication. This message contains the client's responses to the Type 2 challenge, which demonstrate that the client has knowledge of the account password without sending the password directly. The Type 3 message also indicates the domain and username of the authenticating account, as well as the client workstation name.

Note that the flags in the Type 3 message are optional; older clients include neither the session key nor the flags in the message. In this case, the data block begins at offset 52, immediately following the workstation name security buffer. It has been determined experimentally that the Type 3 flags (when included) do not carry any additional semantics in connection-oriented authentication; they do not appear to have any discernable effect on either authentication or the establishment of session security. Clients sending flags typically mirror the established Type 2 settings fairly closely. It is possible that the flags are sent as a "reminder" of established options, to allow the server to avoid caching the negotiated settings. The Type 3 flags are relevant during datagram-style authentication, however.

The LM/LMv2 and NTLM/NTLMv2 responses are security buffers containing replies created from the user's password in response to the Type 2 challenge; the process for generating these responses is outlined in the next section.

The domain name is a security buffer containing the authentication realm in which the authenticating account has membership. This is either Unicode or OEM, depending on the negotiated encoding.

The user name is a security buffer containing the authenticating account name. This is either Unicode or OEM, depending on the negotiated encoding.

The workstation name is a security buffer containing the client workstation's name. This is either Unicode or OEM, depending on the negotiated encoding.

The session key value is used by the session security mechanism during key exchange; this is discussed in more detail in the Session Security section.

When "Negotiate Local Call" has been established in the Type 2 message, the security buffers in the Type 3 message are typically all empty (zero length). The client "adopts" the SSPI context sent in the Type 2 message, effectively circumventing the need to calculate an appropriate response.

Responding to the Challenge

The client creates one or more responses to the Type 2 challenge, and sends these in the Type 3 message. There are six types of responses:

For more detailed information on these schemes, it is highly recommended that you read Christopher Hertel's Implementing CIFS, especially the section on authentication.

The LM Response

The LM response is sent by most clients. This scheme is older than the NTLM response, and less secure. While newer clients support the NTLM response, they typically send both responses for compatibility with legacy servers; hence, the security flaws present in the LM response are still exhibited in many clients supporting the NTLM response.

The LM response is calculated as follows (see Appendix D for a sample implementation in Java):

  1. The user's password (as an OEM string) is converted to uppercase.
  2. This password is either null-padded or truncated to 14 bytes.
  3. This "fixed" password is split into two 7-byte halves.
  4. These values are used to create two DES keys (one from each 7-byte half).
  5. Each of these keys is used to DES-encrypt the constant ASCII string "KGS!@#$%" (resulting in two 8-byte ciphertext values).
  6. These two ciphertext values are concatenated to form a 16-byte value - the LM hash.
  7. The 16-byte LM hash is null-padded to 21 bytes.
  8. This value is split into three 7-byte thirds.
  9. These values are used to create three DES keys (one from each 7-byte third).
  10. Each of these keys is used to DES-encrypt the challenge from the Type 2 message (resulting in three 8-byte ciphertext values).
  11. These three ciphertext values are concatenated to form a 24-byte value. This is the LM response.

This process is best illustrated with a detailed example. Consider a user with the password "SecREt01", responding to the Type 2 challenge "0x0123456789abcdef".

  1. The password (as an OEM string) is converted to uppercase, giving "SECRET01" (or "0x5345435245543031" in hexadecimal).
  2. This password is null-padded to 14 bytes, giving "0x5345435245543031000000000000".
  3. This value is split into two 7-byte halves, "0x53454352455430" and "0x31000000000000".
  4. These two values are used to create two DES keys. A DES key is 8 bytes long; each byte contains seven bits of key material and one odd-parity bit (the parity bit may or may not be checked, depending on the underlying DES implementation). Our first 7-byte value, "0x53454352455430", would be represented in binary as:

    01010011 01000101 01000011 01010010 01000101 01010100 00110000

    A non-parity-adjusted DES key for this value would be:

    01010010 10100010 01010000 01101010 00100100 00101010 01010000 01100000

    (the parity bits are shown in red above). This is "0x52a2506a242a5060" in hexadecimal. Applying odd-parity to ensure that the total number of set bits in each octet is odd gives:

    01010010 10100010 01010001 01101011 00100101 00101010 01010001 01100001

    This is the first DES key ("0x52a2516b252a5161" in hex). We then apply the same process to our second 7-byte value, "0x31000000000000", represented in binary as:

    00110001 00000000 00000000 00000000 00000000 00000000 00000000

    Creating a non-parity-adjusted DES key gives:

    00110000 10000000 00000000 00000000 00000000 00000000 00000000 00000000

    ("0x3080000000000000" in hexadecimal). Adjusting the parity bits gives:

    00110001 10000000 00000001 00000001 00000001 00000001 00000001 00000001

    This is our second DES key, "0x3180010101010101" in hexadecimal. Note that if our particular DES implementation does not enforce parity (many do not), the parity-adjustment steps can be skipped; the non-parity-adjusted values would then be used as the DES keys. In any case, the parity bits will not affect the encryption process.

  5. Each of our keys is used to DES-encrypt the constant ASCII string "KGS!@#$%" ("0x4b47532140232425" in hex). This gives us "0xff3750bcc2b22412" (using the first key) and "0xc2265b23734e0dac" (using the second).
  6. These ciphertext values are concatenated to form our 16-byte LM hash - "0xff3750bcc2b22412c2265b23734e0dac".
  7. This is null-padded to 21 bytes, giving "0xff3750bcc2b22412c2265b23734e0dac0000000000".
  8. This value is split into three 7-byte thirds, "0xff3750bcc2b224", "0x12c2265b23734e" and "0x0dac0000000000".
  9. These three values are used to create three DES keys. Using the process outlined previously, our first value:

    11111111 00110111 01010000 10111100 11000010 10110010 00100100

    Gives us the parity-adjusted DES key:

    11111110 10011011 11010101 00010110 11001101 00010101 11001000 01001001

    ("0xfe9bd516cd15c849" in hexadecimal). The second value:

    00010010 11000010 00100110 01011011 00100011 01110011 01001110

    Results in the key:

    00010011 01100001 10001001 11001011 10110011 00011010 11001101 10011101

    ("0x136189cbb31acd9d"). Finally, the third value:

    00001101 10101100 00000000 00000000 00000000 00000000 00000000

    Gives us:

    00001101 11010110 00000001 00000001 00000001 00000001 00000001 00000001

    This is the third DES key ("0x0dd6010101010101").

  10. Each of the three keys is used to DES-encrypt the challenge from the Type 2 message (in our example, "0x0123456789abcdef"). This gives the results "0xc337cd5cbd44fc97" (using the first key), "0x82a667af6d427c6d" (using the second) and "0xe67c20c2d3e77c56" (using the third).
  11. These three ciphertext values are concatenated to form the 24-byte LM response:

    0xc337cd5cbd44fc9782a667af6d427c6de67c20c2d3e77c56

There are several weaknesses in this algorithm which make it susceptible to attack. While these are covered in detail in the Hertel text, the most prominent problems are:

The NTLM Response

The NTLM response is sent by newer clients. This scheme addresses some of the flaws in the LM response; however, it is still considered fairly weak. Additionally, the NTLM response is nearly always sent in conjunction with the LM response. The weaknesses in that algorithm can be exploited to obtain the case-insensitive password, and trial-and-error used to find the case-sensitive password employed by the NTLM response.

The NTLM response is calculated as follows (see Appendix D for a sample Java implementation):

  1. The MD4 message-digest algorithm (described in RFC 1320) is applied to the Unicode mixed-case password. This results in a 16-byte value - the NTLM hash.
  2. The 16-byte NTLM hash is null-padded to 21 bytes.
  3. This value is split into three 7-byte thirds.
  4. These values are used to create three DES keys (one from each 7-byte third).
  5. Each of these keys is used to DES-encrypt the challenge from the Type 2 message (resulting in three 8-byte ciphertext values).
  6. These three ciphertext values are concatenated to form a 24-byte value. This is the NTLM response.

Note that only the calculation of the hash value differs from the LM scheme; the response calculation is the same. To illustrate this process, we will apply it to our previous example (a user with the password "SecREt01", responding to the Type 2 challenge "0x0123456789abcdef").

  1. The Unicode mixed-case password is "0x53006500630052004500740030003100" in hexadecimal; the MD4 hash of this value is calculated, giving "0xcd06ca7c7e10c99b1d33b7485a2ed808". This is the NTLM hash.
  2. This is null-padded to 21 bytes, giving "0xcd06ca7c7e10c99b1d33b7485a2ed8080000000000".
  3. This value is split into three 7-byte thirds, "0xcd06ca7c7e10c9", "0x9b1d33b7485a2e" and "0xd8080000000000".
  4. These three values are used to create three DES keys. Our first value:

    11001101 00000110 11001010 01111100 01111110 00010000 11001001

    Results in the parity-adjusted key:

    11001101 10000011 10110011 01001111 11000111 11110001 01000011 10010010

    ("0xcd83b34fc7f14392" in hexadecimal). The second value:

    10011011 00011101 00110011 10110111 01001000 01011010 00101110

    Gives the key:

    10011011 10001111 01001100 01110110 01110101 01000011 01101000 01011101

    ("0x9b8f4c767543685d"). Our third value:

    11011000 00001000 00000000 00000000 00000000 00000000 00000000

    Yields our third key:

    11011001 00000100 00000001 00000001 00000001 00000001 00000001 00000001

    ("0xd904010101010101" in hexadecimal).

  5. Each of the three keys is used to DES-encrypt the challenge from the Type 2 message ("0x0123456789abcdef"). This yields the results "0x25a98c1c31e81847" (using our first key), "0x466b29b2df4680f3" (using the second) and "0x9958fb8c213a9cc6" (using the third key).
  6. These three ciphertext values are concatenated to form the 24-byte NTLM response:

    0x25a98c1c31e81847466b29b2df4680f39958fb8c213a9cc6

The NTLMv2 Response

NTLM version 2 ("NTLMv2") was concocted to address the security issues present in NTLM. While its effectiveness in this regard is questionable, it does at least provide a more secure replacement for the LM response. When NTLMv2 is enabled, the NTLM response is replaced with the NTLMv2 response, and the LM response is replaced with the LMv2 response (which we will discuss next).

The NTLMv2 response is calculated as follows (see Appendix D for a sample implementation in Java):

  1. The NTLM password hash is obtained (as discussed previously, this is the MD4 digest of the Unicode mixed-case password).
  2. The Unicode uppercase username is concatenated with the Unicode uppercase authentication target (domain or server name). The HMAC-MD5 message authentication code algorithm (described in RFC 2104) is applied to this value using the 16-byte NTLM hash as the key. This results in a 16-byte value - the NTLMv2 hash.
  3. A block of data known as the "blob" is constructed. The Hertel text discusses the format of this structure in greater detail; briefly:
    Description Content
    0Blob Signature 0x01010000
    4Reserved long (0x00000000)
    8Timestamp Little-endian, 64-bit signed value representing the number of tenths of a microsecond since January 1, 1601.
    16Client Nonce8 bytes
    24Unknown4 bytes
    28Target Information Target Information block (from the Type 2 message).
    (variable)Unknown 4 bytes
  4. The challenge from the Type 2 message is concatenated with the blob. The HMAC-MD5 message authentication code algorithm is applied to this value using the 16-byte NTLMv2 hash (calculated in step 2) as the key. This results in a 16-byte output value.
  5. This value is concatenated with the blob to form the NTLMv2 response.

Let's look at an example. Since we need a bit more information to calculate the NTLMv2 response, we will use the following values from the examples presented previously:

Domain: DOMAIN
Username: user
Password: SecREt01
Challenge: 0x0123456789abcdef
Target Information: 
0x02000c0044004f00
  4d00410049004e00
  01000c0053004500
  5200560045005200
  0400140064006f00
  6d00610069006e00
  2e0063006f006d00
  0300220073006500
  7200760065007200
  2e0064006f006d00
  610069006e002e00
  63006f006d000000
  0000

  1. The Unicode mixed-case password is "0x53006500630052004500740030003100" in hexadecimal; the MD4 hash of this value is calculated, giving "0xcd06ca7c7e10c99b1d33b7485a2ed808". This is the NTLM hash.
  2. The Unicode uppercase username is concatenated with the Unicode uppercase authentication target, giving "USERDOMAIN" (or "0x550053004500520044004f004d00410049004e00" in hexadecimal). HMAC-MD5 is applied to this value using the 16-byte NTLM hash from the previous step as the key, which yields "0x04b8e0ba74289cc540826bab1dee63ae". This is the NTLMv2 hash.
  3. Next, the blob is constructed. The timestamp is the most tedious part of this; looking at the clock on my desk, it's about 6:00 AM EDT on June 17th, 2003. In Unix time, that would be 1055844000 seconds after the Epoch. Adding 11644473600 will give us seconds after January 1, 1601 (12700317600). Multiplying by 107 (10000000) will give us tenths of a microsecond (127003176000000000). As a little-endian 64-bit value, this is "0x0090d336b734c301" (in hexadecimal).

    We also need to generate an 8-byte random "client nonce"; we will use the not-so-random "0xffffff0011223344". Constructing the rest of the blob is easy; we just concatenate:

    0x01010000(the blob signature)
    0x00000000(reserved value)
    0x0090d336b734c301(our timestamp)
    0xffffff0011223344(a random client nonce)
    0x00000000(unknown, but zero will work) 
    0x02000c0044004f00
  4d00410049004e00
  01000c0053004500
  5200560045005200
  0400140064006f00
  6d00610069006e00
  2e0063006f006d00
  0300220073006500
  7200760065007200
  2e0064006f006d00
  610069006e002e00
  63006f006d000000
  0000
    		 (our target information block)
    0x00000000(unknown, but zero will work)

  4. We then concatenate the Type 2 challenge with our blob: 
    0x0123456789abcdef0101000000000000
  0090d336b734c301ffffff0011223344
  0000000002000c0044004f004d004100
  49004e0001000c005300450052005600
  450052000400140064006f006d006100
  69006e002e0063006f006d0003002200
  7300650072007600650072002e006400
  6f006d00610069006e002e0063006f00
  6d000000000000000000

    Applying HMAC-MD5 to this value using the NTLMv2 hash from step 2 as the key gives us the 16-byte value "0xcbabbca713eb795d04c97abc01ee4983".

  5. This value is concatenated with the blob to obtain the NTLMv2 response: 
    0xcbabbca713eb795d04c97abc01ee4983
  01010000000000000090d336b734c301
  ffffff00112233440000000002000c00
  44004f004d00410049004e0001000c00
  53004500520056004500520004001400
  64006f006d00610069006e002e006300
  6f006d00030022007300650072007600
  650072002e0064006f006d0061006900
  6e002e0063006f006d00000000000000
  0000

The LMv2 Response

The LMv2 response is used to provide pass-through authentication compatibility with older servers. It is quite possible that the server with which the client is communicating will not actually perform the authentication; rather, it will pass the responses through to a domain controller for verification. Older servers pass only the LM response, and expect it to be exactly 24 bytes. The LMv2 response was designed to allow such servers to operate properly; it is effectively a "miniature" NTLMv2 response, obtained as follows (see Appendix D for a sample Java implementation):

  1. The NTLM password hash is calculated (the MD4 digest of the Unicode mixed-case password).
  2. The Unicode uppercase username is concatenated with the Unicode uppercase authentication target (domain or server name). The HMAC-MD5 message authentication code algorithm is applied to this value using the 16-byte NTLM hash as the key. This results in a 16-byte value - the NTLMv2 hash.
  3. A random 8-byte client nonce is created (this is the same client nonce used in the NTLMv2 blob).
  4. The challenge from the Type 2 message is concatenated with the client nonce. The HMAC-MD5 message authentication code algorithm is applied to this value using the 16-byte NTLMv2 hash (calculated in step 2) as the key. This results in a 16-byte output value.
  5. This value is concatenated with the 8-byte client nonce to form the 24-byte LMv2 response.

We will illustrate this process with a brief example using our tried-and-true sample values:

Domain: DOMAIN
Username: user
Password: SecREt01
Challenge: 0x0123456789abcdef

  1. The Unicode mixed-case password is "0x53006500630052004500740030003100" in hexadecimal; the MD4 hash of this value is calculated, giving "0xcd06ca7c7e10c99b1d33b7485a2ed808". This is the NTLM hash.
  2. The Unicode uppercase username is concatenated with the Unicode uppercase authentication target, giving "USERDOMAIN" (or "0x550053004500520044004f004d00410049004e00" in hexadecimal). HMAC-MD5 is applied to this value using the 16-byte NTLM hash from the previous step as the key, which yields "0x04b8e0ba74289cc540826bab1dee63ae". This is the NTLMv2 hash.
  3. A random 8-byte client nonce is created. From our NTLMv2 example, we will use "0xffffff0011223344".
  4. We then concatenate the Type 2 challenge with our client nonce:

    0x0123456789abcdefffffff0011223344

    Applying HMAC-MD5 to this value using the NTLMv2 hash from step 2 as the key gives us the 16-byte value "0xd6e6152ea25d03b7c6ba6629c2d6aaf0".

  5. This value is concatenated with the client nonce to obtain the 24-byte LMv2 response:

    0xd6e6152ea25d03b7c6ba6629c2d6aaf0ffffff0011223344

The NTLM2 Session Response

The NTLM2 session response can be employed in conjunction with NTLM2 session security (it is made available with the "Negotiate NTLM2 Key" flag). This is used to provide enhanced protection against precomputed dictionary attacks (particularly Rainbow Table-based attacks) in environments which do not support full NTLMv2 authentication.

The NTLM2 session response replaces both the LM and NTLM response fields as follows (see Appendix D for a sample implementation in Java):

  1. A random 8-byte client nonce is created.
  2. The client nonce is null-padded to 24 bytes. This value is placed in the LM response field of the Type 3 message.
  3. The challenge from the Type 2 message is concatenated with the 8-byte client nonce to form a session nonce.
  4. The MD5 message-digest algorithm (described in RFC 1321) is applied to the session nonce, resulting in a 16-byte value.
  5. This value is truncated to 8 bytes to form the NTLM2 session hash.
  6. The NTLM password hash is obtained (as discussed, this is the MD4 digest of the Unicode mixed-case password).
  7. The 16-byte NTLM hash is null-padded to 21 bytes.
  8. This value is split into three 7-byte thirds.
  9. These values are used to create three DES keys (one from each 7-byte third).
  10. Each of these keys is used to DES-encrypt the NTLM2 session hash (resulting in three 8-byte ciphertext values).
  11. These three ciphertext values are concatenated to form a 24-byte value. This is the NTLM2 session response, which is placed in the NTLM response field of the Type 3 message.

To demonstrate this with our previous example values (a user with the password "SecREt01", responding to the Type 2 challenge "0x0123456789abcdef"):

  1. A random 8-byte client nonce is created; we will use "0xffffff0011223344", as in the previous examples.
  2. The challenge is null-padded to 24 bytes:

    0xffffff001122334400000000000000000000000000000000

    This value is placed in the LM response field of the Type 3 message.

  3. The challenge from the Type 2 message is concatenated with the client nonce, forming a session nonce ("0x0123456789abcdefffffff0011223344").
  4. Applying the MD5 digest to this nonce yields the 16-byte value "0xbeac9a1bc5a9867c15192b3105d5beb1".
  5. This is truncated to 8 bytes to obtain the NTLM2 session hash ("0xbeac9a1bc5a9867c").
  6. The Unicode mixed-case password is "0x53006500630052004500740030003100"; applying the MD4 digest to this value gives us the NTLM hash ("0xcd06ca7c7e10c99b1d33b7485a2ed808").
  7. This is null-padded to 21 bytes, giving "0xcd06ca7c7e10c99b1d33b7485a2ed8080000000000".
  8. This value is split into three 7-byte thirds, "0xcd06ca7c7e10c9", "0x9b1d33b7485a2e" and "0xd8080000000000".
  9. These values are used to create three DES keys (as calculated in our previous NTLM response example, "0xcd83b34fc7f14392", "0x9b8f4c767543685d", and "0xd904010101010101").
  10. Each of these three keys is used to DES-encrypt the NTLM2 session hash ("0xbeac9a1bc5a9867c"). This yields the results "0x10d550832d12b2cc" (using our first key), "0xb79d5ad1f4eed3df" (using the second), and "0x82aca4c3681dd455" (using the third key).
  11. These three ciphertext values are concatenated to form the 24-byte NTLM2 session response:

    0x10d550832d12b2ccb79d5ad1f4eed3df82aca4c3681dd455

    which is placed in the NTLM response field of the Type 3 message.

The Anonymous Response

The Anonymous Response is seen when the client is establishing an anonymous context, rather than a true user-based context. This is typically seen when a "placeholder" is needed for operations that do not require an authenticated user. Anonymous connections are not the same as the Windows "Guest" user (the latter is an actual user account, while anonymous connections are associated with no account at all).

In an anonymous Type 3 message, the client indicates the "Negotiate Anonymous" flag; the NTLM response field is empty (zero-length); and the LM response field contains a single null byte ("0x00").

Type 3 Message Example

Now that we're familiar with the Type 3 responses, we are ready to examine a Type 3 Message: 

    4e544c4d5353500003000000180018006a00000018001800
    820000000c000c0040000000080008004c00000016001600
    54000000000000009a0000000102000044004f004d004100
    49004e00750073006500720057004f0052004b0053005400
    4100540049004f004e00c337cd5cbd44fc9782a667af6d42
    7c6de67c20c2d3e77c5625a98c1c31e81847466b29b2df46
    80f39958fb8c213a9cc6

This message is decomposed as:

0 0x4e544c4d53535000NTLMSSP Signature
8 0x03000000Type 3 Indicator
12 0x180018006a000000 LM Response Security Buffer:

Length: 24 bytes (0x1800)
Allocated Space: 24 bytes (0x1800)
Offset: 106 bytes (0x6a000000)

20 0x1800180082000000 NTLM Response Security Buffer:

Length: 24 bytes (0x1800)
Allocated Space: 24 bytes (0x1800)
Offset: 130 bytes (0x82000000)

28 0x0c000c0040000000 Domain Name Security Buffer:

Length: 12 bytes (0x0c00)
Allocated Space: 12 bytes (0x0c00)
Offset: 64 bytes (0x40000000)

36 0x080008004c000000 User Name Security Buffer:

Length: 8 bytes (0x0800)
Allocated Space: 8 bytes (0x0800)
Offset: 76 bytes (0x4c000000)

44 0x1600160054000000 Workstation Name Security Buffer:

Length: 22 bytes (0x1600)
Allocated Space: 22 bytes (0x1600)
Offset: 84 bytes (0x54000000)

52 0x000000009a000000 Session Key Security Buffer:

Length: 0 bytes (0x0000)
Allocated Space: 0 bytes (0x0000)
Offset: 154 bytes (0x9a000000)

60 0x01020000 Flags:

Negotiate Unicode (0x00000001)
Negotiate NTLM (0x00000200)

64 
0x44004f004d004100
  49004e00
 Domain Name Data ("DOMAIN")
76 0x7500730065007200 User Name Data ("user")
84 
0x57004f0052004b00
  5300540041005400
  49004f004e00
 Workstation Name Data ("WORKSTATION")
106 
0xc337cd5cbd44fc97
  82a667af6d427c6d
  e67c20c2d3e77c56
 LM Response Data
130 
0x25a98c1c31e81847
  466b29b2df4680f3
  9958fb8c213a9cc6
 NTLM Response Data

Analysis of this reveals:

Upon receipt of the Type 3 message, the server calculates the LM and NTLM responses and compares them to the values provided by the client; if they match, the user is successfully authenticated.

NTLM Version 2

NTLM version 2 consists of three new response algorithms (NTLMv2, LMv2, and the NTLM2 session response, discussed previously) and a new signing and sealing scheme (NTLM2 session security). NTLM2 session security is negotiated via the "Negotiate NTLM2 Key" flag; NTLMv2 authentication, however, is enabled through a modification to the registry. Further, the registry setting on the client and domain controller must be compatible in order for authentication to be successful (although it is possible for NTLMv2 authentication to pass through an older server to an NTLMv2 domain controller). The result of the configuration and planning required to deploy NTLMv2 is that many hosts just use the default setting (NTLMv1), and NTLMv2 authentication is underutilized.

Instructions for enabling NTLM version 2 are detailed in Microsoft Knowledge Base Article 239869; briefly, a modification is made to the registry value: 

    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\LSA\LMCompatibilityLevel

(LMCompatibility on Win9x-based systems). This is a REG_DWORD entry, and can be set to one of the following values:

LevelSent by ClientAccepted by Server
0 LM
NTLM 		LM
NTLM
LMv2
NTLMv2
1 LM
NTLM 		LM
NTLM
LMv2
NTLMv2
2 NTLM LM
NTLM
LMv2
NTLMv2
3 LMv2
NTLMv2 		LM
NTLM
LMv2
NTLMv2
4 LMv2
NTLMv2 		NTLM
LMv2
NTLMv2
5 LMv2
NTLMv2 		LMv2
NTLMv2

In all levels, NTLM2 session security is supported and negotiated when available (most available documentation indicates that NTLM2 session security is only enabled on levels 1 and above, but it is seen in practice with Level 0 as well). By default, only the LM response is supported on Windows 95 and Windows 98 platforms; installing the Directory Services client makes NTLMv2 available on these hosts as well (and enables the LMCompatibility setting, although only levels 0 and 3 are available).

In Level 2, clients send the NTLM response twice (in both the LM and NTLM response fields). At Level 3 and higher, the LMv2 and NTLMv2 responses replace the LM and NTLM responses, respectively.

When NTLM2 session security has been negotiated (indicated by the "Negotiate NTLM2 Key" flag), the NTLM2 session response can be used in Levels 0, 1, and 2 as a replacement for the weaker LM and NTLM responses. This offers heightened protection over NTLMv1 against server-based precomputed dictionary attacks; the client's response to a given challenge is made variable by adding a random client nonce to the calculation.

The NTLM2 session response is interesting in that it can be negotiated between a client and server that support the newer schemes, even in the presence of an older domain controller that does not. In a typical scenario, the server in an authentication transaction does not actually possess the user's password hash; that is instead held at the domain controller. When a machine is joined to an NT-style domain, it establishes an encrypted, mutually-authenticated channel to the domain controller (colloquially deemed the "NetLogon pipe"). When a client authenticates to the server using the "vanilla" NTLMv1 handshake, the following transactions occur in the background:

  1. The client sends the Type 1 message, containing flags and other information as discussed previously.
  2. The server generates a challenge for the client and sends the Type 2 message containing the negotiated flag set.
  3. The client responds to the challenge, providing the LM/NTLM responses.
  4. The server sends the challenge and client responses over the NetLogon pipe to the domain controller.
  5. The domain controller uses the stored hashes and the challenge given by the server to reproduce the authentication calculations; if they match the responses, the authentication is successful.
  6. The domain controller calculates and sends the session key to the server, that can be used for subsequent signing and sealing operations between the server and the client.

In the case of the NTLM2 Session Response, it is possible that a client and server have been upgraded to allow the newer protocol, but the domain controller has not. To allow for this contingency, the handshake described above is modified slightly as follows:

  1. The client sends the Type 1 message, in this case indicating the "Negotiate NTLM2 Key" flag.
  2. The server generates a challenge for the client and sends the Type 2 message containing the negotiated flag set (also including the "Negotiate NTLM2 Key" flag).
  3. The client responds to the challenge, providing the client nonce in the LM field, and the NTLM2 Session Response in the NTLM field. Note that the latter is exactly the same calculation as the NTLM response, except instead of encrypting the server challenge the client has encrypted the MD5 hash of the server challenge concatenated with the client nonce.
  4. Instead of sending the server challenge directly over the NetLogon pipe to the domain controller, the server sends the MD5 hash of the server challenge concatenated with the client nonce (lifted from the LM response field). Additionally it sends the client responses (as usual).
  5. The domain controller encrypts the challenge field sent by the server using the stored hash as the key and notes that it matches the NTLM response field; hence, the client is successfully authenticated.
  6. The domain controller calculates and sends the normal NTLM User Session Key to the server; the server uses this in a secondary calculation to obtain the NTLM2 Session Response User Session Key (discussed in a subsequent section)

Essentially, this allows upgraded clients and servers to use the NTLM2 Session Response in networks where the domain controller has not yet been upgraded to NTLMv2 (or where the network administrator has not yet configured the LMCompatibilityLevel registry setting to use NTLMv2).

Related to the LMCompatibilityLevel setting are the NtlmMinClientSec and NtlmMinServerSec settings; these specify minimum requirements for NTLM contexts established by the NTLMSSP. Both are REG_WORD entries, and are bitfields specifying a combination of the following NTLM flags:

While most of these are more applicable to NTLM2 signing and sealing, the "Negotiate NTLM2 Key" is significant to authentication in that it can prevent sessions from being established with hosts that are unable to negotiate NTLM2 session security. This serves to ensure that the LM and NTLM responses are not sent (requiring that authentication will at least use the NTLM2 Session Response in all cases).

NTLMSSP and SSPI

At this point, we will start to look at how NTLM fits into the "big picture".

Windows provides a security framework known as SSPI - the Security Support Provider interface. This is the Microsoft equivalent of the GSS-API (Generic Security Service Application Program Interface, RFC 2743), and allows for a very high-level, mechanism-independent means of applying authentication, integrity, and confidentiality primitives. SSPI supports several underlying providers; one of these is the NTLMSSP (NTLM Security Support Provider), which provides the NTLM authentication mechanism we have been discussing thus far. SSPI supplies a flexible API for handling opaque, provider-specific authentication tokens; the NTLM Type 1, Type 2, and Type 3 messages are such tokens, specific to and processed by the NTLMSSP. The API provided by SSPI abstracts away almost all the details of NTLM. The application developer doesn't even have to be aware that NTLM is being used, and another authentication mechanism (such as Kerberos) can be swapped in with little or no changes at the application level.

We aren't going to delve too deeply into the SSPI framework, but this is a good point to look at the SSPI authentication handshake as applied to NTLM:

  1. The client obtains a representation of the credential set for the user via the SSPI AcquireCredentialsHandle function.
  2. The client calls the SSPI InitializeSecurityContext function to obtain an authentication request token (in our case, a Type 1 message). The client sends this token to the server. The return value from the function indicates that authentication will require multiple steps.
  3. The server receives the token from the client, and uses it as input to the AcceptSecurityContext SSPI function. This creates a local security context on the server to represent the client, and yields an authentication response token (the Type 2 message), which is sent to the client. The return value from the function indicates that further information is needed from the client.
  4. The client receives the response token from the server and calls InitializeSecurityContext again, passing the server's token as input. This provides us with another authentication request token (the Type 3 message). The return value indicates that the security context was successfully initialized; the token is sent to the server.
  5. The server receives the token from the client and calls AcceptSecurityContext again, using the Type 3 message as input. The return value indicates the context was successfully accepted; no token is produced, and authentication is complete.

Local Authentication

We have alluded to the local authentication sequence at various points in our discussion; having a basic understanding of SSPI, we can look at this scenario in more detail.

Local authentication is negotiated through a series of decisions made by the client and server, based on the information in the NTLM messages. It works as follows:

  1. The client calls the AcquireCredentialsHandle function, specifying the default credentials by passing in null to the "pAuthData" parameter. This obtains a handle to the credentials of the logged in user for single sign-on.
  2. The client calls the SSPI InitializeSecurityContext function to create the Type 1 message. When the default credential handle is supplied, the Type 1 message contains the workstation and domain name of the client. This is indicated by the presence of the "Negotiate Domain Supplied" and "Negotiate Workstation Supplied" flags, and the inclusion of populated Supplied Domain and Supplied Workstation security buffers in the message.
  3. The server receives the Type 1 message from the client, and calls AcceptSecurityContext. This creates a local security context on the server to represent the client. The server examines the domain and workstation information sent by the client to determine if the client and server are the same machine. If so, the server initiates local authentication by setting the "Negotiate Local Call" flag in the resultant Type 2 message. The first long in the Context field of the Type 2 message is populated with the "upper" portion of the newly obtained SSPI context handle (specifically, the "dwUpper" field of the SSPI CtxtHandle structure). The second long in the Context field appears to be empty in all cases. (although logically one would assume it should contain the "lower" portion of the context handle).
  4. The client receives the Type 2 message from the server and passes it to InitializeSecurityContext. Having noted the presence of the "Negotiate Local Call" flag, the client examines the server context handle to determine if it represents a valid local security context. If the context cannot be validated, authentication proceeds as usual - the appropriate responses are calculated, and included with the domain, workstation, and username in the Type 3 message. If the security context handle from the Type 2 message can be validated, however, no responses are prepared whatsoever. Instead, the default credentials are internally associated with the server context. The resulting Type 3 message is completely empty, containing zero-length security buffers for the responses as well as the username, domain, and workstation.
  5. The server receives the Type 3 message and uses it as input to the AcceptSecurityContext function. The server verifies that the security context has been associated with a user; if so, authentication has successfully completed. If the context has not been bound to a user, authentication fails.

Datagram Authentication

Datagram-style authentication is used to negotiate NTLM over a connectionless transport. While much of the semantics around the messages remain unchanged, there are a few significant differences:

During "normal" (connection-oriented) authentication, all options are negotiated in the first transaction between the client and the server, during the exchange of the Type 1 and Type 2 messages. The negotiated settings are "remembered" by the server and applied to the client's Type 3 message. Although most clients send the agreed-upon flags with the Type 3 message, they are not used in connection authentication.

In datagram authentication, however, the game changes a bit; to alleviate the server's need to track the negotiated options (which becomes more difficult without a persistent connection), the Type 1 message is removed completely. The server generates a Type 2 message containing all supported flags (as well as the challenge, of course). The client then decides which options it will support, and replies with a Type 3 message containing the responses to the challenge and the set of selected flags. The SSPI handshake sequence for datagram authentication is as follows:

  1. The client calls AcquireCredentialsHandle to obtain a representation of the credential set for the user.
  2. The client calls InitializeSecurityContext, passing the ISC_REQ_DATAGRAM flag as a context requirement via the fContextReq parameter. This starts the construction of the client's security context, but does not produce a request token (Type 1 message).
  3. The server calls the AcceptSecurityContext function, specifying the ASC_REQ_DATAGRAM context requirement flag and passing in a null input token. This creates the local security context and yields an authentication response token (the Type 2 message). This Type 2 message will contain the "Negotiate Datagram Style" flag, as well as all flags supported by the server. This is sent to the client as usual.
  4. The client receives the Type 2 message and passes it to InitializeSecurityContext. The client selects appropriate options from those presented by the server (including "Negotiate Datagram Style", which must be set), creates the responses to the challenge, and populates the Type 3 message. The message is then relayed to the server.
  5. The server passes the Type 3 message into the AcceptSecurityContext function. The message is processed according to the flags selected by the client, and the context is successfully accepted.

When used with SSPI, there is apparently no means of producing a datagram-style Type 1 message. It is interesting to note, however, that we can "induce" datagram semantics at a lower level by subtly manipulating the NTLMSSP tokens to produce our own datagram Type 1 token.

This can be achieved by setting the "Negotiate Datagram Style" flag on the Type 1 message produced by the first InitializeSecurityContext call in a connection-oriented SSPI handshake before passing the token to the server. When the modified Type 1 message is passed into the AcceptSecurityContext function, the server will adopt datagram semantics (even though ASC_REQ_DATAGRAM was not specified). This will produce a Type 2 message with the "Negotiate Datagram Style" flag set, but otherwise identical to the connection-oriented message that would normally have been generated; that is, the Type 1 flags sent by the client are considered during the construction of the Type 2 message, rather than simply offering all supported options.

The client can then call InitializeSecurityContext with this Type 2 token. Note that the client is still in connection-oriented mode; the Type 3 message produced will ignore the "Negotiate Datagram Style" flag applied to the Type 2 message. The server, however, is enforcing datagram semantics, and will now require the Type 3 flags to be set appropriately. Adding the "Negotiate Datagram Style" flag to the Type 3 message manually before sending it to the server allows the server to successfully call AcceptSecurityContext with the modified token.

This results in successful authentication; the "doctored" Type 1 message effectively switches the server into datagram-style authentication, in which the Type 3 flags are observed and enforced. There is no known practical use for this, but it does demonstrate some of the interesting and unexpected behavior that can be observed by strategically manipulating the NTLM messages.

Session Security - Signing & Sealing Concepts

In addition to the SSPI authentication services, message integrity and confidentiality functionality is provided. This is also implemented by the NTLM Security Support Provider. "Signing" is performed by the SSPI MakeSignature function, which applies a Message Authentication Code (MAC) to a message. This can be verified by the recipient, and provides a strong assurance that the message was not modified in transit. The signature is generated using a secret key, known to the sender and receiver; the MAC can only be verified by a party possessing the key (which in turn provides assurance that the signature was created by the sender). "Sealing" is performed by the SSPI EncryptMessage function. This applies encryption to a message to prevent it from being viewed by a third party in transit; the NTLMSSP uses a variety of symmetric encryption mechanisms (the same key is used to decrypt as to encrypt).

The keys used in signing and sealing are established as a by-product of the NTLM authentication process; in addition to verifying a client's identity, the authentication handshake establishes a context between the client and server which includes the key(s) needed to sign and seal messages between the parties. We will discuss the derivation of these keys, and the mechanisms used for signing and sealing by the NTLMSSP.

There are numerous key schemes employed during signing and sealing; we will start with an overview of the different types of keys and core session security concepts.

The User Session Key

This is the basic key type employed in session security. There are many variants:

The method of derivation used depends on the responses sent in the Type 3 message. These variants and their calculations are outlined below.

The LM User Session Key

Used when only the LM response is provided (i.e., with Win9x clients). The LM User Session Key is derived as follows:

  1. The 16-byte LM hash (calculated previously) is truncated to 8 bytes.
  2. This is null-padded to 16 bytes. This value is the LM User Session Key.

The NTLM User Session Key

This variant is used when the client sends the NTLM response. The calculation of the key is fairly straightforward:

  1. The NTLM hash is obtained (the MD4 digest of the Unicode mixed-case password, calculated previously).
  2. The MD4 message-digest algorithm is applied to the NTLM hash, resulting in a 16-byte value. This is the NTLM User Session Key.

The LMv2 User Session Key

Used when the LMv2 response is sent (but not the NTLMv2 response). Deriving this key is a bit more complicated, but not terribly complex:

  1. The NTLMv2 hash is obtained (as calculated previously).
  2. The LMv2 client nonce is obtained (used in the LMv2 response).
  3. The challenge from the Type 2 message is concatenated with the client nonce. The HMAC-MD5 message authentication code algorithm is applied to this value using the NTLMv2 hash as the key, resulting in a 16-byte output value.
  4. The HMAC-MD5 algorithm is applied to this value, again using the NTLMv2 hash as the key. The resulting 16-byte value is the LMv2 User Session Key.

The NTLMv2 User Session Key

Used when the NTLMv2 response is sent. Calculation of this key is very similar to the LMv2 User Session Key:

  1. The NTLMv2 hash is obtained (as calculated previously).
  2. The NTLMv2 "blob" is obtained (as used in the NTLMv2 response).
  3. The challenge from the Type 2 message is concatenated with the blob. The HMAC-MD5 message authentication code algorithm is applied to this value using the NTLMv2 hash as the key, resulting in a 16-byte output value.
  4. The HMAC-MD5 algorithm is applied to this value, again using the NTLMv2 hash as the key. The resulting 16-byte value is the NTLMv2 User Session Key.

The NTLM2 Session Response User Session Key

Used when NTLMv1 authentication is employed with NTLM2 session security. This key is derived from the NTLM2 session response information as follows:

  1. The NTLM User Session Key is obtained as outlined previously.
  2. The session nonce is obtained (discussed previously, this is the concatenation of the Type 2 challenge and the nonce from the NTLM2 session response).
  3. The HMAC-MD5 algorithm is applied to the session nonce, using the NTLM User Session Key as the key. The resulting 16-byte value is the NTLM2 Session Response User Session Key.

The Null User Session Key

The Null User Session Key is employed when Anonymous authentication is performed. This one is simple; it's just 16 null bytes ("0x00000000000000000000000000000000").

The Lan Manager Session Key

The Lan Manager Session Key is an alternative to the User Session Keys, used to derive keys in NTLM1 signing and sealing when the "Negotiate Lan Manager Key" NTLM flag is set. Calculation of the Lan Manager Session Key is as follows:

  1. The 16-byte LM hash (calculated previously) is truncated to 8 bytes.
  2. This is padded to 14 bytes with the value "0xbdbdbdbdbdbd".
  3. This value is split into two 7-byte halves.
  4. These values are used to create two DES keys (one from each 7-byte half).
  5. Each of these keys is used to DES-encrypt the first 8 bytes of the LM response (resulting in two 8-byte ciphertext values).
  6. These two ciphertext values are concatenated to form a 16-byte value - the Lan Manager Session Key.

Note that the Lan Manager Session Key is based on the LM response (rather than simply the LM hash), which means that it will change in response to a different server challenge. This is an advantage over the LM User Session Key, which is based solely on the password hash; the Lan Manager Session Key changes for each authentication operation, while the LM/NTLM User Session Keys remain the same until the user changes his or her password. For this reason, the Lan Manager Session Key is stronger than the LM User Session Key; however, it is still arguably weaker than the NTLM User Session Key (which does not vary on each authentication, but has a full 128-bit keyspace; the LM User Session Key and Lan Manager Session Key both have an effective key strength of 64 bits).

Key Exchange

When the "Negotiate Key Exchange" flag is negotiated, the client and server will agree upon a "secondary" key, used instead of the session key for signing and sealing. This done as follows:

  1. The client selects a random 16-byte key (the secondary key).
  2. The session key (either the User Session Key or Lan Manager Session Key, depending on the state of the "Negotiate Lan Manager Key" flag) is used to RC4-encrypt the secondary key. This results in a 16-byte ciphertext value.
  3. This value is sent to the server in the "Session Key" field of the Type 3 message.
  4. The server receives the Type 3 message and decrypts the value sent by the client (using RC4 with the User Session Key or Lan Manager Session Key).
  5. The resulting value is the recovered secondary key, and is used in place of the session key for signing and sealing.

Additionally, the key exchange process subtly changes the signing protocol in NTLM2 session security (discussed in a subsequent section).

Key Weakening

The key used for signing and sealing is "weakened" in accordance with cryptographic export restrictions. The key strength is determined by the "Negotiate 128" and "Negotiate 56" flags. The strength of the final key used is the maximum strength supported by both the client and server; if neither flag is set, the default key length of 40 bits is used. NTLM1 signing and sealing supports 40-bit and 56-bit keys; NTLM2 session security supports 40-bit, 56-bit, and unweakened 128-bit keys.

NTLM1 Session Security

NTLM1 is the "original" NTLMSSP signing and sealing scheme, used when the "Negotiate NTLM2 Key" flag is not negotiated. Key derivation in this scheme is driven by the following NTLM flags:

Negotiate Lan Manager Key When set, the Lan Manager Session Key is used as the basis for the signing and sealing keys (rather than the User Session Key). If not established, the User Session Key will be used for key derivation.
Negotiate 56 Indicates support for 56-bit keys. If not negotiated, 40-bit keys will be used. This is only applicable in combination with "Negotiate Lan Manager Key"; User Session Keys are not weakened under NTLM1 (as they are already weak).
Negotiate Key Exchange Indicates that key exchange will be performed to negotiate a secondary key for signing and sealing.

NTLM1 Key Derivation

Deriving NTLM1 keys is essentially a three-step process:

  1. Master key negotiation
  2. Key exchange
  3. Key weakening

Master Key Negotiation

The first step is negotiation of the 128-bit "master key" from which the final signing and sealing key will be derived. This is driven by the "Negotiate Lan Manager Key" NTLM flag; if set, the Lan Manager Session Key will be used as the master key. Otherwise, the appropriate User Session Key is employed.

As an example, consider our example user with the password "SecREt01". If the "Negotiate Lan Manager" key is not set, and an NTLM response was provided in the Type 3 message, the NTLM User Session Key will be selected as the master key. This is calculated by taking the MD4 digest of the NTLM hash (which is itself the MD4 hash of the Unicode password):

0x3f373ea8e4af954f14faa506f8eebdc4

Key Exchange

If the "Negotiate Key Exchange" flag is set, the client will populate the "Session Key" field in the Type 3 message with a new master key, RC4-encrypted with the previously selected master key. The server will decrypt this value to receive the new master key.

For example, assume that the client selects the random master key "0xf0f0aabb00112233445566778899aabb". The client will encrypt this value using RC4 with the previously negotiated master key ("0x3f373ea8e4af954f14faa506f8eebdc4") to obtain the value:

0x1d3355eb71c82850a9a2d65c2952e6f3

This is sent to the server in the "Session Key" field of the Type 3 message. The server RC4-decrypts this value using the old master key to recover the new master key selected by the client ("0xf0f0aabb00112233445566778899aabb").

Key Weakening

Finally, the key is weakened to comply with export restrictions. NTLM1 supports 40-bit and 56-bit keys. If the "Negotiate 56" NTLM flag is set, the 128-bit master key will be weakened to 56-bits; otherwise, it will be weakened to 40-bits. Note that key weakening is only employed under NTLM1 when the Lan Manager Session Key is used ("Negotiate Lan Manager Key" is set). The LM and NTLM User Session Keys are based on the password hashes, rather than the responses; a given password will always result in the same User Session Key under NTLM1. Weakening was apparently not deemed necessary, since the User Session Key can be easily recovered given a user's password hash.

The process for key weakening under NTLM1 is as follows:

Using the master key "0x0102030405060708090a0b0c0d0e0f00" as an example, the 40-bit key used for signing and sealing would be "0x0102030405e538b0". If 56-bit keys are negotiated, the final key would be "0x01020304050607a0".

Signing

Once the key has been negotiated it can be used to produce digital signatures, providing message integrity. Support for signing is indicated by the presence of the "Negotiate Sign" NTLM flag.

NTLM1 signing (as done by the SSPI MakeSignature function) is performed as follows:

  1. An RC4 cipher is initialized using the previously negotiated key. This is done once (before the first signing operation), and the keystream is never reset.
  2. The CRC32 checksum of the message is calculated; this is represented as a long (32-bit little-endian value).
  3. A sequence number is obtained; this starts at zero and is incremented after each message is signed. The number is represented as a long.
  4. Four zero bytes are concatenated with the CRC32 value and sequence number to obtain a 12-byte value ("0x00000000" + CRC32(message) + sequenceNumber).
  5. This value is encrypted using the previously initialized RC4 cipher.
  6. The first four bytes of the ciphertext result are overwritten with a pseudorandom counter value (the actual value used is insignificant).
  7. A version number ("0x01000000") is concatenated with the result from the previous step to form the signature.

As an example, assume that we are signing the message "jCIFS" (hexadecimal "0x6a43494653") using the 40-bit key from the previous example:

  1. The CRC32 checksum is calculated (in little-endian hexadecimal, "0xa0310bb7").
  2. A sequence number is obtained. Since this is the first message we have signed, the sequence number is zero ("0x00000000").
  3. Four zero bytes are concatenated with the CRC32 value and sequence number to obtain a 12-byte value ("0x00000000a0310bb700000000").
  4. This value is RC4-encrypted using our key ("0x0102030405e538b0"); this yields the ciphertext "0xecbf1ced397420fe0e5a0f89".
  5. The first four bytes are overwritten with a counter value; using "0x78010900" gives "0x78010900397420fe0e5a0f89".
  6. The version stamp is concatenated with the result to form the final signature:

    0x0100000078010900397420fe0e5a0f89

The next message signed would receive the sequence number 1; also, note again that the RC4 keystream initialized with the first signing is not rese