Category Archives: Windows Server

Ever wonder if your Microsoft product is supported for virtualization?

If so, wonder no more!

The Microsoft Common Engineering Criteria (CEC), a 140-page checklist, lists all the requirements a Microsoft product must fulfill before it can ship. One such requirement is “Must support Hyper-V”. So if you cannot support Hyper-V you cannot launch! Microsoft has been kind enough to list all its server software and the software’s current virtualization support in a frequently updates KB article called: KB957006 Microsoft Server Software and supported virtualization environments. Every piece of server software that is currently under support is listed here, along with its virtualization support. Any caveats or limitations are also listed.

Virtualization is not limited to Hyper-V but also any virtualization platform certified under the Server Virtualization Validation Program (SVVP). And before you ask; Yes, VMWare is validated!

Also remember; supported in this context means Microsoft officially supports virtualizing the software. If your favorite product is not supported, that does not mean it will not run…

Will if fail over, or just fail? Some DNS empirical testing

While out to dinner the other night we got to talking about name resolution in Windows, as one does while out enjoying a fine meal with friends… During the discussion we found that one of us had experienced some strange behavior with regards to the prioritization of DNS servers and failover. Specifically the issue was that the DNS resolver in Windows would not fail over to the next DNS server on the list if the server currently in use became unavailable. This sounded like a major bug and we were amazed that if this was indeed the case; how come we had not heard about it before? Out came the smartphones, the conversation stopped, the food went cold and the beer warm. After quite some time (at this point the waitress had started sending us long gazes, wondering if we were all stricken by some strange debilitating disease) we still had nothing to back up this case. So there was nothing for it: I went back to the hotel to do some testing. This is what I found…

Resolving names on Windows

It is important to understand that we are talking about the DNS name resolution behavior here when the machine is acting as a DNS client, not a DNS Server. Windows uses two components to resolve, register and cache DNS names. These are the DNS Client and the DNS Resolver. The DNS Resolver, aka the Windows resolver, is part of the TCP/IP protocol and cannot be disabled without disabling the protocol itself. The DNS Client is implemented as a service with the friendly name DNS Client, and a service name of Dnscache. Its description reads as follows:

The DNS Client service (dnscache) caches Domain Name System (DNS) names and registers the full computer name for this computer. If the service is stopped, DNS names will continue to be resolved. However, the results of DNS name queries will not be cached and the computer’s name will not be registered. If the service is disabled, any services that explicitly depend on it will fail to start.

So the DNS Client is responsible for caching the results from the DNS Resolver, and register the computer’s FQDN in Dynamic DNS. If you stop the DNS Client you can still resolve names, but they will not be cached and a DNS Server will be queried each time a name needs to be resolved to an IP address. Needless to say, this may impact performance.

Name resolution works the same for both Windows clients; XP, Vista, 7, 8 etc., and servers; 2003, 2008, 2008 R2, 2012. Both servers and clients need to be able to resolve, cache and register network names and they all do it the same way.

The DNS Server list

Each network adapter in a Windows machine that is bound to either the TCP/IPv4 or TCP/IPv6 protocol has a prioritized list of zero or more DNS servers to which queries to resolve names can be sent. The adapters themselves are also prioritized. A name is resolved using this process:

The DNS Resolver queries the DNS servers in the following order:

  1. The DNS Resolver sends the name query to the first DNS server on the preferred adapter’s list of DNS servers and waits one second for a response.
  2. If the DNS Resolver does not receive a response from the first DNS server within one second, it sends the name query to the first DNS servers on all adapters that are still under consideration and waits two seconds for a response.
  3. If the DNS Resolver does not receive a response from any DNS server within two seconds, the DNS Client service sends the query to all DNS servers on all adapters that are still under consideration and waits another two seconds for a response.
  4. If the DNS Resolver still does not receive a response from any DNS server, it sends the name query to all DNS servers on all adapters that are still under consideration and waits four seconds for a response.
  5. If the DNS Resolver does not receive a response from any DNS server, the DNS client sends the query to all DNS servers on all adapters that are still under consideration and waits eight seconds for a response.

If the DNS Resolver receives a positive response, it stops querying for the name, adds the response to the cache (via the DNS Client service) and returns the response to the client.

If the DNS Resolver has not received a response from any server within eight seconds, the DNS Resolver responds with a timeout.

The Case

The case we are exploring here is as follows:

On a Windows machine with one network adapter, which is bound to the TCP/IPv4 protocol, with two or more DNS servers specified in its TCP/IP properties; if the primary DNS server does not respond the machine will not fail over to the secondary DNS server and will be unable to resolve names.

I used this setup to test.

  • 2 Windows Server 2012 DC/DNS servers named MAYA1 and MAYA2 with the addresses 192.168.131.10 and 192.168.131.11 respectively. Both have the same DNS zones and configuration, i.e.. they will both answer with the same information when queried.
  • A Windows 8 Professional client, named MAYA-CLIENT1, with a dynamically assigned IP address from MAYA1, acting as a DHCP server, and 2 dynamically configured DNS servers (MAYA1 and MAYA2) in that order.
  • All machines on the same subnet.
  • No Internet access
  • Resolving names from a zone the two DNS servers were authoritative for
  • Network Monitor on the Windows 8 client used to capture network traffic
  • All servers and client were VMs running on Hyper-V
  • 2 DNS names for testing; nothere1 (1.1.1.1) and nothere2 (1.1.1.2)

image

Testing

The test I did was very simple. With Network Monitor running on the client I first pinged the name nothere1. In Network Monitor I verified that the response had come from MAYA1 (the configured primary DNS server). I emptied the DNS cache on the client and disconnected MAYA1 from the network. I then pinged nothere1 again, using Network Monitor to see which server answered.

It should come as no surprise that this worked exactly as expected. Under normal conditions Windows will use its configured primary DNS server, if that fails it will use the next configured server on the list after a short delay.

Here is the process in Network Monitor

image

Frame Operation
15 Query from client to primary DNS server for the name nothere1
16 Answer from primary DNS server to client with the IP address of nothere1 (1.1.1.1)
<Primary DNS server MAYA1 disconnected from network and the DNS Cache is emptied on the client>
31 New query from client to primary DNS server for the name nothere1
34 Since no reply has been received from the primary DNS server for 1 second a new query is sent to the secondary DNS server MAYA2 (192.168.131.11) for the name nothere1
37 Answer from secondary DNS server to client with the IP address of nothere1 (1.1.1.1)

 

Notes

  • These tests were performed with Windows 8 as the resolving client. It is quite possible that an earlier version of Windows behaves differently, but nothing I have found suggests so. Testing of that will have to wait for the time being, though.
  • I did come across people who reported variations of the problem originally stated in this post, but none with the exact same result. Some people were able to log on to their domain and resolve names successfully when the primary DNS server was offline, but were baffled that nslookup did not work. I guess they didn’t know that nslookup always queries the primary DNS server.
  • Some documentation claims that it is the DHCP Client service (service name Dhcp) that registers the computer’s FQDN with DNS. I have tested this on the setup used in this article and have not been able to reproduce that behavior. This article; How to configure DNS dynamic updates in Windows Server 2003, claims that the DHCP Client service registers the name even for statically configured addresses. A funny thing here is that the service description for the DHCP Client service also claims that it registers addresses: Registers and updates IP addresses and DNS records for this computer. If this service is stopped, this computer will not receive dynamic IP addresses and DNS updates. If this service is disabled, any services that explicitly depend on it will fail to start. I also remember reading about this in the Windows 2000 days, so maybe this was the way it worked before?

Conclusion

So far it looks like Windows behaves exactly as you would expect! That is always nice.

More information:

Fun exercise

To see a demonstration of how the DNS Resolver and the DNS Client work together; try this:

  1. Stop and disable the DNS Client service on a machine
    Just stopping it will not work because Windows will restart it as soon as you start resolving names.
  2. Ping your favorite DNS name and marvel at Windows’ ability to resolve the name without the benefit of the cache.
  3. Run ipconfig /displaydns to display the DNS cache
    Instead of the usual list of cached DNS names you will see this error: Could not display the DNS Resolver Cache.
  4. Run ipconfig /flushdns to clear the DNS cache:
    No dice; this error pops up: Could not flush the DNS Resolver Cache: Function failed during execution.
  5. Run ipconfig /registerdns to register the current computer’s name with a DNS server:
    Fail: Registration of DNS records failed: The binding handle is invalid.
  6. Sit back and enjoy your deep understanding of DNS on Windows.
  7. Re-enable and restart your DNS Client service.

Authentication errors on NLB cluster

I configred a 2 node NLB cluster to load balance Remote Desktop Session Hosts with Windows Server 2008 R2. These were virtual servers running on VMWare so I selected to use multicast mode for the cluster. The cluster IP (.3) correctly resolved to the cluster multicast address with ARP. The cluster formed and converged successfully.

After a few minutes I could no longer access node 2 (.2) from any other computer. The error given by Windows was “The target account name is incorrect”. This is a Kerberos authentication error (KRB5KRB_AP_ERR_MODIFIED)., which means there is a mismatch between the name you entered in your request and the name of the server you are actually contacting. In other words, the name of the machine is different from the name you specified, yet you were sent to that machine by the networking stack. I also experienced another error when trying to remotely administering node 2; The RPC server is unavailable. How could this be?

Turns out the reason is IPv6 6to4. When both NLB nodes were given the same official IPv4 address as a secondary address on their adapters (the cluster address), they also both configured the corresponding IPv6 address for that IPv4 address on their 6to4 adapters. Both machines had the same IPv6 6to4 address. Once they both registered this in DNS the game was up. Whenever I typed in the name of node 2 in an UNC path it randomly resolved to the IPv6 address of node 1. When the connection was attempted on node 1 the actual name of node 1 was different from what was specified in the Kerberos ticket that the connecting machine got from the Domain Controller, thus producing the “mismatch” error, and in turn the RPC error.

To work around the error I disabled 6to4 on both NLB nodes:

netsh interface ipv6 6to4 set state disabled

This is not a perfect solution and I would much rather find a way to disable the registration of 6to4 addresses in DNS. I will have to look into this further…

Another approach would be to disable strict name checking on the hosts, in effect disabling the requirement that the name of the machine must match the name in the Kerberos ticket, but that is a major change with some serious security issues attached to it. Also the effect of having two machines with the same 6to4 address is unknown to me.

By the way, you might not see this error in your network or lab, since 6to4 addresses are only configured for machines with official IPv4 addresses. In other words, if you use any of the RFC1918 addresses (10/8, 172.16/12 or 192.168/16) you will not have 6to4 addresses.

The Case of The Strange Folder Redirection Error

I was enabling Folder Redirection for some Windows 7 Professional machines, or rather, for the users of some Windows 7 Professional machines. The users already had a server based home directory with a My Documents folder, which also had data. The purpose of the operation was to, firstly, enable Folder Redirectin, but also to merge the contents of the My Documents folder on the client machines with the My Documents folder on the network server. First, to see what kind of conflict resolution Folder Redirection had, I created a file with the same name (but different content) in both the local My Documents folder and the one on the server. After logging on the first time the Folder Redirection policy was active I found this error event in the Application log on the client machine:

Log Name:      Application
Source:        Microsoft-Windows-Folder Redirection
Event ID:      502
Level:         Error
User:          <domain><username>
Computer:      <computername>
Description: Failed to apply policy and redirect folder Documents to \<servername><share><username>Documents.
Redirection options=0x1001.
The following error occurred: Failed to copy files from C:Users<username>Documents to  \<servername><share><username>Documents.
Error
details: This function is not supported on this system.

Originally I thought this was a problem with NTFS file permissions on the file server, but these we OK. After all, other clients were redirecting their folders without problem. Since the error details didn’t give me any clue I decided to try to remove my offending duplicate file. I deleted it from the client machine and on the next logon the My Documents folder redirected without problem.

The error in the log should have a better explanation of what is happening. Folder Redirection is definitely supported on Windows 7 Professional. The Folder Redirection specific event logs didn’t contain any more information either. The error text should have said something along the lines “File conflict; file X already exists”. Maybe in Windows 8.

Unfortunately I solved the problem before I had a chance to enable debug logging for the Folder Redirection Client Side Extension. Maybe that would have told me what the problem was. Should you want to enable debug logging for Folder Redirection you can do so with this command:

reg.exe add “HKLMSoftwareMicrosoftWindows NTCurrentVersionDiagnostics” /v FdeployDebugLevel /d 0x0f /t REG_DWORD

If you are on Windows XP/2003 or earlier this will give you a log file: %windir%debugusermodefdeploy.log. If you are running Windows Vista/2008 or newer you will simply get more events in the Windows event logs.

So, should you find yourself staring at this error in the middle of the night (or any other time), see if you have any duplicate files in the folders you are trying to redirect.

Happy redirecting!

UPDATE: The duplicate files I created were created by a different account than the user owning the client computer and home directory on the server. That means that the user actually owning the folders could not delete or move the duplicate file. That could also be the reason for this error. But the fact remains; it is still a very poor error message.

UPDATE 2: I have now had a chance to test the conflict resolution in Folder Redirection and from my tests it seems that the client wins if the same file (but with different content) exists on both the server where you are redirecting to and on the client. I performed the same experiment as outlined above; two files with the same name, one on the server and one on the client. This time they were both created by the user owning the client and the folder on the server. Upon the next logon the folder redirection policy took effect and the local files were copied to the server, merging them with the content that already existed there. But as I say, the copy of the identical file on the server was silently overwritten by the file on the client. So now you know.

If you won’t translate RDS profiles; I will!

Out of pure frustration with the fact that the Active Directory Migration Tool (ADMT) is unable (unwilling is my guess) to do security translation for users’ Remote Desktop Services (RDS) roaming profiles, I decided to take matters into my own hands and created the script below. It is not very refined just now, but I have a lot of ideas for future versions. In the meantime, if you can use it for something; great!

# TranslateRDSProfiles.ps1
# Morgan Simonsen
# http://morgansimonsen.wordpress.com
#
# Script to translate a user profile belonging to a migrated user.
# Primarily intended to solve the problem with ADMT not translating
# Remote Desktop Service roaming profiles.
#
# Version: 1.0 (13.02.2012)
#     Initial version.

#----------------------------------
# User changeable strings
# Only edit in this section!
#
# Root of folder or sharing storing the profiles
$RDSProfileRootDirectory = "Z:" # Include trailing backslash
# NetBIOS Name of source domain; the domain the user was migrated from
$SourceNBTDomainName = "DOMAIN_1" # Include trailing backslash
# NetBIOS Name of target domain; the domain the user was migrated to
$TargetNBTDomainName = "DDS" # Include trailing backslash
# DNS name of target domain
$TargetDNSDomainName = "dds.intern"
# FQDN of Domain Controller in target domain
$TargetDomainDC = "ddsdc1.dds.intern"
# Location of subinacl.exe
$SUBINACLLocation = "C:Program Files (x86)Windows Resource KitsToolssubinacl.exe"
# Location of echoArgs.exe (not actually used by the script)
$echoArgs = ($PSHome+"ModulesPscxAppsechoArgs.exe")
#----------------------------------

Get-ChildItem $RDSProfileRootDirectory | ForEach `
{
    Write-Host ("Processing folder: "+$_.name)
    If (Get-QADUser ($TargetNBTDomainName+$_.name) -service $TargetDomainDC)
    {
        $user = Get-QADUser ($TargetNBTDomainName+$_.name) -service $TargetDomainDC
        Write-Host (" Found matching user: "+$user.Userprincipalname)

        If ( (Test-Path ($_.FullName+"NTUSER.DAT")) -or (Test-Path ($_.FullName+"NTUSER.MAN")) )
        {
            Write-Host (" Found user registry hive: "+($_.FullName+"NTUSER.DAT"))
            Write-Host (" Updating permissions...")
            $entireprofile = ($_.FullName+"*.*")
            $completeusername = ($TargetNBTDomainName+$user.samaccountname)
            Write-Host (" SubInACL.exe File Output:")
            & $SUBINACLLocation /noverbose /subdirectories $entireprofile /grant="$completeusername=f" /setowner=$completeusername  > $null
            Write-Host (" SubInACL.exe Registry Output:")
            reg.exe load ("HKU"+$_.Name) ($_.FullName+"NTUSER.DAT") > $null
            $regkey = ("HKEY_USERS"+$_.Name)
            & $SUBINACLLocation /noverbose /subkeyreg $regkey /grant="$completeusername=f"  > $null
            reg.exe unload ("HKU"+$_.Name) > $null
        }
    }
    Else
    {
        Write-Host " Cannot find user in domain that matches folder name."
        Write-Host " Continuing with next user."
    }
}

	

SHA1 Thumbprints for trusted .rdp publishers

Remote Desktop Connection (RDC) has a Group Policy setting that determines which publishers are to be considered trusted when launching connections (typically .rdp files served in various ways).

The publisher is identified by the SHA1 thumbprint of the certificate of the publisher (the certificate used to sign the .rdp file). You get the thumbprint from the certificate:

image

The setting is located under:
Computer Configuration\Policies\Administrative Templates\Windows Components\Remote Desktop Services\Remote Desktop Connection Client

Setting:
Specify SHA1 thumbprints of certificates representing trusted .rdp publishers

Description:
This policy setting allows you to specify a list of Secure Hash Algorithm 1 (SHA1) certificate thumbprints that represent trusted Remote Desktop Protocol (.rdp) file publishers.

If you enable this policy setting, any certificate with an SHA1 thumbprint that matches a thumbprint on the list is trusted. If a user tries to start an .rdp file that is signed by a trusted certificate, the user does not receive any warning messages when they start the file. To obtain the thumbprint, view the certificate details, and then click the Thumbprint field.

If you disable or do not configure this policy setting, no publisher is treated as a trusted .rdp publisher.

Notes:

You can define this policy setting in the Computer Configuration node or in the User Configuration node. If you configure this policy setting for the computer, the list of certificate thumbprints trusted for a user is a combination of the list defined for the computer and the list defined for the user.

This policy setting overrides the behavior of the “Allow .rdp files from valid publishers and user’s default .rdp settings” policy setting.

If the list contains a string that is not a certificate thumbprint, it is ignored.

As you can see; no mention of how the thumbprint is to be entered!

I found out the hard way that you have to remove all spaces and convert all letters to uppercase for the thumbprint to be valid. You are not informed if the format you enter is incorrect, it is just silently ignored if not recognized as a valid thumbprint.

This quick PowerShell command will do these two operations:

(“<your thumbprint here>”).ToUpper().Replace(” “,””)

If this Group Policy setting is not in effect, either because you have not set it or the thumbprint is incorrect/invalid, your users will get a warning when connecting, even if the certificate used to sign the .rdp file is trusted:

image

Error: A website wants to run a RemoteApp program. Make sure that you trust the publisher before you connect to run the program.

It is interesting to note that the rdpsign.exe command line utility that is used to sign .rdp files manually, requires that the thumbprint of the certificate must be provided in just this way: http://technet.microsoft.com/en-us/library/cc753982(WS.10).aspx

More info:

A note on copying the thumbprint

If you look at the highlighted/selected thumbprint in the image above you will see what looks like a leading whitespace. If you select the whole string (not as above), you will get a strange leading character in your thumbprint. Have a look at this zoomed image:

image

I do not know what character this is, but it invalidates the thumbprint string if you paste it into the SHA1 thumbprint field in your GPO. Even stranger is that it does not show up in the pasted text in the GPO object; it just “looks” right. As I said, I have no explanation, but remember to skip the leading whitespace when you copy your thumbprint.

This is how it should look:

image

Establishing a performance baseline

Introduction

A performance baseline is a vital part of you system documentation. A baseline should be established immediately after a new system has entered production, and should be repeated with regular intervals. That way you can see how your system performs over time and make informed decisions about when a system will have to be upgraded etc.

Creating a baseline

The primary tool you use to capture a baseline on the Windows platform is Performance Monitor (or just Performance or PerfMon). PerfMon is an MMC snap-in that enables you to record various aspects of you system. These are called objects and counters. An object can be e.g. the Processor which in turn has several counters, e.g. % Processor Time.

Which counters?

Which counters you capture depends on the role of the system you are establishing a baseline for. The counters captures for a SQL server are different from those captures on an Exchange Server.
This table lists some important counters. The Role column indicates which server role the counter is applicable to.
ObjectCounter Definition Recommendations Role
Processor% Processor Time % Processor Time is the percentage of elapsed time that the processor spends to execute a non-Idle thread. It is calculated by measuring the duration of the idle thread is active in the sample interval, and subtracting that time from interval duration.  (Each processor has an idle thread that consumes cycles when no other threads are ready to run). This counter is the primary indicator of processor activity, and displays the average percentage of busy time observed during the sample interval. It is calculated by monitoring the time that the service is inactive, and subtracting that value from 100%. A CPU time of more than 90 % for extended periods of time is generally regarded as a problem. All
MemoryPages/sec Pages/sec is the rate at which pages are read from or written to disk to resolve hard page faults. This counter is a primary indicator of the kinds of faults that cause system-wide delays.  It is the sum of Memory\Pages Input/sec and Memory\Pages Output/sec.  It is counted in numbers of pages, so it can be compared to other counts of pages, such as Memory\Page Faults/sec, without conversion. It includes pages retrieved to satisfy faults in the file system cache (usually requested by applications) non-cached mapped memory files. Although it is normal to have some spikes, this counter generally remains at or close to zero. All
PhysicalDiskAvg. Disk Queue Length Avg. Disk Queue Length is the average number of both read and write requests that were queued for the selected disk during the sample interval. The number of requests should not exceed two times the number of spindles constituting the physical disk. If the number of requests is too high, you can add additional disks or replace the existing disks with faster disks. All
PhysicalDiskAvg. Disk sec/Read Avg. Disk sec/Read is the average time, in seconds, of a read of data from the disk. Should not be above 25 ms.
PhysicalDiskAvg. Disk sec/Write Avg. Disk sec/Write is the average time, in seconds, of a write of data to the disk. Should not be above 25 ms.

Tools

Tool name Description Notes
logman.exe CLI utility included in Windows Server 2003 and newer. Logman manages the “Performance Logs and Alerts” service for creating and

managing Event Trace Session logs and Performance logs.
perfmon.exe Performance Monitor Can also be launched by using perfmon.msc
Performance Analysis of Logs (PAL) Tool Open source utility on Codeplex http://pal.codeplex.com/
ExPerfWiz ExPerfWiz is a powershell based script to help automate the collection of performance data on Exchange 2007 and Exchange 2010 servers. http://code.msdn.microsoft.com/ExPerfwiz
LogWiz Automate the collection of Performance monitor logs using logman.exe http://logwiz.codeplex.com/
relog.exe Relog creates new performance logs from data in existing performance logs by changing the sampling rate and/or converting the file format. Supports all performance log formats, including Windows NT 4.0 compressed logs.
Typeperf.exe Typeperf writes performance data to the command window or to a log file.
TraceRpt.exe Tracerpt processes binary Event Trace Session log files or real-time streams from instrumented Event Trace providers

and creates a report or a text (CSV) file describing the events generated.

Commands

Establish a one week performance baseline for a Domain Controller:
logman.exe create counter “Active Directory Baseline (1 Week)” -c “Processor(*)% Processor Time” “NTDS*” “DNS*” “PhysicalDisk(*)Avg. Disk Queue Length” “MemoryPages/sec” -max 50 -rf 168:00:00 -cnf

Links