Category Archives: Windows

Packer for Azure Peculiarities

Building images on Azure with Packer I have discovered some things that might be useful to others:

Cannot locate resource group means something else

Error when doing Azure Managed Disk builds:

Build ‘azure-arm’ errored: Cannot locate the managed image resource group <your RG>

What Packer really means is that your credentials are wrong. Check your values for client_id and client_secret. Remember that it is the Application ID that is your client_id, not the name or the object ID.

full access to this subscription” is not really necessary

The Packers docs state:

“The service principal specified in client_id must have full access to this subscription, unless build_resource_group_name option is specified in which case it needs to have owner access to the existing resource group specified in build_resource_group_name parameter.”

First of all; “full access” is not the name of a role in Azure IAM. Since they go on to mention “owner”, which is a role, later on when discussing the use of a pre-existing resource group, I assume that owner is what they mean. But the Packer service principal does not need owner. It just needs permissions to perform the creation and deletion of resources at the right level. Contributor works fine.

Remember thy OS

Packer commands must use double quotes on Windows and single quotes on Linux. Simple, right? Yet so hard to remember.  AFAICT this is mentioned only once in the Packer docs, in the Intro section, but not in the Template User Variables section. Even the help output of packer.exe (1.2.) on Windows is wrong:

 -var 'key=value' Variable for templates, can be used multiple times.

Will work on Linux, but not on Windows:

$ packer build \
 -var 'aws_access_key=YOUR ACCESS KEY' \
 -var 'aws_secret_key=YOUR SECRET KEY' \
 example.json

Will work on Windows, but not on Linux:

PS> packer build \
 -var "aws_access_key=YOUR ACCESS KEY" \
 -var "aws_secret_key=YOUR SECRET KEY" \
 example.json

Windows images require client_id and object_id

The object_id configuration option is not listed under the required section of the Azure builder, but if you are building a Windows image it is in fact required. This is explained in the object_id explanation, but should be made more clear. After all, if object_id is missing; Packer will not tell you that. Rather it will give you this:

==> azure-arm: ERROR: -> BadRequest : An invalid value was provided for ‘accessPolicies’.

object_id takes as its value the actual object id of the service principal that Packer is using. So client_id needs the application id and object_id needs the object id. What seems strange to me here is that Packer already knows the application id of its service principal and could easily get the object id from that by querying the graph. This is turn could make the whole issue with providing object_id redundant. Packer could just figure it out. This issue is discussed more in this link. If you have the wrong value in object_id you might see this error:

Failed to fetch secret from <KV Name>/packerKeyVaultSecret, HTTP status code=403 (Forbidden)

This is because Packer uses the value provided for object_id in the KeyVault access policy, which is accepted by the Azure ARM API (after all, it is just a GUID), but does not work in the next step when Packer tries to retrieve the value of the secret. Again, the solution is to set the correct value for object_id. People talk more about this issue here.

Happy Packing!

But I _am_ on the Internet!

For some time I have had some strange symptoms on Windows 10 systems when they were accessing the Internet over a VPN connection. In my case the VPN provider was TigerVPN. This issue is unrelated to the provider, but is instead caused by the Windows networking stack when used with the OpenVPN software (which many VPN providers use). More on that later, first let’s explore the symptoms.

In my case the problem manifested itself in many different ways, but it seems they all stem from the inability to sign in to Microsoft Accounts (MSA) when the VPN connection was active.  For example the Microsoft Store app did not work and I was constantly informed that I needed the Internet to do what I wanted to do (hence the title of this article). This was the error the Store gave me;

You’ll need the internet for this.

It doesn’t look like you’re connected to the internet. Please check your connection and

try again.

Ox800704cf

The network location cannot be reached. For information about network

troubleshooting, see Windows Help.

Hitting the “Send Feedback” button would open the Feedback Hub app , which would be stuck at an endlessly looping “Let’s get you signed in” screen. Very helpful…

Other problems include, but are probably not limited to, not being able to change your Windows Insider settings and issues with the Xbox app. All other network operations that I tried worked fine, but there could also, of course, be other issues.

So why did this happen?

It turns out that OpenVPN supports IPv6 connections inside the tunnel by default since version 2.3.0. This is good, we all like IPv6, but not all VPN providers support it on their servers, which is also fine. But not forever, and not without telling you. Windows 10 (and all versions since Vista) also support IPv6 by default and has a dual IPv4/IPv6 stack that will try to use both protocols simultaneously. In addition, Windows also has a number of IPv6 transition technologies built in. This is also good. If everything supported IPv6 through the entire stack and connection these problems would not present themselves. But this time that was not the case. My VPN provider did not support IPv6 and had it turned off. (I am unsure of how, exactly, they disabled IPv6 but I see the message did_ifconfig_ipv6_setup=0 every time I connect and take that to mean that IPv6 has been disabled either as a setting pushed by the server or in the client config file (.ovpn). But Windows did not know that IPv6 was disabled in the OpenVPN software so the tunnel adapter that OpenVPN creates still had IPv6 enabled. So Windows thought that the connection supported IPv6, but it did not. This is usually not a problem. When I looked in network connections this is what I saw:

The TAP adapter (Ethernet 2) is the OpenVPN software tunnel adapter. As you can see the Connectivity status is listed as “No network access”, but network access was working fine except for the issues mentioned. The properties of the adapter looked like this:

Notice that the IPv6 protocol is enabled. When I disabled IPv6 on the adapter, and thus making the connection pure IPv4, the Connectivity status immediately changed to “Internet access”, and all problems were resolved. And there was much rejoicing…

I can’t say exactly what caused this issue to arise. After all, having IPv6 enabled on adapters that are connected to IPv4-only networks works fine. I suspect this has something to do with one of IPv6’s transition technologies, Teredo, 6to4, ISATAP or PortProxy, but I did not investigate this further. This is a case where I am happy to accept the empirical fact that turning off IPv6 resolves the problem.

I have not seen any negative effects of this “fix”. My VPN provider does not support IPv6 yet so I am not loosing anything by disabling it on my adapter. When TigerVPN finally implements IPv6 support I expect that I can leave IPv6 enabled and not experience any issues. Time will tell.

NOTE: While we are on the subject of VPNs and IPv6 I would be remiss if I did not mention what is known as “IPv6 leak”. Very quickly “IPv6 leak” can happen if your VPN provider only supports IPv4 and not IPv6, and worse, just ignores any IPv6 traffic. If your ISP supports IPv6 and you get an IPv6 public address over which you can route traffic; that traffic will not be captured by your VPN provider (since they ignore it), and could thus “leak” and expose your identity, location and activities. Some VPN providers support what they call “IPv6 leak protection” where they still do not support IPv6, but when the VPN connections is established they insert black hole routes to all IPv6 destinations. I strongly recommend you find out exactly how your VPN provider handles IPv6 if you rely on VPNs to maintain your privacy (as you should).

M

More information:

A computer by any other name… would be better;how to change the hostname of a Windows 10 computer during setup

Introduction

Over the years the setup process for the Windows OS has been streamlined and optimized a lot. It used to be that you have to input a lot of information throughout the installation and the actual transferring of the OS files onto the hard drive took a long time. Now, the data copying has become a lot quicker by using things like imaging technology, and the user interaction during setup has been reduced to the bare necessities. So much so in fact, that I personally am now missing a few options that I could previously set. One of those is the computer name or hostname; the name you give your computer. In Windows 10 there is no question during setup what the computer should be called. Instead, Windows Setup automatically generates a name for you that looks something like this: DESKTOP-7KCPLCO. You can of course change this later using e.g. PowerShell’s Rename-Computer cmdlet. The problem is that the original name is often stored in the services you connect to, before you can actually change it. For example, when you join Azure AD during the Windows 10 Out-of-Box-Experience (OOBE), your machine is joined to Azure AD with the name that Windows Setup configured, and even if you change it later, it does not update in Azure AD. This is surely (hopefully) something that Microsoft will fix, so that it works the way it does in local Active Directory, where, if you change the name of a domain joined computer; the name in the directory also changes. But for now, that is not the case, and it can be quite a challenge keeping track of all those DESKTOP-<random number> machines.

The Fix

Luckily there is a workaround available. (In fact, there might be more than one workaround, but one was enough for me.) The one I made work was using the Windows unattended install support to supply a computer name during the specialize phase of Windows Setup. I basically configured an XML file that told Windows Setup which name to give the computer. Since there is no GUI prompt for this it all happens behind the scenes during your install. Using unattended setup of Windows is usually something that is only cost effective if you are going to install a bunch of machine over a long period of time, and it usually requires a lot of configuring a testing and things like driver packages, reference machines, distribution shares and in-depth knowledge of the Windows DISM utility. But in our case we need none of that, the only thing we want is a file with the name of the computer in it. This is how you set that up…

Unattend.xml

The file that holds all the information to install Windows 10 in a highly customized, hands-off fashion is called an answer file and is called unattend.xml by default. It is usually generated with the System Image Manager tool, which is part of the Windows Assessment and Deployment Toolkit (ADK). Hundreds of settings can be added to unattend.xml, but right now we only need the one for the name of the computer. This particular value is called ComputerName and is configured during the specialize phase of setup. Here is the entire chunk of XML you need to set the name of your computer using unattend.xml:

<?xml version="1.0" encoding="utf-8"?>
<unattend xmlns="urn:schemas-microsoft-com:unattend">
 <settings pass="specialize">
  <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <ComputerName>ENTER YOUR DESIRED NAME HERE</ComputerName>
  </component>
 </settings>
</unattend>

You must enter you desired name in the ComputerName element, and also change processorArchitecture from amd64 to x86 if you are using this file to install the 32-bit version of Windows (which you shouldn’t). Remember to adhere to the rules for computer names, which you can find in the information about the ComputerName element provided above.

You need a removable drive

So how do we get Windows Setup to read our unattend.xml file during setup? When we do large deployments of Windows we usually have several XML files for different variations of installs, and they are specified using the deployment tools used. Luckily for us, we need nothing so complex here. Remember, we are talking about the occasional install of a few machines. Windows Setup is configured to implicitly look for an unattend.xml file in several locations when it starts. (It’s called implicit, because you can also explicitly provide an answer file using a command line parameter.) Windows Setup looks in a lot of places for unattend.xml, and also looks for other XML files to use with other phases of setup, but we don’t need to go into detail about that here. Setup will look for unattend.xml in the root of all RW/RO removable drives on the system, and this is by far the easiest method to achieve what we are trying to do here.

After you have configured your desired name in the piece of XML provided about, save it as unattend.xml, in the root of either the USB drive from which you are installing Windows, in the root of another USB drive, of even on a floppy disk (virtual or physical). As long as it is called unattend.xml, is in the root of the drive and the drive is of type removable, it will be picked up by Windows Setup. Start Windows setup as you normally would and wait until OOBE starts, then configure your computer like you want it. The setup process itself will not change and you will have no indication that the machine has been given another name until you can access the desktop and check for yourself. Once that happens you can use e.g. hostname.exe to see if your change was successful.

Note that since setup looks in several places for unattend.xml, it might find more than one. There is a search order and the answer file with the highest precedent is used. Make sure you know which one will be picked in your scenario and that you make your changes in that file.

Good luck!

BTW: I hope someone out there picked up on my attempt at Shakespeare-pun in the title of this post:

“A rose by any other name would smell as sweet”

 -Juliet in Act II, Scene II of Rome and Juliet

 

Fun with Bash on Ubuntu on Windows

Ever since the Windows Subsystem for Linux/Bash on Ubuntu on Windows feature in Windows 10 I have been playing around with it. Canonical, the makers of the Ubuntu Linux distribution, and Microsoft, made the Windows Subsystem for Linux (WSL) together, and it enables bash, which is the de facto default CLI shell on Linux, to run on Windows as a first class citizen. You no longer need things like Cygwin to run  bash and its command language and tools on your Windows 10 computer.Recently I did a little customizing to get a more genuine Ubuntu feeling on my Windows box. Specifically I wanted to make my Windows-based bash window look as much as the real thing as possible.

Windows has its own fonts for console windows, like Command Prompt (Consolas) and PowerShell (Lucida Console). But Ubuntu uses the Ubuntu Mono font for its console windows. Luckily for us the Ubuntu Font Family, of which Ubuntu Mono is a member, is freely available to anyone, in easy to install TTF format. To use it for your bash windows, download the package from the Ubuntu font site, extract its content and right-click each font file (.ttf) you want to install.

Screenshot (1)

Now, open your bash shell and edit its properties. In the Font list, select the newly installed Ubuntu Mono font:

Screenshot (2)

Granted, it’s not a huge difference between the default font and Ubuntu Mono, but the devil is in the details 🙂

Screenshot (3)

Screenshot (4)

 

Error 0x80070001 on Windows 10 when trying to install a new app

This is slightly off topic for me, but because I spent quite a bit of time on figuring it out and could not find this documented anywhere else, I thought I would write it up quickly.

At some point I could no longer install any new apps from the Windows Store on my Surface 3 Pro Windows 10 machine. Apps already installed would update fine, but new ones could not be added. The error was:

“Try that again. Something went wrong. The error code is 0x8007001, in case you need it”.

If we translate that number to a human readable form we get:

Incorrect function.

Not much to go on. I initially thought this was something to do with either the modern app framework or Windows installation and tried things like resetting the store with (WSReset.exe) and scanning the system files with SFC.EXE. None of these things helped. In the end it turned out that is was related to my SD card. I had an  SD card installed and had previously moved a few apps to it. Apps that were so large that I didn’t want them eating up my system drive. Moving these apps somehow caused all new apps from then on to try to install on the SD card, or at least rely on it for something during the install. I shut down the computer, removed the card and could then install apps again. At this point I also reinserted the card and could now also install new apps with the card inserted. Some setting somewhere had obviously been changed. I do not know the root cause of this behavior, which is always annoying, but I am prepared to accept that I made it work.

Updating already installed apps worked because they were all on the correct (C:) drive. The default install location for apps was also set to the C: drive, which makes this even stranger…

Hope this helps someone. Happy installing!

Connecting to an Azure AD joined machine with an Azure AD user account over Remote Desktop

Introduction

Windows 10 introduces the ability to join a computer to the cloud directory service Azure AD. This is very similar to the traditional domain join, where you join a computer to an Active Directory domain, run on-premises by one or more Domain Controllers. Both operations lets the computer operate within a common security context and benefit from Single Sign-On (SSO) to all resources that share the same security context. However, joining Azure AD instead of a traditional domain can break things or make them more difficult. There are many examples of this, but the one I want to discuss here is connecting with Remote Desktop (RDP) to an Azure AD joined computer with a user account from Azure AD. Just to be clear; the connection we want to establish is to an Azure AD joined computer, logging on with an account from Azure AD. This account can either be synced from on-premises or be mastered in the cloud, and both federated and password logons are supported. We do not depend on any local accounts on the computer, using tricks such as adding an Azure AD work account to a local account or a Microsoft Account (MSA), this is pure Azure AD.

Connecting Successfully

There are some obvious prerequisites for this to work:

  • The computer must be joined to Azure AD
  • Remote Desktop connections must be enabled and allowed through the host firewall
  • Any other firewall between you and the computer must allow the Remote Desktop protocol

The key to connecting is having Windows 10 present an desktop login screen:

win10rdp2

That means that we must disable any form of single sign-on or integrated authentication. This requires the following steps:

  • On the Windows 10 computer; disable Network Level Authentication (NLA) for Remote Desktop Connections
    Open System Properties and navigate to the Remote tab. Under Remote Desktop; make sure Allow remote connections to this computer is enabled, and that Allow connections only from computers running Remote Desktop with Network Level Authentication is unchecked.
    win10rdp3
    This will disable the ability on the host to require that authentication happens before a user session is created.
  • On the computer you are connecting from create an RDP file and add the following settings to it:
    enablecredsspsupport:i:0
    authentication level:i:2
    Again, these settings disables sending any credentials automatically to the host computer. Leaving Windows with no choice but to display a desktop logon screen. The easiest way to create an RDP file is to open the remote desktop client, enter the name or IP of the computer you want to connect to and then his Save As. This will produce an RDP file that you can add/edit the necessary settings in. For those interested, most of the settings you can specify in an RDP file are listed here. In theory you could also add these settings on the command line, but I have not worked that out.

The last trick to make this work involves the username you specify on the logon screen. It must be in the following format:

AzureAD\<full UPN in Azure AD>

e.g. AzureAD\morgan.simonsen@langskip.no

This is a non-intuitive format for those of us who have connected to Windows over RDP in the past, but it is what works. I have not been able to connect with any other combination of domain, username, DNS domain or UPN, but this may very well change soon.

UPDATE 2015-11-7: On Windows 10 build 10586 the AzureAD prefix is no longer needed. You can just use your UPN.

Closing remarks

When you are joined to Azure AD we are naturally also authenticating against Azure AD, but it might be that you have federated Azure AD against an ADFS server, in which case authentications are redirected back to on-premises. Depending on your setup for authentication you will see the following differences:

Azure AD authenticated users will display the logged on user as: AzureAD\<concatenated display name>. Federated tenants will display the logged on user as <on-premises NetBIOS domain name>\<on-premises sAMAccountName>. This difference is visible if you use the whoami utility or look at the environment variables. Just to be clear, these differences do not have anything to do with remote desktop connections, they are just a consequence of joining Azure AD.

Connecting with a local account to a Windows 10 computer joined to Azure AD would as it does for any other Windows computer.

This is probably not how Microsoft would like us to connect to Azure AD joined machines so we can expect NLA authenticated connections to work some time in the future.

Happy connecting!

Morgan

The Case of the Missing Technical Preview build

I am trying out the Windows 10 Technical Preview, and have been running build 9926 for some time. Today (19032015) Microsoft released build 10041 and I installed it immediately, of course. Not surprisingly I had some problems which were so bad that I reverted back to the 9926 build. I later figured out that it might not have been the new build that was the problem, but something else. So I wanted to try installing 10041 again to test that theory. Problem was that 10041 was no longer being offered to me in Windows Update. Turns out Windows keeps track of the builds you have reverted from and hides those from Windows Update. Here is how to make them visible again.

In Registry Editor, navigate to HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsSelfHostApplicability.

This is where all the settings for the preview program are stored. Here is what that key looked like on my system:

HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsSelfHostApplicability
BranchName    REG_SZ    fbl_impressive
ThresholdRiskLevel    REG_SZ    low
ThresholdOptedin    REG_DWORD    0x1

HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsSelfHostApplicabilityRecoveredFrom
10041    REG_DWORD    0x1

Notice the key RecoveredFrom and the value 10041 in it. Delete the RecoveredFrom key and do another check for updates in Windows Update. The build should now be listed.

How to opt in to Microsoft Update with PowerShell

For Windows to retreive updates from Microsoft Update, as opposed to Windows Update (which is the default), you have to opt in. This is usually done through the Windows Update GUI by selecting “Get updates for other Microsoft Products”:

image

This becomes tedious very quickly so here’s how you can do it with PowerShell:

https://gist.github.com/morgansimonsen/8039966

To see if this was successful you can query the Services property of the $MU object:

$mu.Services

You should see something like this:

Name                  : Microsoft Update
ContentValidationCert : {}
ExpirationDate        : 11/18/2014 1:27:43 AM
IsManaged             : False
IsRegisteredWithAU    : True
IssueDate             : 11/18/2011 1:27:43 AM
OffersWindowsUpdates  : True
RedirectUrls          : System.__ComObject
ServiceID             : 7971f918-a847-4430-9279-4a52d1efe18d
IsScanPackageService  : False
CanRegisterWithAU     : True
ServiceUrl            : https://fe1.update.microsoft.com/v6/
SetupPrefix           : mu
IsDefaultAUService    : True

Name                  : Windows Update
ContentValidationCert : {}
ExpirationDate        : 6/18/5254 9:21:00 PM
IsManaged             : False
IsRegisteredWithAU    : False
IssueDate             : 1/1/2003 12:00:00 AM
OffersWindowsUpdates  : True
RedirectUrls          : System.__ComObject
ServiceID             : 9482f4b4-e343-43b6-b170-9a65bc822c77
IsScanPackageService  : False
CanRegisterWithAU     : True
ServiceUrl            :
SetupPrefix           :
IsDefaultAUService    : False

If you only have the Windows Update entry something went wrong.

One of the major benefits of this is that you can do it remotely. I have yet to find another way to do this. If you know, leave a comment.

Data Deduplication on Windows 8 Pro

Due to the modular nature of the Windows platform, it is actually possible to move features between SKUs, and even between server and client. A user at the My Digital Life forum has experiemented with this and has extracted the necessary packages to run Windows Server 2012’s Data deduplication feature on Windows 8 Pro!

If you want to test that out have a look here: http://forums.mydigitallife.info/threads/34417-Data-deduplication-for-Windows-8-x64

image

You get no GUI and have to manage Data Dedupe with PowerShell, but that should not be a problem. I was able to save almost 300 GBs on a 500 GB drive storing virual machine images!

image

If you decide to run this on any production machines take special care when hotfixes and future service packs are released.

More info on the excellent disk deduplication feature here:

What’s special about the builtin Administrator account?

Every installation of Windows based on the Windows NT code base has a builtin admin account called Administrator. Every installation of Active Directory Directoy Services also has a builtin admin account called Administrator. (If you are running a version of Windows other than English, your accounts may be named something else.) This account provides complete access to files, directories, services, and other facilities. But are there other things that make these account special?

  • The Relative Identifier (RID) is always 500
    In Windows each Security Principal is identified with a Security Identifier or SID. The SIDs have two parts; the machine or domain component and the Relative Identifier (RID). The RID is simply a whole number incremented with one (1) each time a new Security Principal, typically a group or user, is created. The builtin Administrator accounts, whether they are in a local SAM database or in Active Directory, always have the RID 500. This means that if you know the domain or machine component of the SID, you also know the full SID of the builtin Administrator. From there it is easy to do a “reverse lookup” and find the actual username of the builtin Administrator, and then to start trying to break into it. (Some older code even lets you authenticate with the SID directly, as opposed to a username.) See next bullet.
  • The account cannot be locked out
    The builtin Administrator account cannot be locked out of the system no matter how many failed logon attempts it accumulates. This makes it a prime target for brute force attacks. Auditing can help you find out if someone is trying to do a brute force attack using the builtin Administrator account. Other, manually created, administrator accounts can be locked out, and therefore do not present a similar threat. Renaming your builtin Administrator account will afford you some protection, but be aware of the limitations of this; see previous bullet.
  • The account cannot be deleted
    At least not using the default Windows tools.
  • The account is disabled on client OSs as of Windows XP
    In Windows XP and onwards, the builtin Administrator account does not have a password and is disabled. During setup you are required to create at least one new account, and this account becomes an administrator.