Virtual Machines (IaaS) in the Azure Cloud–Outline

I've recently taken some time to deep dive into the Azure IaaS / VM offering, and created an outline with my notes. I'm posting my findings here in case they are useful to anyone.

  • Intro
    • IaaS - Infrastructure as a service
    • VM - Virtual machine
    • Provision a VM without worrying about underlying hardware
    • Can "magically" create new machines for scaling
    • Runway to the cloud for existing apps
  • Models
    • Full control of the operating system
      • This means you'll need to maintain the OS including patches, etc.
    • Designed for IT pros to not need developers
    • These are virtual machines, not VM Roles. If you heard the term "VM Role", they're gone. VM's have state.
    • Management
        • Live ID's to log in, possible to configure AD, but not trivial* Think through how subscriptions will be structured
    • Think through your dev/test environment
    • Command line tools
    • You can get some information through the server manager in visual studio
    • REST APIs
      • Useful for custom automation
    • There are environment deployment tools available such as the Azure Knife plug-in for Chef
  • VM sizing/pricing
    • Pricing grid
    • Sizing grid
    • Extra small is shared, and should typically only be used for testing, or non-time critical workloads
    • Prices don't include EA discounts, 6/12 month agreement discounts, or pre-pay discounts
    • If you are ever going to use an A6/A7 instance, create it FIRST, so that they can be in the same rack
    • For a detailed architectural overview of Azure, check out the presentation by Mark Russinovich from Build 2013
    • In general, it is better to scale out instead of scaling up. You have more expandability when scaling out because you just stand up additional VM's. You'll hit a ceiling if trying to scale up individual machines.
    • Use your free Azure time included with your MSDN for individual Dev/Test
  • Creating
    • Images
      • There are a number of stock images available including various OS's and applications
      • To use, choose "From Gallery" when creating a new VM
      • Windows Server 2012 and above are Azure optimized
      • Images that you created will show up in the gallery under "My Images"
    • Custom Images
      • You can sysprep a machine that is already in Azure, and convert it into an image
      • Your images will be available in the gallery under "My images"
      • Can upload a VHD
        • Up to 128GB VHD is allowed
        • Create image
        • Sysprep image before upload
        • Need VHD, VHDX does not work (to convert, "Actions" pane, choose "edit disk")
        • VHD must be "fixed size", not "dynamically expanding"
        • Upload using Add-AzureVhd cmdlet
  • Disks
    • When to use each type
      • C: drive is persistent, uses storage as a backing store
      • D: is a temporary drive. Very fast, but is deleted if the machine needs to be migrated.
    • Data Disks
      • The size of the machine affects data disk use cases
        • Larger machines give you more available bandwidth
        • Larger machines allow a higher number of disks
      • Performance & scaling
        • The temp disk is fast, but volatile
        • Attach multiple disks and stripe data between them to increase throughput
    • Disk performance varies significantly from on premise solutions, so test performance early
    • Storage whitepaper
  • Accessing
    • When viewing the VM, click "Connect" for an RDP file
    • Alternatively, run the remote desktop client, use the DNS name or the public virtual IP
      • You must add/view the endpoint for RDP, this works like NAT
      • Personally, I like to obscure port numbers to avoid port-sniffers looking for common endpoints
  • Pay only for what you use
    • Shutdown within the VM does NOT stop billing
    • Shutting down from Azure will deallocate the machine, and stop billing
    • Used to round up to the hour
    • Stopped machines would incur charges
    • Billing is per-minute, with no rounding up
    • If you shutdown, reboot, or crash a VM, it will still incur costs
    • Shutting down a VM from PowerShell or the management portal will shut down and deallocate the machine and will not incur costs.
  • Affinity groups
  • Networking
    • Using internal IP's bypasses the load balancer
    • IP's are assigned via DHCP, but the lease is virtually infinite
    • Rebooting a machine will cause it to get the same IP
    • Deallocating a machine will get a new IP address and lease
    • VPN
      • Allows your cloud infrastructure to be an extension of your on premise network
      • Site-to-site
        • Supports certain brands of routers
        • Can VPN to a Windows Server RAS
      • Point-to-site
        • Individual machines can VPN in to the cloud and get access to that network
  • Availability
    • Use availability sets to tell Azure to create update domains and fault domains to keep your application running during updates and failures.
    • Availability set provides a 99.95% SLA
    • Update domains
      • Represents groups of resources that will be updated together
      • Host OS updates honor service update domains
      • 5 update domains by default, can bump up to 20
    • Fault domains
      • Avoids single points of failure
      • Represent groups of resources anticipated to fail together i.e. Same rack, same server
      • Fabric spreads instances across fault at least 2 fault domains
    • Procedure for Host OS updates:
      1. Remove guest from load balancer
      2. Issue shutdown command
      3. Host updates and reboots
      4. Guest booted
      5. Guest added back to the load balancer

Like this post? Please share it!

See a mistake? Edit this post!

Setting up a DNS Server in Azure IaaS/VMs

With the recent GA release and rapidly growing usage of virtual machines within Windows Azure, many are trying to set up an independent/non-VPN network. The great part of Azure virtual machines is that it can be as flexible as a set of PCs or VMs hosted within your own datacenter. Many of these scenarios require a more flexible local DNS server than what is provided by default in Azure. For example, programmatically building a dynamic cluster of VM's. In this post, I'll show you how to get up an running with a DNS server quickly and easily.

After you've created your Azure network (outside the scope of this post), you can feel free to divide your network into logical subnets. The DNS server will work fine across any subnets within your network.

Installing the DNS Server Role

Typically, a DNS server should be configured with a static IP address. In the case of Azure, the servers must use DHCP, but the lease they receive lasts for the life of the machine, as long as you don't shut down the machine through the management console, which will deallocate the VM. Shutting down the machine through Windows itself, however, will preserve the lease. Here you can see the difference between a VM shutdown through Windows, and one shutdown & deallocated through the Azure management console:


My "Stopped" VM still shows an allocated IP address:


Compare this to a "Deallocated" VM:


When you run "ipconfig /all", you'll notice that the DHCP lease for my machine lasts until 2149. 139 years from now I'll add a post explaining how to work around this. :-)


Create your virtual machine to host the DNS just like you would create any server. I highly recommend choosing Windows Server 2012 or above. It's highly optimized for Azure.

When you first remote into the server, you'll be presented with the Server Manager. From here, you can use the Add roles and features wizard to enable the DNS server and dependencies.


Check the DNS Server option and click Next.


You will get a validation warning that your server should have a static IP address, but recall that our DHCP lease is effectively the same thing, so we can disregard.


Complete the wizard (I kept all settings default).

Configuring the DNS Service

In the left pane of the Server Manger, select the DNS section. If it doesn't show up, click in the pane and hit F5 to refresh.


You'll see the server list in the right pane, and from here you can right-click and choose the DNS Manager option.


Expand the tree and find the Forward Lookup Zones section. A forward lookup zone.aspx) is a map of names and IP addresses, and is used to look up the IP address for one of those particular names. Think of it as a simple associative array that maps names to IP's. Right-click and choose New Zone.


For the zone name, decide on what DNS suffix you would like to use, which means that this DNS server can handle queries with that DNS suffix. For example, if your DNS suffix is, you can use your DNS server to resolve * addresses.


If you are not using the Azure VPN functionality, and you're building an Azure-only solution, you'll most likely want to allow non-secure updates (see my note on how to update the records later in this post). Secure updates are typically used on a domain, and are a useful way to ensure that only clients with the correct user permissions can update DNS. In our case, it's a pure server environment so the security concern is minimized (we won't have users mucking around in here).


Testing your DNS Server

To test our DNS server, we're simply going to create a record, and then query the server to ensure it gives a proper response.

First, right click in the forward lookup zone and choose New Host.


Create a test "A" record. The name is not important, we'll just need to remember what it is for the next step.


Open up a command prompt and run "nslookup", which will put us into interactive mode for queries. Set the DNS server by typing "server", which tells the session to use the local server (same machine that we're on). Now, type "" to lookup the record. Confirm that the IP address returned matches what you entered for the DNS record. This would also be a good time to test a public address such as to ensure that addresses that the DNS server cannot resolve are passed to the upstream DNS server (this should work by default).


Next, you'll want to test your DNS server using the same steps, but from a different computer within the same network. Keep in mind that you'll need to use the IP address of the DNS server instead of since we're no longer on the same VM as the DNS server.

Configure DHCP

Azure provides a DHCP server out of the box, which hands out the Azure DNS server address by default. We want to go to the Configure tab within our virtual network properties. Under the dns servers section, enter a name to reference the DNS server (the name really doesn't matter). On the same line, enter the IP address of the server you configured in the previous steps. The next time a server within that network does a DHCP pull, or reboots, it will get the DNS server IP and start using the server you built.


You may have noticed that it is also possible to define DNS servers from the standard New/Create feature in Azure. This only creates the DNS server as an option that can be selected on the Network-Configure screen. I have not found any particular reason to create entries this way.


Configure DNS Suffix

If you want to use short DNS names (eg: "test") instead of fully qualified names (eg: ""), you'll need to update the DNS suffix of your virtual machine. Access the Advanced TCP/IP Settings (Network and Sharing Center->Change Adapter Settings->Ethernet Adapter->Properties->Internet Protocol Version 4->Advanced->DNS Tab) and set your DNS suffix as shown in the screenshot below.


Backup DNS

You should always configure at least 2 DNS servers and configure Azure to use both. This ensures that if one is rebooting, fails, etc. the other server can answer the DNS queries. It also ensures that if one machine becomes deallocated, your servers will failover and give you time to update the DNS settings for the newly assigned IP address of the re-built replacement server.

Dynamic DNS Updating

In a typical on-premise DNS installation, the records are kept up-to-date by the DHCP server. Since we don't have control over our DHCP server, we have to update the records ourselves. In the interest of article length, I'll keep out the full details. This should get you started:

Like this post? Please share it!

See a mistake? Edit this post!

Upgrading CRM 4 Attachments to 2011 Using a Linked Server

We ran into a situation where our CRM 4 database would take days to upgrade to 2011. It turns out we had accumulated over 150GB of attachments from CRM tracked emails. In order to drastically speed up the migration process, we decided to truncate ActivityMimeAttachment the table prior to the upgrade process. I then wrote SQL scripts to pull the data over after the conversion using the linked server functionality in SQL Server. The advantage is that data can be pulled even while one or both of the systems are in use.

If you're not familiar with linked servers, it's a useful feature that allows you to set up one SQL Server database to connect to another and access both within the same query or script. To link a server, you'll want to go to the destination server, and log in. You can actually link either direction, I just chose to write the script for the destination server. The source server you'll be linking must have the proper permissions set up. In this case, since the migration process was temporary, so I created a temporary sysadmin account. I was then able to link the source server with this SQL (if it's not obvious, there are some tokens you need to substitute): Exec sp_addlinkedserver '{source_server}', 'SQL Server'; Exec sp_addlinkedsrvlogin '{source_server}', 'false', null, '{source_server_user}', '{source_server_password}';

Once the servers are linked, you can start pulling over the data. This is where things get really hairy. In CRM 4, the ActivityMimeAttachment table stored all of the attachments and the associated metadata. In CRM 2011, this information is split apart, and there are extra columns that we need to supply data for. First, we'll need to populate the Attachments table:

Insert Into Attachment
Select Body, [Subject], FileSize, MimeType, [FileName], Null as 'VersionNumber',
ActivityMimeAttachmentId as 'AttachmentId'
From {source_server}.{source_crm_database_name}.dbo.ActivityMimeAttachment

After that, we can pull over the metadata. One piece of information that we're missing is the "solutionId". I sent a test email from 2011 and inspected the row it inserted to get the solutionId.

Insert into ActivityMimeAttachment
Select AttachmentNumber, ActivityMimeAttachmentId, Null as [VersionNumber],
    {solution_id} as SolutionId, ama.ActivityMimeAttachmentId as 'AttachmentId',
    Null as 'SupportingSolutionId', ActivityTypecode as 'ObjectTypeCode',
    0 as 'IsManaged', 0 as 'ComponentState', '1900-01-01 00:00:00.000' as 'OverwriteTime',
    (newid()) as 'ActivityMimeAttachmentIdUnique', ama.ActivityId as 'ObjectId'
From {source_server}.{source_crm_database_name}.dbo.ActivityPointerBase apb
Join {source_server}.{source_crm_database_name}.dbo.ActivityMimeAttachment ama
    on ama.ActivityId = apb.ActivityId</pre>

When you're ready to unlink the source server, use the drop commands:

Exec sp_droplinkedsrvlogin '{source_server}', null
Exec sp_dropserver '{source_server}'</pre>

At this point, I was concerned that the script may crash and/or run out of memory. To avoid this possibility, I decided to pull the data over in batches. I won't get into the specifics, but you can reference my complete script below. From a high-level, I'm adding a column to the source table to mark rows that I'm pulling over, and what the migration state is. I made it nullable so that it doesn't affect normal CRM operations.

Declare @LinkedServerUser Varchar(Max)
Declare @LinkedServerPassword Varchar(Max)

Set @LinkedServerUser = {linked_server_user};
Set @LinkedServerPassword = '{linked_server_password}';

Print 'Linking Remote Server...'
Exec sp_addlinkedserver '{linked_server}', 'SQL Server';
Exec sp_addlinkedsrvlogin '{linked_server}', 'false', null, @LinkedServerUser, @LinkedServerPassword;

Create a column in the source database to track which rows have been converted.

--1=Queued to move to attachments
--2=Moved to attachments
--3=Queued to move to mime attachments
--4=Fully migrated
EXECUTE {linked_server}.{crm_database_name}.dbo.sp_executesql
    N'Alter Table ActivityMimeAttachment Add T_Processed Int'

Declare @SolutionId UniqueIdentifier
Set @SolutionId = '{solution_id}';

Declare @Rows Int

Print 'Copying attachments...'
Set @Rows = 1
While(@Rows > 0)
    --Queue up records we haven't moved
    Update Top (1000)
    Set T_Processed = 1
    Where T_Processed Is Null

    Insert Into Attachment
    Select Body, [Subject], FileSize, MimeType, [FileName], Null as 'VersionNumber',
        ActivityMimeAttachmentId as 'AttachmentId'
    From {linked_server}.{crm_database_name}.dbo.ActivityMimeAttachment
    Where T_Processed = 1

    Set @Rows = @@RowCount
    RAISERROR ('Attachment Rows Updated...', 0, 1) WITH NOWAIT

    --Mark the records as moved
    Update {linked_server}.{crm_database_name}.dbo.ActivityMimeAttachment
    Set T_Processed = 2
    Where T_Processed = 1

Print 'Copying attachment metadata...'
Set @Rows = 1
While(@Rows > 0)
    --Queue up records we haven't moved
    Update Top (1000)
    Set T_Processed = 3
    Where T_Processed = 2

    Insert into ActivityMimeAttachment
    Select AttachmentNumber, ActivityMimeAttachmentId, Null as [VersionNumber],
        @SolutionId as SolutionId, ama.ActivityMimeAttachmentId as 'AttachmentId',
        null as 'SupportingSolutionId', ActivityTypecode as 'ObjectTypeCode',
        0 as 'IsManaged', 0 as 'ComponentState', '1900-01-01 00:00:00.000' as 'OverwriteTime',
        (newid()) as 'ActivityMimeAttachmentIdUnique', ama.ActivityId as 'ObjectId'
    From {linked_server}.{crm_database_name}.dbo.ActivityPointerBase apb
    Join {linked_server}.{crm_database_name}.dbo.ActivityMimeAttachment ama
        on ama.ActivityId = apb.ActivityId
    Where ama.T_Processed = 3

    Set @Rows = @@RowCount
    RAISERROR ('ActivityMimeAttachment Rows Updated...', 0, 1) WITH NOWAIT

    --Mark the records as moved
    Update {linked_server}.{crm_database_name}.dbo.ActivityMimeAttachment
    Set T_Processed = 4
    Where T_Processed = 3

Print 'Dropping temp column'
EXECUTE {linked_server}.{crm_database_name}.dbo.sp_executesql  N'Alter Table ActivityMimeAttachment Drop Column T_Processed'

Print 'Unlinking remote server...'
Exec sp_droplinkedsrvlogin '{linked_server}', 'sa'
Exec sp_droplinkedsrvlogin '{linked_server}', null
Exec sp_dropserver '{linked_server}'

Print 'Migration complete.'

Like this post? Please share it!

See a mistake? Edit this post!

Jason Young I'm Jason Young, software engineer. This blog contains my opinions, of which my employer - Microsoft - may not share.

@ytechieGitHubLinkedInStack OverflowPersonal VLOG