Thursday, October 13, 2016

Security in the cloud - Disk encryption in Azure

Security in the cloud is a priority for every organization planning to adopt public cloud for mission critical applications. In Azure, these security concerns are addressed at different layers starting from the platform layer up to the VM OS layer. This picture shows an overview of the different layers of security in Azure

Any traffic directed to your applications hosted in Azure will first hit the platform's native DDOS protection mechanism. If a DOS attack is happening a specific IP is targeted, the DDOS protection mechanism will blackhole the traffic and the endpoint will be brought down. Thereby the surrounding resources will be protected. If you have resiliency built in, you can bring up another endpoint and ensure that your service is available
At the next layer you have endpoints, ie traffic will be received only at the designated endpoints in case of classic model, or as defined in NSGs in case of ARM model. The VMs can be placed in different virtual networks which are isolated from each other by default. Inside Vnet you can use NSGs and UDRs to manage traffic. If you want to notch up the security one level higher , you can introduce  Virtual Network Appliances in your network.For eg: Firewalls, IDS/IPS devices etc that are available in the Azure marketplace.
In this blog , I am going to talk about security at the VM layer, ie encryption of your data at rest using disk encryption. This is achieved using Bitlocker encryption of Windows VMs and DMcrypt in Linux VMs. It leverages Azure KeyVault services to store your encryption keys which is again an additional level of security. The process is quite straight forward and encryption can be done for new as well as existing VMs. In case of Windows IaaS VMs, we can encrypt both the OS and Data disk, while in case of Linux IaaS VMs the data disk can be encrypted.You will have to use the ARM deployment model to leverage this feature, since it does not work with the classic model. There are ARM templates readily available in GitHub that can readily help you with the encryption of existing and New VMs.
 Let us take  a look at the steps to be done to enable OS and Data disk encryption of IaaS windows VMs in Azure
1) Create an Azure KeyVault
This can be now done from the ARM portal itself. KeyVault is available in the Azure Portal in preview mode. You can create a new keyVault  by providing basic information like KeyVault Name , Resource group Name,Location etc. In addition to that , by default your user account will have access in the "Access Policies". You can edit the "Advanced Access Policy" and enable all the three options given there
Alternately, you can also use the ARM template available at this location to create a KeyVault:
This template will create a KeyVault for you with all the three advanced access policies including the Volume encryption policy that we need for Disk encryption.
2) Create an application in Azure AD with permission to access KeyVault:
This is a very important step since you will be using this application id and key during VM encryption
1. Select the organization's active directory from the classic portal and select the application tab

2.:Click on add from the bottom menu to add a new application. Select the first option, ie add an application my organization is developing

3. Provide name of the application in the next step

3. Add the Sign-on URL and App URI. You can enter any value here in URI format, it need not have to be an existing application. Only requirement is that it should be unique for an organization

4.Now click on configure, and copy the client Id of the application

5.Next we need the application key. This can be generated from the portal from under the keys session. Select duration as 1 year from the drop down. Once you save the configuration, a key will be displayed which can be copied over

6. Next step is to provide this application access to keyvault. It can be done from an Azure PowerShell window using the following command
Set-AzureRmKeyVaultAccessPolicy -VaultName $keyVaultName -ServicePrincipalName $aadClientID -PermissionsToKeys 'WrapKey' -PermissionsToSecrets 'Set' -ResourceGroupName $rgname

You have to set the variables $keyVaultName,$aadClientID,$rgname  to have values of your keyvault name, client id of the application that we got at Step 4 above and Resource group name

Now you have the client id and the key that you will need during the ARM template execution. Now lets proceed with the VM disk encryption

Create a new encrypted VM using ARM template and KeyVault

Deploy the following template available in GitHub to encrypt a new VM :

Click on the "deploy to Azure" option in the page to deploy the ARM template directly in Azure

Provide the mandatory parameter values like VM name, admin username, password, Storage account name, Virtual network, subnet,keyvault name and keyvault resource group along with the client ID and Client Secret  of the Azure AD application that we created earlier. Additionally there will be two options for Key Encryption key and URL.It is not mandatory and we are not using that in this example. You can then agree to the terms and conditions , click on purchase and the deployment of encrypted VM will start

Encrypt an existing VM using ARM template and keyvault

Deploy the following template available in GitHub to encrypt an existing VM : :

Deploy the template to Azure and provide details of the VM that you want to encrypt.'Volume type' can be OS,Data or All(default value) depending on which disk you want to encrypt

Check encryption status

Now that you have enabled the encryption you might want to verify the status. There are few ways to check this.

Easiest way is to check from the Azure portal.Navigate to the Disks information of the VM from the portal. It will show the OS disk status as Encrypted.

The data disk encryption status is currently not reflected in the portal. However, you can check the data disk encryption status by using Azure PowerShell command Get-AzureRmVmDiskEncryptionStatus and providing resource group name and Vmname as parameters

Get-AzureRmVmDiskEncryptionStatus  -ResourceGroupName $rgname -VMName $Vmname

You will see OSVolumeEncrypted and DataVolumesEncrypted status as True if encryption is enabled

You can also check the status of disk encryption from within the VM using ‘manage-bde’ command and providing the drive letter as parameter. Sample output given below

Also, it can been seen from the GUI of the server , the drives will have a lock signal associated

Now you know how to enable disk encryption for protection of data at rest in Azure!!!!

Disk encryption is only one of the security measures available in Azure. There are many other options like NSGs,Virtual appliances, UDRs etc that we discussed in the introduction. Also, there is a new service called Azure Security Center that takes cares of your Azure environment security requirements holistically . Keep watching this space for more information on it!!!

Friday, September 16, 2016

#MyAzureLabs: DRaaS using Azure: Test your DR strategy

This is Part 2 of my blog post on DRaaS using Azure. You can view first part of the blog here

In first part, we discussed how to protect your on-prem physical servers using Azure Site Recovery services. Having a DR strategy and enabling protection is not always enough. You should ensure that your DR strategy will work as expected when a disaster strikes. In case of usual DR solutions, it is not always possible to test the DR strategy without downtimes. However, Azure Site Recovery provides you with an option to test your DR strategy and keep it well oiled and battle ready!! Test failover to the rescue..  Lets see how we can do a test failover of on-prem physical services to Azure.

Select the vault where your replicated data resides. Select the settings, and choose the replicated items.

Select the option "Test failover"

Select the settings of the test failover. The failover direction will be automatically selected, ie from on-prem to Azure. You can select the recovery point and the virtual network. Recovery point can be the latest point in time, latest processed point in time, latest application consistent point in time or even a custom restore point. The virtual network should not be same as what you will be using in your production, ie the one that you specified in the target configuration while enabling the replication

Click ok and the test failover will begin!!

Now, if you go and check the job status you can see a number of things happening. The test failover will create a test environment, spin up your VM and at one point wait for your input to complete testing

The VM created by the test failover will be listed in the portal now. You will have to do some additional configurations though to connect to the VM and test your applications

- Go to the settings of the VM, select the NIC card and associate a public address to it
- Now associate a NSG to the NIC card and create the rules that you require, for eg: RDP access, access to a web site that you might be running in the VM etc

After that the connect button of the VM will be enabled and you can connect to it and test your applications.

All done now and if you are happy with the DR test, you can clear the failover environment with a single click. Go back to the test failover job, click on  "complete failover" . You will have to provide a reason, and then click ok.

That's and the test environment will be magically cleaned up by ASR !! Now that is a very easy and clean way of testing your DR strategy without disturbing any of your existing environments..

Monday, August 15, 2016

#MyAzureLabs : Azure Point to site VPN configuration for existing Vnet

Azure Point-to-Site enables VPN connectivity from client machines to Azure Vnet. This is especially useful for mobile users,  who could be travelling and is not connected to your office network. There is a very good documentation available on how to configure Point 2 site VPN for a new Vnet, both for classic and new portal . It  can be found here :

What if  you already have a Vnet in Azure with resources connected to it ? In this blog, I will elaborate on how to enable Point-to-Site VPN for an existing Vnet . It is documented based on the testing done in new portal. The Vnet was already existing, and a VPN gateway was created from the new portal using the graphical interface and connected to the Vnet. For the remaining steps, PowerShell was used.

1. Create VPN gateway . Go to new portal->Virtual network gateway and create new. You will have to select the Vnet for which you want to create the gateway, provide a gateway subnet IP range , select a public Ip address , select the Gateway type as VPN and VPN type as route based. Please note that for enabling Point-to-Site VPN connectivity , the gateway type should be route based.

2. Create an IP pool , from which IP will be allocated to VPN clients. This should be done using Azure PowerShell. First login to your Azure account


Create VPN Client Address pool

Set-AzureRmVirtualNetworkGatewayVpnClientConfig -VirtualNetworkGateway $Gw -VpnClientAddressPool ""

3. Create a self signed certificate for your client machine using the following steps mentioned in the following link

4. Once the client certificate is created installed , next steps is to export the root certificate in .cer format. You can do that from the certificate management mmc. Ensure that you select the option "Base-64 encoded X.509 (.cer)" in the certificate export wizard

5.You need to copy over this .cer file to the machine from which the Azure PowerShell commands will be executed. Run the following command from Azure PowerShell .

$Gw = Get-AzureRmVirtualNetworkGateway -Name "p2sgwtst" -ResourceGroupName "p2stest"

 Here p2sgwtst is name if my Virtual Network Gateway and p2stest is Resource Group Name

6. Get information about your root certificate

$Text = Get-Content -Path "\<path of root certificate>\Root.cer"

7. Add the root certificate to the VPN gateway

$rootCert = Add-AzureRmVpnClientRootCertificate -VpnClientRootCertificateName "RootCertificateNp.cer" -PublicCertData ($CertificateText | out-string) -VirtualNetworkGatewayName $gw.Name -ResourceGroupName "p2stest"

 8. Download the Azure VPN package

Get-AzureRmVpnClientPackage -ResourceGroupName "p2stest" -VirtualNetworkGatewayName $gw.Name -ProcessorArchitecture Amd64

You will get a download link for the exe as output. Copy it over to the client machine and install using administrative privileges.  If it is all installed correctly, you will see the network in the list of VPNs in your machine


Sunday, July 3, 2016

DRaaS using Azure: How to protect your on-prem physical machines.... #MyAzureLabs

BC/DR is a key consideration for all organizations big or small. Thanks to Azure, we now have an affordable and easy to implement BC/DR solution . Azure site recovery service(ASR) can be used for a multitude of disaster recovery scenarios, with an economic pay-as-you-go costing model. The DR scenarios catered to by ASR currently are:

DR site in Azure
- Physical machines to Azure
-VMware environment to Azure
- Hyper-v(with or without VMM) to Azure

DR site in a secondary DC, and orchestration by ASR
-VMM site to site
-VMware/Physical to VMware
-VMM to VMM(SAN replication)

This week in my Azure labs, I tried out the first scenario, ie DR from On-Prem Physical machines to Azure. This blog is all about my little experiment and some tips and tricks that I learned during the same.

The following link , which explains the procedure for protecting Physical/VMware environment is a good starting point:

I will use this article as reference point, which is very detailed and well written. I will be going into further more details on few of the areas mentioned in the link .Based on my experience,  I think it will be helpful for someone trying to set up a Physical server to Azure replication for the first time.

You should ensure that the prerequisites for physical server protection mentioned in the link are taken care of. You need to run the Site Recovery Unified SetUp for installing the configuration and process server. Refer to "Step 2: Set up the source environment" in the link above for details on initial set up of the vault, setting up configuration server, registering it in Azure etc. These steps are  pretty straight forward. Detailed explanation of the configuration server setup on-prem  is also mentioned in  Step 2 of the article

Lets assume that you done the initial vault creation , configuration server setup , created target environment in Azure(Resource group, storage, network etc) and have also created the replication policies to be used. All these come under "Step 1: Prepare your infrastructure" in your site recovery vault. These steps are again clearly explained in the official documentation :

Now lets see what needs to be done at the physical server end to enable the protection

Steps to be done on Physical server:

1)Set up the registry key entry

2)Enable the following in Allow an app or feature through Firewall.
    > File and print sharing
    >Windows Management Instrumentation

3)Add an account that has admin privilege in the target physical machine in the cspsconfigtool.  It can be found in the following location in the configuration server

Click on Add account

4)In my case, the physical machine was not added to domain. Hence I added a local admin user. The friendly name can be anything, it is just for identifying that account in Azure portal.

5)Now you can install the mobility agent on the physical server. The installer can again be found in the configuration server at the following location. You need to select the installer based on the operating system type. In my case I selected the Windows installer

Select option to install Mobility service

Enter Configuration server IP and Passphrase

Specify install location. That is all that is required. You can go to the next step and wait for the installation to be complete

Steps to be done in Azure portal:

Now that the mobility agent is installed, you can refresh the configuration server in the Azure management portal

Go to <recovery services vault> -> Settings->Site recovery infrastructure->Servers
select the configuration server and click refresh server

Click ok on the message and wait for the refresh to be completed.

Once the refresh is completed, ideally the new physical server will be reflected in the connected agents list

Now you can go ahead and enable replication for your physical server. In the Management portal, go to <Recovery services vault>->Settings->Site Recovery->Enable replication

Enter the source. This will be your configuration server .Machine type will be Physical machines and Process server in this installation is same as configuration server

Configure the target environment in Azure.

You need to select the target physical server at the next step. Click on the + sign

Enter details of your on prem physical server, ie server name , IP and the OS type

Click ok and wait for the server to be added
Once the server is added, it will be listed in the blade. Select the server and click ok

In the next step configure properties. If the agent is installed correctly and is detected by the portal, you will be able to select the disks that you want to backup . ie , disks other than the OS disks
From the account dropbox you can select the account that you had created earlier in the cspsconfigtool.(Refer step no: 3)

In the configure replication settings page, select the replication policy that you had created earlier

Now all the steps are done, and you can click "Enable replication" to protect your on-prem physical server

You  can click on notifications to see progress of the task. You can also go to <site recovery vault>->Jobs->"Site Recovery Jobs"-> and select the "Enable protection" job to see the status

If you see all green ticks, your machine protection is enabled . You can see the status of replication from  site recovery vault>->Replicated Items. Once the replication is completed, the status will be shown as protected

Now that the Physical server is replicated and protected, we might want to test if everything will work as expected during a Disaster. Right? That is where the Test failover feature will help. I will cover that in my next blog post. Keep watching this space for more!!

Update: The second part of the blog is published, which covers the test failover process. You can find it here

Friday, June 17, 2016

The cloud has got your back(up): A primer on Azure Backup

 Azure backup offers a comprehensive cloud based  hybrid backup solution that enables backup of not only your Azure VMs, but your files, folders, applications etc both on prem and in Azure. This solution can be used to replace your chunky on-prem backup solutions,  tape drives , backup tapes and the likes. In this blog, I will give a brief overview of the Azure backup service, its advantages and scenarios that it can cater to currently

Service highlights:

Azure backup is offered as a complete backup as a service offering. Lets take a look at few highlights of the service

Cost effective

You need not own any backup infrastructure, say services , tools and devices to use this service. You can directly subscribe to the service and pay based on your usage. There are no additional compute charges involved in the service. You pay a fixed charge for each protected instance, and also for the storage that you consume in the cloud for storing your backup data. The egress traffic for restore is also free, in addition to the free ingress backup traffic to cloud. Only the first backup is full backup  The data being backed up from on prem and transferred to Azure is compressed before the transfer . This will reduce the storage space used in azure for storing the backup, thereby reducing the storage cost


 It offers the flexibility of centralized backup operations management from the cloud. Since the backup is stored in the cloud, you can leverage its unlimited scale and high availability capabilities.The backed up data can be stored in either an locally redundant storage or a globally redundant storage. LRS will keep three copies of your data in a given location, and will be suitable for cost conscious customers. GRS , in addition to the 3 local copies will store 3 additional copies in a different geography. This provides additional resiliency incase of an Azure site level disaster


Ample emphasis is given on the security aspect as well while designing the service. The backup data is encrypted using a passphrase that will be available only locally. The data in transit and rest is encrypted. Only an administrator who possesses the passphrase can decrypt the data.


The backup data can be application consistent, file consistent or crash consistent depending on your backup scenario . Application consistent backups in windows ensure that you need not do additional fixes in your application when you restore it. This greatly reduces the recovery time in case of a disaster. This makes use of VSS technology in windows. Since VSS is not present in Linux, backup of Linux machines will be file consistent. Crash consistent backups are those backups taken when your machine is shutdown

Long term retention

You can store the backup data in cloud for as long as 99 years!!

 Backup scenarios

When you sign up for azure backup service, you will first create a backup vault in the cloud. It is nothing but a storage space for your backup. You can choose LRS or GRS storage depending on your resiliency preferences. Azure backup makes use of different components in different backup scenarios. For eg, file and folder level backup needs a different tool than a VM level backup. Let us take a look at the different components of Azure backup

Azure backup agent

This is a standalone agent that can be installed for taking file, folder and volume level backup on a Windows OS. The machine can be physical or virtual and can reside either on-prem or in Azure. You can download the agent from the management interface of you backup service in Azure and install in target location. The agent should be registered with the vault using a vault credentials. Also a passphrase is created during the installation that will encrypt data in transit and at rest. You can restore the data to either the same machine or to a different machine. You will have to provide the passphrase to initiate the restore process

System center Data protection manager + Azure backup agent

System center DPM can work in conjunction with the Azure backup agent to backup your workload to Azure. It supports all major MS workloads like SQL, SharePoint, AD, Exchange etc in addition to file/folder backup and VM backups.This option is more suited for customers who already have an investment on system center suite of tools. They can install the backup agent in the DPM server and take backup of files, folders, VMs and applications to Azure. The DPM can be hosted either in on-prem or in Azure. It also supports VM level backup of Linux machines hosted in Hyper-V. It makes use of app aware VSS snapshots to ensure consistency of backed up data.

Azure backup server

This can be considered as a stripped down version of the DPM option. It provides all the functionality of DPM+ Backup agent, except the following
- It doesn't not need a system center integration
- Tape drive is not supported
- An azure backup subscription is required

Azure backup server supports pretty much all workloads supported by DPM. If you don't want to backup to cloud, you can even use it for an on-prem disk to disk protection. You can consider it a subscription based backup service where you are charged based on the number of protected instances. If you are backing up to cloud you will be charged for the cloud storage as well

Azure IaaS VM backup

This is very straight forward VM level backup of VMs that you host in Azure using the backup service. You can backup both Linux and Windows VMs using this service with no additional agent installation

That is Azure backup in a nutshell. You can refer the official Azure documentation here to understand more about each scenarios and the service capabilities

Keep watching this space for more articles on Azure!!!

Wednesday, June 1, 2016

Azure VM migration using PowerShell

Microsoft recommends usage of ARM for all new deployments in Azure. All new developments/features/services will be available in ARM going forward.  But  there are lot of services that are yet to be migrated to ARM. What if one of the services that you want to use is not currently available in ARM and you have already set up rest of your environment in ARM?  In such a scenario, you can always set up a site to site VPN between the classic V1 VNET and the ARM VNET. This process is also well documented:

That being the case,  what if we want to test the interoperability of services and you want to move few already set up VMS in ARM to classic? I know that it is not a very common scenario. Also it is not a recommended approach for production deployment, ARM is definitely the way to go. However, for enabling that test run you might very badly want to do before taking the plunge, we will look at the process of creating a new VM in classic portal from a hard disk of VM in ARM portal using Azure PowerShell.

First you will have to login to your Azure account


Enter the Source blob uri, ie location of the ARM VM's VHD

$sourceBlobUri = https://<Source-storagename><vhdname>.vhd

Set the Source context

$sourceContext = New-AzureStorageContext  –StorageAccountName <Source-Storagename> StorageAccountKey <Storage access key>

In the destination context, give the name of your classic storage and its key

$destinationContext = New-AzureStorageContext  –StorageAccountName <dest-storagename> -StorageAccountKey <Storage access key>

Copy the vhd to the destination storage

Start-AzureStorageBlobCopy -srcUri $sourceBlobUri -SrcContext $sourceContext -DestContainer "vhds" -DestBlob "rds1201647182929.vhd" -DestContext $destinationContext

This command will copy the VHD from the source storage to the container named 'vhds' in the destination storage. Ensure that your VM is in a stopped deallocated  status during this procedure. Also if you want to The copy over took only a few minutes in my experience

 Now we need to add this VHD as an OS disk in the gallery. Start with importing the publish settings file of the subscription


Download the publish settings file and import it

 Import-AzurePublishSettingsFile  '<Publish settings file name>'

Set the current subscription

Set-AzureSubscription -SubscriptionName "<Subscription name>"

Now add the OS disk

Add-AzureDisk -DiskName "OSDisk" -MediaLocation "https://<Source-Storagename><vhdname>.vhd" -Label "My OS Disk" -OS "Windows"

Refresh Azure classic portal

OS disk will be listed in the gallery. Now you can go ahead and create new VM from the disk!!

Note:  The reverse of what is explained here is also possible, that is Migration from classic to ARM.  There are well documented tools available for the same:


Sunday, May 29, 2016

Azure automation:Using Graphical runbooks

Azure automation can be an Azure administrator's best friend and can  ease up your day to day administration work. There are three options available in Azure automation- Graphical runbooks, PowerShell Work flow and Powershell based runbooks
If you want to play around with Azure automation and want to quickly automate some daily mundane tasks, graphical runbooks are the easiest to start with . You can find many templates in the runbooks gallery in Azure that can easily get the job done easily  for you. Lets start with the basics. One of the common tasks that needs to be done is to start or stop VM at a scheduled time, say for eg: your Dev/Test machines that should be shutdown after office hours. The runbooks for this are readily available in the gallery. In this blogpost we will focus on a graphical runbook available in the gallery that can be used to start or stop VMs at a scheduled time.
                 Schedule automated start and stop of VMS using graphical runbook

Create new Azure automation account from portal using the default settings. Go to New->management->Automation->new automation account
You will get a confirmation  message as follows

Browse Automation accounts and select the newly created account. Click on runbooks tile->Browse gallery

Search for  the graphical runbook  “Start Azure V2 VM”. For stopping you can use the graphical template “Stop Azure V2 VM”


Import the template







 Click on edit and then “test pane” to do a test run of the template


If you want to start a single VM, give the name of the VM as input parameter. Click on start to do a test run



Publish the runbook to make it available in your automation account


Now you can add a schedule to the runbook

Configure the time when you want the schedule to run


You can select the time and time zone here when you want the schedule to run


Configure input parameters for the schedule, for eg: the VM name or resource group name.
If you want to restart a single VM in a given resource group, specify the name of the resource group as well as the VM name. If you want to start all VMs in a resource group, give the name of the resource group alone