Microsoft Azure StorSimple : Hybrid Cloud Storage - Part 2

This is the part 2 in my blog series about StorSimple. In the first part, we introduced StorSimple, the different models and the pricing. You can find the blog here. In this blog we will look into the key features of StorSimple device in detail.

Automatic Tiering

StorSimple uses three layers of storage - SSD, HDD and Cloud storage. The read & write of fresh data will always happen in the SSD tier. When data gets aged and is  accessed less, it is tiered to HDD layer. The cold data, ie the least accessed data will be tiered to the Azure Cloud storage tier. With this architecture, organizations need not worry about local storage capacity management and planning since cloud storage is integrated with the solution and archival data is automatically tiered to it.

Now let us take a look into the automated tiering workflow. When data is written first to the device it goes to the SSD tier. Inline deduplication and compression will be active but the archival procedure doesn't kick in until much later. So the data continues to be written in this tier until first a defined low threshold limit and  later a high threshold is reached. At this point the system starts identifying the non-working set of data, ie the oldest data and this  data is spilled over to the next tier ie HDD. Lower and higher threshold is always kept empty because we want to keep some buffer space available if the user wants to restore some archival data. at a later point. The threshold limit  is  94% for 8000 series,  after which data is migrated from SSD to HDD and from HDD to AzureStorage . These processes are transparent to users and applications, and there is no impact on how they access the data.

In the case of  StorSimple Virtual array, we do not have a concept of SSD and HDD tiers. Hence the data is tiered directly to Cloud from the local storage ie the virtual hard disk .It is done based on a data heat map, which tracks usage of data , its age and relationship with other data. The active data or hot data is stored locally and cold /inactive data is tiered to cloud storage

Deduplication, Compression and Encryption

Dedeplication is enabled by default in StorSimple devises and there are no special licenses associated with it. When data comes into the StorSimple device, it  is written as 64 Kb blocks. For every block, hashkeys are built and a metadata map is created. SSD tier consists of raw storage and this metadata map. Deduplication happens in the SSD layer, thereby ensuring performance. When data comes in, it is matched with the metadata map. If  block exists it discarded and the pointers are updated. Same is the case with data being read. This helps in optimal utilization of  local capacity and makes operations like data migrations time efficient.

As mentioned earlier , data is spilled over to HDD layer once the high watermark is reached in SSD tier. It is in HDD tier that lossless compression of the data sets are done. The type of compression used is deflate compression. That means, data residing  in the HDD tier is fully deduped and compressed . However the users can continue to access the data in the device without any noticeable difference as the entire process is transparent


When data is tiered out from StorSimple array local storage to Cloud it is encrypted using AES-256 encryption. Customer holds the encrypted data. Data is converted to iscsi blocks , deduped ,compressed and then encrypted before sending to Azure. The data is sent to Azure over HTTPS. Data residing in Azure is further protected by mechanisms RBAC, login password, auditing, Access keys etc.  These are the different layers of security for your data in StorSimple.
 
To summarize the process , deduplication happens in SSD tier and when SSD reaches capacity the data is compressed and moved to HDD tier. When the data is ready to be tiered to cloud it is encrypted and send to cloud storage over HTTPS. However in case of virtual array there is a small difference wherein the data that resides in the local storage is not deduplicated and compressed .The deduplication, compression and encryption happens before data is tiered  to Cloud storage .

Local Snapshots and Cloud Snapshots
 
Snapshots refer to the inbuilt backup mechanism of StorSimple devices. There are two types of backups -  local snapshots and cloud snapshots

Local snapshots are point in time copies of data in StorSimple local storage. They are usually scheduled on daily and weekly basis with shorter retention periods. They are useful for restoring any recently deleted data. Local Snapshots uses Copy reference On Write(CROW) method. It makes use of volume metadata references for creating storage efficient snapshots and is stored locally in the devices

Cloud snapshots are point in time copies of data in Azure Cloud storage. Cloud snapshots are typically scheduled with longer retention periods, like weeks and months and are useful in DR scenarios. In case of cloud snapshots, entire data and the metadata is copied over to cloud when the snapshot is taken for the first time. All subsequent snapshots are incremental , ie only the changed data and metadata is copied over to cloud thereby optimizing the cloud storage  usage
 
StorSimple physical array supports both local and cloud snapshots. However Virtual array supports only Cloud snapshots

In the next part of this blog series we will look into the different management tools and some important StorSimple terminologies /concepts
 

Picture courtesy:http://www.slideshare.net/MicrosoftSuomi/microsoft-azure-storsimple-paul-chiola
 

Microsoft Azure StorSimple : Hybrid cloud Storage - Part 1

Exponential data growth is one of the biggest challenges faced by organizations today. Traditional storage solutions are giving way to more robust cloud based storage systems. Azure StrorSimple is Microsoft's offering in the Hybrid cloud storage space that is capable of catering to all key storage requirements like primary storage, data archival, tape replacement, intelligent tiering, offsite storage etc. It has automatic storage tiering built in , and can tier all your less used and archival data to Azure cloud storage without any operational overhead. This cloud integrated storage mechanism was developed by a company called StorSimple which was acquired my Microsoft. Now this solution is offered under the umbrella of Azure Hybrid Cloud storage solution

 
Some of the key features of this storage solution are: 
 

• Seamless integration of cloud storage with local storage
• Automated tiering mechanism
• Combination of SAS and SSD drives for local storage
• Deduplication and compression of data
• Thin provisioning
• Local snapshots and Cloud snapshots for backup
• Certified support from VMware
• Inbuilt resiliency for hardware device with dual controller, hot swappable
• Non-disruptive software upgrade
• Integrated DR capabilities for recovery from cloud storage
• Deterministic thin restores to download only the working data set

Now let us take a look at the  different types of StorSimple devices available
 
StorSimple is available as  physical rack mountable storage device, a virtual appliance and a cloud  appliance
 

StorSimple Physical devices - 8100 & 8600 models:
 
There are mainly two models - 8100 series and 8600 series. 8100 is a 2U device with a total usable local capacity of 15 TB and a maximum capacity of 200 TB including cloud storage. 8600 is a 4U device with an Extended Bunch of Device(EBOD) enclosure and related components. The total usable local capacity is 40 TB and a maximum capacity is 500 TB including cloud storage. The storage enclosures includes a combination of SSD and HDD drives for local storage. There are 12 disk drive slots per enclosure, with 8600 devices having additional 12 slots in the EBOD enclosure. The drive slots support SAS disk drives, which can be combination of SSD and HDD. The devices had resiliency built in , with share processors and storage along with mirrored controllers in active passive mode

StorSimple Cloud appliance - 8010 & 8020 models :

StorSimple Cloud appliance, as the name indicates, runs in Azure as a Virtual machine. In the Microsoft StorSimple documentation, you may find this being referred to as StorSimple Virtual device. It comes in two models - 8010 and 8020. 8010 devices can support a maximum capacity of 30 TB and 8020 devices upto 64 TB. 8020 is the latest model and has the capability to support premium storage for high performance work loads. You can connect volumes exposed by the StorSimple  cloud appliance to your Virtual machines in Azure using iscsi protocol.Another important target use case for cloud appliance is Disaster Recovery(DR). You can failover from your physical StorSimple device to a cloud appliance in the event of a disaster and bring up your machines in the cloud.

Imp: StorSimple Cloud appliance is always used in conjunction with a  StorSimple Physical appliance. One of the prerequistes of using a StorSimple Cloud appliance is that you should have a physical device registered with the StorSimple manager service running from the Azure portal.


 StorSimple Virtual array - 1200 model :

StorSimple virtual array is an ova format of the StorSimple Physical device that can be deployed both in VMware and Hyper-V hypervisors.It can provide a maximum local storage capacity of 6.4 TB and total capacity including cloud storage upto 64 TB. This solution targets enterprises who want to go for a cost-effective , but hybrid cloud based storage solution. It supports volumes being exposed as iscu targets along with SMB/CIFS based file shares. You can use it the same way that you would use a physical StorSimple device in terms of management and confirmation. There are certain limitations though when compared to the Physical device. For eg: it supports lesser storage capacity including cloud storage , ie 64 TB, when compared to 200 TB or 500 TB supported by   8100 and 8600 devices respectively. Also being a virtual device the performance has a dependency on the underlying Infra and Virtualization platform. However it supports all major use cases like cloud based storage, backup and DR. It can also be registered and managed from the Azure Portal like the Physical device

StorSimple Pricing

StorSimple Physical array and Cloud appliance requires you to have an Enterprise Agreement . Once you have registered the Physical device you can create the cloud appliance as well in Azure . This Cloud appliance is nothing but a Virtual machines of type Standard_A3 in case of 8010 model and Standard_DS3 if you go for 8020 model. Deploying these appliances will incur additional VM compute and Storage charges like in the case of a normal  Azure VM.

StorSimple Virtual Array on the other hand, is now available on a pay as you go model also. You can find the latest announcement regarding the same here :

https://azure.microsoft.com/en-in/blog/storsimple-new-offers/

You can download the OVA from the Azure portal , deploy it on you on-prem infra, and register it with the management service in Azure portal. The Virtual array uses a pay as you go model, where as you will be charged on a per day basis. In addition to that you will have to pay for Azure blob storage and data egress charges. For latest pricing of the virtual array, please refer to the following link : https://azure.microsoft.com/en-in/pricing/details/storsimple/

 

Stay tuned for the next part of this blog series, where we will do a deep dive into the  salient features of StorSimple that makes it stand apart from the crowd!!
 
Update : You can read part 2 of the blog series here