Virtualization and the “Datacenter Operating System” of the Future

Is hypervisor based server virtualization basically a “useful” tactical consolidation “play” or does it form the basis for much more? For several years, server virtualization has been used with good success by many organizations to consolidate legacy workloads as well as to cost-effectively provision test and development virtual machines (VMs). During the same timeframe, there has been significant discussion pertaining to “the datacenter of the future” – a centrally managed “dynamic” datacenter where resources will be dynamically assigned “on demand” to workloads in an agile manner thereby allowing the datacenter operations teams to quickly adapt to fluctuating demands for compute resources. So, “where is the connection?” you ask.

The recently concluded VMworld 2007 may offer some hints. At that conference, VMware Chief Scientist, Dr. Mendel Rosenblum, demonstrated some key capabilities that show where compute virtualization is headed in its evolution to becoming the “datacenter operating system” of the future. Dr. Rosenblum demonstrated “storage live migration” and “lockstep VMs”. Transparent migration of storage “owned” by a VMs from one disk (LUN) to another (potentially on a different storage frame) without impacting application availability was demonstrated in the first demo. He also demonstrated the ability to transparently failover an executing VM to a “secondary” VM – on another host - that has been executing in lockstep with the primary VM! If you did not have a chance to see Dr. Rosenblum’s keynote, a webcast is still accessible here.

Per the VMworld demonstrations mentioned above, the hypervisor based virtualization capability and upcoming hardware assists for virtualization in combination with a “compute virtualization manager” that collectively enable policy-based management of the datacenter including virtualization hosts, VMs, storage is well on its way to evolving to the dynamic “Datacenter OS” of the future. This powerful Datacenter OS of the future – evolved from today’s “tactical” virtualization capabilities - will dynamically manage compute, memory, power, storage and network resources of the Datacenter in an automated, highly efficient manner. Pre-packaged applications in portable VMs will be landed on these dynamic datacenters in a flexible and agile manner.

While the hypervisor based virtualization based evolutionary path may not be applicable to highly specialized datacenters (e.g., HPC), its applicability to Business computing environments is clear. Am I oversimplifying and underestimating the technical challenges (security and scaling to name a few) along this evolutionary path? Probably. Am I glossing over the organizational and business process related challenges? Almost certainly. On the other hand, are all the major building blocks like efficient hypervisors, virtualization hardware assists, VM live migration, storage live migration, power management, record and replay, lockstep VMs…) successfully prototyped or better (in production)? The answer is an emphatic “Yes”. Do you agree or do you forsee insurmountable barriers that will thwart attempts to evolve the virtualization capabilities of today into the dynamic data-center of the future along the lines suggested by the ground-breaking capabilities demonstrated at VMworld? Either way, I’d really like to hear your perspective.