Intel Rolls Out Enhanced Lustre* File System

For High Performance Computing (HPC) users who leverage open-source Lustre* software, a good file system for big data is now getting even better. That’s a key takeaway from announcements Intel is making this week at ISC15 in Frankfurt, Germany.

Building on its substantial contributions to the Lustre community, Intel is rolling out new features that will make the file system more scalable, easier to use, and more accessible to enterprise customers. These features, incorporated in Intel® Enterprise Edition for Lustre* 2.3, include support for Multiple Metadata Targets in the Intel® Manager for Lustre* GUI.

The Multiple Metadata Target feature allows Lustre metadata to be distributed across servers. Intel Enterprise Edition for Lustre 2.3 supports remote directories, which allow each metadata target to serve a discrete sub-directory within the file system name space. This enables the size of the Lustre namespace and metadata throughput to scale with demand and provide dedicated metadata servers for projects, departments, or specific workloads.

This latest Enterprise Edition for Lustre release supports clients running Red Hat Enterprise Linux (RHEL) 7.1 as well as client nodes running SUSE Linux Enterprise 12.

The announcements don’t stop there. Looking ahead a bit, Intel is preparing to roll out new security, disaster recovery, and enhanced support features in Intel® Cloud Edition for Lustre 1.2, which will arrive later this year. Here’s a quick look at these coming enhancements:

  • Enhanced security— The new version of Cloud Edition adds network encryption using IPSec to provide enhanced security. This feature can be automatically configured to ensure that the communication of important data is always secure within the file system, and when combined with EBS encryption (released in version 1.1.1 of Cloud Edition) provides a complete and robust end-to-end security solution for cloud-based I/O.
  • Disaster recovery—Existing support for EBS snapshots is being expanded to support the recovery of a complete file system. This feature enhances file system durability and increases the likelihood of recovering important data in the case of failure or data corruption.
  • Supportability enhancements—Cloud Edition supportability has been enhanced with the addition of client mounting tools, updates to instance and target naming, and added network testing tools. These changes provide a more robust framework for administrators to deploy, manage, and troubleshoot issues when running Cloud Edition.

Making adoption and use of Lustre easier for organizations is a key driver behind the Intel Manager for Lustre software. This management interface includes easy-to-use tools that provide a unified view of Lustre storage systems and simplify the installation, configuration, monitoring, and overall management of the software. Even better, the Intel Distribution includes an integrated adapter for Apache Hadoop*, which enables users to operate both Lustre and Apache Hadoop within a shared HPC infrastructure.

Enhancements to the Intel Distributions for Lustre software products are a reflection of Intel’s commitment to making HPC and big data solutions more accessible to both traditional HPC users and mainstream enterprises. This commitment to the HPC and big data space is also evident in Intel’s HPC scalable system framework. The framework, which leverages a collection of leading-edge technologies, enables balanced, power-efficient systems that can support both compute- and data-intensive workloads running on the latest Intel® Xeon processors and Intel® Xeon Phi™ coprocessors.

For a closer look at these topics, visit Intel Solutions for Lustre Software and Intel’s HPC Scalable System Framework.

Intel, the Intel logo, Xeon and Xeon Phi are trademarks of Intel Corporation in the United States and other countries.
* Other names and brands may be claimed as the property of others.