The Future is NowHPC Clusters Pair with Advanced Simulation Software to Spawn New Age of Innovative Design.

Produced by the editors of Desktop Engineering.Sponsored by Dassault Systèmes SIMULIA and IBM.

The Future is Now 1

INTRODUCTION

I t doesn’t matter if you’re talking about a state-of-the-art jetliner or a common piece of industrial machinery: Companies of all sizes across most industries are making more wide-spread use of advanced simulation far earlier in the design process. Physically accurate simu-

lation is leading their march toward greater product innovation and more cost-effective and efficient development cycles.

Yet simulation studies are only as robust as the hardware on which they are deployed. For years, companies’ analysis efforts have been hampered by workstations not powerful enough to support more extensive use of detailed simulation studies. At the same time, high performance computing (HPC) platforms that could handle the load have historically been too costly and hard to imple-ment and manage, putting them out of reach for mainstream engineering organizations.

The advent of less expensive, easy to deploy and manage HPC clusters paired with realistic simulation software optimized to exploit the cluster architecture has the potential to rewrite the rules. The marriage of the two technologies gives engineering teams a considerable competi-tive advantage, allowing them to meet the demands of shrinking design cycles while outpacing their rivals with a profitable stream of higher quality, less costly, and more innovative products.

SETTING THE STAGE FOR CHANGE

M anufacturers in the automotive, aerospace & defense, high-tech consumer electronics, marine, industrial machinery, and other high-growth industry segments are facing a daunting array of obstacles as they navigate the product development cycle while try-

ing to remain competitive on a worldwide scale. New technologies and the Internet are leveling the playing field, allowing smaller players

to enter diverse markets and opening the door to competition in new areas around the globe. Against this global backdrop is a complex regulatory climate where rules governing every aspect of safety, energy efficiency, and environmental sustainability are continuously evolving and vary depending on the country and target market.

The products themselves are also getting far more complex. The integration of more elec-tronics and software into mainstream offerings such as cars and appliances, shrinking product sizes, and advanced materials like composites are taxing manufacturers’ existing design and optimization processes. Factor in shorter time-to-market cycles and reduced R&D and en-gineering budgets, and manufacturers trying to do more with less are anxious for new tech-

nologies and design workflows that can help them rise to the challenge and carve out a competitive edge.

Realistic simulation is one of the core technologies that can assist companies in meeting new design imperatives. With realistic simula-

tion, companies can cost-effectively evaluate more design options, improve their efficiency, and help reduce overall risk. By

simulating the behavior of products in a virtual world, engineering organizations are better equipped to

make design recommendations in the areas of strength, weight, reliability, or material choices. They can do so without having to spend pro-tracted amounts of time or investing significant

capital on building physical prototypes for real-world testing. As a result, realistic simulation prac-

tices ensure that companies introduce the right products to the right markets in a more expedient fashion.

Coupled with this heightened interest in realistic simulation is a transition on the hardware front. Companies are starting to shift en-gineering and simulation work away from individual workstations to

Abaqus combined with high performance computing accelerates the simulation of large-scale linear dynamics workflows such as this automotive noise and vibration analysis.

The Future is Now 2

HPC clusters that have become increasingly more accessible to technical users who are not necessarily experts in computer and software technology.

While high-performance workstations have made huge strides in allowing engineers to con-duct robust simulations at the desktop, there are inefficiencies to this approach. Workstations must be individually maintained and data has to be moved to and from these systems, often causing delays or making it challenging to have a common source of data. Further, large simula-tion jobs can tie up workstations for days, even weeks for highly detailed and complex analyses, which isn’t practical for engineers who depend on their systems for other work. As a result of this logjam, the analysis process becomes limited by the practicality of the model development cycle. Engineering organizations get caught in the trap of running far fewer simulations and far less detailed analyses so they don’t bring their departments to a standstill—a scenario that’s hardly conducive to efficient and effective design optimization.

What is an HPC Cluster?

HPC clusters are a group of servers built from off-the-shelf components like multicore x86 servers and standard inter-connect technologies such as InfiniBand or Ethernet.

The cluster serves as a pool of technical computing resources formed by linking together multiple computers (or nodes) via the interconnect technologies. The HPC cluster aggregates computing power from the pool of processors, each with many cores—ranging from as few as 24 cores, up to hundreds or even thousands—to meet the processing demands of complex engineering software like real-istic simulation and rendering.

HPC clusters’ other big advantage is support for parallel processing, which breaks up a complex computing problem like a simulation into smaller jobs that are run across the cluster’s many nodes and multiple cores. The result is shortened runtimes when compared to individual workstations as the HPC cluster’s many cores work together to solve the problem in parallel.

HPC clusters built on this architecture can deliver a superior price/performance advantage over alternative solutions such as custom-built systems or supercomputers. As a result, HPC clusters are fast becoming the optimal choice for many small- and mid-sized engineering-based businesses as a way to accelerate complex simulations and rendering tasks.

Part of what makes this generation of HPC clusters more accessible is a new breed of intelligent cluster management software. Historically, clusters have been deployed, configured, and maintained using a variety of open-source software tools that were cobbled together to get a cluster up and running. While this approach required at least one, if not many, IT experts versed in the various tools and languages, new integrated clus-ter and workload management solutions are making the systems easier to deploy and manage. These new all-in-one solutions provide everything from comprehensive workload and resource management capabili-ties to monitoring features and parallel file system functionality. All of this advanced functionality is dressed in a common Web interface so mainstream engineers can schedule simulation jobs, provision HPC resources, and tend to other administrative tasks on their own without having to rely on IT or HPC specialists.

The combination of more affordable and simplified management and deployment means that small- and mid-sized companies previously shut out from HPC horsepower can now tap an HPC cluster architecture to drive their own realistic simulation efforts. Historically, HPC has been a strategic part of the engineering and design tool arsenal for only the largest engineering organizations, academic institutions, and government entities that have the deep pockets and on-staff expertise necessary to support the technology.

Along with changes on the HPC hardware landscape, simulation software has also evolved over the last few years. In one of the more important shifts, the leading simulation vendors have optimized their simulation tools to exploit the 64-bit and parallel processing capabilities of the multicore HPC cluster architecture. This affords engineering teams the ability to perform much larger and more highly detailed and realistic simulations than were previously possible in an individual workstation environment. Moreover, the parallelized simulation software also delivers a significant performance boost for many types of realistic simulations, from fine-grained structural analysis to fluid dynamics studies and everything in between.

IBM’s Web interface allows non-specialists to manage cluster tasks.

The Future is Now 3

Enter HPC clusters built on a modular, multi-core x86 architecture, which increases the amount of computing power a system can deliver while enabling organizations to invest in far less hardware than in the past. (See “What is an HPC Cluster” on page 2.) This latest generation of highly scalable HPC clusters support parallel processing, creating a cost effective and highly accessible platform on which to conduct realistic simulation compared with the big iron HPC systems of the past.

ABAQUS BECOMES CLUSTER-FRIENDLY

Abaqus, part of Dassault Systèmes’ SIMULIA realistic simulation portfolio, is one of the leading simulation platforms that’s been optimized for an HPC cluster architecture. Abaqus delivers structural and multiphysics simulation capabilities based on the finite

element method. Used by leading aerospace, automotive, biomedical, and related engineering and research organizations, Abaqus includes capabilities that tackle pre-and post-processing along with linear and nonlinear stress, noise and vibration, computational fluid dynamics (CFD), heat transfer, crack initiation, fracture and failure, electromagnetics, and a host of other analyses related to realistic simulation. The family of applications includes Abaqus/CAE, Abaqus/Standard, Abaqus/Explicit, and Abaqus/CFD.

Companies across a wide range of industries are employing Abaqus as part of their realistic simulation efforts. Manufacturers of consumer electronics gear, for example, are leveraging the simulation software to do drop testing, while au-tomotive OEMs put the Abaqus suite through its paces to explore crashworthiness or ballistic impact. Abaqus is also used to accurately analyze nonlinear behavior such as contact, making it a viable option for simulating many quasi-static events such as the rolling of hot metal or the slow crushing of energy absorbing devices. The software’s CFD capabilities, coupled with its ex-tensive support for pre- and post-processing, are

primed to address a broad range of multiphysics problems, including fluid-thermal and fluid-structural scenarios.

SIMULIA embarked on a three-stage effort to optimize Abaqus for parallel processing, be-ginning in the mid-1990s. Because code is tightly coupled in the world of finite element analysis

In this example, Abaqus is used to simulate the risk of neck injury during a rear-end collision. This helps automotive OEMs and seat suppliers to accelerate the design of head restraints systems that minimize whiplash injuries.

The TCO (total cost of ownership) on cluster management software for a 32-node cluster is about 35% more for a self-assembled solu-tion compared with an integrated tool like Platform HPC.

— Source: IBM

“”

The Future is Now 4

HPC-Driven Simulation Steers Dana on the Road to Virtual Engineering

A s Dana Holding Corp. builds out its vision of a virtual engineering environment, the vehicular supplier credits the marriage of high performance computing (HPC) clusters coupled with simulation software optimized for that environment as a key enabling technology steering it closer toward its goal.

Dana, which manufacturers an array of automotive-related products, including axles, drive shafts, trans-missions, sealing, and thermal-management products, leans on HPC-driven simulation in a multitude of ways, according to Frank Popielas, Dana’s senior manager of global CAE for its Power Technologies Group and the lead for Dana’s CAE efforts.

The company, which operates an HPC environment built on IBM System x® clusters, employs SIMULIA Abaqus, from Dassault Systèmes as its primary structural analysis and multiphysics simulation software. The combination of technologies lets Dana simulate its sealing products, driveline components, and heat exchangers to explore a variety of system behaviors, including stress and strength limitations, different sets of boundary conditions, sealing pressure, torque, and noise vibration and harshness conditions, among other factors.

Abaqus, which is optimized to exploit the parallel processing capabilities of HPC clusters, gives Dana an edge in its simulation efforts, allowing it to do more sophisticated simulations, explore larger assemblies, and drastically improve simulation turnaround time, Popielas says.

More importantly, HPC-driven simulation is moving Dana closer to its vision of a complete virtual engi-neering environment where it can complete most testing in a digital world, reducing the amount it spends on physical testing and bringing cost efficiencies to its overall engineering efforts. “Because we can simulate large assemblies with more details, it helps us avoid more and more physical testing,” Popielas explains. “We still use physical testing for validation at the end of the design cycle, but this process lets us eliminate more of the in-between testing.”

As a pioneer in HPC-driven simulation, Popielas agrees that new thinking was needed that treats soft-ware and hardware as a system and that “we as the end user had to play our part in connecting both and testing it out,” he says. “This is where Dana devotes a significant amount of time and resources to the ini-tiative. Sure, the cost of the HPC hardware, and more significantly the simulation software, played a major role during the early days before the benefits of this new technology took hold—as did having the dedicated manpower to learn not just the technology, but how to apply it effectively as part of the early design pro-cess. But in the end the benefits justified the cost and efforts.”

Having key partners like IBM and Dassault Systèmes SIMULIA was critical to Dana making the transi-tion to HPC-driven simulation. “In the past, whoever on the CAE vendor side had the best package from a cost perspective would win, but it’s different today,” Popielas explains. “You need a strategic partnership because you’re talking about a complete infrastructure and a system solution, not just a point solution. You have to select partners that are capable of looking into the future and that have a strategy in place to get you there.”

(FEA), work was required to rebalance Abaqus workloads and logically separate tasks to make the most efficient use of multiple cores and processors. The software’s data management capabilities were also recalibrated to ensure top performance in an HPC cluster environment.

In the first stage, Abaqus was rearchitected to support group parallelism, where a small number of critical routines were optimized to achieve modest performance gains. Support for symmetric multiprocessing (SMP) parallel-ism was the next step, allowing Abaqus simulations to run in parallel within a single machine. The optimization effort culminated with full support for distributed memory, an approach that allows Abaqus to leverage the multiple cores and processors specific to an x86-based HPC cluster environment to achieve significant performance improvements while boosting the productiv-

ity of engineering users. Currently, Abaqus’ high-performance solvers can utilize up to 128 cores for rapid turnaround, and SIMULIA is continuously refining its optimization efforts to increase the number of cores supported.

Parallelized simulation software delivers a significant performance boost for many types of realistic simulations.

The Future is Now 5

Just how dramatic are the performance improvements when running the optimized Abaqus on an HPC cluster? Based on results from benchmarks conducted by SIMULIA, the gains are pretty substan-tial. For example, one analysis took 19.4 hours to complete using a customer’s own workstation with

six cores. In contrast, that same analysis run in-house on optimized cluster was reduced to 2.2 hours with a 64-core configuration.

In addition to better raw performance, the HPC cluster environment provides a cost-effective platform for deploying Abaqus. In that same benchmark scenario, the Abaqus analysis tied up six cores and 10 license to-kens for 19.4 hours when run on the cus-tomer’s hardware, resulting in a cost of about $55.32. Transitioning the analysis job to Abaqus on a 64-core HPC cluster tied up 64 cores and 28 token licenses, but only for 2.2 hours, resulting in a reduced cost of opera-

tion at $21.54.Similarly in another example, a customer ran an Abaqus simulation on 10 cores, consum-

ing 18.83 hours of operation and tying up 13 tokens for an effective hardware/software cost of $71.39. That same analysis run on a 64-core HPC cluster at SIMULIA completed in only 3.39 hours on 64 cores for an effective cost of $33.13.

HPC CLUSTERS HEAD TO THE MAINSTREAM

Such measureable performance gains are sparking interest in HPC clusters among main-stream engineering shops, which have historically been shut out of the HPC equation for a variety of reasons.

For one thing, HPC clusters have been notoriously difficult to implement, requiring spe-cially trained IT personnel to set up, manage, and configure jobs. In a typical HPC cluster scenario, a variety of open source tools are employed—one program might handle provision-ing of the cluster nodes while another package deals with network configurations and yet another tool takes on scheduling and load balancing. Each package is equipped with a different

user interface and corresponding learning curve, which means someone in IT has to be versed in the entire portfolio of tools or a firm must have several HPC specialists on staff, which is cost prohibitive for most small- and mid-sized firms.

To add to the challenge, clusters have become increasingly com-plex as the number of nodes and cores grows and the mix of compo-nents on which to distribute jobs expands to include both CPUs and graphics accelerators. Finally, the lack of software optimized for the HPC cluster architecture has given smaller engineering organizations little reason to invest in the technology let alone rethink their design workflows to take advantage of HPC horsepower.

Yet advances in HPC configurations and cluster management software are addressing these barriers to entry, making it far easier for smaller companies to get on board. IBM Technical Computing is now delivering high performance systems integrated with work-load and resource management tools that put HPC capabilities well within reach of small- and mid-size firms from both a price/perfor-mance and ease of manageability perspective.

For example, IBM offers a number of cluster options depending on customer need. Users can create their own HPC cluster using IBM

IBM Application Ready Solution for Abaqus provides customers with a reference architec-ture and high-performance cluster solution that is easy to deploy and manage. With optimized clusters, hidden costs are reduced, helping to accelerate faster time to results and provide our clients with a competitive edge.

— Matt Dunbar, SIMULIA R&D Software Architecture Director, Dassault Systèmes

“”

IBM Platform™ HPC is all-in-one cluster and workload management software that features a centralized Web interface.

The Future is Now 6

NeXtScale System™, a modular, high performance, x86-based system that can be easily scaled based on need, or opt for IBM FleX System™ — high performance blades that offer a higher degree of integration be-tween servers, storage, and networking functionality. Either offering can be purchased as an IBM Intelligent Cluster™ configuration, which means it is factory integrated and fully-tested, further simplifying the configura-tion and deployment burden for customers or partners.

Accompanying the high-performance systems is IBM’s integrated cluster management software, technology gleaned from its 2012 acquisi-tion of Platform Computing. IBM Platform HPC is all-in-one cluster and workload management software that replaces the smorgasbord of open source tools. It features a centralized Web interface that makes it easy for non-IT experts to set up, manage, and provision a cluster. For ex-ample, through a Web-style common interface, engineers could prioritize workloads based on policies to optimize cluster performance; provision and monitor the hardware; and submit jobs without having to involve IT or HPC experts.

While an integrated solution like IBM Platform HPC initially costs more than open source management software, the total cost of owner-ship favors an integrated cluster management approach. According to IBM, the total cost of ownership (TCO) on cluster management soft-

ware for a 32-node cluster is about 35% more for a self-assembled solution compared with an inte-grated tool like Platform HPC. A TCO tool is available to let you determine your own savings: http://goo.gl/QQ4y05.

INTRODUCING IBM APPLICATION READY SOLUTION FOR ABAQUS

While easier-to-manage HPC clusters and cluster-friendly simulation software can have a huge impact on accelerating realistic simulation on their own, the value proposition is greatly enhanced when the two technologies are paired in a “black box” solution,

especially for resource-constrained small- and mid-sized shops.That is the primary objective of the IBM Application Ready Solution for Abaqus, a tightly integrated,

high-performance platform for realistic simulation co-developed by IBM and Dassault Systèmes’, com-panies which have enjoyed a strategic partnership for more than 30 years. The easy-to-use solution offers small, medium, and large configurations that the companies say have been optimized for the best performance at affordable prices. Based on expert configured and tested reference architectures, these solutions deliver requisite systems and software1 components for deploying, running, and managing a high-performance Abaqus computing environment. It allows engineering organizations to take full ad-vantage of realistic simulation as part of their design optimization process in a much more expedient and cost-effec-tive fashion.

The IBM Application Ready Solu-tion for Abaqus provides a high-perfor-mance integrated cluster based on IBM Flex System or IBM NeXtScale System, IBM Platform HPC management soft-ware, storage capabilities, solution-level support from IBM Technical Comput-

1. Application software sold separately

Abaqus provides cut-through visualization for meshes and geometry, which allows the interior of models — such as this turbomachinery assembly — to be viewed, making it easier to position components and assign analysis attributes.

You have to select partners that are capable of looking into the future and that have a strategy in place to get you there.

— Frank Popielas, Senior Manager of Global CAE, Power Technologies Group,

Dana Holding Corp.

“”

The Future is Now 7

ing industry experts, optimized to accelerate Abaqus FEA and multiphysic realistic simulations.

A core part of the IBM Application Ready Solution for Abaqus is a reference architecture designed to take the guesswork out of properly configuring the environment to support the needs of an engineering organization. The reference architecture defines optimized solution configu-rations for both the cluster hardware and the software, including sizing considerations, guidelines, and sample use cases. There are also pre-integrated Abaqus applica-tion templates that can be deployed right out of the box to reduce setup time and simplify job submission; intelligent workload scheduling capabilities for accelerating applica-tion throughput and improving resource utilization; easy access to Abaqus job-related data and remote job manage-ment for shortening the deployment time; and advanced cluster file system capabilities to help reduce data-related bottlenecks and improve performance.

The reference architecture addresses the deployment of Abaqus in two primary technical computing architectures: A high-performance cluster or for remote 2D/3D visualization. More importantly, it is not a one-size-fits-all solution. Unlike other “black box” approaches, such as an HPC appliance, for ex-ample, the reference architecture is flexible, outlining a range of

recommended configurations that account for the differences in organizational size, budgets, and simulation workloads. IBM maintains a bill of materials for each of these validated configurations to further simplify the ordering and procurement process.

For example, for relatively small simulation workloads, defined as four large jobs with between 2 and 5 million degrees of freedom (MDOF) each, the Reference Architecture rec-ommends six compute nodes. Medium workloads, consisting of 10 large jobs with between 2 and 5 MDOF each, would run ideally on 10 compute nodes while a large workload of 15 jobs comprised of between 10 and 20 MDOF each would be best suited to run on 42 compute nodes. For customers with moderate-sized models using the standard solvers, GPUs may also be used. The reference architecture also details the suggested memory, disk storage, file system, and network configurations, among other accompanying components, for each level of workload. This further helps organizations quickly and easily zero in on the optimal HPC cluster configuration tuned to meet their specific Abaqus simulation requirements.

ENTER THE NEW AGE OF INNOVATIVE DESIGN

G iven the constant pressure to innovate against a backdrop of shrinking time-to-delivery cy-cles, the ability to perform advanced and realistic simulation far earlier in the design process is a mandate for companies, regardless of industry or size. Now that HPC cluster resources

are far more accessible from both a cost and manageability standpoint, and because realistic simula-tion software has been optimized to fully capitalize on the HPC cluster architecture, there is nothing standing in the way of any company tapping into HPC horsepower to fuel more realistic simulation.

The IBM Application Ready Solution for Abaqus further closes the gap, reducing any remain-ing complexities surrounding HPC procurement and deployment, while delivering shorter time to results with realistic simulation at a lower TCO. The future is now for companies of all shapes and sizes to get onboard with realistic simulation and give their products a competitive edge.

To learn more about how your organization can gain a competitive advantage by evaluating more design options, reducing physical testing and bringing products to market faster, watch IBM’s Application Ready Solution for Abaqus webcast: http://goo.gl/FDe7Ny.

In this simulation of a pressure vessel, the Extended Finite Element Analysis (XFEM) capability in Abaqus is used to predict crack growth along arbitrary paths that do not correspond to element boundaries.

The Future is Now 8

Appendix

ADDITIONAL RESOURCES

➜ Dassault Systèmes’ SIMULIA Abaqus: 3ds.com/products-services/simulia/portfolio/abaqus/overview

➜ IBM Technical Computing: ibm.com/technicalcomputing

➜ Accelerating Realistic Simulation – Application Ready Solutions brief: public.dhe.ibm.com/common/ssi/ecm/en/swd14004usen/SWD14004USEN.PDF

➜ Watch IBM’s Application Ready Solution for Abaqus webcast: engage.vevent.com/index.jsp?eid=556&seid=64724&code=DEWP

➜ Download IBM’s Application Ready Solution for Abaqus reference architecture: www14.software.ibm.com/webapp/iwm/web/signup.do?source=stg-web&S_PKG=ov20293

This white paper is sponsored by Dassault Systèmes SIMULIA and IBM Technical Computing. It has been produced by Desktop Engineering as an educational resource for the engineering community.

Copyright Dassault Systèmes 2014, all rights reserved. 3DEXPERIENCE, CATIA, SOLIDWORKS, ENOVIA, SIMULIA, DELMIA, 3DVIA, 3DSwYm, EXALEAD, and Netvibes are registered trademarks of Dassault Systèmes or its subsidiaries.