Testing & Validation HIGH PERFORMANCE COMPUTING FRAMEWORK FOR - - PowerPoint PPT Presentation

▶
testing amp validation
SMART_READER_LITE
LIVE PREVIEW

Testing & Validation HIGH PERFORMANCE COMPUTING FRAMEWORK FOR - - PowerPoint PPT Presentation

Jun 09, 2023 421 likes •680 views

Modeling & Simulation, Testing & Validation HIGH PERFORMANCE COMPUTING FRAMEWORK FOR CO-SIMULATION OF VEHICLE TERRAIN INTERACTION Radu Serban, Nicholas Olsen, Dan Negrut University of Wisconsin Madison 8/22/2018 Modeling &

slide-1
SLIDE 1

Modeling & Simulation, Testing & Validation

8/22/2018

HIGH PERFORMANCE COMPUTING FRAMEWORK FOR CO-SIMULATION OF VEHICLE – TERRAIN INTERACTION

Radu Serban, Nicholas Olsen, Dan Negrut University of Wisconsin – Madison

slide-2
SLIDE 2

Modeling & Simulation, Testing & Validation

8/22/2018

Acknowledgements

  • University of Wisconsin – Madison

– Antonio Recuero [Goodyear Tire & Rubber Co.] – Hammad Mazhar [NVIDIA] – Michael Taylor [UW and Harley-Davidson]

  • Outside collaborators

– Hiroyuki Sugiyama [University of Iowa] – Bryan Peterson [University of Iowa]

slide-3
SLIDE 3

Modeling & Simulation, Testing & Validation

8/22/2018

Off-Road Vehicle Mobility

slide-4
SLIDE 4

Modeling & Simulation, Testing & Validation

8/22/2018

Off-Road Vehicle Mobility: A Multi-Physics Problem

  • Multibody vehicle system

– Rigid body models of ground vehicles – Full vehicle subsystems (suspensions, steering, driveline, anti-roll bars, etc.) – Models for powertrain, driver (open/close loop)

  • Tire subsystem

– Rigid tire – Empirical tire models (Pacejka, Fiala) – Flexible FEA tire models

  • Deformable terrain system

– Semi-empirical soil model (Bekker-Wong) – Granular terrain (DEM) – Continuum soil models (FEA-based)

  • Fluid-Solid Interaction

– Lagrangian-Lagrangian approach

Chassis Wheels Tires Terrain Suspension Joints Contact

Internal Terrain Forces Internal Tire Forces

slide-5
SLIDE 5

Modeling & Simulation, Testing & Validation

8/22/2018

Project Chrono

  • Growing ecosystem of software tools
  • Multi-physics simulation engine
  • Open source, under permissive BSD-3 license
  • Provides support for simulation of

– Many-body dynamics – Nonlinear Finite Element Analysis – Fluid-Solid Interaction Problems

  • Middleware: can be embedded in third-party applications
  • Modular: based on optional linking of specialized modules
  • Expandable: via C++ inheritance
  • Efficient: fast and robust data structures and algorithms
  • Cross-platform: builds on Windows, Linux, OS X (MSVC, GCC, ICC, clang)

Chrono: An open source multi-physics dynamics engine, HPC in Sci. and Eng. – Lecture Notes in CS, Springer, 2016

slide-6
SLIDE 6

Modeling & Simulation, Testing & Validation

8/22/2018

Project Chrono - Organization

Hardware CPU, Multicore Hardware Multiple GPU Hardware Multiple Nodes

HPC Chrono API

MBD API FEA API FSI API … API

Support for Classical Multi-Body Dynamics Support for Structural And Volumetric Elements Support for Fluid-Solid Interaction Future Chrono Expansion

Advanced Chrono Use Low-Entry Point Chrono Use

Chrono Vehicle Chrono Granular Chrono Robotics Chrono FEA Chrono FSI Chrono … Chrono MBD

Chrono::Engine

Equation Formulation Equation Solution Collision Detection HPC Support Pre/Post Processing Chrono Distributed Chrono Parallel

MPI CUDA OpenMP

slide-7
SLIDE 7

Modeling & Simulation, Testing & Validation

8/22/2018

Bottleneck – Computational Times But what about everything in one simulation?

slide-8
SLIDE 8

Modeling & Simulation, Testing & Validation

8/22/2018

Why Co-Simulation? Flexibility and efficiency consideration:

  • Allow any combination of formulations for physics modeling

– no suitable integration scheme for NSC involving FEA

  • Use different integration schemes as called for by the particular dynamics problem:

– implicit, adaptive HHT scheme for FEA tires – a semi-implicit Euler scheme for the granular terrain

  • Allow each subsystem to advance its state using a suitable integration time step
  • Leverage different and independent parallelization techniques, as dictated by the

structure of each subsystem:

– multi-core (OpenMP) evaluation of FEA internal forces and Jacobians – multi-core (OpenMP), GPU (CUDA), or distributed (MPI) granular terrain simulation

slide-9
SLIDE 9

Modeling & Simulation, Testing & Validation

8/22/2018

Co-Simulation of Single Tire on Granular Terrain

High fidelity approach for vehicle mobility simulation: Nonlinear finite element tires operating on granular material,

  • J. Terramechanics, 2017
  • Lower tire inflation pressure leads to enhanced mobility capabilities
  • Curve corresponding to rigid-mesh tire acts as limit envelope
slide-10
SLIDE 10

Modeling & Simulation, Testing & Validation

8/22/2018

Co-Simulation of Full Vehicle on Granular Terrain

  • Vehicle mass: 2300 kg
  • Terrain: 932,000 particles

(penalty)

  • Tires: 90x24 ANCF multilayer shell

elements

  • Simulation: 7.6 s
  • Step size: 0.04 ms
  • Run time: 5.5 days
  • 40 core Intel Xeon CPU E5-2650 v3

@ 2.30GHz

slide-11
SLIDE 11

Modeling & Simulation, Testing & Validation

8/22/2018

Co-Simulation of Full Vehicle on Granular Terrain

A co-simulation framework for high-performance, high-fidelity simulation of ground vehicle—terrain interaction,

  • Intl. J. Vehicle Performance, 2018

Rear tires fall onto front tires initial footprints Initial drop causes large vertical contact forces Once rear tires run over settled terrain, they have larger net forces and lower resistance forces

slide-12
SLIDE 12

Modeling & Simulation, Testing & Validation

8/22/2018

Performance Bottleneck: Granular Terrain Simulation

Straight line maneuver over granular terrain Moving patch granular terrain option

slide-13
SLIDE 13

Modeling & Simulation, Testing & Validation

8/22/2018

Chrono::Distributed – Philosophy and Design

  • Philosophy and Assumptions

– Chrono::Distributed is intended first and foremost for providing high- resolution terrain for mobility studies – Restricts to the penalty-based frictional contact model – Assumes load balancing is not a concern – Minimally wrap Chrono::Parallel – Create nearly-disjoint Chrono::Parallel simulations on each MPI rank

  • Let Chrono::Parallel handle simulation as it normally would with a

synchronization step

– Critically: Allow each Chrono::Parallel sub-system to perform its usual algorithm for collision detection on only the bodies in its sub-domain

slide-14
SLIDE 14

Modeling & Simulation, Testing & Validation

8/22/2018

Chrono::Distributed – Domain Decomposition

  • At setup:

– Statically divide a predefined domain into subdomains for each MPI rank – Minimize inter-node communication with: – Non-blocking MPI – Point-to-point communications – No global collective operations

  • Three levels of parallelism

– SIMD vectorization from AVX – Multi-core shared-memory parallelism from OpenMP – Multi-node distributed-memory parallelism from MPI

slide-15
SLIDE 15

Modeling & Simulation, Testing & Validation

8/22/2018

Chrono::Distributed – Strong Scaling Results

Ranks Number Particles Ratio Wall-clock Time (s / s) Parallel Efficiency 1 1,236,372 1 23,490 – 2 1,236,372 1 11,827 0.993 4 1,236,372 1 5,954 0.986 8 1,236,372 1 2,773 1.059 16 1,236,372 1 1,440 1.020 32 1,236,372 1 712 1.031 Parallel Efficiency 𝐹𝑇 𝑜 =

𝑈(1) 𝑜𝑈(𝑜)

Cray XC30; dedicated Cray Aries network nodes: 2 x 12 core Intel Xeon CPU E5-2697 v2 @ 2.7 GHz

slide-16
SLIDE 16

Modeling & Simulation, Testing & Validation

8/22/2018

Chrono::Distributed – Weak Scaling Results

Ranks Number Particles Ratio Wall-clock Time (s / s) Parallel Efficiency 1 1,236,372 1 23,490 – 2 2,472,744 2 23,901 0.983 4 4,944,700 4 23,998 0.979 8 9,889,400 8 24,099 0.975 16 19,778,012 16 24,407 0.962 32 39,555,236 32 24,481 0.960 Parallel Efficiency 𝐹𝑋 𝑜 =

𝑈(1) 𝑈(𝑜)

Cray XC30; dedicated Cray Aries network 64 nodes; 2 x 12 core Intel Xeon CPU E5-2697 v2 @ 2.7 GHz

slide-17
SLIDE 17

Modeling & Simulation, Testing & Validation

8/22/2018

Chrono::Distributed – Wave Tank Demonstration

slide-18
SLIDE 18

Modeling & Simulation, Testing & Validation

8/22/2018

Distributed Co-Simulation Framework

slide-19
SLIDE 19

Modeling & Simulation, Testing & Validation

8/22/2018

Full Vehicle Simulations – Performance

Maneuver Acceleration Acceleration DLC DLC Tire model ANCF ANCF Rigid mesh Rigid mesh Domain size [m x m] 8 x 3 8 x 3 110 x 6 110 x 6 Particle radius [mm] 12.5 10.0 12.5 12.5 Number particles 283,162 591,090 6,513,518 6,513,518 Step size [ms] 0.04 0.04 0.04 0.04 Number MPI ranks 5 + 8 5 + 16 5 + 16 5 + 32 Average timing information [ms] Vehicle 0.93 0.93 0.92 0.92 Terrain 348.03 391.57 3953.67 1992.25 Tire (max overall) 3483.08 3488.10 1.46 1.41 Total 3547.53 3545.19 3966.21 2002.35

slide-20
SLIDE 20

Modeling & Simulation, Testing & Validation

8/22/2018

Double Lane Change Maneuver on Granular Terrain

  • Terrain: 6.5 million particles
  • Tires: rigid mesh
  • Simulation: 12 s
  • Step size: 0.05 ms
  • Run time: 66 hours on 5+64 nodes

Extrapolation to higher resolution:

  • Terrain: 22 million particles
  • Tires: rigid mesh
  • Simulation: 12 s
  • Step size: 0.05 ms
  • Run time: 223 hours on 5+64 nodes

Cray XC30; dedicated Cray Aries network nodes: 2 x 12 core Intel Xeon CPU E5-2697 v2 @ 2.7 GHz

slide-21
SLIDE 21

Modeling & Simulation, Testing & Validation

8/22/2018

Acceleration Maneuver on Granular Terrain

Extrapolation to higher resolution:

  • Terrain: 22 million particles
  • Tires: ANCF mesh
  • Simulation: 10 s
  • Step size: 0.04 ms
  • Tire: ~3.4 s / step
  • Terrain: ~3.4 s / step
  • Run time: 236 hours on 5+64 nodes

*Animation obtained with multi-core granular terrain simulation

Cray XC30; dedicated Cray Aries network nodes: 2 x 12 core Intel Xeon CPU E5-2697 v2 @ 2.7 GHz

slide-22
SLIDE 22

Modeling & Simulation, Testing & Validation

8/22/2018

Conclusions and Future Work

  • Conclusions

– Chrono::Distributed scales very well in both strong and weak scaling – Good scaling characteristics carry over to the co-simulation framework – Moved the bottleneck to the FEA component:

  • With enough computational resources, computational cost is that of a single

flexible tire

  • Future Work

– Chrono::Distributed additions & enhancements

  • Dynamic load balancing
  • Higher-dimensional domain-decomposition
  • Complementarity-based contact

– Ongoing parallel effort to improve performance of Chrono::FEA

slide-23
SLIDE 23

Modeling & Simulation, Testing & Validation

8/22/2018

nicholas.olsen@wisc.edu Simulation Based Engineering Lab University of Wisconsin-Madison

Thank You

Project website http://projectchrono.org Chrono animations http://sbel.wisc.edu/Animations http://projectchrono.org/gallery