IEEE Computer Society

Computing Research Association and Computing Community Consortium

Research Interests

My interest is in the use of high performance computing to solve problems that are too hard for other techniques. I have concentrated in two areas: the development of scalable numerical algorithms for partial differential equations (PDEs), especially for the linear systems that arise from approximations to PDEs, and the development of programming models and systems for expressing and implementing highly scalable applications. In each of these areas, I have led the development of software that has been widely adopted. PETSc is a powerful numerical library for the solution of linear and nonlinear systems of equations. MPI is the mostly widely used parallel programming system for large scale scientific computing. The MPICH implementation of MPI is one of the most widely used and is the implementation of choice for the world's fastest machines.

Current Major Research Projects

• I lead several projects that have deployed cyberintrastructure that supports research. The major projects are two GPU-rich supercomputers. Delta is funded by the National Science Foundation and is designed to accelerate the adoption of new computing technologies such as GPUs and non-POSIX file systems by the computational science community. This resource is available through the ACCESS program. DeltaAI was awarded by NSF in May 2023, and will provide an AI/ML optimized supercomputer that will be connected to Delta, allowing users to take advantage of the capabilities of both systems.

  A smaller system for AI/ML for Illinois researchers, deployed in 2016, is HAL, Deep Learning Major Research Instrument, which combines my interests in HPC software and numerics and high-performance I/O with the revolution in machine learning.

  A related project is Illinois Computes, which provides both systems and expert support for Illinois Researchers.

• The Center for Exascale-enabled Scramjet Design is funded by the US Dept. of Energy in the Predictive Science Academic Alliance Program (PSAAP III). The project has openings for graduate students with interests in numerical analysis, scientific computing, parallel program, performance and I/O, among others.

• The Midwest Big Data Hub (MBDH) is one of four NSF-funded regional big data an innovation hubs, which serve to bring communities together to apply data science to a wide range of challenges. I led the MBDH from 2017 through mid-2020, and currently am one of the co-principal investigators.

• The MPI Forum is the organization that continues to develop the Message-Passing Interface standard, which is the dominant programming system for massively parallel applications in science and engineering. I'm lead several of the chapter committees and am the overall editor. I also do work in MPI implementations and design.

• High performance I/O is another focus of my research, and I have several projects looking at everything from using and implementing collective I/O to using database ideas to better manage data from simulations. One feature that these have in common is that they do not require POSIX I/O semantics - specifically, the strong consistency semantics that contribute to complex implementations and that lead to performance and robustness problems. The Delta systems, mentioned above, will feature a fast, non-POSIX file system.

Research Opportunities

I have an active research program and currently have openings for graduate students, both Ph.D. and Masters students, and scientific programmers. A brief description of these is given below; more information can be found here.

Achieving high performance requires paying close attention to data movement and taking a quantitative approach to performance analysis. These projects address different challenges in achieving high performance in applications.

1. Process mapping for MPI programs. How MPI processes are assigned to cores, sockets, and nodes can impact the amount of data that moves between each level of that hierarchy - and thus the scalability and performance of an application. This project looks at ways to improve that mapping of MPI processes and thus provide better performance and scalability with little to no work by application programmer.
2. Data movement optimizations for I/O and data management. Modern systems have complex and hierarchical data storage systems and applications rarely make good use of these capabilities. This project is part of a larger effort to improve tools for managing data for science applications.
3. Programming tools for heterogeneous systems. While there has been much research in this area, there are few working systems. The first project is to evaluate the current technology by creating a version of the Top500 HPL benchmark for multi-GPU nodes. This will enable use of "evergreen" (frequently updated) of parallel hardware for challenging applications.
4. Code transformations for performance. The reality is that compilers often don't have enough information to produce the best performing code. This project uses code transformations to improve performance of applications, particularly for codes limited by memory performance.
5. MPI datatypes provide a language to describing data movement, particularly for non-continuous data. This project seeks to improve performance of applications that make use of MPI datatypes, especially for applications using GPUs and multicore CPUs.

Of Special Interest