My interests are in high performance scientific computing, with particular emphasis on parallel computing. These include the development of scalable algorithms, programming systems to enable the efficient and correct expression of algorithms, and tools for writing, understanding, and tuning parallel applications. I am also involved in defining and providing the infrastructure for high performance computing and for a large science project, the Large Synoptic Survey Telescope.

A common underlying issue is matching the algorithms and software to the computer architecture, particularly as architectures adapt, possibly with radical innovations, to the end of Dennard scaling (around 2004) and the end of Moore's Law (in rigourous terms, also already ended, in practice, slowly fading since it is really an engineering imperative rather than a natural law).

Programming systems
Effective ways to program parallel computers require (a) a standard approach, (b) a high-quality, ubiquitous implemenation of that standard, and (c) effective ways to get high performance on each core and node.
MPI Standard
The Message Passing Interface (MPI) standard is the dominant programming system for highly parallel HPC applications. I have been involved from the beginning of MPI process and currently serve as the overall standard editor and the co-chair of the RMA chapter.
MPICH implementation
The MPICH implementation of MPI is the most widely used implementation of MPI. I began this implementation while MPI was originally being defined, and it continues with the major work at Argonne National Laboratory. My recent contributions have included better implementations for datatypes, remote memory access, collective communications, and process topologies.
Code transformations
Parallelism is only one approach to performance. Real HPC applications ofter require non-trivial code transformations to go beyond what a compiler is willing or able to perform. Recent projects in this area include the Illinois Coding Environment (ICE) and a Just-In-Time (JIT) compilation approach for HPC applications called Moya.
Parallel I/O
I/O in parallel applications often performs spectacurally poorly. I have worked on measuring I/O performance and establishing performance expectations, contributing to the ROMIO implementation of MPI-IO, developing application-level libraries (such as MeshIO) for better performance, and exploring the possibilities for post POSIX I/O specifications in HPC.
Performance Models and Tools
Quantifing performance and developing performance models and especially performance expectations. Recent work has included a new performance model that better matches the performance of internode communication than the classical postal model; this new model is called the max rate model. Software to measure communication performance include mpptest and nodecomm. FPMPI is a software library that uses the MPI profiling interface to gather details about the use of MPI in an application without requiring any source code changes. Baseenv is a collection of routines and programs for parallel programs. Included in baseenv is a comprehensive set of tests of compiler vectorization. Some introductory papers on performance modeling
Scalable Numerical Methods
  • PETSc is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations.
  • At extreme scale, synchronizing operations such as reductions (dot products) can limit scalability. I have investigated ways to use non-blocking collective operations in MPI to implement scalable Krylov Methods.

Current Grants

Most of these grants are team projects in which my role runs from being the principle investigator to one of several co-investigators.
  • Center for the Exascale Simulation of Plasma Coupled Combustion (XPACC), a >$20M, 6 year project in the DOE Predictive Science program.
  • LSST; NCSA leads the data facility for the LSST project.
  • Blue Waters is NSF's most powerful general purpose supercomputer, and has sustained more than a petaFLOP on a wide variety of applications.
  • Midwest Big Data Hub (MBDH) is one of the four regional data hubs funded by the NSF
  • Deep Learning Major Research Instrument is an instrument to support deep learning researchers across the University of Illinois Urbana-Champaign campus. This is a development MRI, in partnership with IBM and NVIDIA
  • Scalable solvers for reservoir simulation, a joint project with Luke Olson and funded by ExxonMobil.
  • Lattice QCP Exascale Computing Project, one of the DOE Exascale Computing Projects.
  • Scalable and Highly Accurate Methods for Metagenomics, led at Illinois by Tandy Warnow
  • While not a grant in the conventional sense, I administer a Siebel Fellowship to a student at UIUC; I've used this for students pursuing topics in big data and bioinformatics.
In additional to grants of funds, I have several grants of supercomputer time. These grants have significant value in dollars but are rarely included in lists of grants because no funds are exchanged (and for the institutions, there is no overhead collected).
  • As a Blue Waters Professor, I have a grant of time on the Blue Waters system that I use for a variety of projects
  • As part of the PEAC-INCITE award, I have access to time on supercomputers operated by DOE's Office of Science.
Computer Science Department
University of Illinois Urbana-Champaign