Projects

My interests are in high performance scientific computing, with particular emphasis on parallel computing. These include the development of scalable algorithms, programming systems to enable the efficient and correct expression of algorithms, and tools for writing, understanding, and tuning parallel applications. I am also involved in defining and providing the infrastructure for high performance computing.

I am looking for students to work on several projects. These projects are described here.

A common underlying issue is matching the algorithms and software to the computer architecture, particularly as architectures adapt, possibly with radical innovations, to the end of Dennard scaling (around 2004) and the end of Moore's Law (in rigourous terms, also already ended, in practice, slowly fading since it is really an engineering imperative rather than a natural law).

Programming systems
Effective ways to program parallel computers require (a) a standard approach, (b) a high-quality, ubiquitous implemenation of that standard, and (c) effective ways to get high performance on each core and node.
MPI Standard
The Message Passing Interface (MPI) standard is the dominant programming system for highly parallel HPC applications. I have been involved from the beginning of MPI process and currently serve as the overall standard editor and the co-chair of the RMA chapter. Note that MPI and collective algorithms developed for HPC have been adopted in the machine learning community.
MPICH implementation
The MPICH implementation of MPI is the most widely used implementation of MPI. I began this implementation while MPI was originally being defined, and it continues with the major work at Argonne National Laboratory. My current work looks at better implementation of process mapping and MPI datatypes.
Code transformations
Parallelism is only one approach to performance. Real HPC applications ofter require non-trivial code transformations to go beyond what a compiler is willing or able to perform.
Parallel I/O
I/O in parallel applications often performs spectacurally poorly. I have worked on measuring I/O performance and establishing performance expectations. Post POSIX I/O is a current focus, with Delta and DeltaAI providing good testbeds.
Performance Models and Tools
Quantifing performance and developing performance models and especially performance expectations. Recent work has included a new performance model that better matches the performance of internode communication than the classical postal model; this new model is called the max rate model. Software to measure communication performance include mpptest and nodecomm. FPMPI is a software library that uses the MPI profiling interface to gather details about the use of MPI in an application without requiring any source code changes. Baseenv is a collection of routines and programs for parallel programs. Included in baseenv is a comprehensive set of tests of compiler vectorization.

Some introductory papers on performance modeling.

Scalable Numerical Methods
  • PETSc is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations.
  • At extreme scale, synchronizing operations such as reductions (dot products) can limit scalability. I have investigated ways to use non-blocking collective operations in MPI to implement scalable Krylov Methods.
Infrastructure
  • Delta and DeltaAI are supercomputers funded by the National Science Foundation that provide computing for the nation's researchers.
  • The Deep Learning Major Research Instrument Project is an NSF-funded project, hosted at NCSA, to provide Illinois researchers with a powerful deep learning resource.
  • Illinois Computes, a major commitment from the University of Illinois to support research computing and data.
  • As NCSA Director, I guide the cyberinfrastructure that NCSA deploys. This includes both systems and software. On the system side, this includes HPC systems (Delta and DeltaAI), systems for AI (Delta, DeltaAI, HAL, HOLL-I), systems for regulated data, e.g., data subject to HIPAA, CUI, or FERPA (Nightingale), and an on-prem cloud system (Radiant). Also included is a tiered data storage system (Taiga and Granite).
Policy
I participate in community discussions about the directions for HPC. Some of the most important are:

Current Grants

Most of these grants are team projects in which my role runs from being the principle investigator to one of several co-investigators.
  • Delta and DeltaAI are NSF-funded supercomputers that provide GPU-rich cyberinfrastructure for AI/ML and simulations.
  • End-to-end Object-focused Software-defined Data Management for Science is a DOE project addressing the challenges of working with data in today's complex architectures.
  • Midwest Big Data Hub (MBDH) is one of the four regional data hubs funded by the NSF.
  • The Center for Exascale-enabled Scramjet Design is funded by the US Dept. of Energy in the Predictive Science Academic Alliance Program (PSAAP III).
  • Deep Learning Major Research Instrument is an instrument to support deep learning researchers across the University of Illinois Urbana-Champaign campus. This is a development MRI, in partnership with IBM and NVIDIA.
  • While not a grant in the conventional sense, I administer a Siebel Fellowship to a student at UIUC; I've used this for students pursuing topics in big data and bioinformatics.
Computer Science Department
University of Illinois Urbana-Champaign