index.json

[{"authors":null,"categories":null,"content":"I am a Metropolis Postdoctoral Fellow at Los Alamos National Laboratory studying astrophysical plasmas using exascale GPU-based resources. My work at Los Alamos is currently focused on developing AthenaPK, an exascale magnetohydrodynamics code, and Parthenon, the adaptive mesh refinement framework that powers AthenaPK. With AthenaPK, I run simulations of astrophysical jets in various scenarios on Frontier, the world\u0026rsquo;s first exascale supercomputer.\nMy computational intrests focus on enabling scientific computing on GPUs via the development of performance-portable codes: codes that are capable of high performance on various GPU and CPU architectures.\nFeel free to contact me for collaboration via email.\nDownload my resume.\n","date":1618272000,"expirydate":-62135596800,"kind":"term","lang":"en","lastmod":1618272000,"objectID":"2525497d367e79493fd32b198b28f040","permalink":"","publishdate":"0001-01-01T00:00:00Z","relpermalink":"","section":"authors","summary":"I am a Metropolis Postdoctoral Fellow at Los Alamos National Laboratory studying astrophysical plasmas using exascale GPU-based resources. My work at Los Alamos is currently focused on developing AthenaPK, an exascale magnetohydrodynamics code, and Parthenon, the adaptive mesh refinement framework that powers AthenaPK.","tags":null,"title":"Forrest Glines","type":"authors"},{"authors":[],"categories":null,"content":" Click on the Slides button above to view the built-in slides feature. Slides can be added in a few ways:\nCreate slides using Wowchemy\u0026rsquo;s Slides feature and link using slides parameter in the front matter of the talk file Upload an existing slide deck to static/ and link using url_slides parameter in the front matter of the talk file Embed your slides (e.g. Google Slides) or presentation video on this page using shortcodes. Further event details, including page elements such as image galleries, can be added to the body of this page.\n","date":1906549200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1906549200,"objectID":"a8edef490afe42206247b6ac05657af0","permalink":"https://forrestglines.github.io/talk/example-talk/","publishdate":"2017-01-01T00:00:00Z","relpermalink":"/talk/example-talk/","section":"event","summary":"An example talk using Wowchemy's Markdown slides feature.","tags":[],"title":"Example Talk","type":"event"},{"authors":null,"categories":null,"content":"Myself and a group of collaborators were awarded an INCITE 2023 Award to perform exascale simulations of galaxy clusters with magnetized jets powered by a central active galactic nuclei (AGN). Galaxy clusters, as the largest gravitationally bound structures, provide a unique probe of large scale structure in the universe. The magnetized AGN jets play a key role in the dynamics of baryonic matter in galaxy clusters and thus observational signatures of clusters.\nThese simulations are enabled by AthenaPK, our exascale capable performance portable astrophysics code. I am currently working on optimizing these AthenaPK simulations on the exascale supercomputer Frontier.\nRead MSU\u0026rsquo;s reporting on the award here.\n","date":1689379200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1689379200,"objectID":"4442c21f90b669acdf8e996533aabb2e","permalink":"https://forrestglines.github.io/project/incite_2023/","publishdate":"2023-07-15T00:00:00Z","relpermalink":"/project/incite_2023/","section":"project","summary":"INCITE 2023 Award to run exascale simulations of magnetized AGN jets in galaxy clusters","tags":["Exascale"],"title":"Exascale simulations of magnetized AGN jets on Frontier","type":"project"},{"authors":null,"categories":null,"content":"AthenaPK is an in-development performance-portable conversion of Athena++ build on the Parthenon adaptive mesh refinement (AMR) framework using the Kokkos performance portability library. I am one of the main developers for AthenaPK and a co-developer for Parthenon. The Parthenon framework is designed to be massively scalable and efficient on both CPUs and GPUs, enabling next-generation AMR simulations on a variety of hardware architectures. Kernels and data are managed by Kokkos, which enables high performance on any architecture supported by Kokkos, including CPUs, NVIDIA and AMD GPUs, and future Intel GPUs. AthenaPK uses the robust solvers from Athena++ within the Parthenon framework to enable future exascale astrophysical simulations.\nParthenon is publicly available on github.\nAthenaPK is publicly available on gitlab.\n","date":1623715200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1623715200,"objectID":"03113d3bd847c3181dcebd06f114ede6","permalink":"https://forrestglines.github.io/project/athenapk_and_parthenon/","publishdate":"2021-06-15T00:00:00Z","relpermalink":"/project/athenapk_and_parthenon/","section":"project","summary":"Fully Fledged Performance-Portable Astrophysics Simulations with Adaptive Mesh Refinement","tags":["Code Development","Performance Portability","AthenaPK"],"title":"AthenaPK and Parthenon","type":"project"},{"authors":null,"categories":null,"content":"K-Athena is a partial conversion of Athena++, using Kokkos for performance portability, meaning that it runs efficiently on CPUs and GPUs. The code is a precursor to the Parthenon and AthenaPK projects, implementing only uniform grids efficiently when running on GPUs. However, the code was a valuable proof of concept for a performance-portable magnetohydrodynamics code, allowing future exascale simulations to be unconstrained by niche architectures.\nK-Athena is publicly available on gitlab.\nAs part of the development effort, we quantified the performance portability of code using roofline models. We constructed roofline models on each of the CPU and GPU devices on which we tested K-Athena. Roofline models allow estimations of the theoretical peak throughput of a code as limited by its arithmetic intensity (the number of operations execute per byte loaded) and by the bandwidths and computational throughputs of the hardware. By comparing the actual efficiency achieved to the theoretical efficiency for each architecture, we obtain a performance efficiency for each machine that can be directly compared, even if the architectures are very different.\nRoofline model of an NVIDIA Tesla V100 with the arithmetic intensity of K-Athena, showing performance in TFLOPS versus arithmetic intensity in floating point operations execute per byte loaded and written. Throughputs appear as horizontal ceilings, bandwidths of the different memory spaces of the hardware appear as diagonal ceilings, and arithmetic intensities of the code appear as vertical lines. The intersect of an arithmetic intensity with a bandwidth or throughput ceiling show the theoretical throughput ceiling imposed by that bandwidth or throughput. We generated rooflines for all architectures on which we profiled K-Athena.\nEfficiency achieved on each architecture on which we profiled K-Athena, showing the percentage performance achieved out of the theoretical performance as limited by the DRAM and L1 memory for each architecture. By taking the harmonic mean of these efficiencies we arrive at a performance portability measure. The implementation of K-Athena (and similar MHD codes) is typically limited by the DRAM bandwidth, leading to a performance portability of 62.8%. Less efficiency utilization of the L1 cache on almost all architectures leads to a 7.7% performance portability with respect to the L1 cache.\nOur full method description and performance analysis can be found in IEEE Transactions on Parallel and Distributed Systems.\n","date":1623715200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1623715200,"objectID":"78201ad733022c32fe805fccd0854ae4","permalink":"https://forrestglines.github.io/project/kathena/","publishdate":"2021-06-15T00:00:00Z","relpermalink":"/project/kathena/","section":"project","summary":"Proof-of-Concept Performance-Portable Uniform Grid Astrophysics Code","tags":["Code Development","Performance Portability","K-Athena"],"title":"K-Athena","type":"project"},{"authors":null,"categories":null,"content":"To investigate the development of turbulence in intermittently driven plasmas such as the intracluster medium, we used our newly developed K-Athena code to run a series of simulations of the magnetized Taylor-Green vortex. The turbulence that arises from the unsteady flow of the magnetized Taylor-Green vortex models the turbulence that develops from large scale infrequent events such as galaxy cluster mergers disturbing the intracluster medium. In our paper, we examine the magnitude and the spectra of the kinetic, magnetic, and thermal energy reservoirs within the plasma. Additionally, we apply an energy transfer analysis to study the movement of energy between different length scale and between different reservoirs\nOur full setup, explanation, and analysis can be found in Physical Review E.\n","date":1623715200,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1623715200,"objectID":"51fe917e765dd95437e9149db141a5a3","permalink":"https://forrestglines.github.io/project/taylor_green/","publishdate":"2021-06-15T00:00:00Z","relpermalink":"/project/taylor_green/","section":"project","summary":"Investigating the development of MHD turblence in the absence of driving forces","tags":["Astrophysics","MHD Turbulence","K-Athena"],"title":"Magnetized Turbulence in the Taylor-Green Vortex","type":"project"},{"authors":["Forrest Glines","Philipp Grete","Brian W. O'Shea"],"categories":null,"content":"Magnetohydrodynamic (MHD) turbulence affects both terrestrial and astrophysical plasmas. The properties of magnetized turbulence must be better understood to more accurately characterize these systems. This work presents ideal MHD simulations of the compressible Taylor-Green vortex under a range of initial subsonic Mach numbers and magnetic field strengths. We find that regardless of the initial field strength, the magnetic energy becomes dominant over the kinetic energy on all scales after at most several dynamical times. The spectral indices of the kinetic and magnetic energy spectra become shallower than $k^{−5/3}$ over time and generally fluctuate. Using a shell-to-shell energy transfer analysis framework, we find that the magnetic fields facilitate a significant amount of the energy flux and that the kinetic energy cascade is suppressed. Moreover, we observe nonlocal energy transfer from the large-scale kinetic energy to intermediate and small-scale magnetic energy via magnetic tension. We conclude that even in intermittently or singularly driven weakly magnetized systems, the dynamical effects of magnetic fields cannot be neglected.\n","date":1618272000,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1618272000,"objectID":"0fdd3a4c3450ccf3caeb05f091cd712a","permalink":"https://forrestglines.github.io/publication/taylor_green/","publishdate":"2020-01-01T00:00:00Z","relpermalink":"/publication/taylor_green/","section":"publication","summary":"Magnetohydrodynamic (MHD) turbulence affects both terrestrial and astrophysical plasmas. The properties of magnetized turbulence must be better understood to more accurately characterize these systems. This work presents ideal MHD simulations of the compressible Taylor-Green vortex under a range of initial subsonic Mach numbers and magnetic field strengths.","tags":[],"title":"Magnetized decaying turbulence in the weakly compressible Taylor-Green vortex ","type":"publication"},{"authors":["Deovrat Prasad","G. Mark Voit","Brian W. O'Shea","Forrest Glines"],"categories":null,"content":"In the universe\u0026rsquo;s most massive galaxies, active galactic nucleus (AGN) feedback appears to limit star formation. The accumulation of cold gas near the central black hole fuels powerful AGN outbursts, keeping the ambient medium in a state marginally unstable to condensation and formation of cold gas clouds. However, the ability of that mechanism to self-regulate may depend on numerous environmental factors, including the depth of the potential well and the pressure of the surrounding circumgalactic medium (CGM). Here we present a suite of numerical simulations, with halo mass ranging from $2\\times10^{12} M_\\odot$ to $8\\times10^{14} M_\\odot$, exploring the dependence of AGN feedback on those environmental factors. We include the spatially extended mass and energy input from the massive galaxy\u0026rsquo;s old stellar population capable of sweeping gas out of the galaxy if the confining CGM pressure is sufficiently low. Our simulations show that this feedback mechanism is tightly self-regulating in a massive galaxy with a deep central potential and low CGM pressure, permitting only small amounts of multiphase gas to accumulate and allowing no star formation. In a similar-mass galaxy with shallower central potential and greater CGM pressure the feedback mechanism is more episodic, producing extended multiphase gas and allowing small rates of star formation ($\\sim0.1 M_\\odot \\text{yr}^{−1}$). At the low-mass end, the mechanism becomes implausibly explosive, perhaps because the CGM initially has no angular momentum, which would have reduced the amount of condensed gas capable of fueling feedback.\n","date":1606780800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1606780800,"objectID":"c5e7b7c54c8712294b180674c34cb047","permalink":"https://forrestglines.github.io/publication/self_regulating_bh_feedback/","publishdate":"2020-09-01T00:00:00Z","relpermalink":"/publication/self_regulating_bh_feedback/","section":"publication","summary":"In the universe\u0026rsquo;s most massive galaxies, active galactic nucleus (AGN) feedback appears to limit star formation. The accumulation of cold gas near the central black hole fuels powerful AGN outbursts, keeping the ambient medium in a state marginally unstable to condensation and formation of cold gas clouds.","tags":[],"title":"Environmental Dependence of Self-regulating Black Hole Feedback in Massive Galaxies","type":"publication"},{"authors":["Forrest Glines","Brian W. O'Shea","G. Mark Voit"],"categories":null,"content":"In cool-core galaxy clusters with central cooling times much shorter than a Hubble time, condensation of the ambient central gas is regulated by a heating mechanism, probably an active galactic nucleus. Previous analytical work has suggested that certain radial distributions of heat input may result in convergence to a quasi-steady global state that does not substantively change on the timescale for radiative cooling, even if the heating and cooling are not locally in balance. To test this hypothesis, we simulate idealized galaxy cluster halos using the ENZO code with an idealized, spherically symmetric heat input kernel intended to emulate. Thermal energy is distributed with radius according to a range of kernels, in which total heating is updated to match total cooling every 10 Myr. Some heating kernels can maintain quasi-steady global configurations, but no kernel we tested produces a quasi-steady state with central entropy as low as those observed in cool-core clusters. The general behavior of the simulations depends on the proportion of heating in the inner 10 kpc, with low central heating leading to central cooling catastrophes, high central heating creating a central convective zone with an inverted entropy gradient, and intermediate central heating resulting in a flat central entropy profile that exceeds observations. The timescale on which our simulated halos fall into an unsteady multiphase state is proportional to the square of the cooling time of the lowest-entropy gas, allowing more centrally concentrated heating to maintain a longer-lasting steady state.\n","date":1598918400,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1598918400,"objectID":"00b2adc01b14e3977c600d973d086363","permalink":"https://forrestglines.github.io/publication/agn_thermal/","publishdate":"2020-09-01T00:00:00Z","relpermalink":"/publication/agn_thermal/","section":"publication","summary":"In cool-core galaxy clusters with central cooling times much shorter than a Hubble time, condensation of the ambient central gas is regulated by a heating mechanism, probably an active galactic nucleus.","tags":[],"title":"Tests of AGN Feedback Kernels in Simulated Galaxy Clusters ","type":"publication"},{"authors":["Philipp Grete","Forrest Glines","Brian W. O'Shea"],"categories":null,"content":"Large scale simulations are a key pillar of modern research and require ever-increasing computational resources. Different novel manycore architectures have emerged in recent years on the way towards the exascale era. Performance portability is required to prevent repeated non-trivial refactoring of a code for different architectures. We combine ATHENA++, an existing magnetohydrodynamics (MHD) CPU code, with KOKKOS, a performance portable on-node parallel programming paradigm, into K-ATHENA to allow efficient simulations on multiple architectures using a single codebase. We present profiling and scaling results for different platforms including Intel Skylake CPUs, Intel Xeon Phis, and NVIDIA GPUs. K-ATHENA achieves $\u0026gt; 10^8$ cell-updates/s on a single V100 GPU for second-order double precision MHD calculations, and a speedup of 30 on up to 24,576 GPUs on Summit (compared to 172,032 CPU cores), reaching $1:94\\times10^12$ total cell-updates/s at 76 percent parallel efficiency. Using a roofline analysis we demonstrate that the overall performance is currently limited by DRAM bandwidth and calculate a performance portability metric of 62.8 percent. Finally, we present the implementation strategies used and the challenges encountered in maximizing performance. This will provide other research groups with a straightforward approach to prepare their own codes for the exascale era. K-ATHENA is available at https://gitlab.com/pgrete/kathena.\n","date":1594944000,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1594944000,"objectID":"fd0ebe8c6cf30d0608d8cacfbb5adf86","permalink":"https://forrestglines.github.io/publication/kathena/","publishdate":"2020-07-17T00:00:00Z","relpermalink":"/publication/kathena/","section":"publication","summary":"Large scale simulations are a key pillar of modern research and require ever-increasing computational resources. Different novel manycore architectures have emerged in recent years on the way towards the exascale era.","tags":[],"title":" K-Athena: a performance portable structured grid finite volume magnetohydrodynamics code ","type":"publication"},{"authors":[],"categories":[],"content":"Create slides in Markdown with Wowchemy Wowchemy | Documentation\nFeatures Efficiently write slides in Markdown 3-in-1: Create, Present, and Publish your slides Supports speaker notes Mobile friendly slides Controls Next: Right Arrow or Space Previous: Left Arrow Start: Home Finish: End Overview: Esc Speaker notes: S Fullscreen: F Zoom: Alt + Click PDF Export: E Code Highlighting Inline code: variable\nCode block:\nporridge = \u0026quot;blueberry\u0026quot; if porridge == \u0026quot;blueberry\u0026quot;: print(\u0026quot;Eating...\u0026quot;) Math In-line math: $x + y = z$\nBlock math:\n$$ f\\left( x \\right) = ;\\frac{{2\\left( {x + 4} \\right)\\left( {x - 4} \\right)}}{{\\left( {x + 4} \\right)\\left( {x + 1} \\right)}} $$\nFragments Make content appear incrementally\n{{% fragment %}} One {{% /fragment %}} {{% fragment %}} **Two** {{% /fragment %}} {{% fragment %}} Three {{% /fragment %}} Press Space to play!\nOne **Two** Three A fragment can accept two optional parameters:\nclass: use a custom style (requires definition in custom CSS) weight: sets the order in which a fragment appears Speaker Notes Add speaker notes to your presentation\n{{% speaker_note %}} - Only the speaker can read these notes - Press `S` key to view {{% /speaker_note %}} Press the S key to view the speaker notes!\nOnly the speaker can read these notes Press S key to view Themes black: Black background, white text, blue links (default) white: White background, black text, blue links league: Gray background, white text, blue links beige: Beige background, dark text, brown links sky: Blue background, thin dark text, blue links night: Black background, thick white text, orange links serif: Cappuccino background, gray text, brown links simple: White background, black text, blue links solarized: Cream-colored background, dark green text, blue links Custom Slide Customize the slide style and background\n{{\u0026lt; slide background-image=\u0026quot;/media/boards.jpg\u0026quot; \u0026gt;}} {{\u0026lt; slide background-color=\u0026quot;#0000FF\u0026quot; \u0026gt;}} {{\u0026lt; slide class=\u0026quot;my-style\u0026quot; \u0026gt;}} Custom CSS Example Let\u0026rsquo;s make headers navy colored.\nCreate assets/css/reveal_custom.css with:\n.reveal section h1, .reveal section h2, .reveal section h3 { color: navy; } Questions? Ask\nDocumentation\n","date":1549324800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1549324800,"objectID":"0e6de1a61aa83269ff13324f3167c1a9","permalink":"https://forrestglines.github.io/slides/example/","publishdate":"2019-02-05T00:00:00Z","relpermalink":"/slides/example/","section":"slides","summary":"An introduction to using Wowchemy's Slides feature.","tags":[],"title":"Slides","type":"slides"},{"authors":["Forrest Glines","Matthew Anderson","David Neilsen"],"categories":null,"content":"A shift is underway in high performance computing (HPC) towards heterogeneous parallel architectures that emphasize medium and fine grain thread parallelism. Many scientific computing algorithms, including simple finite-differencing methods, have already been mapped to heterogeneous architectures with order-of-magnitude gains in performance as a result. Recent case studies examining high-resolution shock-capturing (HRSC) algorithms suggest that these finite-volume methods are good candidates for emerging heterogeneous architectures. HRSC methods form a key scientific kernel for compressible inviscid solvers that appear in astrophysics and engineering applications and tend to require enormous memory and computing resources. This work presents a case study of an HRSC method executed on a heterogeneous parallel architecture utilizing hundreds of GPU enabled nodes with remote direct memory access to the GPUs for a non-trivial shock application using the relativistic magnetohydrodynamics model.\n","date":1441065600,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1441065600,"objectID":"5c253b9a909c5813c1d2e9c8d01157d2","permalink":"https://forrestglines.github.io/publication/scalable_shock_capturing/","publishdate":"2015-09-01T00:00:00Z","relpermalink":"/publication/scalable_shock_capturing/","section":"publication","summary":"A shift is underway in high performance computing (HPC) towards heterogeneous parallel architectures that emphasize medium and fine grain thread parallelism. Many scientific computing algorithms, including simple finite-differencing methods, have already been mapped to heterogeneous architectures with order-of-magnitude gains in performance as a result.","tags":[],"title":"Scalable Relativistic High-Resolution Shock-Capturing for Heterogeneous Computing","type":"publication"},{"authors":null,"categories":null,"content":"","date":-62135596800,"expirydate":-62135596800,"kind":"page","lang":"en","lastmod":-62135596800,"objectID":"f26b5133c34eec1aa0a09390a36c2ade","permalink":"https://forrestglines.github.io/admin/config.yml","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/admin/config.yml","section":"","summary":"","tags":null,"title":"","type":"wowchemycms"}]