Computing Systems Group (CSG) - Prof. Dr. Holger Fröning
(formerly Computer Engineering Group)
This site is outdated, please go here instead:
https://csg.ziti.uni-heidelberg.de
This site is outdated, please go here instead:
https://csg.ziti.uni-heidelberg.de
The Computing Systems Group (CSG) at the Institute of Computer Engineering at Ruprecht-Karls University of Heidelberg is focussing on improving the performance, energy-efficiency and programmability of accelerated computing systems. The group’s expertise covers parallel computer architecture, high-performance computing, machine learning and deep learning, high-performance analytics and reconfigurable logic.
Today, research in computing systems is most concerned with specialized forms of computing in combination with seamless integration into existing systems. Specialized computing, for instance based on GPUs (as known for gaming) or FPGAs (field programmable gate arrays) or ASICs (not the shoe brand but “application-specific integrated circuits”), is motivated by diminishing returns from CMOS technology scaling and hard power constraints. Notably, for a given fixed power budget p, energy efficiency e defines performance perf[ops/sec] = p[Watt] * e[ops/Joule]. Thus, a sustained performance scaling based on CMOS technology requires to improve the energy efficiency of compute and memory operations substantially, which is typically being done using the previously mentioned specialized forms of computing. However, any specialization stands in contrast to generality, thus raising various questions related to programmability and algorithmic innovation.
Particular research foci include
- resource-efficient ML such as model compression for edge, mobile and embedded systems,
- code analysis and generation as for instance based on CLANG/LLVM and targeting (multi-)GPU systems,
- HW/SW codesign to meet application objectives by a comprehensive treatment of software and hardware components, and
- specialized processor architectures under performance, energy efficiency and programmability constraints.
The group has gained international notice for designing the first communication model that is completely in-line with a thread-collaborative processor’s execution model (GGAS), yielding substantial savings for clustered GPUs in terms of energy and time. Other efforts on GPU Computing include compilation techniques for automated partitioning and resource aggregation, and performance and power modeling of scalable heterogeneous computing systems. The group leads the GPU Education and Research Center at Heidelberg University, sponsored by NVIDIA. The group has a rich history on reconfigurable logic for specialization, including interconnection networks and memory architectures. Within the DeepChip project, we collaborate with Graz University of Technology and Materials Center Leoben to optimize Deep Learning for resource-constrained embedded platforms, and to automate the use of hybrid processors like ARM and FPGA for such tasks. Similarly, our collaboration with CERN explores real-time data acquisition using commodity technology. Also, related efforts include GPU-accelerated ray-tracing for real-time treatment planning and optimizing graph processing for high-performance analytics by extending columnar in-memory databases.
News
- 09/2023 Two papers to be presented at ITEM workshop (collocated with ECML 2023), and one paper at PMBS workshop (collocated with SC 2023)!
- 06/2023 The HiPEAC Info Magazine covers a couple of our PhD students, some in flight and some who already graduated!
- 04/2023 New course on "Embedded Machine Learning" for this summer term - https://moodle.uni-heidelberg.de/course/view.php?id=16549
- 04/2023 The fourth ITEM workshop is accepted as a full-day workshop at ECML2023 in Turin, Italy! - https://www.item-workshop.org/
- 04/2023 Guest lecture at Xi'an University of Technology on "HW and SW support for Embedded Machine Learning"
- 03/2023 WEML2023 took place in Heidelberg, after a long COVID break! - https://www.deepchip.org/weml2023
- 01/2023 Congrats to Torben and Bernhard for an accepted paper on "Towards Hardware-Specific Automatic Compression of Neural Networks" at the Practical-DL workshop at the AAAI conference! - https://practical-dl.github.io, https://github.com/UniHD-CEG/galen, https://arxiv.org/abs/2212.07818
- Additional kudos for bringing a best paper award back home!
- 01/2023 Congrats to Hendrik and Bernhard for an accepted paper on "Walking Noise: Understanding Implications of Noisy Computations on Classification Tasks" at the AccML workshop at the HiPEAC conference! - https://accml.dcs.gla.ac.uk, https://arxiv.org/abs/2212.10430
- 01/2023 We are co-organizing the Workshop on "Future of FPGAs in HPC and Datacenter" - http://bit.ly/f4hd-2023
- 01/2023 The next Workshop on Embedded Machine Learning (WEML2023) will take place March 2nd, 2023! Read more: https://www.deepchip.org/weml2023
- 09/2022 The third ITEM workshop (first non-virtual one) took place at ECML2022 in Grenoble, France! - https://www.item-workshop.org
- 09/2022 We are local arrangement chairs for the renowned IEEE CLUSTER conference, which takes place in Heidelberg from Sept 6 to 9. The conference includes multiple workshops and tutorials, parallel conference sessions, a poster session and various social events in the vicinity of Heidelberg’s Old City. Read more: https://clustercomp.org/2022
- 07/2022 Congrats to Jonas for an accepted paper at FPL! - https://fpl.org/agenda/#accelerators-i
- 06/2022 HiPEAC conference (Budapest) aftermath: Dennis, Hendrik and Holger were presenting on "Joint Program and Layout Transformations to Enable Convolutional Operators on Specialized Hardware Based on Constraint Programming", "QONNX: Representing Arbitrary-Precision Quantized Neural Networks", "HW-Aware Initialization of DNN Auto-Tuning to Improve Exploration Time and Robustness", and "DeepChip and its use of FPGAs for Embedded Machine Learning"! [evidence: 1, 2]
- 05/2022 ITEM2022 workshop accepted at ECML-PKDD2022, check the CfP! - https://www.item-workshop.org, https://2022.ecmlpkdd.org
- 04/2022 New TACO publication on constraint programming for convolutional operators on specialized processors - kudos to Dennis!
- 02/2022 We are organizing a thematic session on Collaborative Machine Learning across IoT, Edge, Fog and Cloud devices for Improved Privacy and Resilience at HiPEAC Computing Week in Tampere, Finnland!
- 02/2022 Update by Hendrik on his XILINX internship: slides and recording
- 02/2022 Please welcome two new group members: Kazem Shekofteh has joined us again, but this time as post-doctoral research fellow, and Hendrik Borras is the newest PhD student in the set.
- 02/2022 The HiPEAC Info magazine is covering our ITEM2021 workshop! Also, the keynote presentations are meantime online: https://www.item-workshop.org
Main Research Projects
- Khunjerab: resistive RAM is a promising option in the light of diminishing returns from CMOS technology scaling, however, RRAM devices are inherently unstable and require amortization effort to compensate for process variations, wear-off, and stochastic effects during read and write. Khunjerab gears to exploit resistive memory for ML applications, in particular with tradeoffs regarding performance, power, and reliability. It is a joint project with colleagues from Xi’an University of Technology, China, Chinese Academy of Sciences, and the ECML group of Heidelberg University. Since 2021.
- DeepChip - Deep learning on resource-constrained systems: gathers methods that support the implementation of deep convolutional networks for machine learning on embedded platforms. DeepChip heavily relies on extreme forms of quantization and related unsafe optimizations to match the computational and memory requirements of deep neural networks to available hardware resources. By integrating these techniques in existing tool stacks for machine learning, it allows specialists in machine learning to leverage the ubiquitous availability and high energy-efficiency of advanced embedded systems for an improved use of classification and regression methods. This D-A-CH (DFG/FWF) project is a collaboration with the group of Franz Pernkopf from Technical University of Graz. While we already started our efforts with the help of research assistants and master students earlier, the funded project has just started in December 2016. Since 2015. More: http://www.deepchip.org
- CERN ATLAS/LHCb: for the ATLAS and LHCb high-energy physics experiments at CERN we are contributing to the data acquisition system, in particular the data collection manager. In this project, a commodity Ethernet network is used and upper-level software layers like the data collection manager guarantee minimal collection latencies by traffic shaping techniques. In addition, a complete data-flow messaging library is designed and optimized for this special application. This is a collaboration with colleagues from CERN and University of Castilla-La Mancha, Spain, and currently being extended by system modeling and data compression techniques. Since 2013.
Sponsors
We gratefully acknowledge the generous support that we are receiving. Current and past sponsors include Helmholtz, FWF, BMBF, DFG, Google, NVIDIA, XILINX, Intel, SAP, and Carl-Zeiss Stiftung.
Public code repository
Please find our public code here: https://github.com/UniHD-CEG
Principal Investigator
Prof. Dr. Holger Fröning
holger.froening(at)ziti.uni-heidelberg.de
Fon: +49 6221 54-16442 (email preferred)
Office: INF368, Room 529
Office hours: by appointment
Office assistance: Andrea Seeger
andrea.seeger(at)ziti.uni-heidelberg.de
Fon: +49 (0)6221-54-16361
Office: INF368, Room 502
Office hours: by appointment
Post-Doctoral Research Fellow
- Kazem Shekofteh - INF368, Room 526 - +49 6221 54-16425, kazem.shekofteh(at)ziti.uni-heidelberg.de
PhD Students
- Bernhard Klein - INF368, Room 530 - +49 6221 54-16443, bernhard.klein(at)ziti.uni-heidelberg.de
- Vahdaneh Kiani - INF368, Room 528 - +49 6221 54-16441, vahdaneh.kiani(at)ziti.uni-heidelberg.de
- Hendrik Borras - INF368, Room 530 - +49 6221 54-16443, hendrik.borras(at)ziti.uni-heidelberg.de
- Daniel Barley - INF368, Room 528 - +49 6221 54-16441
- Kevin Stehle, INF368, Room 527
External PhD Students
- Jonas Dann, SAP, jonas.dann(at)sap.com
- Aleksandra Poreba, CERN, aleksandra.poreba(at)cern.ch
Research Assistants and Graduate Students - INF368, Room 531/527
- Lisa Kuhn, MSc student, kuhn(at)cl.uni-heidelberg.de
- Falk Selker, MSc student, selker(at)stud.uni-heidelberg.de
- Dennis Jakob, research assistant
- Eric Kern, MSc student, eric.kern(at)stud.uni-heidelberg.de
- Dimitri Butov, MSc student
- Xiao Wang, MSc student
- Christian Simonides, MSc student, christian.simonides(at)stud.uni-heidelberg.de
Visiting scientists and PhD students
- Juan José Garcia-Castro Crespo (visiting PhD student, University Castilla-La Mancha), exploring the hardware costs of dynamic network reconfiguration, 09/2020-02/2021
- Kazem Shekofteh (visiting PhD student, Ferdowsi University of Mashhad, Iran), 11/2016-05/2017
- Francisco Andujar (visiting PhD student, Universidad Castilla-La Mancha, Spain, 2014)
- Pedro Garcia (visiting scientist, Universidad Castilla-La Mancha, Spain, 2013)
- Jesus Escudero Sahuquillo (visiting scientist, Universidad Castilla-La Mancha, Spain, 2012)
Internships and research stays of group members
- Hendrik Borras, XILINX Research, Dublin, Ireland, 05-12/2021 (sponsored)
- Felix Zahn, University of Castilla-La Mancha, Spain, 01-02/2017 (sponsored)
- Holger Fröning, NVIDIA Research, Santa Clara, CA, US, 05-10/2016 (sponsored)
- Benjamin Klenk, NVIDIA Research, Santa Clara, CA, US, 02-07/2016 (sponsored)
- Felix Zahn, University of Castilla-La Mancha, Spain, 10/2015 (sponsored)
- Holger Fröning, Technical University of Graz, Austria, 03/2015 (sponsored)
- Benjamin Klenk, NVIDIA Research, Santa Clara, CA, US, 03-06/2015 (sponsored)
- Alexander Matz, Intel Labs, Hillsboro, OR, US, 01-05/2015 (sponsored)
Former members & visiting students (date of leave/visit)
- Christian Alles (MSc student, 07/2023, first appointment at John Deere)
- Yannick Emonds (leaving to Fraunhofer)
- Dr. Dennis Rieber (PhD student, defended 03/2023, first appointment at Bosch)
- Constantin Nicolai (BSc student, 05/2022)
- Torben Krieger (MSc student, 12/2022, first appointment at SAP)
- Lorenz Braun (PhD student, first appointment at Bosch, 07/2022)
- Dr. Laura Promberger (PhD student, defended 08/2022, first appointment at CERN)
- Tobias Richstein, tobias.richstein(at)stud.uni-heidelberg.de (MSc student, 2022, first appointment at SAP)
- Dr. Günther Schindler (Post-Doc and PhD student, defended 06/2021, first appointment at SAP, 01/2022)
- Royden Wagner (MSc student, 2021, leaving as PhD candidate to Bioquant, Heidelberg University)
- Benjamin Maier (MSc student, 2021)
- Chenyang Zhu (MSc student, 2021)
- Florian Nowak (BSc student, 2021)
- Dr. Felix Zahn (PhD student, defended 11/2020, first appointment at CERN, 09/2020)
- Florian Brunner (MSc student, 2021)
- Georg Weisert (BSc student, 2020)
- Raphael Kirchholtes (BSc student, 2020)
- Otto von Zastrow-Marcks (BSc student, 2020)
- Kevin Stehle (MSc student, leaving as PhD candidate to CERN, 03/2020)
- Paul Bethge (MSc student, 02/2020)
- Dilan Canpolat (MSc student, 01/2020)
- Berra Sayin (visiting student, Sabancı University, Turkey, 05-08/2019)
- Shreyas Ravishankar (visiting student, Birla Institute of Technology and Science Pilani (BITS Pilani), India, 05-08/2019)
- Dr. Alexander Matz (PhD student, defended 11/2020, first appointment at IMC, Amsterdam, Netherlands, 06/2019)
- Alejandro Santos (PhD student, 2019)
- David Marquant (MSc student, 2019), first appointment at HD Vision Systems GmbH
- Michael Harbarth (MSc student, 2019)
- Himanshu Tiwari (MSc student, 2018)
- Antsa Andriamboavonjy (MSc student, 2018), first appointment at SAP (Walldorf, Germany)
- Sven Nobis (MSc student, 2018)
- Andreas Melzer (MSc student, 2018), first appointment at SAP (Walldorf, Germany)
- Armin Schäffer (MSc student, 2018), first appointment at HMS Analytical Software (Heidelberg, Germany)
- Klaus Neumann (MSc student, 2018)
- Dennis Rieber (MSc student, 2017), first appointment at Robert Bosch GmbH (Renningen, Germany)
- Dr. Benjamin Klenk, (PhD student, 2017), first appointment at NVIDIA Research, Santa Clara (US)
- Steffen Lammel (MSc student and research assistant, 2017), first appointment at SAP (Walldorf, Germany)
- Julian Schwing (MSc student, 2017), first appointment at SAP (Walldorf, Germany)
- Artur Kühlwein (MSc student, 2017)
- Dr. Tommaso Colombo (PhD student, 2016), first appointment at CERN
- Dominik Sterk, (MSc student, 2016)
- Christoph Klein, (MSc student, 2016), first appointment as PhD student at Heidelberg University
- Daniel Schlegel (MSc student, 2016), first appointment at EVS Broadcast Equipment
- Benjamin Baumann (MSc student, 2016)
- Eugen Rusakov (MSc student, 2016), first appointment as PhD student at TU Dortmund
- Julian Romera (MSc student, 2016)
- JProf. Dr. Lena Oden (PhD, defended 04/2015), first appointment at Postdoc at Argonne National Labs, Chicago, IL, US.
- Hector Montaner (PhD student, defended 04/2013 with highest degrees, Technical University of Valencia, Spain, 2013)
- Manuel Dewald (MSc student, 2013), first appointment at SAP
- Elena Kuss (MSc student, 2012), first appointment as PhD student at University of Mannheim
