Sébastien Rouault

About my work

I am a PhD candidate in the Distributed Computing Laboratory of EPFL, working on the robustness of (distributed) machine learning (ML) algorithms. In particular, my academic work focuses on developing and testing practical algorithms, that can be used to improve actual ML systems.

My scientific background spans from mathematics and statistics to design principles of computer systems. My interests in understanding (and later building) automated systems goes back to my childhood, and has only kept growing with the studying of mathematics, foundations of software, software system engineering and actual networked system administration.

Academic achievements

I have co-published conference papers at the following venues1:

  1. Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?
    Rachid Guerraoui, Nirupam Gupta, Rafaël Pinot, Sébastien Rouault, John Stephan
    PODC 2021 — ACM 40th Symposium on Principles of Distributed Computing
    Selerno, Italy, July 26–30, 2021
  2. Garfield: System Support for Byzantine Machine Learning
    Rachid Guerraoui, Arsany Guirguis, Jérémy Max Plassmann, Anton Alexandre Ragot, Sébastien Rouault
    DSN 2021 — 51st IEEE/IFIP International Conference on Dependable Systems and Networks
    Taipei, Taiwan, June 21–24, 2021
  3. Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent
    El-Mahdi El-Mhamdi, Rachid Guerraoui, Sébastien Rouault
    ICLR 2021 — 9th International Conference on Learning Representations
    Vienna, Austria, May 4–8, 2021
  4. Aksel: Fast Byzantine SGD
    Amine Boussetta, El-Mahdi El-Mhamdi, Rachid Guerraoui, Alexandre Maurer, Sébastien Rouault
    OPODIS 2020 — 24th International Conference on Principles of Distributed Systems
    Strasbourg, France, December 14–16, 2020
    Best Student Paper award
  5. Fast and Robust Distributed Learning in High Dimension
    El-Mahdi El-Mhamdi, Rachid Guerraoui, Sébastien Rouault
    SRDS 2020 — IEEE 39th International Symposium on Reliable Distributed Systems
    Shanghai, China, September 21–24, 2020
  6. Genuinely Distributed Byzantine Machine Learning
    El-Mahdi El-Mhamdi, Rachid Guerraoui, Arsany Guirguis, Lê Nguyen Hoang, Sébastien Rouault
    PODC 2020 — ACM 39th Symposium on Principles of Distributed Computing
    Selerno, Italy, August 3–7, 2020
  7. AggregaThor: Byzantine Machine Learning via Robust Gradient Aggregation
    Georgios Damaskinos, El-Mahdi El-Mhamdi, Rachid Guerraoui, Arsany Guirguis, Sébastien Rouault
    MLSys 2019 — 1st Conference on Machine Learning and Systems
    Palo Alto, CA, USA, March 31–April 2, 2019
  8. The Hidden Vulnerability of Distributed Learning in Byzantium
    El-Mahdi El-Mhamdi, Rachid Guerraoui, Sébastien Rouault
    ICML 2018 — 35th International Conference on Machine Learning
    Stockholm, Sweden, July 10–15, 2018
    Accepted with a "Long Talk"
  9. On The Robustness of a Neural Network
    El-Mahdi El-Mhamdi, Rachid Guerraoui, Sébastien Rouault
    SRDS 2017 — IEEE 36th Symposium on Reliable Distributed Systems
    Hong Kong, China, September 26–29, 2017

1Please note that, in my unit, the author names are always written in alphabetical order.


I gave the following talks:

  • Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent
    ICLR 2021 — 9th International Conference on Learning Representations
  • Genuinely Distributed Byzantine Machine Learning
    PODC 2020 — ACM 39th Symposium on Principles of Distributed Computing
    First half (background on ML + asynchronous algorithm)
  • Byzantine Resilient Machine Learning: Algorithms to cure poisoned SGD
    EcoCloud 2019 — A Center for Sustainable Cloud Computing
    Invited talk

I have (co-)implemented the following research software:

  • Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent
    ICLR 2021 — 9th International Conference on Learning Representations
    Authored everything
    (Substantial) parts were also reused in:
    • Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?
    • Garfield: System Support for Robust Machine Learning
    • Aksel: Fast Byzantine SGD (best student paper award)
    • Fast and Robust Distributed Learning in High Dimension
    • Genuinely Distributed Byzantine Machine Learning
  • Garfield: System Support for Byzantine Machine Learning
    DSN 2021 — 51st IEEE/IFIP International Conference on Dependable Systems and Networks
    Authored the parallelized CPU and CUDA (GPU) Byzantine-resilient GAR implementations for both PyTorch and TensorFlow
    In addition, this paper reuses code I wrote for:
    • Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent
    • AggregaThor: Byzantine Machine Learning via Robust Gradient Aggregation
  • AggregaThor: Byzantine Machine Learning via Robust Gradient Aggregation
    MLSys 2019 — 1st Conference on Machine Learning and Systems
    Everything, except: MPI and UDP-related code additions, and TensorFlow optional patches
    (Substantial) parts were also reused in:
    • Garfield: System Support for Robust Machine Learning
    • Genuinely Distributed Byzantine Machine Learning
  • On The Robustness of a Neural Network
    SRDS 2017 — IEEE 36th Symposium on Reliable Distributed Systems
    Authored everything


I started my PhD in September 2017, and I will most likely graduate from my PhD around end of August 2021.
On a side note, I hold two MSc degrees:

  • the MSc in Computer Science (specialization "Software Systems") from EPFL
  • the "Diplôme d'Ingénieur" from Supélec (now CentraleSupélec)

Also, I remember as a MSc student that there were two kinds of course projects: those with an automated submission system, and those without any. In my teaching assistant duties, I decided to build a submission system for the EPFL course Concurrent Algorithms. In the project, students have to write their own Software Transactional Memory library, and test it against a common workload on a shared high-performance, multi-core server. The submission system manages the (sequential) testing of the student implementations on the hardware, besides many good-to/must-have features (tight sandboxing via firejail, extensive logging, submission rate limitation, automated credential broadcast to student via emails, student can overwrite/download their own/best submission). The satisfaction of both obtaining (very) positive student feedbacks and at the same time (substantially) "optimizing" my TA workload 😉 seem enough to justify the place of this work in this section.

Personal work

I have been pretty busy with my PhD, and other projects, in the last 3 and an half years. I guess this was expected. There currently are two projects I've (slowly but thoroughly) been working on, but I doubt I will be able to finish and release any of them in the coming months or so.

I have been officially involved in the following student associations:

  • EPIC
    The graduate student association of the faculty. It organizes mountain outings, board game nights, informative panels and talks, and more. I was the treasurer for the year 2019; to be fair it was nothing overwhelming 🙂. The 6 members of the association won the EPFL IC faculty's Distinguished Service Award for our steadfast help and involvement during the EDIC Open House 2019.
  • Supélec Rézo Rennes
    I have been a member of the association for both of my two years at Supélec. I was elected president for my second academic year, from April 2014 to March 2015. This association acts as an Internet and service provider for the students on the campus (5 residential buildings + 1 "social" building), and manages its own hardware and software. This was a great experience, and the knowledge I acquired there still serves me today. During my mandate, the association renewed its stock of aging (even failing) switchs, added the TV via multicast and a few other services. In particular, we worked on a quality of service problem: the upstream bandwidth provided by our (many) ADSL modems was not always shared evenly between our many users, sometimes leading to increased latencies for everyone. My part of the project was to design the algorithms and write the kernel module ensuring fair upstream sharing, while my teammate worked on the integration with the existing software architecture.

Acknowledgments

Do you know this methaphor, to "stand on the shoulders of giants"? That's what I do all day long.

About my PhD work, its (upper layer) foundations were largely laid by the authors of Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent. One of the authors, El Mahdi El Mhamdi, advised me on all the scientific aspects in my junior time, under the supervision of Rachid Guerraoui, who left us carte blanche on this then-new line of research for his unit.

About my personal work, well, there are definitely too many factors that contributed to what I have done so far, and what I can do today. Assuredly, growing in a "fully developed" country, sheltered from all the vital needs, and with parents who introduced me to Mr. Computer very early (age 4 for the keyboard, age 8 for coding), have been a tremendous chance. Stellar resources out there are likely another factor, so perhaps I could take advantage of this section to highlight some pieces of work that (I think) are really worth casting a glance at, to learn about specific subjects:

  • Preshing on Programming
    I find the blog entries on C11/C++11 concurrent programming limpid, which really is a feat for such a technical subject. And there is more to this blog than only concurrent programming.
  • Ralf's Ramblings
    The author is an expert (PhD!) on Rust, and I find that his posts on this language tackle specification/internal details which can be critical to know to properly write unsafe Rust.
  • Git Internals
    Just in case you have missed it. If in general you very much prefer to understand things in details first, to be able to reason about and use them better, regarding git, the documentation is excellent.
  • Distill
    From the about page: "Machine Learning Research Should Be Clear, Dynamic and Vivid. Distill Is Here to Help.".
  • WordReference.com
    For word/expression translation, this is my go-to resource (I use it mostly for EN/FR).

This list is by no mean exhaustive, I put entries as I think about them. There is no particular order.

GPG keys

Master key (certify only)

-----BEGIN PGP PUBLIC KEY BLOCK-----

mDMEYDBG+RYJKwYBBAHaRw8BAQdAm4efi1S5KcS7/paOxvg4GBA6vsh41b+ueM4U
UJJUoJO0ElPDqWJhc3RpZW4gUm91YXVsdIiQBBMWCAA4AhsBBQsJCAcCBhUKCQgL
AgQWAgMBAh4BAheAFiEEdgPygd8ofGqet/4IBSy1bvRxi6cFAmAwWokACgkQBSy1
bvRxi6dG8gEA/uukjPv469mRzIXyAYYzvZ7JjxGrX6SXKmJVujbGKnMA/ipoNZfW
IdCWbLDkyapQV4iQjUO2/az5JtLTeK0OymcA
=b2Dd
-----END PGP PUBLIC KEY BLOCK-----

Signatures
  • El Mahdi El Mhamdi
    97F00A86DB26223BDC77481D9D0DB658ADF6B78E
Current keys

-----BEGIN PGP PUBLIC KEY BLOCK-----

mDMEYM9tihYJKwYBBAHaRw8BAQdAWCPE4OMKC0JQrsTYBqqRsvs8ZmdxNTJObSyA
rwaLYs20LFPDqWJhc3RpZW4gUm91YXVsdCA8c2ViYXN0aWVuQGNhbGljYXJwYS5j
b20+iJYEExYIAD4WIQTT0gbWB8cbUduTMd26dh7snVRArwUCYM9ucQIbAwUJA8Jn
AAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRC6dh7snVRAr/VqAP9Q2WkJU6yA
OXkCqwWeZ5Ez2fqhaz1eC+ysBHgDUOlLQAEAqY57PtTPhzKFfy+LflGwBy07ASX8
uSe7vUI3yimcRQuIeAQQFggAIBYhBHYD8oHfKHxqnrf+CAUstW70cYunBQJgz3r+
AgcAAAoJEAUstW70cYun0dABAOveFNPrUDjvqUJkQum/ovimGPj1AySh+c4uFBnH
zY5CAQC549zY2A8JtnijcoYEzfHkw/KF9MZVB9pXpm/CuoHuALQnU8OpYmFzdGll
biBSb3VhdWx0IDxzZWJhc3RpZW5Acm91YXUubHQ+iJYEExYIAD4WIQTT0gbWB8cb
UduTMd26dh7snVRArwUCYM9tigIbAwUJA8JnAAULCQgHAgYVCgkICwIEFgIDAQIe
AQIXgAAKCRC6dh7snVRArxSsAQDQFqFbw2EDD16TX1syIl5KLyzauUhygxaGK6uj
nJIvTAEApea9EdfU4jNV2naUwc1xhPPIdLIsl++pXZsO+EunmgeIeAQQFggAIBYh
BHYD8oHfKHxqnrf+CAUstW70cYunBQJgz3r+AgcAAAoJEAUstW70cYunO3wBAIQ9
pul0nj6PYb6OwPTAXtQ3Y0nnPNwZlfan7ywCES8OAQC/mC436a2Rl6Pi9nfmLs7f
lYlbtkR0L6iS4m3AiCtLDrQ1U8OpYmFzdGllbiBSb3VhdWx0IDxzZWJhc3RpZW4u
cm91YXVsdEBhbHVtbmkuZXBmbC5jaD6IlgQTFggAPhYhBNPSBtYHxxtR25Mx3bp2
HuydVECvBQJgz25nAhsDBQkDwmcABQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJ
ELp2HuydVECv4PoBAJ91hwkYNKaG+EGmTxzkYJR7U0Po20RPiyBo4SQOUqJ6AQCo
5pKs66TUpl4teQhjuheA137YBiRN1rLlC0eI5hqXDoh4BBAWCAAgFiEEdgPygd8o
fGqet/4IBSy1bvRxi6cFAmDPev8CBwAACgkQBSy1bvRxi6dlwQD9Ek6b6p6okVWT
p27AaEN+7lGu92Y6LCv/OflWrNhliksBAPKKZQVeCPGME0nH/Kud/z/PzXirbhFe
/jpgZwGDe0kEuDgEYM9tihIKKwYBBAGXVQEFAQEHQNFTYjn/BpgeS2nYKEX0nZk9
7Xp6e7vbQ58736dMJrwQAwEIB4h+BBgWCAAmFiEE09IG1gfHG1HbkzHdunYe7J1U
QK8FAmDPbYoCGwwFCQPCZwAACgkQunYe7J1UQK/lNAD/Q2Y1wU2APwzI64+Kt1zq
hnnAe89/x9JDgiE1MGn+rYAA/ja5GYutE8hW7mrt188AESsrxfAl6wJQ1B7trPJd
i0kC
=+WeU
-----END PGP PUBLIC KEY BLOCK-----