Matei Zaharia
Started Apache Spark, helped develop MLflow and co-authored DSPy; UC Berkeley professor and Databricks CTO
- Associate Professor of EECS, University of California, Berkeley1
- Co-founder and CTO, Databricks1
Matei Zaharia is an associate professor at UC Berkeley, working on computer systems and AI in the Sky Lab, and a co-founder and the CTO of Databricks. During PhD work Zaharia started the Apache Spark project and co-started Apache Mesos and Spark Streaming; at Databricks, Zaharia helped develop MLflow and Delta Lake. Zaharia's recent projects include the DSPy framework for LLM programs, the GEPA optimizer and the LOTUS engine for semantic queries, and Zaharia is a co-author of the DSPy paper.12
Work
MLflowOpen-source code
Zaharia's homepage says Zaharia helped develop MLflow at Databricks. The mlflow/mlflow repository is published under the Apache-2.0 license.14
DSPyOpen-source code
Co-author of "DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines" (2023); Zaharia's homepage lists DSPy among Zaharia's recent projects. The DSPy repository is MIT-licensed.213
Apache Spark (external site: spark.apache.org)Open-source code
Started the Apache Spark project during PhD work, according to Zaharia's homepage.1
This profile covers public professional work only, from the sources below. It is not an endorsement or a ranking. To correct or remove it, use the correction form.