Question: What is distilation?

Yee Whye Teh answered on 16 Nov 2019:

I’m not sure what is the context of this question. Are you asking about the technical meaning of distillation in machine learning?

In standard machine learning, we get systems to learn from data. Distillation is a different process, where a first system learns from data, then a second system (the student) learns from the first system (the teacher). This process somethings works better than having the student learn directly from data. If we think of the teacher’s learning as gaining knowledge from data, the transfer of knowledge to the student can be thought of as the teacher distilling its knowledge when it teachers the student.