A lot of my work is figuring out components of a machine learning system. So if I have a hypothesis that a component (that maybe I designed, or somebody else did in a different system) is useful, then I’ll design computer experiments which test whether the whole system will perform better with or without the component. However the signal from a single experiment is often noisy, so I have to replicate the experiment lots of times, maybe with different parameters. This means that the experiments cannot be run on my computer, and I have to run it “in the cloud” across lots and lots of computers.
Comments