Winter is coming, the 3rd AI Winter.
- The 2nd industrial revolution may end up being more like the 1st than many would like. Steam power was properly invented in 1698 and took 140 years to change the world. So far, AI is following the same time-line and the emerging challenges may see it continue to do so.
- Hype. RFM thinks that the biggest problem facing AI is the expectations that have been set.
- Deep Learning algorithms have no understanding of the functions they perform. This means that if the data set changes for any reason, the algorithm will often catastrophically fail.
- Compute power. AlphaGo Zero was trained using 300,000x the compute resource compared to AlexNet which set a standard for image recognition in 2012. RFM thinks that AlphaGo Zero does not offer 300,000x the value of AlexNet strongly implying diminishing returns from investments in compute This is significantly exacerbated by the ending of Moore’s Law.
- Perfect data. Deep learning does best in environments which are both finite and stable (e.g. games) as even most sophisticated deep learning algorithms are incapable of extrapolation due to their lack of causal understanding.