machine-learning.md

Machine learning

{% embed url="https://www.youtube.com/watch?v=ZJixNvx9BAc&list=WL&index=14&t=0s" %}

Models

• https://arxiv.org/abs/1907.07355 Probing Neural Network Comprehension of Natural Language Arguments
• https://arxiv.org/abs/1902.01007 Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference

GPU

What is GPU https://course.fast.ai/gpu_tutorial.html

Cost

• https://twitter.com/eturner303/status/1143174828804857856 Holy crap: It costs $245,000 to train the XLNet model (the one that's beating BERT on NLP tasks..512 TPU v3 chips 2.5 days $8 a TPU -see the above thread
• The cost of training MT models: https://medium.com/syncedreview/the-staggering-cost-of-training-sota-ai-models-e329e80fa82
• Mozilla's struggles with Baidu model training - hardware issues
  • https://blog.mozilla.org/blog/2017/11/29/announcing-the-initial-release-of-mozillas-open-source-speech-recognition-model-and-voice-dataset/
  • https://hacks.mozilla.org/2017/11/a-journey-to-10-word-error-rate/

Ethics

Open AI and their ethical(?) release of model that can create fake news https://openai.com/blog/better-language-models/#task2

• news https://techxplore.com/news/2019-02-openai-gpt-algorithm-good-fake.html
• criticism https://www.theregister.co.uk/2019/02/14/open_ai_language_bot/
• https://thegradient.pub/openai-please-open-source-your-language-model/
• GPT‑3: Its Nature, Scope, Limits, and Consequences Luciano Floridi1,2 · Massimo Chiriatti

{% embed url="https://www.theguardian.com/technology/2019/may/28/a-white-collar-sweatshop-google-assistant-contractors-allege-wage-theft" %}

{% embed url="https://boingboing.net/2019/06/03/wage-theft-2.html" %}