---
marp: true
theme: uncover
color: #eee
colorSecondary: #333
backgroundColor: #111
paginate: true
---

# Théorie : Models,
# Models _everywhere_

---
## Perceptron

<!-- Base fondamentale -->

<!-- Little more than a learning thermostat -->

---

> _perceptron may eventually be able to learn, make decisions, and translate languages._

**Frank Rosenblatt**, 1958

---

<!-- 
Was researched until Minsky's killer book Perceptrons:
> acknowledge some of the perceptrons' strengths while also showing major limitations
-->

...until Marvin Minsky's book 
## _Perceptrons_ (1969)

![bg right:38% fit](https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fweltbild.scene7.com%2Fasset%2Fvgw%2Fperceptrons-195682049.jpg&f=1&nofb=1&ipt=802cd208ee9f59170d9fb973f0c820f23265e3d6c23b60529d332b7536fec005&ipo=images)

---

<!---

-> Focus on "Symbolic systems"

-> huge limits to perceptrons abilities
-> killed the field: almost stopped research on the connectionnist paradigm for the next fifteen years, until it gets a REVIVAL
-->

- Pour aller plus loin: [_WP History of AI | Perceptrons and the attack on connectionism_](https://en.wikipedia.org/wiki/History_of_artificial_intelligence#Perceptrons_and_the_attack_on_connectionism)

--- 

![bg fit](https://upload.wikimedia.org/wikipedia/en/5/52/Mark_I_perceptron.jpeg)
<!-- Mark I Perceptron machine, 1958 | Cornell University Library website -->
<!-- 
It was connected to a camera with 20×20 cadmium sulfide photocells to make a 400-pixel image.
The main visible feature is a patch panel that set different combinations of input features. 
To the right, arrays of potentiometers that implemented the adaptive weights. -->

---

# Formalisme: un Perceptron

<!-- _backgroundColor: 
_color: -->
$$ f(\mathbf{x}) = \begin{cases}1 & \text{if }\ \mathbf{w} \cdot \mathbf{x} + b > 0,\\0 & \text{otherwise}\end{cases} $$

---

### Intuition: apprentissage d'un Perceptron

<!-- _backgroundColor: 
_color: -->

![bg right 95%](https://upload.wikimedia.org/wikipedia/commons/thumb/8/8a/Perceptron_example.svg/1280px-Perceptron_example.svg.png)
<!-- Diagram by Elizabeth Goodspeed -->

$$ f(\mathbf{x}) = \begin{cases}1 & \text{if }\ \mathbf{w} \cdot \mathbf{x} + b > 0,\\0 & \text{otherwise}\end{cases} $$

---


![bg 90%](https://cdn-images-1.medium.com/max/1600/1*ZafDv3VUm60Eh10OeJu1vw.png)

---

![bg left fit](https://cdn-images-1.medium.com/max/1600/1*ZafDv3VUm60Eh10OeJu1vw.png)
<!-- Image credit: Shruti Jadon, Introduciton to Different Activation Functions for Deep Learning
https://medium.com/@shrutijadon/survey-on-activation-functions-for-deep-learning-9689331ba092 -->


- Aller plus loin: [CodeX - Ansh David: _intro to Perceptrons and different type of activation functions_
](https://medium.com/codex/single-layer-perceptron-and-activation-function-b6b74b4aae66)

---
## Multi Layer Perceptron


![bg right:55% fit](https://rasbt.github.io/mlxtend/user_guide/classifier/NeuralNetMLP_files/neuralnet_mlp_1.png)
<!-- Image credit & reference: Raschka, Sebastian (2018) MLxtend: Providing machine learning and data science utilities and extensions to Python's scientific computing stack.
https://rasbt.github.io/mlxtend/
 -->

--- 

## MLP == Neural Network!

---
## Recurrent Neural Networks

<!-- Recurrence -> loops -> temporal & dynamic behavior -->

<!-- internal state as input == kind of 'memory'

-> Works for understanding streams of stuff, e.g. cursive handwriting, or stock market data, or audio -> speech to text, music analysis, etc  
 -->

![bg fit right](https://www.skynettoday.com/assets/img/overviews/2020-09-27-a-brief-history-of-neural-nets-and-deep-learning/34568.gif)

- See [The Unreasonable Effectiveness of RNNs](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)

---
## LSTMS

<!-- how can RNNS handle long term dependencies?” -->

<!-- Let's learn from Christopher Olah: 
co-founders of Anthropic, an AI lab focused on the safety of large models. Previously, I led interpretability research at OpenAI
worked at Google Brain
co-founded Distill, a scientific journal focused on outstanding communication. 
-->
- See [Understanding LSTM networks](https://colah.github.io/posts/2015-08-Understanding-LSTMs/) by **Christopher Olah**

---
# Deep Learning

--- 
### Layers, Layers, Layers!

--- 
### Attention! It's all you need
- https://arxiv.org/pdf/1706.03762.pdf

--- 
### TRANSFORMERS

![bg right fit](https://external-content.duckduckgo.com/iu/?u=http%3A%2F%2Fwww.wikialpha.org%2Fmediawiki%2Fimages%2Fthumb%2F1%2F17%2FSteppernightstick-united.jpg%2F480px-Steppernightstick-united.jpg&f=1&nofb=1&ipt=f023244f8ef45b4cba81ffdd819cada10f55dea2c207d715f0f10498b001582f&ipo=images)

--- 
### Language Models

- [BERT: Bidirectional
Encoder Representations from Transformers](https://arxiv.org/pdf/1810.04805v2.pdf)

---
<!-- _backgroundColor: 
_color: -->
### Reinforcement Agents

![bg right 90%](./img/01-reinforcement.png)
<!-- Image Credit: [Megajuice](https://en.wikipedia.org/wiki/Reinforcement_learning#/media/File:Reinforcement_learning_diagram.svg) -->

- [Exemple d'AlphaZero](https://arxiv.org/pdf/1712.01815.pdf)

---
# Pratique : Choisir un Modèle

---

# Supervisé ou non ?

--- 

![bg 93%](https://miro.medium.com/max/1400/0*p3zNp9D8YFmX32am.webp)
<!-- Reference et source: https://medium.com/@dkatzman_3920/supervised-vs-unsupervised-learning-and-use-cases-for-each-8b9cc3ebd301 -->

--- 

### Selon l'objectif
- [Overview of Supervised ML Algorithms](https://towardsdatascience.com/overview-of-supervised-machine-learning-algorithms-a5107d036296)

---

![bg](https://miro.medium.com/max/1400/1*31IsgRs2QZ7H4kl8tNqjdQ.webp)

---

### Selon le tooling à disposition


<!-- 
Target:

Learning and deploy, or continuous learning?
GPU? CPU? Embarqué ?
Mobile ?


Language/frameworks:
CUDA
PyTorch
etc

Deployment/ops:
Own model deployed and monitored ?
ML as a Service, e.g. HuggingFace inference API
 -->


