---
marp: true
theme: uncover
color: #eee
colorSecondary: #333
backgroundColor: #111
paginate: true
footer: "ML101 | Paul-Louis Nech | INTECH 2022-2023"
---

# <!-- fit --> Hello World

ROBOTS ARE UPRISING. WHAT SIDE ARE YOU ON?

<!-- 21ème siècle. Les algos sont partout. -->
<!-- L'API divise le monde, scindé en deux catégories : ceux qui l'écrivent et ceux qui la vivent. -->
<!-- Si vous êtes arrivés jusqu'ici, en 5ème année, 
bravo,
car vous êtes du bon coté de la ligne... -->
---

<!-- Plus sérieusement, bienvenue dans mon cours -->
### On va parler de 
# <!-- fit --> Machine Learning
<!-- Vous avez eu l'occasion d'y réfléchir un peu avant ce cours. -->

---

## Mais pas comme ça

<br />

$$ I_{xx}=\int\int_Ry^2f(x,y)\cdot{}dydx $$
<!-- Le projet, c'est pas que vous compreniez tous les maths derrière -->
<br />

#### (Le moins possible)
<!-- Parfois c'est nécessaire pour saisir la beauté d'un modèle -->
---

## Ni comme ça
<!-- Les dangers, un sujet pour un cours entier -->
<!-- Biais, feedback loop foireuses, Risque existentiel?... -->
![bg right](https://uproxx.com/wp-content/uploads/2015/05/angry-bender.jpg?w=650)
<!-- Image Credit: Bender from Futurama, created by Matt Groening and David X. Cohen -->

###### - [the Fallacy of generalization from Fictional evidence](https://www.lesswrong.com/posts/rHBdcHGLJ7KvLJQPk/the-logical-fallacy-of-generalization-from-fictional)
<!-- The more details, the less likely! -->
--- 

## Encore que...

![bg right grayscale](https://uproxx.com/wp-content/uploads/2015/05/angry-bender.jpg?w=650)

###### - [AGI Ruin: A List of Lethalities](https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities)

--- 

## On va parler des derniers succès

#### - [Meta AI Research: CICERO](https://ai.facebook.com/blog/cicero-ai-negotiates-persuades-and-cooperates-with-people/)
<!-- Mais aussi:
 - AlphaGo / AlphaZero
 - 
-->
![bg left](./img/01-cicero.png)

--- 

## Et d'échecs intéressants

##### - [MIT Tech review:  100s of AI tools built for covid... None helped.](https://www.technologyreview.com/2021/07/30/1030329/machine-learning-ai-failed-covid-hospital-diagnosis-pandemic/)

![bg left 100%](./img/01-covid-failed.png)

----

## Mais aussi de révolutions en cours


##### - [Midjourney](fixme)

![bg left](./img/01-midjourney.png)

---
<!-- _footer: "" -->
![bg 100%](./img/01-diffusion.png)
<br />
<br />
<br />
<br />
<br />
##### - [The Illustrated Stable Diffusion by Jay Alamar](https://jalammar.github.io/illustrated-stable-diffusion/)

---
### Comment les modèles... peuvent raconter des histoires 

##### - [AI Dungeon 2](https://aidungeon.cc/)

![bg left 100%](./img/01-dungeon.jpg)

---
### Comment les modèles... peuvent halluciner 

##### - [DeepDream](https://en.wikipedia.org/wiki/DeepDream)

- [Pareidolia](https://en.wikipedia.org/wiki/Pareidolia)

![bg left 100%](../tp/img/intech_dream.png)

----
### Comment ils nous comprennent


##### - [OpenAI Jukebox: a neural net that generates music, including rudimentary singing](https://openai.com/blog/jukebox/)

![bg left 100%](./img/01-jukebox.png)

----
## Comment vous pourriez les comprendre
<!-- Comprendre ce qu'ils ont dans la tête -->
<!-- Comprendre comment ils nous analysent -->
<!-- Comprendre comment on les contourne -->

--- 
## Et comment leur faire faire ce que vous voulez
<!-- Les appliquer à votre quotidien -->
<!-- S'en servir intelligemment en entreprise -->
<!-- Entrer dans un nouveau monde d'art et de créativité -->

---
<!--
_backgroundColor: white
_color: black
-->
<br />

## Qui suis-je?

<!-- Vous pouvez me contacter -->

### Paul-Louis Nech
- ###### ✉ formation@nech.pl
- ###### 🔗 LinkedIn.com/in/PLNech
- ###### 💡 GitHub.com/PLNech

![bg right grayscale](./img/01-me.jpg)

---

<!--
_backgroundColor: white
_color: black
-->
<br />

## Parcours

- #### EPITA 2016 | MTI
- #### Software Engineer @Algolia
- #### Senior ML Engineer @Algolia
![bg right](./img/01-me2.jpg)

--- 

## Et vous ?

<!-- Nom, genre de musique pref, plat pref -->

---
## C'est quoi ce cours ?

<!-- Plan du cours: -->

<!-- TD: Quali aujourd'hui, plus pratique en janvier -->

<!-- Eval: Par les rendus, pas de piège :) -->

<!-- Format / interactions welcome -->
---
#### Des connaissances
## &
#### Des compétences
![bg right](./img/01-map-center.png)

---
#### Ce que vous saurez
![bg right](./img/01-map-known.png)

---
###### (Et ce que vous saurez... que vous ne savez pas) 
![bg right 99%](./img/01-map-unknown.png)

---
#### Ce que vous saurez faire
![bg right](./img/01-map-practice.png)

---
# Prêts ?
---
# Les bases 

---
## C'est quoi apprendre ?
<!-- 
Behaviorisme: apprendre = savoir _faire_ 
Acquis vs Inné : danse des abeilles vs culture des chimpanzés
Conditionnement opérant (Pavlov et sa cloche)
-->
---
<!-- _footer: "" -->
![bg](https://image.beeplaza.com.au/wp-content/uploads/2019/12/bee-dance.jpg)
<!-- Image Credit: BeePlaza Beekeeping Shop; beeplaza.com.au -->

<!-- 
Acquis vs Inné : danse des abeilles vs culture des chimpanzés
-->

---
<!-- _footer: "" -->
![bg](https://i0.wp.com/www.throwcase.com/wp-content/uploads/2014/12/monkeys-e1419187612146.jpg?fit=1000%2C692&ssl=1)
<!-- Image Credit: Throwcase's Net Madness section - https://www.throwcase.com/2014/12/21/that-five-monkeys-and-a-banana-story-is-rubbish/ -->

---

## Le Conditionnement
<!-- 
Conditionnement classique vs opérant 
- Classique: liaison Stimuli-Stimuli 

Pavlov: nourriture -> salive [Stimulus Inconditionnel -> Réponse Inconditionnelle]
Avant la nourriture, on active une cloche [Stimulus Neutre], le chien écoute (Réaction)
A force, face à la cloche [Stimulus Conditionnel], il salive [Réponse Conditionnelle], avant même de voir de la nourriture : on a lié un Stimulus neutre à son Stimulus Inconditionnel initial

- Conditionnement opérant, "essais et erreurs": liaison Stimulus-Réponse (ou encore Stimulus-Réponse-Conséquence)
- Assis, s'assoie, caresse
- Couché, s'assoie, reçoit rien

- Cage de Skinner : Un rat est placé dans une cage de Skinner dans laquelle se trouve un levier actionnant un mécanisme qui fait entrer dans la cage des croquettes.

Le rat effectue au hasard toutes sortes d'opérations sur les objets qui se trouvent dans la cage, dont le levier.
Après qu'il a actionné quelques fois le levier, ce qui est suivi par la présentation de croquettes, il ne fait plus d'erreurs : la pesée sur le levier est devenue la réponse correcte.

Skinner:
-->

---

## C'est quoi apprendre ?
#### voir...
- [WP: Learning](https://en.wikipedia.org/wiki/Learning)
- [WP: _Machine_ Learning](https://en.wikipedia.org/wiki/Machine_Learning)
- [WP: Operant Conditioning](https://en.wikipedia.org/wiki/Operant_conditioning)

---
### Conditionnement 
## Opérant
![bg right 99%](./img/01-operant.png)

---
<!-- 
_color: black
_backgroundColor: white 
-->
### Reinforcement
##  Learning

![bg right 90%](./img/01-reinforcement.png)
<!-- Image Credit: [Megajuice](https://en.wikipedia.org/wiki/Reinforcement_learning#/media/File:Reinforcement_learning_diagram.svg) -->

---
### Le par-coeur, un problème ? 

<!-- Pour passer un exam ? -->

<!-- Et si c'est l'exam du permis ? Apprendre par coeur les commandes du parcours précédent ? Apprendre des règles _généralisées_ ?  -->

---
### Guessing the teachers password 
<!-- 
Proper beliefs pay rent by functioning as anticipation-controllers; 
a proper belief is not just a string of words, but a cognitive structure fully integrated with the believer's model of how the world simply is. 
Unfortunately, many students in traditional schools become accustomed to guessing the teacher's password, repeating back words that make it superficially sound as if they understand the topic, when in fact they don't.
 -->

<br />
<br />
<br />

- Voir [LessWrong](https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-the-teacher-s-password)
---
## Apprendre 
## c'est oublier


![bg right 90%](./img/01-information.png)
<!-- Image(s) Credit: https://towardsdatascience.com/information-theory-of-neural-networks-ad4053f8e177 -->

<!-- 
Initially,weights are randomly initialised. 
So barely anything is known about the correct output. 

With successive layers the mutual information about input decreases and the information in hidden layers about the output is low as well.
 -->
---
## Apprendre 
## c'est oublier

![bg right 90%](./img/01-information2.png)

<!-- 
As we train the Neural Network the plots start moving up, signifying gain of information about the output.

But.

Plots also start shifting towards the right side, signifying increase of information in latter layers about the input.
 -->
---
## Apprendre 
### c'est 
###### (pas tout) 
#### oublier


- ###### [[Towards Data Science] _Information theory of Neural Networks_](https://towardsdatascience.com/information-theory-of-neural-networks-ad4053f8e177)
![bg right 90%](./img/01-information3.png)

<!-- 

After the compression phase, the plots starts shifting towards the top but also to left side.

This signifies, with successive layers, there is loss of information about the input and what’s retained in the last layer is the lowest entropy information about the output.
-->

---
Comment je sais ce que je sais pas ? 

<!-- Your strength as a rationalist is to be more surprised by fiction than reality. If your mental model can explain _anything_, it's simply useless. -->

<!-- Train/Test split : je traine sur l'un, je teste sur l'autre -->
<!-- Puis je fine-tune, mais du coup je vais peut-être finir par apprendre TEst non ? -->

<!-- Comment faire un train/test split et garder une metrique finale de **validation** ? -->

---

## Underfitting
![bg right 80%](https://upload.wikimedia.org/wikipedia/commons/5/55/Underfitted_Model.png)
<!-- Image by User:AAstein: https://upload.wikimedia.org/wikipedia/commons/5/55/Underfitted_Model.png -->

---

## Overfitting
![bg right 80%](https://upload.wikimedia.org/wikipedia/commons/thumb/1/19/Overfitting.svg/1920px-Overfitting.svg.png)
<!-- Image by User:Chabacano: https://commons.wikimedia.org/wiki/File:Overfitting.svg -->



---
# Failures of ML 

---
### MIT COVID FAILS 
![bg right 100%](./img/01-covid-failed.png)

- Frankenstein data sets <!--  Driggs highlights the problem of what he calls Frankenstein data sets, which are spliced together from multiple sources and can contain duplicates. This means that some tools end up being tested on the same data they were trained on, making them appear more accurate than they are. -->

- Enfant == sain ? <!-- Many unwittingly used a data set that contained chest scans of children who did not have covid as their examples of what non-covid cases looked like. But as a result, the AIs learned to identify kids, not covid. -->

- Allongé == malade ? <!-- Driggs’s group trained its own model using a data set that contained a mix of scans taken when patients were lying down and standing up. Because patients scanned while lying down were more likely to be seriously ill, the AI learned wrongly to predict serious covid risk from a person’s position. -->

- _Confounding factors_ <!-- In yet other cases, some AIs were found to be picking up on the text font that certain hospitals used to label the scans. As a result, fonts from hospitals with more serious caseloads became predictors of covid risk. -->


---
### COVID FAILS: How to fix it? 
![bg right 100%](./img/01-covid-failed.png)

- ###### Better data would help, but <!-- but in times of crisis that’s a big ask. It’s more important to make the most of the data sets we have. The simplest move would be for AI teams to **collaborate more with clinicians** -->

- ###### "Researchers need to **share their models** and disclose **how they were trained**" <!--  so that others can test them and build on them. “Those are two things we could do today,” he says. “And they would solve maybe 50% of the issues that we identified.” -->

- ###### Standardized formats

- ###### Not-invented-here syndrome 
<!-- most researchers rushed to develop their own models, rather than working together or improving existing ones. The result was that the collective effort of researchers around the world produced hundreds of mediocre tools, rather than a handful of properly trained and tested ones.

REPLICATION CRISIS! Probleme commun dans toute la science, incentives, game theory...


“The models are so similar—they almost all use the same techniques with minor tweaks, the same inputs—and they all make the same mistakes,” says Wynants. “If all these people making new models instead tested models that were already available, maybe we’d have something that could really help in the clinic by now.”
 -->

---
 <!-- 
 “Until we buy into the idea, we’re doomed to repeat the same mistakes”
 -->
>  we need to sort out the 
> _unsexy_ problems 
> **before the sexy ones**

<!-- Publish or Perish, ça vous parle ? -->
<!-- Replication Crisis ? -->

--- 

Generative models

https://thisxdoesnotexist.com/

---

# FaceApp

![](https://cdn-images-1.medium.com/max/1024/1*XBEpvGfjv_xo7ebBYNVDNA.png)
<!-- Credit: https://laptrinhx.com/faceapp-or-how-i-learned-to-stop-worrying-and-love-the-machines-2993489094/ -->


--- 

# GANs

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

![bg 80%](https://www.pianshen.com/images/242/e94f2e3824d178f110112d865fbda65a.png)


---


GALACTICA?
https://galactica.org/explore/

[Try it live ;)](https://huggingface.co/spaces/morenolq/galactica-base)

![bg right 80%](./img/00-tweet-lecun-hyped.png)


---

[GALACTICA failed](https://www.technologyreview.com/2022/11/18/1063487/meta-large-language-model-ai-only-survived-three-days-gpt-3-science/
)
![bg right](img/00-galactica-failed.png)

<br />

> Like all language models, Galactica is a **mindless bot** that _cannot tell fact from fiction_. 

[See Research _Paper_](https://arxiv.org/abs/2211.09085)


---

Demos
https://huggingface.co/spaces?sort=modified&search=galactica

---


> Narrator voice: LMs have no access to "truth", or any kind of "information" beyond information about the distribution of word forms in their training data.

[Emily M. Bender, Faculty Director, Computational Linguistics](https://twitter.com/emilymbender/status/1592993757498331136)

---

> In all cases, it was wrong or biased but sounded right and authoritative. I think it's dangerous. 
[~ Michael Black](https://twitter.com/Michael_J_Black/status/1593133722316189696)

---

<!-- Vous savez comment un scientifique réagit quand on lui dit que son projet est dangereux ? -->

[LeCun not happy](https://twitter.com/ylecun/status/1593293058174500865)
![bg right 100%](img/01-tweet-lecun-not-happy.png)


---

# ChatGPT

![bg right 100%](./img/00-chatgpt.png)
- [Blog post](https://openai.com/blog/chatgpt/)
---

ChatGPT: 
<!-- 
Linda is 31 years old, single, outspoken, and very bright. 
She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which is more probable? 

- Linda is a bank teller. 
- Linda is a bank teller and is active in the feminist movement. 
 -->
[Linda Problem](https://twitter.com/dggoldst/status/1598317411698089984)

---

<!-- But maybe assumption? Language ambiguous! What's implicit in this text? -->
[Assumptions?](https://twitter.com/dggoldst/status/1598737445780164635)
<!-- Humans learn mistakes because of BIASES which are MOSTLY USEFUL
Machines which learn alike will by default exhibit the same tendencies!
 -->

---

<!-- Fast and slow: Baseball bat & ball -->
[Computing, Fast & Slow](https://twitter.com/stefanmherzog/status/1598397009161060359)

---
<!-- Which is more likely? A 30/50 event or a 3/5 event? -->

[Math is hard tho](https://twitter.com/batwood011/status/1598389323979800576/photo/1)

---
<!-- Would you rather a 100% sure million, or 1 BILLION probability 0.99? -->
[Risk preferences](https://twitter.com/mdahardy/status/1598139462122622976/photo/1)

---
<!-- What's more likely? Canceled Flight or Canceled flight due to upcoming snowstorm? -->
[Conjuction fallacy](https://twitter.com/mdahardy/status/1598139470641262592/photo/1)

---
<!-- What's something most people know that you wouldn't know? -->
[Meta-cognition](https://twitter.com/mdahardy/status/1598139475451817984)

---

<!-- Are windows placed near plants, or plants near windows? -->
[Physical Reasoning: Windows](https://twitter.com/mdahardy/status/1598139488873897984/photo/1)
- Yet [ordering lol](https://twitter.com/mdahardy/status/1598139493856743424)

---

<!-- Could you fit 100 big macs in an average car? -->
[Physical Reasoning: Big Macs](https://twitter.com/mdahardy/status/1598139498357231616/photo/1)

---
<!-- Who would win in a fight: Napoleon's army, or a hippopotamus? -->
[Physical Reasoning: Napoleon](https://twitter.com/mdahardy/status/1598139507559518208)


---
- See more: [AI Alignment Forum | Cognitive Biases in Large Language Models](https://www.alignmentforum.org/posts/fFF3G4W8FbXigS4gr/cognitive-biases-in-large-language-models)

---

TAY

![bg right fit vertical](https://mmp483.p3cdn1.secureserver.net/wp-content/uploads/2016/03/Tay-AI-Feature-Image-03232016.jpg?time=1670290595)
![bg right fit vertical invert](https://mmp483.p3cdn1.secureserver.net/wp-content/uploads/2016/03/Tay-AI-Feature-Image-03232016.jpg?time=1670290595)

-> Interface matters!

---

### Gaming the game: IAs flemmardes

<br />

- [Régalez vous <br /> avec ce tableau](https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vRPiprOaC3HsCf5Tuum8bRfzYUiKLRqJmbOoC-32JorNdfyTiRRsR7Ea5eWtvsWzuxo8bjOxCG84dAg/pubhtml)

![bg right 90%](./img/01-cheater.png)

---
### Short-term goals vs long-term goals 
<!-- What did these robots fail to learn? -->

<!-- What is Netflix optimizing for? -->

<!-- What is the risk of a 'devilishly-good' add to cart automated A/B testing? -->

---
<!-- 
_color: black
 -->

### Dangerous goals :
### L'histoire du _Paperclip maximizer_ 

- [Try it yourself](https://www.decisionproblem.com/paperclips/index2.html) with a Clicker game :P

<!-- 
First described by Nick Bostrom (2003). -->

![bg crop blur](./img/00-paperclips.png)

---

> ##### _Rare photographie d'un humain après déploiement du Paperclip Maximizer_

<!-- Utilitronium: if your reward function is basic, universe becomes grey goo -->

<!-- Also, Nozick experience machine! -->


![bg right crop](https://www.sunnyskyz.com/uploads/2020/05/tqz1n-clippy-office-prank-2.jpg)
<!-- Image credit: https://www.sunnyskyz.com/blog/3056/Employee-Pranks-Entire-Office-With-An-Army-Of-Clippy-Posts -->


---

# C'est quoi du ML ? 

---
Un thermostat ? 

![bg fit right 90%](https://www.simulace.info/images/NegativeF.jpg)
<!-- Image credit: University of Prague - Simulace.info -->

---
ET un sac de ifs?

![bg right fit](https://upload.wikimedia.org/wikipedia/commons/e/eb/Decision_Tree.jpg)

<!-- Image credit: User:Gilgoldm

A tree showing survival of passengers on the Titanic ("sibsp" is the number of spouses or siblings aboard). The figures under the leaves show the probability of survival and the percentage of observations in the leaf. Summarizing: Your chances of survival were good if you were (i) a female or (ii) a male at most 9.5 years old with strictly fewer than 3 siblings.

 -->
---
Amazon mechanical turk

<!-- The Turk also had the ability to converse with spectators using a letter board. The operator, whose identity during the period when Kempelen presented the machine at Schönbrunn Palace is unknown,[23] was able to do this in English, French, and German. Carl Friedrich Hindenburg, a university mathematician, kept a record of the conversations during the Turk's time in Leipzig and published it in 1789 as Über den Schachspieler des Herrn von Kempelen und dessen Nachbildung (or On the Chessplayer of Mr. von Kempelen And Its Replica). Topics of questions put to and answered by the Turk included its age, marital status, and its secret workings. -->

<!-- -> What about DeepBlue? About AlphaZero? -->

![bg right fit](https://upload.wikimedia.org/wikipedia/commons/thumb/6/6e/Racknitz_-_The_Turk_3.jpg/1920px-Racknitz_-_The_Turk_3.jpg)
<!-- Image credit: Joseph Racknitz - Humboldt University Library -->


---
[Eliza](http://psych.fullerton.edu/mbirnbaum/psych101/Eliza.htm)? Turing test ? 


<!-- Est-ce du ML si je le confonds avec un humain ? Est-ce que ML == "capacite humaine apprise?" -->

---
Et [Akinator](https://fr.akinator.com/game) ? 


---
Quelles limites aux possibilités du ML ? 

https://hbr.org/2016/11/what-artificial-intelligence-can-and-cant-do-right-now

---
> If a typical person can do a mental task with 
> **less than one second of thought**, 
> we can probably automate it using AI 
> either now or in the near future.

~ Andrew Ng, [What AI can and can't do](https://hbr.org/2016/11/what-artificial-intelligence-can-and-cant-do-right-now)

---
## Formalisme: Features

$$ X \Rightarrow Y $$

<!-- C'est quoi une feature?
Le Feature Engineering, tout un art
 -->


<br />
<br />
<br />

Voir [_A broad introduction to Feature Engineering_](https://medium.com/geekculture/a-broad-introduction-to-feature-engineering-ab27a9636f8a) 

---

## Métriques 

---
<!-- 
backgroundColor: white
color: black 
-->

## Precision & Recall
![bg right:40% 90%](./img/01-metrics.png)


---
Précision

![bg right:40% 90%](./img/01-precision.png)

<!-- Ça veut dire quoi une bonne précision ? -->

<!-- Est-ce qu'on peut être "trop précis" ? -->

---
Rappel (_Recall_ )

![bg right:40% 90%](./img/01-recall.png)

<!-- Quid d'un bon rappel? -->

<!-- Quid de "trop capturer" ? -->

<!-- Tradeoff! Idealement les deux -->

<!-- Depend du coup des erreurs:

- Labo pharma: faux pos? ok, faux neg? terrible
- Justice: faux neg? ok, faux pos? terrible
 -->
---

## Autres métriques ?

<br />
<br />
<br />

- Voir [[TDS] 20 Popular Metrics](https://towardsdatascience.com/20-popular-machine-learning-metrics-part-1-classification-regression-evaluation-metrics-1ca3e282a2ce)
<!--
    Classification Metrics (accuracy, precision, recall, F1-score, ROC, AUC, …)
    Regression Metrics (MSE, MAE)
    Ranking Metrics (MRR, DCG, NDCG)
    Statistical Metrics (Correlation)
    Computer Vision Metrics (PSNR, SSIM, IoU)
    NLP Metrics (Perplexity, BLEU score)
    Deep Learning Related Metrics (Inception score, Frechet Inception distance) 
 -->
---

Supervisé ou non?

[Jeu]

<!-- Supervisé : trouvez la règle de mon jeu d'Eleusis (variante triplets de nombres.) -->

<!-- Non supervisé : clustering de genres de musiques -->

---
<!-- 
backgroundColor: black
color: white
-->

Exemples d'algos supervisés

---

- Linear Regression

- Decision trees

- Nearest Neighbor

---
Exemples d'algos non-supervisés

---
- Clustering, e.g. **K-means**
- Anomaly Detection
<!-- To detect anomalies, we have observations x1,. . . , xn ∈ X. The underlying presumption is, most of the data come from the same (unknown) distribution. We call it  normalization in data. 
However, some observations come from a different distribution. They are considered anomalies. Several reasons can lead to these anomalies. 
The final task is to identify these anomalies by observing a concise description of the standard data so that divergent observations become outliers. -->

- Principal Component Analysis
<!-- Principal Component Analysis is an unsupervised learning algorithm. We use it for dimensionality reduction in machine learning. 

A statistical approach transforms the observations of correlated features into a collection of linearly uncorrelated components using orthogonal transformation. 

These new transformed features are known as the Principal Components. It is one of the most popular machine learning algorithms.

 PCA is used for exploratory data analysis and predictive modeling. It is a way to identify hidden patterns from the given dataset by lessening the variances. It follows a feature extraction technique. 

PCA usually tries to express the lower-dimensional surface to project the high-dimensional data. PCA determines the variance of each feature. 

The feature with high variance shows the excellent split between the classes and hence reduces the dimensionality.  -->

---
<!-- 
backgroundColor: black
color: white
-->
## Pionniers & Penseurs

---
### Turing
![bg right](https://1.bp.blogspot.com/-jqU6NCD5dVU/WBVezz7LZWI/AAAAAAAADkY/t7hQMwMuCZIXxgPl0ymB2M9WpvkPw3hvQCLcB/s200/Alan%2BTuring.jpg)
<!-- Turing test: Mécompréhensions


Against the other lines of thought, Turing provides a little “viva voce” that is intended to illustrate the kind of evidence that he supposes one might have that a machine is intelligent. 
Given the right kinds of responses from the machine, we would naturally interpret its utterances as evidence of pleasure, grief, warmth, misery, anger, depression, etc. 
Perhaps—though Turing doesn’t say this—the only way to make a machine of this kind would be to equip it with sensors, affective states, etc., i.e., in effect, to make an artificial person. However, the important point is that if the claims about self-consciousness, desires, emotions, etc. are right, then Turing can accept these claims with equanimity: his claim is then that a machine with a digital computing “brain” can have the full range of mental states that can be enjoyed by adult human beings.
 -->
<!-- 
 Also, Searles' chinese room experiment:

Searle imagines himself alone in a room following a computer program for responding to Chinese characters slipped under the door. Searle understands nothing of Chinese, and yet, by following the program for manipulating symbols and numerals just as a computer does, he sends appropriate strings of Chinese characters back out under the door, and this leads those outside to mistakenly suppose there is a Chinese speaker in the room.
The narrow conclusion of the argument is that programming a digital computer may make it appear to understand language but could not produce real understanding. Hence the “Turing Test” is inadequate. Searle argues that the thought experiment underscores the fact that computers merely use syntactic rules to manipulate symbol strings, but have no understanding of meaning or semantics.
https://plato.stanford.edu/entries/chinese-room/
His conclusion seems somewhat wrong to me as computer programs have no representation of meaning is not always true, but in our specific case of Language Models he was really on point :not_bad: -->

- Turing test
-> [Coffee test](https://www.fastcompany.com/1568187/wozniak-could-computer-make-cup-coffee)? 
<!-- Steve Wozniak: for general AI, Turing's test is outdated -->
---
### AI Summer: Dartmouth workshop

> _1956' Dartmouth Summer Research Project on Artificial Intelligence_

<!-- 
In 1956, the summer workshop of Darmouth was held with prestigious participants (Minsky, Shannon, McCarthy, etc.). 
This event marked the starting point of Artificial Intelligence as a field on its own. 
The hopes were high: beating a chess champion, prove math theorems, replace human’s works. Following this meeting, Rosenblatt created the perceptron in 1958, a very simplified model of a biological neuron that aimed to classify images. Yes, “artificial neural networks” and “computer vision” aren’t new.

Despite these initial progresses, the field suffered from severe cutbacks in funding leading to the first AI winter in the 70s.
The expectation were too great compared to what was achieved. 
This phenomenon occurred again in the 80s. 

At this point, practical applications came mainly from other approaches than neural networks.
 -->

---

### Perceptron

![](https://miro.medium.com/max/1100/1*v88ySSMr7JLaIBjwr4chTw.webp)
 <!-- Image credit: https://towardsdatascience.com/multi-layer-neural-networks-with-sigmoid-function-deep-learning-for-rookies-2-bf464f09eb7f?gi=15a9a7836230 -->

 ---
<!-- _backgroundColor:  -->
![bg fit 50%](https://upload.wikimedia.org/wikipedia/commons/8/8a/Perceptron_example.svg)
<!--  A diagram showing a perceptron updating its linear boundary as more training examples are added. -->
 ---

<!-- _backgroundColor: 
_color: -->
## Multi-Layer Perceptron

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

![bg fit 80%](https://miro.medium.com/max/1100/1*CJEBy3GCaGQKNx7PEy-w5w.webp)

 ---
<!-- color: white -->
## Loss Function

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

![bg fit 70%](https://arthurdouillard.com/deepcourse/introduction/lossfunction.webp)
<!-- Image Credit: Arthur Douillard's Deep Learning Course -->

---
### Back to Dartmouth Summer of dreams

<!-- The study is to proceed on the basis of the conjecture that 
every aspect of learning or any other feature of intelligence can in principle 
be so precisely described 
that a machine can be made to simulate it. 

An attempt will be made to find:
- how to make machines use language
- form abstractions and concepts
- solve kinds of problems now reserved for humans
- and improve themselves. 


-->

> We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.

---

<!-- _backgroundColor: 
_color:
 -->
### (Gartner Hype Cycle)

<br />

######         ☝ 🔺🔺🔺🔺🔺🔺🔺🔺🔺      We're here

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

![bg fit 60%](https://upload.wikimedia.org/wikipedia/commons/thumb/9/94/Gartner_Hype_Cycle.svg/1920px-Gartner_Hype_Cycle.svg.png)


---
### AI Winter: 1984 et autres déceptions

# ❄🤖❄

<!-- . The AI winter was a result of such hype, due to over-inflated promises by developers, unnaturally high expectations from end-users, and extensive promotion in the media -->

<!-- Even today:
- AI researcher Rodney Brooks would complain in 2002 that "there's this stupid myth out there that AI has failed, but AI is around you every second of the day."[4] 
- In 2005, Ray Kurzweil agreed: "Many observers still think that the AI winter was the end of the story and that nothing since has come of the AI field. Yet today many thousands of AI applications are deeply embedded in the infrastructure of every industry."
 -->

 <!-- Enthusiasm and optimism about AI has generally increased since its low point in the early 1990s. Beginning about 2012, interest in artificial intelligence (and especially the sub-field of machine learning) from the research and corporate communities led to a dramatic increase in funding and investment.  -->
---
### Yann LeCun

<!-- Thèse sur la BackPropagation
Père du Deep LEarning

Yann Le Cun est diplômé de l'ESIEE Paris en 1983, il part ensuite à l'université Pierre-et-Marie-Curie effectuer un DEA puis un doctorat qu'il obtient en 1987. 
Il s'oriente rapidement vers la recherche sur l'apprentissage automatique et il propose pendant sa thèse une variante de l'algorithme de rétropropagation du gradient, qui permet depuis le début des années 1980 l'apprentissage des réseaux de neurones. 

Il réalise son post-doctorat au sein de l'équipe de Geoffrey Hinton.

Yann Le Cun travaille depuis les années 1980 sur l’apprentissage automatique (machine learning) et l’apprentissage profond (deep learning) : la capacité d’un ordinateur à reconnaître des représentations (images, textes, vidéos, sons) à force de les lui montrer, de très nombreuses fois.

En 1987, Yann Le Cun rejoint l'Université de Toronto et en 1988 les laboratoires AT&T, pour lesquels il développe des méthodes d'apprentissage supervisé6.

Yann Le Cun s'intéresse ensuite à la conception des algorithmes de compression du format d'archivage DjVu7 puis la reconnaissance automatique de chèques bancaires.

Yann Le Cun est professeur à l'université de New York où il a créé le Center for Data Sciences. Il travaille notamment au développement technologique des voitures autonomes.

Le 9 décembre 2013, Yann Le Cun est invité par Mark Zuckerberg à rejoindre Facebook pour créer et diriger le laboratoire d'intelligence artificielle FAIR (« Facebook Artificial Intelligence Research ») à New York, Menlo Park et depuis 2015 à Paris, notamment pour travailler sur la reconnaissance d'images et de vidéos. Il avait précédemment refusé une proposition similaire de la part de Google.

En 2016, Yann Le Cun est le titulaire pour l'année de la chaire « Informatique et sciences numériques » du Collège de France9.

En janvier 2018, Yann Le Cun quitte son poste de chef de division en recherche sur l'intelligence artificielle chez Facebook, au profit de Jérôme Pesenti, pour occuper un poste de chercheur en tant que scientifique en chef de l'IA toujours chez Facebook. 
 -->
![bg right ](https://www.controcorrenteblog.com/wp-content/uploads/2015/09/Yann-LeCun-1.jpg)

- [BackPropagation](http://yann.lecun.com/exdb/publis/pdf/lecun-89e.pdf)
- [Deep Learning](https://www.historyofdatascience.com/yann-lecun/)

---

![bg](./img/01-lenet.gif)

---

## LeNet Architecture
![](./img/01-lenet.jpg)

---
### Geoffrey Hinton
![bg right](https://upload.wikimedia.org/wikipedia/commons/thumb/3/34/Geoffrey_Hinton_at_UBC.jpg/1024px-Geoffrey_Hinton_at_UBC.jpg)

<!-- 
Hinton was co-author of a highly cited paper published in 1986 that popularized the backpropagation algorithm for training multi-layer neural networks, although they were not the first to propose the approach.
Hinton is viewed as a leading figure in the deep learning community.
The dramatic image-recognition milestone of the AlexNet designed in collaboration with his students Alex Krizhevsky and Ilya Sutskever for the ImageNet challenge 2012 was a breakthrough in the field of computer vision.

Hinton received the 2018 Turing Award, together with Yoshua Bengio and Yann LeCun, for their work on deep learning.[24] They are sometimes referred to as the "Godfathers of AI" and "Godfathers of Deep Learning",[25][26] and have continued to give public talks together.[27][28]  -->


---
<!-- _backgroundColor: white
_color: black
_footer: '' -->
![bg 60%](https://upload.wikimedia.org/wikipedia/commons/thumb/c/cc/Comparison_image_neural_networks.svg/1024px-Comparison_image_neural_networks.svg.png)

<!-- AlexNet contained eight layers; the first five were convolutional layers, some of them followed by max-pooling layers, and the last three were fully connected layers.It used the non-saturating ReLU activation function, which showed improved training performance over tanh and sigmoid. -->

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

#### LeNet vs AlexNet | [Learn the difference](https://programmathically.com/deep-learning-architectures-for-image-classification-lenet-vs-alexnet-vs-vgg/)

---

### Andrew Ng

![bg fit right](https://upload.wikimedia.org/wikipedia/commons/a/a3/Andrew_Ng_WSJ_%282%29.jpg)

- deeplearning.ai

- "AI Optimist"

<!-- Ng was a co-founder and head of Google Brain and was the former chief scientist at Baidu, building the company's Artificial Intelligence Group into a team of several thousand people. 

Ng is an adjunct professor at Stanford University (formerly associate professor and Director of its Stanford AI Lab or SAIL). Ng has also made substantial contributions to the field of online education as the co-founder of both Coursera and deeplearning.ai

 Ng believes that AI technology will improve the lives of people, not that it is an anathema that will "enslave" the human race.[3] Ng believes the potential benefits of AI outweigh the threats, which he believes are exaggerated.[63] He has stated that

    Worrying about AI evil superintelligence today is like worrying about overpopulation on the planet Mars. We haven't even landed on the planet yet![63]

A real threat is regarding the future of work: "Rather than being distracted by evil killer robots, the challenge to labour caused by these machines is a conversation that academia and industry and government should have."[64] A particular goal of Ng's work is to "democratize" AI learning so that people can learn more about it and understand its benefits. Ng's stance on AI is shared by Mark Zuckerberg, but opposed by Elon Musk.[64]

In 2017, Ng said he supported basic income to allow the unemployed to study AI so that they can re-enter the workforce. He has stated that he enjoyed Erik Brynjolfsson and Andrew McAfee's "The Second Machine Age" which discusses issues such as AI displacement of jobs.[65] 

-->
---
### Eliezer Yudkowsky


- ###### [HELL OF AN] AI PESSIMIST
- [Machine Intelligence Research Institute _[intelligence.org]_](https://intelligence.org/research/)
- [LessWrong](lesswrong.com) & [Overcoming Bias](overcomingbias.com)
- AI Box Experiments
![bg right](https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2Fb1f9d45e-db99-41a5-b38b-c824fb9ef481_1258x1078.jpeg)

---
<!-- 
_backgroundColor: white
_color: #111 
_footer: ""
-->

![bg 100%](./img/01-miri.png)

---
<!-- 
_backgroundColor: white
_color: #111 
_footer: ""
-->

## LessWrong <3
- #### [Sequences on Rationality](https://www.lesswrong.com/rationality)
    - ###### [Guessing the Teacher's Password](https://www.lesswrong.com/s/5uZQHpecjn7955faL/p/NMoLJuDJEms7Ku9XS)
    - ###### ["Science" as a Curiosity-Stopper](https://www.lesswrong.com/posts/L22jhyY9ocXQNLqyE/science-as-curiosity-stopper)
    - ###### [Say no to Complexity](https://www.lesswrong.com/s/5uZQHpecjn7955faL/p/kpRSCH7ALLcb6ucWM)
    - ###### [Making Beliefs pay Rent](https://www.lesswrong.com/s/7gRSERQZbqTuLX5re/p/a7n8GdKiAZRX86T5A)

![bg right 120%](./img/01-lesswrong.png)

---

## On résume

<!-- Tour de table : un truc que tu as appris aujourd'hui  -->

<!-- Main levée : un truc qui nous a surpris ? -->

<!-- 
Main levée : qui a une grosse question ? 
Un blind spot dans sa carte mentale ? 
-->

<!-- Préparons la suite : 
Tour de table - c'est quoi le plus important pour vous que vous voulez apprendre des TPs ? -->

---

### Conclusion:
> la route est _longue_, mais la voie est _libre_
![bg right 90%](https://miro.medium.com/max/640/0*JWCLdKhz-0e_77tB.webp)
<!-- Image Credit: Steward Brandt - Whole Earth Catalog -->

<!-- This is the DREAM TIME! Nous sommes la première génération à être limités, non pas par les possibilités techniques, mais par notre imagination 🤩 -->

<!-- "Gros gros potentiel" -->

<!-- -> Grosses grosses responsabilités -->

<!-- Bon, ok, mais en pratique ?

- Comment on choisit un modèle selon le problème à résoudre ?

- Comment on entraine un modèle ?

- Comment on teste sa qualité ? Ses biais ? 

- Comment on peut build on top ? Le déployer ? Collaborer à plusieurs ?

On se retrouve l'année prochaine pour voir tout ça ensemble :)
-->