---
marp: true
theme: uncover
color: #eee
colorSecondary: #333
backgroundColor: #111
paginate: true
transition: wipe
footer: "ML101 par Paul-Louis Nech | Présenté à l'INTECH Info | © 2022-2023"
---

<style>
li  {
  font-size: 0.8em;
}
</style>

# <!-- fit --> Hello World

ROBOTS ARE UPRISING. WHAT SIDE ARE YOU ON?

<!-- 21ème siècle. Les algos sont partout. -->
<!-- L'API divise le monde, scindé en deux catégories : ceux qui l'écrivent et ceux qui la vivent. -->
<!-- Si vous êtes arrivés jusqu'ici, en 5ème année, 
bravo,
car vous êtes du bon coté de la ligne... -->
---

<!-- Plus sérieusement, bienvenue dans mon cours -->
### On va parler de 
# <!-- fit --> Machine Learning
<!-- Vous avez eu l'occasion d'y réfléchir un peu avant ce cours. -->

---

## Mais pas comme ça

<br />

$$ {\displaystyle g(x):=f^{L}(W^{L}f^{L-1}(W^{L-1}\cdots f^{1}(W^{1}x)\cdots ))} $$
<!-- Le projet, c'est pas que vous compreniez tous les maths derrière -->
<br />

#### (Le moins possible)
<!-- Parfois c'est nécessaire pour saisir la beauté d'un modèle -->
---

## Ni comme ça
<!-- Les dangers, un sujet pour un cours entier -->

<!-- 
> les tergiversations sur les inquiétudes hollywoodiennes du grand public
 -->

<!-- > Réponse d'un élève :  -->
> _À part des films de SF, j'ai pas d'idée_


<!-- Biais, feedback loop foireuses, Risque existentiel?... -->
![bg right](https://uproxx.com/wp-content/uploads/2015/05/angry-bender.jpg?w=650)
<!-- Image Credit: Bender from Futurama, created by Matt Groening and David X. Cohen -->

- [the Fallacy of generalization from Fictional evidence](https://www.lesswrong.com/posts/rHBdcHGLJ7KvLJQPk/the-logical-fallacy-of-generalization-from-fictional)
<!-- The more details, the less likely! -->
--- 

## Encore que...

![bg right grayscale](https://uproxx.com/wp-content/uploads/2015/05/angry-bender.jpg?w=650)

- [AGI Ruin: A List of Lethalities](https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities)

--- 

## On va parler des derniers succès

- [Meta AI Research: CICERO](https://ai.facebook.com/blog/cicero-ai-negotiates-persuades-and-cooperates-with-people/)
<!-- Mais aussi:
 - AlphaGo / AlphaZero
 - 
-->
![bg left](./img/01-cicero.png)

--- 

## Et d'échecs intéressants

- [MIT Tech review:  100s of AI tools built for covid... None helped.](https://www.technologyreview.com/2021/07/30/1030329/machine-learning-ai-failed-covid-hospital-diagnosis-pandemic/)

![bg left 100%](./img/01-covid-failed.png)

----

## Mais aussi de révolutions en cours


- [Midjourney](https://midjourney.com/)

![bg left](./img/01-midjourney.png)
<!-- Image Credit: MidJourney's Showcase -->

---
<!-- _footer: "" -->
![bg 93%](./img/01-diffusion.png)
<!-- Image Credit: Jay Allamar, The illustrated Stable Diffusion -->
<br />
<br />
<br />
<br />
<br />
- [The Illustrated Stable Diffusion by Jay Alamar](https://jalammar.github.io/illustrated-stable-diffusion/)

---
### Comment les modèles... peuvent raconter des histoires 

- [AI Dungeon 2](https://aidungeon.cc/)

![bg left 100%](./img/01-dungeon.jpg)

---
### Comment les modèles... peuvent halluciner 

- [DeepDream](https://en.wikipedia.org/wiki/DeepDream)

- [Pareidolia](https://en.wikipedia.org/wiki/Pareidolia)

![bg left 100%](../tp/img/intech_dream.png)

---
### Comment ils nous battent à plate couture...
![bg right fit](./img/00-alphago-movie.png)

- [AlphaGo](https://www.nature.com/articles/nature16961)

<!-- 
  
Mastering the game of Go with deep neural networks and tree search

    The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.
 -->

---
### ...puis se battent eux-mêmes

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

- [AlphaZero](https://arxiv.org/abs/1712.01815)

<!-- In December 2017, AlphaZero beat the 3-day version of AlphaGo Zero by winning 60 games to 40, and with 8 hours of training it outperformed AlphaGo Lee on an Elo scale. AlphaZero also defeated a top chess program (Stockfish) and a top Shōgi program (Elmo).[7][8] -->

![bg 40%](./img/00-alphazero-learn.png)

<!-- 
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

     The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case. 
 -->

----
### Comment ils nous comprennent
<!-- Nous catégorisent -->
<!-- Percoivent très biens des trucs infimes (analyse morpho, ) -->
<!-- Ratent des trucs énormes (cohérence, spacial reasoning...)) -->
<!-- Bref: sont ALIEN -->

- [OpenAI Jukebox: a neural net that generates music, including rudimentary singing](https://openai.com/blog/jukebox/)

![bg left 100%](./img/01-jukebox.png)

----
## Comment vous pourriez les comprendre
<!-- Comprendre ce qu'ils ont dans la tête -->
<!-- Comprendre comment ils nous analysent -->
<!-- Comprendre comment on les contourne -->

--- 
## Et comment leur faire faire ce que vous voulez
<!-- Les appliquer à votre quotidien -->
<!-- S'en servir intelligemment en entreprise -->
<!-- Entrer dans un nouveau monde d'art et de créativité -->

---
<!--
_backgroundColor: white
_color: black
-->
<br />

## Qui suis-je?

<!-- Vous pouvez me contacter -->

### Paul-Louis Nech
- ###### ✉ etudiants@nech.pl
- ###### 🔗 LinkedIn.com/in/PLNech
- ###### 💡 GitHub.com/PLNech

![bg right grayscale](./img/01-me.jpg)

---

<!--
_backgroundColor: white
_color: black
-->
<br />

## Parcours

-  **EPITA** 2016 | Spé **MTI**
- Software Engineer @**Algolia**
- Senior _ML_ Engineer @**Algolia**
![bg right](./img/01-me2.jpg)

--- 

## Et vous ?

<!-- Anglais OK? -->

<!-- Tutoiement OK ? -->

<!-- Tour de table :
- Prénom
- un genre de musique <3 
- Un plat <3 
-->


---


# Votre expérience du ML ?

---
<!-- 
_backgroundColor: #333
_footer: ""
 -->

![bg fit](./img/00-questionnaire-dev.png)
![bg fit](./img/00-questionnaire-user.png)

---

<!-- 
_backgroundColor: #333
_footer: ""
 -->

![bg fit](./img/00-questionnaire-feel.png)

<!-- Aucun 'ça peut pas marcher' ! On a passé cette phase -->
---
# Le positif

---
#### Le positif

> Dall-E: "_c'est trop bien **à condition de bien formuler** ce que l'on veut_"

---
#### Le positif


> GitHub Copilot: "_arrive parfois à deviner entièrement un paragraphe de code parfois **sans même faire d'efforts** supplémentaire_"

---
### Le positif

> la **rapidité** de suggestions de traduction et la **multitude** de propositions

---

## Le négatif


---
#### Le négatif

> AI-Dungeon: "la complexité parfois mal interprétée"


---
#### Le négatif

> des librairies python à n'en plus finir 

---
#### Le négatif

> Niveau morale c'est pas forcément ouf car ils utilisent du code open source 

---
#### Le négatif

> Ça reprenait vraiment bcp bcp bcp trop les exemples fournis. Les phrases était majoritairement reformulée mais pas nouvelles !

---
#### Le négatif

> Copilot: parfois des suggestions totalement inappropriées 


---
#### Le négatif
> La quantité de donnée à télécharger pour que ce ça deviennent fiable.


---
#### Le négatif

> l'écriture intutive de mon téléphone...


---
#### Le négatif

> le programme perdait vite le cours de la discussion


---
#### Le négatif

> les deepfakes où ils ont utilisé des personnes sans leur consentement dans des vidéos


---
#### Le négatif

> Mon ancienne entreprise voulait tellement "optimiser" le moteur de leurs jeux, qu'ils en ont cassé plus d'un (FDJ).


---
# Espoirs

---
### Espoirs

> la médecine

---
### Espoirs
> Permettre une communication fluide entre personnes de langages différentes, handicapés ou non 


--- 

# Dangers 

---
### Dangers

> les IA en machine learning qui ont été utilisé par de nombreuse personnes sur internet et qui ont été rendu inutile à cause des trolls

---
### Dangers

- [GPT4-chan](https://huggingface.co/ykilcher/gpt-4chan) and the [Ai Gating debate](https://medium.com/geekculture/gpt-4-chan-and-the-ai-gating-debate-41c3eb54ec32) 
- ou [ChatGPT sur Stack Overflow](https://huggingface.co/ykilcher/gpt-4chan)
  > _Temporary policy: ChatGPT is banned_

<!-- En quoi c'est un problème ? -->

![bg right fit](https://thegradient.pub/content/images/size/w1600/2022/06/main-3.png)
<!-- Image credit: https://thegradient.pub/gpt-4chan-lessons/ -->
---
### Dangers
>  par optimiser on peut entendre ajouter des fonctionnalités qui induisent une difficultés d'utilisation du produit à long terme...

---
### Dangers
> La non régression ne marche pas toujours 

---
### Dangers
> Les robots tueurs 
![bg right fit](https://i.ytimg.com/vi/Ofmg-4D5PoA/hqdefault.jpg)

---

<br />
<br />

### Les robots tueurs

<br />

- [Stop Killer Robots Campaign](https://www.stopkillerrobots.org/)

<!-- Technology should be used to empower all people, not to reduce us – to stereotypes, labels, objects, or just a pattern of 1’s and 0’s.

With growing digital dehumanisation, the Stop Killer Robots coalition works to ensure human control in the use of force. Our campaign calls for new international law on autonomy in weapons systems.
 -->

- Trois axes de campagne:
    - `DEHUMANISATION`
    - `A.I. AND RACE`
    - `#KEEPCTRL`

<!-- DEMUNANIZATION: 
From smart homes and targeted advertising to the use of robot dogs by police enforcement, artificial intelligence technologies and automated decision-making are now playing a significant role in our lives. 

Technology can be amazing. But just because we can build something, it doesn’t mean we should. 

Many technologies with varying degrees of autonomy are already being widely rolled out without pausing to consider the consequences of normalising their use. Why do we need to talk about this?

Because machines don’t see us as people, just another piece of code to be processed and sorted.
 -->

<!-- AI AND RACE:

From social media to the use of robot dogs by police enforcement, artificial intelligence technologies and automated decision making are now playing a significant role in our lives.The prejudices in our society live in the algorithms we design, in the data sets we compile and in the labels we prescribe one another.

Emerging technologies like facial and vocal recognition draw on already biased training datasets and often fail in recognising people of colour, persons with disabilities and women, favouring light-skinned and outwardly masculine faces over darker-skinned and outwardly feminine faces.

And while efforts will be made to diversify data sets, this is not just a problem of unrepresentative data.

 -->

<!--  KEEPCTRL:
Autonomy in weapons systems is a profoundly human problem. Killer robots change the relationship between people and technology by handing over life and death decision-making to machines. They challenge human control over the use of force, and where they target people, they dehumanise us – reducing us to data points.

But, technologies are designed and created by people. We have a responsibility to establish boundaries between what is acceptable and what is unacceptable. We have the capacity to do this, to protect our humanity and ensure that the society we live in, that we continue to build, is one in which human life is valued – not quantified.

  -->
<br />
<br />
<br />

![bg right fit](./img/00-killerrobots.png)


---
### Dangers
> _Que le machine learning soit utiliser pour contrôler les humains_
<!-- non pas dans le sens d'un film de science-fiction mais dans le sens ou les personnes aux commandes pourrait fortement influencer les décisions de certaines personnes. Ex: les dernière élections présidentielles au Etats-Unis -->


---
### Dangers
> _TikTok / instagram_ :'( 


---
### Dangers
> _Les pubs qui **s'adaptent** en fonction des recherches Intenrnet des personnes._

---
### Dangers
> _la manipulation des fois non prévu des algorithmes comme la radicalisation de certaines personnes_


---
### Dangers
> _Que les machines contrôlent le monde ?_

---
### Dangers
> CYBERPUNK 2077

![bg right](https://preview.redd.it/fftof4hmu2b71.jpg?width=640&crop=smart&auto=webp&s=238d105bb0ce2441d6e20ad1a203a65cfc153ca2)

---
## C'est quoi ce cours ?

<!-- Plan du cours: -->

<!-- TD: Quali aujourd'hui, plus pratique en janvier -->

<!-- Eval: Par les rendus, pas de piège :) -->

<!-- Format / interactions welcome -->
---
#### Des connaissances
## &
#### Des compétences
![bg right](./img/01-map-center.png)

---
#### Ce que vous saurez
![bg right](./img/01-map-known.png)

---
###### (Et ce que vous saurez... que vous ne savez pas) 
![bg right 99%](./img/01-map-unknown.png)

---
#### Ce que vous saurez faire
![bg right](./img/01-map-practice.png)

---
# Prêts ?
---
# Les bases 

---
## C'est quoi apprendre ?
<!-- 
Behaviorisme: apprendre = savoir _faire_ 
Acquis vs Inné : danse des abeilles vs culture des chimpanzés
Conditionnement opérant (Pavlov et sa cloche)
-->
---
<!-- _footer: "" -->
<!-- 
Acquis vs Inné : danse des abeilles vs culture des chimpanzés
-->

![bg](https://image.beeplaza.com.au/wp-content/uploads/2019/12/bee-dance.jpg)
<!-- Image Credit: BeePlaza Beekeeping Shop; beeplaza.com.au -->


---
<!-- _footer: "" -->
![bg](https://i0.wp.com/www.throwcase.com/wp-content/uploads/2014/12/monkeys-e1419187612146.jpg?fit=1000%2C692&ssl=1)
<!-- Image Credit: Throwcase's Net Madness section - https://www.throwcase.com/2014/12/21/that-five-monkeys-and-a-banana-story-is-rubbish/ -->

---

## Le Conditionnement
<!-- 
Conditionnement classique vs opérant 
- Classique: liaison Stimuli-Stimuli 

Pavlov: nourriture -> salive [Stimulus Inconditionnel -> Réponse Inconditionnelle]
Avant la nourriture, on active une cloche [Stimulus Neutre], le chien écoute (Réaction)
A force, face à la cloche [Stimulus Conditionnel], il salive [Réponse Conditionnelle], avant même de voir de la nourriture : on a lié un Stimulus neutre à son Stimulus Inconditionnel initial

- Conditionnement opérant, "essais et erreurs": liaison Stimulus-Réponse (ou encore Stimulus-Réponse-Conséquence)
- Assis, s'assoie, caresse
- Couché, s'assoie, reçoit rien

- Cage de Skinner : Un rat est placé dans une cage de Skinner dans laquelle se trouve un levier actionnant un mécanisme qui fait entrer dans la cage des croquettes.

Le rat effectue au hasard toutes sortes d'opérations sur les objets qui se trouvent dans la cage, dont le levier.
Après qu'il a actionné quelques fois le levier, ce qui est suivi par la présentation de croquettes, il ne fait plus d'erreurs : la pesée sur le levier est devenue la réponse correcte.

Skinner:
-->

---

## C'est quoi apprendre ?
#### voir...
- [WP: Learning](https://en.wikipedia.org/wiki/Learning)
- [WP: _Machine_ Learning](https://en.wikipedia.org/wiki/Machine_Learning)
- [WP: Operant Conditioning](https://en.wikipedia.org/wiki/Operant_conditioning)

---
### Conditionnement 
## Opérant
![bg right 99%](./img/01-operant.png)

---
<!-- 
_color: black
_backgroundColor: white 
-->
### Reinforcement
##  Learning

![bg right 90%](./img/01-reinforcement.png)
<!-- Image Credit: [Megajuice](https://en.wikipedia.org/wiki/Reinforcement_learning#/media/File:Reinforcement_learning_diagram.svg) -->

---
### Le par-coeur, un problème ? 

<!-- Pour passer un exam ? -->

<!-- Et si c'est l'exam du permis ? Apprendre par coeur les commandes du parcours précédent ? Apprendre des règles _généralisées_ ?  -->

---
### Guessing the teachers password 
<!-- 
Proper beliefs pay rent by functioning as anticipation-controllers; 
a proper belief is not just a string of words, but a cognitive structure fully integrated with the believer's model of how the world simply is. 
Unfortunately, many students in traditional schools become accustomed to guessing the teacher's password, repeating back words that make it superficially sound as if they understand the topic, when in fact they don't.
 -->

<br />
<br />
<br />

- Voir [LessWrong](https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-the-teacher-s-password)
---
## Apprendre 
## c'est oublier


![bg right 90%](./img/01-information.png)
<!-- Image(s) Credit: https://towardsdatascience.com/information-theory-of-neural-networks-ad4053f8e177 -->

<!-- 
Initially,weights are randomly initialised. 
So barely anything is known about the correct output. 

With successive layers the mutual information about input decreases and the information in hidden layers about the output is low as well.
 -->
---
## Apprendre 
## c'est oublier

![bg right 90%](./img/01-information2.png)

<!-- 
As we train the Neural Network the plots start moving up, signifying gain of information about the output.

But.

Plots also start shifting towards the right side, signifying increase of information in latter layers about the input.
 -->
---
## Apprendre 
### c'est 
###### (pas tout) 
#### oublier


- ###### [[Towards Data Science] _Information theory of Neural Networks_](https://towardsdatascience.com/information-theory-of-neural-networks-ad4053f8e177)
![bg right 90%](./img/01-information3.png)

<!-- 

After the compression phase, the plots starts shifting towards the top but also to left side.

This signifies, with successive layers, there is loss of information about the input and what’s retained in the last layer is the lowest entropy information about the output.
-->

---
Comment je sais ce que je sais pas ? 

<!-- Your strength as a rationalist is to be more surprised by fiction than reality. If your mental model can explain _anything_, it's simply useless. -->

<!-- Train/Test split : je traine sur l'un, je teste sur l'autre -->
<!-- Puis je fine-tune, mais du coup je vais peut-être finir par apprendre TEst non ? -->

<!-- Comment faire un train/test split et garder une metrique finale de **validation** ? -->

---

## Underfitting
![bg right 80%](https://upload.wikimedia.org/wikipedia/commons/5/55/Underfitted_Model.png)
<!-- Image by User:AAstein: https://upload.wikimedia.org/wikipedia/commons/5/55/Underfitted_Model.png -->

---

## Overfitting
![bg right 80%](https://upload.wikimedia.org/wikipedia/commons/thumb/1/19/Overfitting.svg/1920px-Overfitting.svg.png)
<!-- Image by User:Chabacano: https://commons.wikimedia.org/wiki/File:Overfitting.svg -->


---
<!-- -> Occam's razor! There's always a more complex rule that matches all observations perfectly.
 -->

## ⟶ Rasoir d'Occam
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

![bg 50%](https://probmods.org/assets/img/Curve_fitting.png)
<!-- Image source: Probabilistic models of Cognition - https://probmods.org/chapters/occams-razor.html -->

---
# Failures of ML 

---
### MIT COVID FAILS 
![bg right 100%](./img/01-covid-failed.png)

- Frankenstein data sets <!--  Driggs highlights the problem of what he calls Frankenstein data sets, which are spliced together from multiple sources and can contain duplicates. This means that some tools end up being tested on the same data they were trained on, making them appear more accurate than they are. -->

- Enfant == sain ? <!-- Many unwittingly used a data set that contained chest scans of children who did not have covid as their examples of what non-covid cases looked like. But as a result, the AIs learned to identify kids, not covid. -->

- Allongé == malade ? <!-- Driggs’s group trained its own model using a data set that contained a mix of scans taken when patients were lying down and standing up. Because patients scanned while lying down were more likely to be seriously ill, the AI learned wrongly to predict serious covid risk from a person’s position. -->

- _Confounding factors_ <!-- In yet other cases, some AIs were found to be picking up on the text font that certain hospitals used to label the scans. As a result, fonts from hospitals with more serious caseloads became predictors of covid risk. -->


---
### COVID FAILS: How to fix it? 
![bg right 100%](./img/01-covid-failed.png)

- ###### Better data would help, but <!-- but in times of crisis that’s a big ask. It’s more important to make the most of the data sets we have. The simplest move would be for AI teams to **collaborate more with clinicians** -->

- ###### "Researchers need to **share their models** and disclose **how they were trained**" <!--  so that others can test them and build on them. “Those are two things we could do today,” he says. “And they would solve maybe 50% of the issues that we identified.” -->

- ###### Standardized formats

- ###### Not-invented-here syndrome 
<!-- most researchers rushed to develop their own models, rather than working together or improving existing ones. The result was that the collective effort of researchers around the world produced hundreds of mediocre tools, rather than a handful of properly trained and tested ones.

REPLICATION CRISIS! Probleme commun dans toute la science, incentives, game theory...


“The models are so similar—they almost all use the same techniques with minor tweaks, the same inputs—and they all make the same mistakes,” says Wynants. “If all these people making new models instead tested models that were already available, maybe we’d have something that could really help in the clinic by now.”
 -->

---
 <!-- 
 “Until we buy into the idea, we’re doomed to repeat the same mistakes”
 -->
>  we need to sort out the 
> _unsexy_ problems 
> **before the sexy ones**

<!-- Publish or Perish, ça vous parle ? -->
<!-- Replication Crisis ? -->

--- 

# Generative models

https://thisxdoesnotexist.com/

---

## FaceApp

![](https://cdn-images-1.medium.com/max/1024/1*XBEpvGfjv_xo7ebBYNVDNA.png)
<!-- Credit: https://laptrinhx.com/faceapp-or-how-i-learned-to-stop-worrying-and-love-the-machines-2993489094/ -->


--- 

## GANs

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

![bg 80%](https://www.pianshen.com/images/242/e94f2e3824d178f110112d865fbda65a.png)


---


## GALACTICA
https://galactica.org/explore/

[Try it live ;)](https://huggingface.co/spaces/morenolq/galactica-base)

![bg right 80%](./img/00-tweet-lecun-hyped.png)


---

[GALACTICA failed](https://www.technologyreview.com/2022/11/18/1063487/meta-large-language-model-ai-only-survived-three-days-gpt-3-science/
)
![bg right](img/00-galactica-failed.png)

<br />

> Like all language models, Galactica is a **mindless bot** that _cannot tell fact from fiction_. 

[See Research _Paper_](https://arxiv.org/abs/2211.09085)


---

Demos
https://huggingface.co/spaces?sort=modified&search=galactica

---


> Narrator voice: LMs have no access to "truth", or any kind of "information" beyond information about the distribution of word forms in their training data.

[Emily M. Bender, Faculty Director, Computational Linguistics](https://twitter.com/emilymbender/status/1592993757498331136)

---

> In all cases, it was wrong or biased but sounded right and authoritative. I think it's dangerous. 
[~ Michael Black](https://twitter.com/Michael_J_Black/status/1593133722316189696)

---

<!-- Vous savez comment un scientifique réagit quand on lui dit que son projet est dangereux ? -->

[LeCun not happy](https://twitter.com/ylecun/status/1593293058174500865)
![bg right 100%](img/01-tweet-lecun-not-happy.png)


---

# ChatGPT

![bg right 100%](./img/00-chatgpt.png)
- [Blog post](https://openai.com/blog/chatgpt/)
---

ChatGPT: 
<!-- 
Linda is 31 years old, single, outspoken, and very bright. 
She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which is more probable? 

- Linda is a bank teller. 
- Linda is a bank teller and is active in the feminist movement. 
 -->
[Linda Problem](https://twitter.com/dggoldst/status/1598317411698089984)

---

<!-- But maybe assumption? Language ambiguous! What's implicit in this text? -->
[Assumptions?](https://twitter.com/dggoldst/status/1598737445780164635)
<!-- Humans learn mistakes because of BIASES which are MOSTLY USEFUL
Machines which learn alike will by default exhibit the same tendencies!
 -->

---

<!-- Fast and slow: Baseball bat & ball -->
[Computing, Fast & Slow](https://twitter.com/stefanmherzog/status/1598397009161060359)

---
<!-- Which is more likely? A 30/50 event or a 3/5 event? -->

[Math is hard tho](https://twitter.com/batwood011/status/1598389323979800576/photo/1)

---
<!-- Would you rather a 100% sure million, or 1 BILLION probability 0.99? -->
[Risk preferences](https://twitter.com/mdahardy/status/1598139462122622976/photo/1)

---
<!-- What's more likely? Canceled Flight or Canceled flight due to upcoming snowstorm? -->
[Conjuction fallacy](https://twitter.com/mdahardy/status/1598139470641262592/photo/1)

---
<!-- What's something most people know that you wouldn't know? -->
[Meta-cognition](https://twitter.com/mdahardy/status/1598139475451817984)

---

<!-- Are windows placed near plants, or plants near windows? -->
[Physical Reasoning: Windows](https://twitter.com/mdahardy/status/1598139488873897984/photo/1)
<!-- - Yet [ordering lol](https://twitter.com/mdahardy/status/1598139493856743424) -->

---

<!-- Could you fit 100 big macs in an average car? -->
[Physical Reasoning: Big Macs](https://twitter.com/mdahardy/status/1598139498357231616/photo/1)

---
<!-- Who would win in a fight: Napoleon's army, or a hippopotamus? -->
[Physical Reasoning: Napoleon](https://twitter.com/mdahardy/status/1598139507559518208)


---
- See more: [AI Alignment Forum | Cognitive Biases in Large Language Models](https://www.alignmentforum.org/posts/fFF3G4W8FbXigS4gr/cognitive-biases-in-large-language-models)

---

TAY

![bg right fit vertical](https://mmp483.p3cdn1.secureserver.net/wp-content/uploads/2016/03/Tay-AI-Feature-Image-03232016.jpg?time=1670290595)
![bg right fit vertical invert](https://mmp483.p3cdn1.secureserver.net/wp-content/uploads/2016/03/Tay-AI-Feature-Image-03232016.jpg?time=1670290595)

-> Interface matters!


---

# OpenAI Codex

# GitHub Copilot

...et autres modèles génératifs de code

---

![bg](https://venturebeat.com/wp-content/uploads/2021/06/Blog-Hero.png?w=1601&strip=all)
<!-- Image credit: https://venturebeat.com/business/openai-warns-ai-behind-githubs-copilot-may-be-susceptible-to-bias/ -->

<!-- 
> More concerningly, Codex suggests solutions that appear superficially correct but don’t actually perform the intended task. 
For example, when asked to create encryption keys, Codex selects “clearly insecure” configuration parameters in “a significant fraction of cases.” The model also recommends compromised packages as dependencies and invokes functions insecurely, potentially posing a safety hazard. 


-> STACKOVERFLOW COPYPASTA ON STEROIDS
-->

--- 

> _"I'm afraid this will create a mis-re-licensing hell"_
> ~[Bruno Hebling Viera](https://twitter.com/HeblingVieira/status/1410972761241968641)
<!-- > I'm afraid this will create a mis-re-licensing hell, where everyone will claim they've been fooled by Copilot, and MS will say licensees should revise the code for licenses themselves. -->

---

- Efficacité == Intelligence ?

![bg right fit](https://www.encora.com/hs-fs/hubfs/Picturegit2.png?width=1280&name=Picturegit2.png)

---

## Recitation?

- [GitHub Copilot: Parrot or Crow?](https://github.blog/2021-06-30-github-copilot-research-recitation/)

<!-- "when it does, it mostly quotes code that everybody quotes" 
-> Appel à la foule :facepalm:
"50 million smokers can't be wrong"

Cela dit, reconnaissent que:
> However, there’s still one big difference between GitHub Copilot reciting code and me reciting a poem: I know when I’m quoting

-->

---

> However, there’s still one big difference between 
> **GitHub Copilot reciting code** 
> and **me reciting a poem**: 
>
> I _know_ when I’m quoting

<br />

~ _Albert Ziegler, GitHub Copilot: Parrot or Crow?_



---

![bg fit](https://i0.wp.com/user-images.githubusercontent.com/4434330/165667874-2f04b14c-909e-4bf5-9639-ff346da960f1.gif?ssl=1)

---

> "I don't want to say anything but that's not the right license Mr Copilot."

- [Tweet](https://twitter.com/i/status/1410886329924194309)

---

<!-- FAST INVERT SQUARE ROOT -->
<!-- QUAKE <3 -->

![bg fit](./img/00-copilot-verbatim.gif)

<!-- Image source: previous tweet -->

--- 

- Question de température ? Pas assez hot pour oser ses propres idées ?

---

> I hacked the Copilot code temperature generation to 1.0 (to allow for **wilder** output), 
> yet the original code **still persisted**, 
> with the alternate approaches being **less helpful**.
- [Max Woolf](https://twitter.com/minimaxir/status/1411005120695865347/photo/1)

![bg fit right:40%](https://pbs.twimg.com/media/E5TlafWVcAAJNRZ?format=jpg&name=4096x4096)
<!-- Image source: Max woolf's tweet -->

---

<!-- _footer: '' -->

![bg fit](./img/00-codex.png)
<!-- Image source: Anonymous internet meme warrior -->

---

### Gaming the game: IAs flemmardes

<br />

- [Régalez vous <br /> avec ce tableau](https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vRPiprOaC3HsCf5Tuum8bRfzYUiKLRqJmbOoC-32JorNdfyTiRRsR7Ea5eWtvsWzuxo8bjOxCG84dAg/pubhtml)

![bg right 90%](./img/01-cheater.png)

---
### Short-term goals vs long-term goals 

<!-- Chemin le plus court vers le goal, probleme ? -->
<!-- What did these robots fail to learn? -->

<!-- What is Netflix optimizing for? -->

<!-- What is the risk of a 'devilishly-good' add to cart automated A/B testing? -->

---
<!-- 
_color: black
 -->

### Dangerous goals :
### L'histoire du _Paperclip maximizer_ 

<!-- 
First described by Nick Bostrom (2003). -->

![bg crop blur](./img/00-paperclips.png)

---

> ##### _Rare photographie d'un humain après déploiement du Paperclip Maximizer_

<!-- Utilitronium: if your reward function is basic, universe becomes grey goo -->

<!-- Also, Nozick experience machine! -->



![bg right crop](https://www.sunnyskyz.com/uploads/2020/05/tqz1n-clippy-office-prank-2.jpg)
<!-- Image credit: https://www.sunnyskyz.com/blog/3056/Employee-Pranks-Entire-Office-With-An-Army-Of-Clippy-Posts -->


---
<!-- 
_color: white
 -->

### Conclusions sur le _Paperclip maximizer_ 

- Essayez vous-même avec un [Clicker game](](https://www.decisionproblem.com/paperclips/index2.html)) 😛
- Voir Nick Bostrom's [_Ethical Issues in Advanced Artificial Intelligence_](https://nickbostrom.com/ethics/ai) 💡

![bg crop brightness:0.2](./img/00-paperclips.png)

---


# C'est quoi du ML ? 

---

#  D'après vous...

![bg right](./img/00-cloud.png)

---

<!-- _footer: "" -->

![bg](./img/00-cloud.png)

<!-- 

Le ML C'est:
- gros jeux de données

- l’automatisation de tache

- réseaux de neurones

- simplement des algorithmes

- des algorithmes qui vont à partir d'un ensemble de données venir s'améliorer d'eux même

- des méthodes d'apprentissage

- automatiser des taches grâce à des training data

- évoluer en fonction des information qu'il digère

- évoluer en gardant à chaque fois les meilleurs résultats 

- apprendre ou reconnaître des patterns

- Plus la data est imposante, plus la machine sera performante

- ...créer des connexions logique pour qu'un automate sorte un résultat à partir d'une entrée

- ...donner à un algorithme la possibilité d'être "presque autonome" dans l'interprétation des données qu'il reçoit

- une methode qui permet a notre application d'évolué d'elle même a chaque utilisation

- Un programme accumulant de la donnée via expérience pour apprendre et atteindre son but de plus en plus efficacement.

- un programme capable d'apprendre des comportement typiques et réagir en conséquence de manière plus efficace à la prochaine présentation de cette situation.

- la capacité d'un système à prendre des informations, les traiter, et les utiliser pour faire des choix. Ces choix s'affineront au fur et à mesure qu'on intégrera de nouvelles inforamtions dans le sytème.

- entrainer des algorithmes à prendre des décisions sur la base d'échantillon de données

- c'est l'utilisation d'un réseau de neurone qui utilise énormément de données afin d'apprendre

- permettre à un programme de s'améliorer avec les utilisations que l'on en fait 

- s'améliorer au fil de son utilisation

- l'application de méthodes statistiques aux algorithmes pour les rendre plus intelligents.

- le programme apprend par lui même à réaliser une tâche à partir d'exemple ou de récompense 

- le fait d’apprendre à apprendre à un ordinateur  -->

---

Un thermostat ? 

![bg fit right 90%](https://www.simulace.info/images/NegativeF.jpg)
<!-- Image credit: University of Prague - Simulace.info -->

---
Et un sac de ifs?

![bg right fit](https://upload.wikimedia.org/wikipedia/commons/e/eb/Decision_Tree.jpg)

<!-- Image credit: User:Gilgoldm

A tree showing survival of passengers on the Titanic ("sibsp" is the number of spouses or siblings aboard). The figures under the leaves show the probability of survival and the percentage of observations in the leaf. Summarizing: Your chances of survival were good if you were (i) a female or (ii) a male at most 9.5 years old with strictly fewer than 3 siblings.

 -->
---
Amazon mechanical turk

<!-- The Turk also had the ability to converse with spectators using a letter board. The operator, whose identity during the period when Kempelen presented the machine at Schönbrunn Palace is unknown,[23] was able to do this in English, French, and German. Carl Friedrich Hindenburg, a university mathematician, kept a record of the conversations during the Turk's time in Leipzig and published it in 1789 as Über den Schachspieler des Herrn von Kempelen und dessen Nachbildung (or On the Chessplayer of Mr. von Kempelen And Its Replica). Topics of questions put to and answered by the Turk included its age, marital status, and its secret workings. -->

<!-- -> What about DeepBlue? About AlphaZero? -->

![bg right fit](https://upload.wikimedia.org/wikipedia/commons/thumb/6/6e/Racknitz_-_The_Turk_3.jpg/1920px-Racknitz_-_The_Turk_3.jpg)
<!-- Image credit: Joseph Racknitz - Humboldt University Library -->


---
[Eliza](http://psych.fullerton.edu/mbirnbaum/psych101/Eliza.htm)? Turing test ? 


<!-- Est-ce du ML si je le confonds avec un humain ? Est-ce que ML == "capacite humaine apprise?" -->

---
Et [Akinator](https://fr.akinator.com/game) ? 


---
Quelles limites aux possibilités du ML ? 

<!-- Qu'est-ce que vous pensez qu'une AI ne pourra jamais faire ? -->

<!-- Peindre un tableau? -->

<!-- Conduire une voiture? -->

<!-- Calmer un enfant qui pleure? -->

<!-- Savoir ce qu'elle ne sait pas, meta-cognition? -->

---
> _If a typical person can do a mental task with 
> **less than one second of thought**, 
> we can probably automate it using AI 
> either now or in the near future._

~ Andrew Ng, [What AI can and can't do](https://hbr.org/2016/11/what-artificial-intelligence-can-and-cant-do-right-now)

---
## Formalisme: Features

$$ X \Rightarrow Y $$

<!-- C'est quoi une feature?
Le Feature Engineering, tout un art
 -->


<br />
<br />
<br />

---

Exemple: a Neural Network

<br />
<br />

$$ {\displaystyle g(x):=f^{L}(W^{L}f^{L-1}(W^{L-1}\cdots f^{1}(W^{1}x)\cdots ))} $$

![invert w:15em](https://www.moonbooks.org/media/images/thumbnails_1000_1000/neural-network-01.PNG?lastmod=1600539785.861)

--- 

## Features: which one?

Voir  
[_A broad introduction to Feature Engineering_](https://medium.com/geekculture/a-broad-introduction-to-feature-engineering-ab27a9636f8a) 

---

## Métriques 

---
<!-- 
backgroundColor: white
color: black 
-->

## Precision & Recall
![bg right:40% 90%](./img/01-metrics.png)


---
Précision

![bg right:40% 90%](./img/01-precision.png)

<!-- Ça veut dire quoi une bonne précision ? -->

<!-- Est-ce qu'on peut être "trop précis" ? -->

---
Rappel (_Recall_ )

![bg right:40% 90%](./img/01-recall.png)

<!-- Quid d'un bon rappel? -->

<!-- Quid de "trop capturer" ? -->

<!-- Tradeoff! Idealement les deux -->

<!-- Depend du coup des erreurs:

- Labo pharma: faux pos? ok, faux neg? terrible
- Justice: faux neg? ok, faux pos? terrible
 -->
---

## Autres métriques ?

<br />
<br />
<br />

- Voir [[TDS] 20 Popular Metrics](https://towardsdatascience.com/20-popular-machine-learning-metrics-part-1-classification-regression-evaluation-metrics-1ca3e282a2ce)
<!--
    Classification Metrics (accuracy, precision, recall, F1-score, ROC, AUC, …)
    Regression Metrics (MSE, MAE)
    Ranking Metrics (MRR, DCG, NDCG)
    Statistical Metrics (Correlation)
    Computer Vision Metrics (PSNR, SSIM, IoU)
    NLP Metrics (Perplexity, BLEU score)
    Deep Learning Related Metrics (Inception score, Frechet Inception distance) 
 -->
---

Supervisé ou non?

[2 Jeux pour avoir une intuition]

<!-- Non supervisé : clustering de genres de musiques -->

<!-- Supervisé : trouvez la règle de mon jeu d'Eleusis (variante triplets de nombres.) -->

<!-- Regle? E.g. differences croissantes ou egales d1 <= d2 <= d3  -->

<!-- Did it feel different? -->
---
<!-- 
backgroundColor: black
color: white
-->

Exemples d'algos supervisés

---

- Linear Regression

- Decision trees

- Nearest Neighbor

---
Exemples d'algos non-supervisés

---
- Clustering, e.g. **K-means**
- Anomaly Detection
<!-- To detect anomalies, we have observations x1,. . . , xn ∈ X. The underlying presumption is, most of the data come from the same (unknown) distribution. We call it  normalization in data. 
However, some observations come from a different distribution. They are considered anomalies. Several reasons can lead to these anomalies. 
The final task is to identify these anomalies by observing a concise description of the standard data so that divergent observations become outliers. -->

- Principal Component Analysis
<!-- Principal Component Analysis is an unsupervised learning algorithm. We use it for dimensionality reduction in machine learning. 

A statistical approach transforms the observations of correlated features into a collection of linearly uncorrelated components using orthogonal transformation. 

These new transformed features are known as the Principal Components. It is one of the most popular machine learning algorithms.

 PCA is used for exploratory data analysis and predictive modeling. It is a way to identify hidden patterns from the given dataset by lessening the variances. It follows a feature extraction technique. 

PCA usually tries to express the lower-dimensional surface to project the high-dimensional data. PCA determines the variance of each feature. 

The feature with high variance shows the excellent split between the classes and hence reduces the dimensionality.  -->

---
<!-- 
backgroundColor: black
color: white
-->
## Pionniers & Penseurs

---
### Turing

- Turing Machine
![bg right](https://1.bp.blogspot.com/-jqU6NCD5dVU/WBVezz7LZWI/AAAAAAAADkY/t7hQMwMuCZIXxgPl0ymB2M9WpvkPw3hvQCLcB/s200/Alan%2BTuring.jpg)
<!-- Turing test: Mécompréhensions

Against the other lines of thought, Turing provides a little “viva voce” that is intended to illustrate the kind of evidence that he supposes one might have that a machine is intelligent. 
Given the right kinds of responses from the machine, we would naturally interpret its utterances as evidence of pleasure, grief, warmth, misery, anger, depression, etc. 
Perhaps—though Turing doesn’t say this—the only way to make a machine of this kind would be to equip it with sensors, affective states, etc., i.e., in effect, to make an artificial person. However, the important point is that if the claims about self-consciousness, desires, emotions, etc. are right, then Turing can accept these claims with equanimity: his claim is then that a machine with a digital computing “brain” can have the full range of mental states that can be enjoyed by adult human beings.
 -->
<!-- 
 Also, Searles' chinese room experiment:

Searle imagines himself alone in a room following a computer program for responding to Chinese characters slipped under the door. Searle understands nothing of Chinese, and yet, by following the program for manipulating symbols and numerals just as a computer does, he sends appropriate strings of Chinese characters back out under the door, and this leads those outside to mistakenly suppose there is a Chinese speaker in the room.
The narrow conclusion of the argument is that programming a digital computer may make it appear to understand language but could not produce real understanding. Hence the “Turing Test” is inadequate. Searle argues that the thought experiment underscores the fact that computers merely use syntactic rules to manipulate symbol strings, but have no understanding of meaning or semantics.
https://plato.stanford.edu/entries/chinese-room/
His conclusion seems somewhat wrong to me as computer programs have no representation of meaning is not always true, but in our specific case of Language Models he was really on point :not_bad: -->

- Turing test
-> [Coffee test](https://www.fastcompany.com/1568187/wozniak-could-computer-make-cup-coffee)? 
<!-- Steve Wozniak: for general AI, Turing's test is outdated -->
---
### AI Summer: Dartmouth workshop

> _1956' Dartmouth Summer Research Project on Artificial Intelligence_

<!-- 
In 1956, the summer workshop of Darmouth was held with prestigious participants (Minsky, Shannon, McCarthy, etc.). 
This event marked the starting point of Artificial Intelligence as a field on its own. 
The hopes were high: beating a chess champion, prove math theorems, replace human’s works. Following this meeting, Rosenblatt created the perceptron in 1958, a very simplified model of a biological neuron that aimed to classify images. Yes, “artificial neural networks” and “computer vision” aren’t new.

Despite these initial progresses, the field suffered from severe cutbacks in funding leading to the first AI winter in the 70s.
The expectation were too great compared to what was achieved. 
This phenomenon occurred again in the 80s. 

At this point, practical applications came mainly from other approaches than neural networks.
 -->

---

### Perceptron

![](https://miro.medium.com/max/1100/1*v88ySSMr7JLaIBjwr4chTw.webp)
 <!-- Image credit: https://towardsdatascience.com/multi-layer-neural-networks-with-sigmoid-function-deep-learning-for-rookies-2-bf464f09eb7f?gi=15a9a7836230 -->

 ---
<!-- _backgroundColor:  -->
![bg fit 50%](https://upload.wikimedia.org/wikipedia/commons/8/8a/Perceptron_example.svg)
<!--  A diagram showing a perceptron updating its linear boundary as more training examples are added. -->
 ---

<!-- _backgroundColor: 
_color: -->
## Multi-Layer Perceptron

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

![bg fit 80%](https://miro.medium.com/max/1100/1*CJEBy3GCaGQKNx7PEy-w5w.webp)

 ---
<!-- color: white -->
## Loss Function

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

![bg fit 70%](https://arthurdouillard.com/deepcourse/introduction/lossfunction.webp)
<!-- Image Credit: Arthur Douillard's Deep Learning Course -->

---
### Back to Dartmouth Summer of dreams

<!-- The study is to proceed on the basis of the conjecture that 
every aspect of learning or any other feature of intelligence can in principle 
be so precisely described 
that a machine can be made to simulate it. 

An attempt will be made to find:
- how to make machines use language
- form abstractions and concepts
- solve kinds of problems now reserved for humans
- and improve themselves. 


-->

> We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.

---

<!-- _backgroundColor: 
_color:
 -->
### (Gartner Hype Cycle)

<br />

######         ☝ 🔺🔺🔺🔺🔺🔺🔺🔺🔺      We're here

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

![bg fit 60%](https://upload.wikimedia.org/wikipedia/commons/thumb/9/94/Gartner_Hype_Cycle.svg/1920px-Gartner_Hype_Cycle.svg.png)


---
### AI Winter: 1984 et autres déceptions

# ❄🤖❄

<!-- . The AI winter was a result of such hype, due to over-inflated promises by developers, unnaturally high expectations from end-users, and extensive promotion in the media -->

<!-- Even today:
- AI researcher Rodney Brooks would complain in 2002 that "there's this stupid myth out there that AI has failed, but AI is around you every second of the day."[4] 
- In 2005, Ray Kurzweil agreed: "Many observers still think that the AI winter was the end of the story and that nothing since has come of the AI field. Yet today many thousands of AI applications are deeply embedded in the infrastructure of every industry."
 -->

 <!-- Enthusiasm and optimism about AI has generally increased since its low point in the early 1990s. Beginning about 2012, interest in artificial intelligence (and especially the sub-field of machine learning) from the research and corporate communities led to a dramatic increase in funding and investment.  -->
---
### Yann LeCun

<!-- Thèse sur la BackPropagation
Père du Deep LEarning

Yann Le Cun est diplômé de l'ESIEE Paris en 1983, il part ensuite à l'université Pierre-et-Marie-Curie effectuer un DEA puis un doctorat qu'il obtient en 1987. 
Il s'oriente rapidement vers la recherche sur l'apprentissage automatique et il propose pendant sa thèse une variante de l'algorithme de rétropropagation du gradient, qui permet depuis le début des années 1980 l'apprentissage des réseaux de neurones. 

Il réalise son post-doctorat au sein de l'équipe de Geoffrey Hinton.

Yann Le Cun travaille depuis les années 1980 sur l’apprentissage automatique (machine learning) et l’apprentissage profond (deep learning) : la capacité d’un ordinateur à reconnaître des représentations (images, textes, vidéos, sons) à force de les lui montrer, de très nombreuses fois.

En 1987, Yann Le Cun rejoint l'Université de Toronto et en 1988 les laboratoires AT&T, pour lesquels il développe des méthodes d'apprentissage supervisé6.

Yann Le Cun s'intéresse ensuite à la conception des algorithmes de compression du format d'archivage DjVu7 puis la reconnaissance automatique de chèques bancaires.

Yann Le Cun est professeur à l'université de New York où il a créé le Center for Data Sciences. Il travaille notamment au développement technologique des voitures autonomes.

Le 9 décembre 2013, Yann Le Cun est invité par Mark Zuckerberg à rejoindre Facebook pour créer et diriger le laboratoire d'intelligence artificielle FAIR (« Facebook Artificial Intelligence Research ») à New York, Menlo Park et depuis 2015 à Paris, notamment pour travailler sur la reconnaissance d'images et de vidéos. Il avait précédemment refusé une proposition similaire de la part de Google.

En 2016, Yann Le Cun est le titulaire pour l'année de la chaire « Informatique et sciences numériques » du Collège de France9.

En janvier 2018, Yann Le Cun quitte son poste de chef de division en recherche sur l'intelligence artificielle chez Facebook, au profit de Jérôme Pesenti, pour occuper un poste de chercheur en tant que scientifique en chef de l'IA toujours chez Facebook. 
 -->
![bg right ](https://www.controcorrenteblog.com/wp-content/uploads/2015/09/Yann-LeCun-1.jpg)

- [BackPropagation](http://yann.lecun.com/exdb/publis/pdf/lecun-89e.pdf)
- [Deep Learning](https://www.historyofdatascience.com/yann-lecun/)

---

![bg](./img/01-lenet.gif)

---

## LeNet Architecture
![](./img/01-lenet.jpg)

---
### Geoffrey Hinton
![bg right](https://upload.wikimedia.org/wikipedia/commons/thumb/3/34/Geoffrey_Hinton_at_UBC.jpg/1024px-Geoffrey_Hinton_at_UBC.jpg)

<!-- 
Hinton was co-author of a highly cited paper published in 1986 that popularized the backpropagation algorithm for training multi-layer neural networks, although they were not the first to propose the approach.
Hinton is viewed as a leading figure in the deep learning community.
The dramatic image-recognition milestone of the AlexNet designed in collaboration with his students Alex Krizhevsky and Ilya Sutskever for the ImageNet challenge 2012 was a breakthrough in the field of computer vision.

Hinton received the 2018 Turing Award, together with Yoshua Bengio and Yann LeCun, for their work on deep learning.[24] They are sometimes referred to as the "Godfathers of AI" and "Godfathers of Deep Learning",[25][26] and have continued to give public talks together.[27][28]  -->


---
<!-- _backgroundColor: white
_color: black
_footer: '' -->
![bg 60%](https://upload.wikimedia.org/wikipedia/commons/thumb/c/cc/Comparison_image_neural_networks.svg/1024px-Comparison_image_neural_networks.svg.png)

<!-- AlexNet contained eight layers; the first five were convolutional layers, some of them followed by max-pooling layers, and the last three were fully connected layers.It used the non-saturating ReLU activation function, which showed improved training performance over tanh and sigmoid. -->

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />

#### LeNet vs AlexNet | [Learn the difference](https://programmathically.com/deep-learning-architectures-for-image-classification-lenet-vs-alexnet-vs-vgg/)

---

### Andrew Ng

![bg fit right](https://upload.wikimedia.org/wikipedia/commons/a/a3/Andrew_Ng_WSJ_%282%29.jpg)

- deeplearning.ai

- "AI Optimist"

<!-- Ng was a co-founder and head of Google Brain and was the former chief scientist at Baidu, building the company's Artificial Intelligence Group into a team of several thousand people. 

Ng is an adjunct professor at Stanford University (formerly associate professor and Director of its Stanford AI Lab or SAIL). Ng has also made substantial contributions to the field of online education as the co-founder of both Coursera and deeplearning.ai

 Ng believes that AI technology will improve the lives of people, not that it is an anathema that will "enslave" the human race.[3] Ng believes the potential benefits of AI outweigh the threats, which he believes are exaggerated.[63] He has stated that

    Worrying about AI evil superintelligence today is like worrying about overpopulation on the planet Mars. We haven't even landed on the planet yet![63]

A real threat is regarding the future of work: "Rather than being distracted by evil killer robots, the challenge to labour caused by these machines is a conversation that academia and industry and government should have."[64] A particular goal of Ng's work is to "democratize" AI learning so that people can learn more about it and understand its benefits. Ng's stance on AI is shared by Mark Zuckerberg, but opposed by Elon Musk.[64]

In 2017, Ng said he supported basic income to allow the unemployed to study AI so that they can re-enter the workforce. He has stated that he enjoyed Erik Brynjolfsson and Andrew McAfee's "The Second Machine Age" which discusses issues such as AI displacement of jobs.[65] 

-->
---
### Eliezer Yudkowsky


- ###### [HELL OF AN] AI PESSIMIST
- [Machine Intelligence Research Institute _[intelligence.org]_](https://intelligence.org/research/)
- [LessWrong](lesswrong.com) & [Overcoming Bias](overcomingbias.com)
- AI Box Experiments
![bg right](https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2Fb1f9d45e-db99-41a5-b38b-c824fb9ef481_1258x1078.jpeg)

---
<!-- 
_backgroundColor: white
_color: #111 
_footer: ""
-->

![bg 100%](./img/01-miri.png)

---
<!-- 
_backgroundColor: white
_color: #111 
_footer: ""
-->

## LessWrong <3
- #### [Sequences on Rationality](https://www.lesswrong.com/rationality)
    - ###### [Guessing the Teacher's Password](https://www.lesswrong.com/s/5uZQHpecjn7955faL/p/NMoLJuDJEms7Ku9XS)
    - ###### ["Science" as a Curiosity-Stopper](https://www.lesswrong.com/posts/L22jhyY9ocXQNLqyE/science-as-curiosity-stopper)
    - ###### [Say no to Complexity](https://www.lesswrong.com/s/5uZQHpecjn7955faL/p/kpRSCH7ALLcb6ucWM)
    - ###### [Making Beliefs pay Rent](https://www.lesswrong.com/s/7gRSERQZbqTuLX5re/p/a7n8GdKiAZRX86T5A)

![bg right 120%](./img/01-lesswrong.png)


---

### But also...

['Death with Dignity' strategy](https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy)

> "tl;dr:  It's obvious at this point that humanity **isn't going to solve the alignment problem**
> or even try very hard
> or even go out with much of a fight  

--- 


### Also 'Death with Dignity' 
### (╯°□°）╯︵ ┻━┻

['Death with Dignity' strategy](https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy)

> Since **survival is unattainable**, 
> we should shift the focus of our efforts to helping humanity 
> **die** with with **slightly more dignity**."

---

# Les Acteurs de l'Écosystème

---

## "Research" labs

- DeepMind
- OpenAI
- HuggingFace

---

### DeepMind

![bg right fit](./img/00-deepmind.png)

- [Lab](https://www.deepmind.com)
- [AI Safety Research](https://deepmindsafetyresearch.medium.com)

Exemples:
- [On Meta's CICERO model](https://www.deepmind.com/blog/ai-for-the-board-game-diplomacy)
- [On Policy Regularization](https://deepmindsafetyresearch.medium.com/your-policy-regulariser-is-secretly-an-adversary-14684c743d45)

---
### "Open"AI


![bg right fit](./img/00-openai.png)

- [Research](https://openai.com/research/)
- [AI Alignment](https://openai.com/alignment/)

<br />

---

Debate : [_Is OpenAI still Open?_](https://becominghuman.ai/is-openai-still-open-81f6839f756b)
<!-- 
This is a key turning point for OpenAI and everything following from the 1 Billion Dollar Investment by Microsoft to releasing GPT-3 as a commercial product 

and now the exclusivity deal with Microsoft stem from this change in corporate structure and leadership.
-->

- API-based [examples](https://gpt3demo.com/apps/algolia)
- [Safety Research](https://openai.com/blog/our-approach-to-alignment-research/)
    - [Yudkowsky on OpenAI](https://www.lesswrong.com/posts/oEC92fNXPj6wxz8dd/how-to-think-about-and-deal-with-openai)
    - [Common misconceptions thread](https://www.lesswrong.com/posts/3S4nyoNEEuvNsbXt8/common-misconceptions-about-openai)

<br />
<!-- Above:
    Nothing else Elon Musk has done can possibly make up for how hard the "OpenAI" launch trashed humanity's chances of survival; 
    previously there was a nascent spirit of cooperation, which Elon completely blew up to try to make it all be about who, which monkey, got the poison banana, and by spreading and advocating the frame that everybody needed their own "demon" (Musk's old term) in their house, 
    and anybody who talked about reducing proliferation of demons must be a bad anti-openness person who wanted to keep all the demons for themselves.
-->

![bg right fit](./img/00-openai.png)

---

## HuggingFace
> "The Underdogs"

- [Models](https://huggingface.co/models)
- [Datasets](https://huggingface.co/datasets)
- [APIs](https://huggingface.co/pricing)

![bg right fit](./img/00-huggingface.png)

---

## Cloud Vendors

<!-- Interet: ils vendent leur compute, nous habituent à leur plateforme, etc -->

- [AWS](https://aws.amazon.com/machine-learning/)
- [GCP](https://cloud.google.com/solutions/ai)
- [Azure](https://azure.microsoft.com/en-us/solutions/ai/#benefits)

![bg right fit](https://www.whizlabs.com/blog/wp-content/uploads/2020/07/aws-machine-learning-tools.png)

---

## Hardware shops

<!-- Interet: ils justifient leurs nouvelles chips! -->

- [NVIDIA](https://www.nvidia.com/en-us/deep-learning-ai/solutions/machine-learning/)
- [AMD, you still here?](https://www.amd.com/en/technologies/deep-machine-learning)
    - [Acquisition strategy](https://www.xilinx.com/applications/video-imaging/video-ai-analytics.html)
- [Google?](https://cloud.google.com/tpu/)

---

## On résume

<!-- Tour de table : un truc que tu as appris aujourd'hui  -->

<!-- Main levée : un truc qui nous a surpris ? -->

<!-- 
Main levée : qui a une grosse question ? 
Un blind spot dans sa carte mentale ? 
-->

<!-- Préparons la suite : 
Tour de table - c'est quoi le plus important pour vous que vous voulez apprendre des TPs ? -->

---

### Conclusion:
> la route est _longue_, mais la voie est _libre_

<!-- This is the DREAM TIME! Nous sommes la première génération à être limités, non pas par les possibilités techniques, mais par notre imagination 🤩 -->

<!-- "Gros gros potentiel" -->

<!-- -> Grosses grosses responsabilités -->

<!-- Bon, ok, mais en pratique ?

- Comment on choisit un modèle selon le problème à résoudre ?

- Comment on entraine un modèle ?

- Comment on teste sa qualité ? Ses biais ? 

- Comment on peut build on top ? Le déployer ? Collaborer à plusieurs ?

On se retrouve l'année prochaine pour voir tout ça ensemble :)
-->

![bg right 90%](https://miro.medium.com/max/640/0*JWCLdKhz-0e_77tB.webp)
<!-- Image Credit: Steward Brandt - Whole Earth Catalog -->

---

### Outro

- AI Box experiments!

--- 


### Outro
- `tp-00`

https://nech.pl/ml101-00-tp

--- 


### Outro
- Let's play with [StableDiffusion](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/diffusers_intro.ipynb#scrollTo=e5MYkuhcRGAS)
- [using Diffusers](https://github.com/huggingface/diffusers)
- [Go further on your own!](https://github.com/Stability-AI/stablediffusion#image-inpainting-with-stable-diffusion)

--- 


### Outro
- Let's play with [Neural Networks](https://colab.research.google.com/github/google/eng-edu/blob/main/ml/cc/exercises/intro_to_neural_nets.ipynb?hl=en#scrollTo=9n9_cTveKmse)

--- 

### Outro
- Let's play with [Neural Networks: LVL2 🛠](https://www.sitepoint.com/keras-digit-recognition-tutorial/)

--- 


### Outro
- Let's play with [LSTM lyric generation](https://git.plnech.fr/pln/BabelZoo/tree/master/LeBoulbiNet)
- Or [Rebuild your own 🛠](https://colab.research.google.com/github/tensorflow/examples/blob/master/courses/udacity_intro_to_tensorflow_for_deep_learning/l10c03_nlp_constructing_text_generation_model.ipynb#scrollTo=2LmLTREBf5ng)
<!-- From https://www.udacity.com/course/intro-to-tensorflow-for-deep-learning--ud187 -->

![vertical bg right](https://git.plnech.fr/pln/BabelZoo/raw/master/LeBoulbiNet/boulbi.jpg)
![vertical bg right fit](https://git.plnech.fr/pln/BabelZoo/raw/master/LeBoulbiNet/first.png)
![vertical bg right fit](https://git.plnech.fr/pln/BabelZoo/raw/master/KoozDawa/first.png)
![vertical bg right fit](./img/00-turfu.png)