Skip to content

M
ML101
Commit 11fea5d6 by PLN (Algolia)
Options

Content: slides1; tp1 bases

parent 467051bb
Showing with 194 additions and 12 deletions
slides/.vscode/settings.json 0 → 100644
{
"gitlens.blame.line.enabled": false,// was working in previous versions
"gitlens.currentLine.enabled": false,// in modern version
"gitlens.codeLens.authors.enabled": false,
"gitlens.codeLens.recentChange.enabled": false,
}
\ No newline at end of file
slides/00-intro.md
...@@ -5,7 +5,7 @@ color: #eee ...@@ -5,7 +5,7 @@ color: #eee
colorSecondary: #333 colorSecondary: #333
backgroundColor: #111 backgroundColor: #111
paginate: true paginate: true
footer: "ML101 | Paul-Louis NECH | INTECH 2022-2023" footer: "ML101 | Paul-Louis Nech | INTECH 2022-2023"
--- ---
# <!-- fit --> Hello World # <!-- fit --> Hello World
...@@ -42,9 +42,10 @@ $$ I_{xx}=\int\int_Ry^2f(x,y)\cdot{}dydx $$ ...@@ -42,9 +42,10 @@ $$ I_{xx}=\int\int_Ry^2f(x,y)\cdot{}dydx $$
<!-- Les dangers, un sujet pour un cours entier --> <!-- Les dangers, un sujet pour un cours entier -->
<!-- Biais, feedback loop foireuses, Risque existentiel?... --> <!-- Biais, feedback loop foireuses, Risque existentiel?... -->
![bg right](https://uproxx.com/wp-content/uploads/2015/05/angry-bender.jpg?w=650) ![bg right](https://uproxx.com/wp-content/uploads/2015/05/angry-bender.jpg?w=650)
<!-- Image Credit: Bender from Futurama, created by Matt Groening and David X. Cohen -->
###### - [the Fallacy of generalization from Fictional evidence](https://www.lesswrong.com/posts/rHBdcHGLJ7KvLJQPk/the-logical-fallacy-of-generalization-from-fictional) ###### - [the Fallacy of generalization from Fictional evidence](https://www.lesswrong.com/posts/rHBdcHGLJ7KvLJQPk/the-logical-fallacy-of-generalization-from-fictional)
<!-- The more details, the less likely! -->
--- ---
## Encore que... ## Encore que...
...@@ -98,6 +99,13 @@ $$ I_{xx}=\int\int_Ry^2f(x,y)\cdot{}dydx $$ ...@@ -98,6 +99,13 @@ $$ I_{xx}=\int\int_Ry^2f(x,y)\cdot{}dydx $$
![bg left 100%](./img/01-dungeon.jpg) ![bg left 100%](./img/01-dungeon.jpg)
---
### Comment les modèles... peuvent halluciner
##### - [DeepDream](https://en.wikipedia.org/wiki/DeepDream)
![bg left 100%](../tp/img/intech_dream.png)
---- ----
### Comment ils nous comprennent ### Comment ils nous comprennent
...@@ -153,7 +161,13 @@ _color: black ...@@ -153,7 +161,13 @@ _color: black
--- ---
## C'est quoi ce cours ? ## C'est quoi ce cours ?
<!-- Plan du cours / TDs / eval -->
<!-- Plan du cours: -->
<!-- TD: Quali aujourd'hui, plus pratique en janvier -->
<!-- Eval: Par les rendus, pas de piège :) -->
<!-- Format / interactions welcome --> <!-- Format / interactions welcome -->
--- ---
#### Des connaissances #### Des connaissances
...@@ -188,6 +202,8 @@ Conditionnement opérant (Pavlov et sa cloche) ...@@ -188,6 +202,8 @@ Conditionnement opérant (Pavlov et sa cloche)
--- ---
<!-- _footer: "" --> <!-- _footer: "" -->
![bg](https://image.beeplaza.com.au/wp-content/uploads/2019/12/bee-dance.jpg) ![bg](https://image.beeplaza.com.au/wp-content/uploads/2019/12/bee-dance.jpg)
<!-- Image Credit: BeePlaza Beekeeping Shop; beeplaza.com.au -->
<!-- <!--
Acquis vs Inné : danse des abeilles vs culture des chimpanzés Acquis vs Inné : danse des abeilles vs culture des chimpanzés
--> -->
...@@ -195,6 +211,7 @@ Acquis vs Inné : danse des abeilles vs culture des chimpanzés ...@@ -195,6 +211,7 @@ Acquis vs Inné : danse des abeilles vs culture des chimpanzés
--- ---
<!-- _footer: "" --> <!-- _footer: "" -->
![bg](https://i0.wp.com/www.throwcase.com/wp-content/uploads/2014/12/monkeys-e1419187612146.jpg?fit=1000%2C692&ssl=1) ![bg](https://i0.wp.com/www.throwcase.com/wp-content/uploads/2014/12/monkeys-e1419187612146.jpg?fit=1000%2C692&ssl=1)
<!-- Image Credit: Throwcase's Net Madness section - https://www.throwcase.com/2014/12/21/that-five-monkeys-and-a-banana-story-is-rubbish/ -->
--- ---
...@@ -241,6 +258,7 @@ _backgroundColor: white ...@@ -241,6 +258,7 @@ _backgroundColor: white
## Learning ## Learning
![bg right 90%](./img/01-reinforcement.png) ![bg right 90%](./img/01-reinforcement.png)
<!-- Image Credit: [Megajuice](https://en.wikipedia.org/wiki/Reinforcement_learning#/media/File:Reinforcement_learning_diagram.svg) -->
--- ---
### Le par-coeur, un problème ? ### Le par-coeur, un problème ?
...@@ -268,6 +286,7 @@ Unfortunately, many students in traditional schools become accustomed to guessin ...@@ -268,6 +286,7 @@ Unfortunately, many students in traditional schools become accustomed to guessin
![bg right 90%](./img/01-information.png) ![bg right 90%](./img/01-information.png)
<!-- Image(s) Credit: https://towardsdatascience.com/information-theory-of-neural-networks-ad4053f8e177 -->
<!-- <!--
Initially,weights are randomly initialised. Initially,weights are randomly initialised.
...@@ -364,6 +383,8 @@ REPLICATION CRISIS! Probleme commun dans toute la science, incentives, game theo ...@@ -364,6 +383,8 @@ REPLICATION CRISIS! Probleme commun dans toute la science, incentives, game theo
> _unsexy_ problems > _unsexy_ problems
> **before the sexy ones** > **before the sexy ones**
<!-- Publish or Perish, ça vous parle ? -->
<!-- Replication Crisis ? -->
--- ---
...@@ -372,19 +393,20 @@ https://galactica.org/explore/ ...@@ -372,19 +393,20 @@ https://galactica.org/explore/
[Try it live ;)](https://huggingface.co/spaces/morenolq/galactica-base) [Try it live ;)](https://huggingface.co/spaces/morenolq/galactica-base)
[LeCun hyped](https://twitter.com/ylecun/status/1592619400024428544) ![bg right 80%](./img/00-tweet-lecun-hyped.png)
--- ---
[GALACTICA failed](https://www.technologyreview.com/2022/11/18/1063487/meta-large-language-model-ai-only-survived-three-days-gpt-3-science/ [GALACTICA failed](https://www.technologyreview.com/2022/11/18/1063487/meta-large-language-model-ai-only-survived-three-days-gpt-3-science/
) )
![bg right](img/00-galactica-failed.png)
<br /> <br />
> Like all language models, Galactica is a mindless bot that cannot tell fact from fiction.
> ~_MIT Tech Review_
[_Paper_](https://arxiv.org/abs/2211.09085) > Like all language models, Galactica is a **mindless bot** that _cannot tell fact from fiction_.
[See Research _Paper_](https://arxiv.org/abs/2211.09085)
--- ---
...@@ -394,20 +416,68 @@ https://huggingface.co/spaces?sort=modified&search=galactica ...@@ -394,20 +416,68 @@ https://huggingface.co/spaces?sort=modified&search=galactica
--- ---
---
> Narrator voice: LMs have no access to "truth", or any kind of "information" beyond information about the distribution of word forms in their training data. > Narrator voice: LMs have no access to "truth", or any kind of "information" beyond information about the distribution of word forms in their training data.
Emily M. Bender, Faculty Director, Computational Linguistics [Emily M. Bender, Faculty Director, Computational Linguistics](https://twitter.com/emilymbender/status/1592993757498331136)
---
https://twitter.com/emilymbender/status/1592993757498331136 > In all cases, it was wrong or biased but sounded right and authoritative. I think it's dangerous.
[~ Michael Black](https://twitter.com/Michael_J_Black/status/1593133722316189696)
https://twitter.com/Michael_J_Black/status/1593133722316189696 ---
<!-- Vous savez comment un scientifique réagit quand on lui dit que son projet est dangereux ? -->
[LeCun not happy](https://twitter.com/ylecun/status/1593293058174500865) [LeCun not happy](https://twitter.com/ylecun/status/1593293058174500865)
![bg right 100%](img/01-tweet-lecun-not-happy.png)
---
# ChatGPT
![bg right 100%](./img/00-chatgpt.png)
- [Blog post](https://openai.com/blog/chatgpt/)
---
ChatGPT:
[Linda Problem](https://twitter.com/dggoldst/status/1598317411698089984)
---
<!-- But maybe assumption? Language ambiguous! -->
[Assumptions?](https://twitter.com/dggoldst/status/1598737445780164635)
---
[Computing, Fast & Slow](https://twitter.com/stefanmherzog/status/1598397009161060359)
---
[Math is hard tho](https://twitter.com/batwood011/status/1598389323979800576/photo/1)
---
[Risk preferences](https://twitter.com/mdahardy/status/1598139462122622976/photo/1)
---
[Conjuction fallacy](https://twitter.com/mdahardy/status/1598139470641262592/photo/1)
---
- See more: https://www.alignmentforum.org/posts/fFF3G4W8FbXigS4gr/cognitive-biases-in-large-language-models
---
[Physical Reasoning: Windows](https://twitter.com/mdahardy/status/1598139488873897984/photo/1)
- But https://twitter.com/mdahardy/status/1598139493856743424
---
[Physical Reasoning: Big Macs](https://twitter.com/mdahardy/status/1598139498357231616/photo/1)
---
[Physical Reasoning: Napoleon](https://twitter.com/mdahardy/status/1598139507559518208)
--- ---
TAY TAY
...@@ -632,4 +702,8 @@ footer: "" ...@@ -632,4 +702,8 @@ footer: ""
![bg right 120%](./img/01-lesswrong.png) ![bg right 120%](./img/01-lesswrong.png)
---
Conclusion:
--- ---
\ No newline at end of file
slides/yarn-error.log 0 → 100644
Arguments:
/usr/bin/node /usr/bin/yarn add marp
PATH:
/home/pln/.rvm/gems/ruby-2.6.0/bin:/home/pln/.rvm/gems/ruby-2.6.0@global/bin:/home/pln/.rvm/rubies/ruby-2.6.0/bin:/home/pln/.gem/ruby/3.0.0/bin:/home/pln/.local/bin:/home/pln/scripts:/usr/local/bin:/usr/lib/safe-rm:/opt/jython/bin/:/opt/google-cloud-sdk/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/pln/.rvm/bin:/home/pln/.local/share/JetBrains/Toolbox/scripts:/home/pln/.antigen/bundles/robbyrussell/oh-my-zsh/lib:/home/pln/.antigen/bundles/zsh-users/zsh-syntax-highlighting:/home/pln/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/pip:/home/pln/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/python:/home/pln/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/node:/home/pln/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/npm:/home/pln/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/tmux:/home/pln/.antigen/bundles/robbyrussell/oh-my-zsh/plugins/gradle:/home/pln/Android/Sdk//tools:/home/pln/Android/Sdk//tools/bin:/home/pln/Android/Sdk//platform-tools:/home/pln:/home/pln/.rvm/bin
Yarn version:
1.22.19
Node version:
16.18.0
Platform:
linux x64
Trace:
Error: https://registry.yarnpkg.com/marp: Not found
at Request.params.callback [as _callback] (/usr/lib/node_modules/yarn/lib/cli.js:66145:18)
at Request.self.callback (/usr/lib/node_modules/yarn/lib/cli.js:140890:22)
at Request.emit (node:events:513:28)
at Request.<anonymous> (/usr/lib/node_modules/yarn/lib/cli.js:141862:10)
at Request.emit (node:events:513:28)
at IncomingMessage.<anonymous> (/usr/lib/node_modules/yarn/lib/cli.js:141784:12)
at Object.onceWrapper (node:events:627:28)
at IncomingMessage.emit (node:events:525:35)
at endReadableNT (node:internal/streams/readable:1358:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
npm manifest:
{
"name": "ml101-slides",
"version": "1.0.0",
"description": "Slides pour le cours ML101@INTECH 2022-2023",
"main": "index.js",
"author": "PLN",
"license": "MIT",
"private": false,
"dependencies": {
"reveal.js": "^4.4.0"
}
}
yarn manifest:
No manifest
Lockfile:
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
reveal.js@^4.4.0:
version "4.4.0"
resolved "https://registry.yarnpkg.com/reveal.js/-/reveal.js-4.4.0.tgz#bff589987a842d99bc8f6eb9b2b59f6d9941bf87"
integrity sha512-jIV6C9V2NEUjGzU8L6dUFGpk1KJmq7/EzP2fOW67ggc2c0Cp/PdprWxZ9Qgp46F0T2ZWDCjQ1p3Ytzy5jA6a2w==
tp/00-intro-tp.md 0 → 100644
---
marp: true
theme: uncover
color: #eee
colorSecondary: #333
backgroundColor: #111
paginate: true
footer: "ML101 | TP0: Introduction | Paul-Louis Nech | INTECH 2022-2023"
---
# <!-- fit --> TP1: Introduction
<!-- ### Définitions & Histoire du ML
### méthodes et outils ;
### critiques, biais et limites
### applications et perspectives d'avenir -->
---
Objectifs :
- Se familiariser avec les bases du ML
- Se faire une intuition de ses applications possibles
- Découvrir ses limites avec quelques gros fails
---
Format: Rendu écrit (fichier Markdown ou Doc)
Sur l'intranet ou à formation@nech.pl
**DEADLINE : 15 Décembre 23:59:59**
<br />
> _Le cachet de mon mailserver faisant foi_.
---
## Lvl 0: Jouer avec des LMs
<br />
<br />
> Consigne: Entrez dans le [AI Dungeon](https://play.aidungeon.io/), faites une partie, et notez:
- Une phrase de son output que vous trouvez excellente _(expliquez en quelques mots pourquoi celle là)_
- Une phrase de son output que vous trouvez risible _(expliquez pourquoi : qu'est-ce qu'il a si mal compris ?)_
---
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment