From c517c68925cef8c0d59ec38ef413e179fb92814f Mon Sep 17 00:00:00 2001 From: Rohan Kumar Date: Sat, 18 Feb 2023 22:28:09 -0800 Subject: [PATCH] New note: ChatGPT reliance considered harmful --- content/notes/chatgpt-considered-harmful.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 content/notes/chatgpt-considered-harmful.md diff --git a/content/notes/chatgpt-considered-harmful.md b/content/notes/chatgpt-considered-harmful.md new file mode 100644 index 0000000..ed63a80 --- /dev/null +++ b/content/notes/chatgpt-considered-harmful.md @@ -0,0 +1,14 @@ +--- +title: "ChatGPT reliance considered harmful" +date: 2023-02-18T22:28:09-08:00 +--- +Designing tools to make people feel convenienced (the opposite of inconvenienced) is sometimes different from designing tools to make people's lives better. + +ChatGPT is _very_ useful for tasks such as re-phrasing ideas and coherently assembling scattered thoughts, assuming that's the sort of thing you're willing to outsource. But current language models are detrimental for ideation: they're designed to generate the least interesting, most obvious response to a prompt. That's not my opinion; the GPT family of language models analyzes patterns in language, and generates predictable outputs. It's often "correct" when a correct answer is the most predictable, least noteworthy response. Unfortunately, it's often convincingly incorrect: [it even defeends wrong answers](https://simonwillison.net/2023/Feb/15/bing/#prompt-leaked). + +Use it to think, and your ideas will be disposable. Don't live your life by a model with billions of parameters optimized to help you be as predictable as possible. It's the equivalent of sending your thoughts through a smoothing algorithm that treats interesting ideas as anomalies. + +People have worried about certain trends in investor-backed technology helping normalize helplessness to [create captive consumers ripe for domestication](https://seirdy.one/posts/2021/01/27/whatsapp-and-the-domestication-of-users/). This is the logical conclusion of that process. Language models are often developed by and for companies who sell access to [the means to change preferences at scale](https://en.wikipedia.org/wiki/Online_advertising). What happens when people ask such a language model for _its_ preferences? + +Use ChatGPT as a cop-out. Sometimes, _it's okay to cop out._ But don't cop-out of what you love to do, or want to do well. +