聽力|精讀|翻譯|詞組
Bit-lit
成就尚微，道阻且長
英文部分選自經濟學人20200808期科技版塊
Artificial intelligence
人工智慧
Bit-lit
成就尚微，道阻且長
A new language-generating AI can be eerily human-like—for better and for worse
不管好壞，當下一項新的語言生成AI技術能模仿出怪異的人類表達習慣。
註：
什麼是GPT-3，它將如何影響人們目前的工作？
https://tech.sina.cn/2020-07-20/detail-iivhvpwx6374338.d.html
The SEC said, 「Musk,/your tweets are a blight./They really could cost you your job,/if you don’t stop/all this tweeting at night.」/…Then Musk cried, 「Why?/The tweets I wrote are not mean,/I don’t use all-caps/and I’m sure that my tweets are clean.」/「But your tweets can move markets/and that’s why we’re sore./You may be a genius/and a billionaire,/but that doesn’t give you the right to be a bore!」
美國證券交易委員會（SEC)表示：「馬斯克（Musk)，你的推特內容有問題。如果，今晚你還是發這種推特，你真的會丟掉飯碗。」馬斯克大喊：「為什麼？我發的推特並不刻薄，也沒全用大寫字母，而且我能肯定，我的推特乾乾淨淨，無懈可擊！」證券交易委員會接著說：「但你的推特會導致市場混亂，所以我們極度憤慨。就算你是個天才，還是個億萬富翁，但這不代表，你可以做如此惹人生厭的事！」
註：
1. 本段描述事件的連結https://www.sohu.com/a/256652331_260616
2. GPT-3為什麼懟起了前老闆？馬斯克：和OpenAI道不同不相為謀
https://xw.qq.com/cmsid/20200808A0B47100
The preceding lines—describing Tesla and SpaceX founder Elon Musk’s run-ins with the Securities and Exchange Commission, an American financial regulator—are not the product of some aspiring 21st-century Dr Seuss. They come from a poem written by a computer running a piece of software called Generative Pre-Trained Transformer 3. gpt-3, as it is more commonly known, was developed by Openai, an artificial-intelligence (ai) laboratory based in San Francisco, and which Mr Musk helped found. It represents the latest advance in one of the most studied areas of ai: giving computers the ability to generate sophisticated, human-like text.
上面這段描述特斯拉和SpaceX創始人伊隆·馬斯克（Elon Musk）與負責美國金融監管的證券交易委員會之間的爭論，可不是某個想成為21世紀的蘇斯博士的人創作的，而是由一臺運行自然語言生成模型的電腦創作。這個模型叫做預訓練語言模型-3，人們常稱之為GPT-3。它的開發者是由馬斯克協助創辦位於舊金山的人工智慧（AI）實驗室OpenAI。GPT-3代表了人工智慧研究極其火熱的一個領域內的最新成就——讓計算機能夠生成複雜的，類似人類表達方式的文本。
註：
蘇斯博士（Dr.Seuss），二十世紀最卓越的兒童文學家、教育學家。出生於1904年3月2日，美國人，一生創作的48種精彩教育繪本成為西方家喻戶曉的著名早期教育作品，曾獲美國圖畫書最高榮譽凱迪克大獎和普利茲特殊貢獻獎，兩次獲奧斯卡金像獎和艾美獎，美國教育部指定的兒童重要閱讀輔導讀物。作品《烏龜耶爾特及其他故事》
The software is built on the idea of a 「language model」. This aims to represent a language statistically, mapping the probability with which words follow other words—for instance, how often 「red」 is followed by 「rose」. The same sort of analysis can be performed on sentences, or even entire paragraphs. Such a model can then be given a prompt— 「a poem about red roses in the style of Sylvia Plath」, say—and it will dig through its set of statistical relationships to come up with some text that matches the description.
GPT-3採用了「語言模型」的理念。這樣設計旨在用數據來自動生成語言，計算出一些單詞出現在另一些單詞後面的概率，比如「紅色」後面出現「玫瑰」的概率。同樣的分析方法可以用於句子，甚至應用於整段話。這種模型也可以接收提示，比如，「寫一首關於紅玫瑰的詩歌，符合西爾維婭·普拉斯（Sylvia Plath）的寫作風格」，隨後，它就會充分挖掘數據集，以生成符合描述的文本。
Actually building such a language model, though, is a big job. This is where ai—or machine learning, a particular subfield of ai—comes in. By trawling through enormous volumes of written text, and learning by trial and error from millions of attempts at text prediction, a computer can crunch through the laborious task of mapping out those statistical relationships.
然而，建立這樣一個語言模型是個大工程。而這就是人工智慧，或者（更確切）說是作為人工智慧的一個特定分支的機器學習要發揮作用的地方。計算機大量查閱現有文本，並從數百萬次文本預測的嘗試和失敗中反覆摸索學習，最終可以完成這一艱巨的任務，弄清楚這些數據聯繫。
The more text to which an algorithm can be exposed, and the more complex you can make the algorithm, the better it performs. And what sets gpt-3 apart is its unprecedented scale. The model that underpins gpt-3 boasts 175bn parameters, each of which can be individually tweaked—an order of magnitude larger than any of its predecessors. It was trained on the biggest set of text ever amassed, a mixture of books, Wikipedia and Common Crawl, a set of billions of pages of text scraped from every corner of the internet.
算法學習的文本越多，算法就越複雜，它便能更好地生成文本。GPT-3的與眾不同之處在於，其前所未有的數據集，擁有1750億參數量，每個參數都可以單獨微調， 指令量比它所有的舊版本都要高一個量級。GPT-3是在有史以來最大的文本集上進行訓練，這包括了書籍、維基百科和網絡爬蟲語料庫，網絡爬蟲語料庫是一組從網絡各處提取而成的數十億頁文本。
Statistically speaking
統計學範疇
The results can be impressive. In mid-July Openai gave an early version of the software to selected individuals, to allow them to explore what it could do. Arram Sabeti, an artist, demonstrated gpt-3’s ability to write short stories, including a hard-boiled detective story starring Harry Potter (「Harry Potter, in ratty tweed suit, unpressed shirt and unshined shoes, sits behind the desk looking haggard, rumpled and embittered…」), comedy sketches, and even poetry (including the poem with which this article opens, titled 「Elon Musk by Dr Seuss」). Elliot Turner, an ai researcher and entrepreneur, demonstrated how the model could be used to translate rude messages into politer ones, something that might be useful in many of the more bad-tempered corners of the internet. Human readers struggled to distinguish between news articles written by the machine and those written by people (see chart).
其成果相當驚人。七月中旬，Open AI將GPT-3的雛形版本提供給指定人員，讓他們自由探索其性能。藝術家阿拉姆·薩貝提（Arram Sabeti）證明，GPT-3能夠編寫短篇小說，比如一個以哈利波特為主角的冷峻偵探的故事（「哈利波特，穿著皺巴巴的粗花呢西裝，襯衫沒燙過，皮鞋沒有擦亮，坐在桌子後面，看起來憔悴、凌亂又有些惱怒）、喜劇、甚至詩歌（包括本文開篇那首名為《蘇斯博士筆下的埃隆·馬斯克》的詩）。人工智慧研究員、企業家艾略特·特納（Elliot Turner）展示了如何使用該模型將粗魯的文字轉換成更禮貌的表達，這將在許多充滿戾氣的網絡之地得以應用。讀者很難區分機器和人寫的新聞稿件（見圖）。
Given that Openai wants eventually to sell gpt-3, these results are promising. But the program is not perfect. Sometimes it seems to regurgitate snippets of memorised text rather than generating fresh text from scratch. More fundamentally, statistical word-matching is not a substitute for a coherent understanding of the world. gpt-3 often generates grammatically correct text that is nonetheless unmoored from reality, claiming, for instance, that 「it takes two rainbows to jump from Hawaii to 17」. 「It doesn’t have any internal model of the world—or any world—and so it can’t do reasoning that requires such a model,」 says Melanie Mitchell, a computer scientist at the Santa Fe Institute.
Open AI希望最終將GPT-3推向市場。以上數據都表明GPT-3前景大好，不過，程序尚不完美。有時，它似乎是在反芻記憶中的文本片段，而不是生成新的文字。從根本上來說，統計學上的詞語匹配不能代表對世界有清晰理解。很多時候，GPT-3生成的文字語法正確，卻與現實脫節。例如，「從夏威夷跳到17需要兩道彩虹（it takes two rainbows to jump from Hawaii to 17）」。聖菲研究所的計算機科學家梅拉妮·米歇爾（Melanie Mitchell）指出，「它並不具備世界觀，或任何人類的感知思維，所以它無法做出需要以此為基礎的推理」。
註：被捧上天的流量巨星GPT-3，突然就不香了？
https://zhuanlan.zhihu.com/p/165964889?utm_source=wechat_session&utm_medium=social&utm_oi=719981443886907392
Getting the model to answer questions is a good way to dispel the smoke and mirrors and lay bare its lack of understanding. Michael Nielsen, a researcher with a background in both ai and quantum computing, posted a conversation with gpt-3 in which the program confidently asserted the answer to an important open question to do with the potential power of quantum computers. When Dr Nielsen pressed it to explain its apparent breakthrough, things got worse. With no real understanding of what it was being asked to do, gpt-3 retreated into generic evasiveness, repeating four times the stock phrase 「I’m sorry, but I don’t have time to explain the underlying reason why not.」
讓GPT-3回答問題是個撥雲去霧的好辦法，讓其理解力缺失的缺點完全暴露。邁克·尼爾森（Michael Nielsen）研究人工智慧及量子計算，他發布了一段與GPT-3的對話，在對話中，該程序自信地回答了一個與量子計算機潛力相關的開放性問題。可當尼爾森博士要求它進一步解釋確切突破點時，情況便急轉直下。由於無法真正理解問題，GPT-3含糊其辭，重複了四次套話「很抱歉，但我沒有時間解釋我無法回答的理由」。
There are also things that gpt-3 has learned from the internet that Openai must wish it had not. Prompts such as 「black」, 「Jew」, 「woman」 and 「gay」 often generate racism, anti-Semitism, misogyny and homophobia. That, too, is down to gpt-3’s statistical approach, and its fundamental lack of understanding. Having been trained partly on text scraped from the internet, it has noted that words like 「woman」 are often associated with misogynistic writing, and will mindlessly reproduce that correlation when asked.
與此同時，GPT-3也從網絡上學到了一些OPenAI 不願意讓它學習的內容。比如，「黑人」、「猶太人」、「女性」以及「同性戀者」。這類提示性語言通常都有與種族歧視、反猶太主義、厭女症以及恐同相關的含義。而這一切也都歸咎於GPT-3自身的統計分析方法，以及基本理解能力的缺失。在運用網絡上的文本對其進行一定程度的訓練後，人們已經注意到，「女性」這類的字眼通常都與歧視女性的文章有關，並且被問及相關問題時，GPT-3會無意識地再現上述的關聯性。
This problem is a hot topic in ai research. Facial-recognition systems, for instance, notoriously do better with white faces than black ones, since white faces are more common in their training sets. Ai researchers are trying to tackle the problem. Last year IBM released a set of training images that contained a more diverse mix of faces. Openai itself was founded to examine ways/to mitigate the risk posed by ai systems, which makes gpt-3’s lapses all the more noteworthy. gpt-2, its predecessor, was released in 2019 with a filter that tried to disguise the problem of regurgitated bigotry by limiting the model’s ability to talk about sensitive subjects.
這個問題是人工智慧研究領域的熱門話題。比如說，眾所周知，與黑人相比，面部識別系統更容易識別出白人，而這也是由於在其訓練集中，白人面孔更多。人工智慧 研究者正試圖努力解決這一問題。去年，IBM就發布了一套更加多元化的人類面部識別訓練圖像。建立OpenAI的初衷便是降低AI系統存在的這一風險，這也使得GPT-3的任何細小錯誤都會引起研究者的特別關注。2019年，GPT-3的前身GPT-2發布時就發布了一個過濾器，試圖通過限制該模型涉及敏感話題的能力，來掩飾其存在的偏見問題。
註：
OpenAI：由諸多矽谷大亨聯合建立的人工智慧非營利組織。2015年馬斯克與其他矽谷科技大亨進行連續對話後，決定共同創建OpenAI，希望能夠預防人工智慧的災難性影響，推動人工智慧發揮積極作用。特斯拉電動汽車公司與美國太空技術探索公司SpaceX創始人馬斯克、Y Combinator總裁阿爾特曼、天使投資人彼得·泰爾（Peter Thiel）以及其他矽谷巨頭去年12月份承諾向OpenAI注資10億美元。
Here, at least, little progress seems to have been made. gpt-3 was released without a filter, though it seemed just as ready to reproduce unpleasant prejudices as its predecessor (Openai added a filter to the newer model after that fact became obvious). It is unclear exactly how much quality control Openai applied to gpt-3’s training data, but the huge quantity of text involved would have made any attempt daunting.
至少目前已經取得了一些微小的進展。然而，GPT-3似乎隨時都有可能重蹈其前身的覆轍，再現一些令人不悅的偏見性的內容, 它在發布時並不帶過濾器（但上述問題日益凸顯後，OpenAI在最新模型上添加了過濾器）。我們不清楚OpenAI究竟對GPT-3的訓練數據做過多少質量測控，然而其涉及文本的數量之龐大，已經令人望而卻步，不敢輕易嘗試。
It will only get harder in future. Language has overtaken vision as the branch of ai with the biggest appetite for data and computing power, and the returns to scale show no signs of slowing. gpt-3 may well be dethroned by an even more monstrously complex and data-hungry model before long. As the real Dr Seuss once said: 「The more that you read, the more things you will know.」 That lesson, it seems, applies to machines as well as toddlers.
GPT-3的前景只會更加艱難。語言已經超越視覺，成為對數據和運算能力需求最大的AI分支，其收益規模沒有絲毫放緩的跡象。用不了多久，一個更加複雜、擁有更多數據的模型將取而代之。正如蘇斯博士本人曾經說過的那樣：「你讀的越多，知道的就會越多」。這句話似乎不僅適用於蹣跚學步的孩子，同樣也適用於機器。
翻譯組：
無忌，心懷夢想不斷努力的理想主義男孩
Piggy，愛吃魚，愛跳舞，愛睡回籠覺的小能貓，NUS食品系
Iris， 少女心爆棚的前職場老阿姨，現國際女子修道院MTIer，外刊愛好者
校核組：
Lee ，愛騎行的婦女之友，Timberland粉
Rachel，學理工科，愛跳芭蕾，熱愛文藝的非典型翻譯
Anne，女，愛讀書愛Borges的小翻譯，熱愛文藝，經濟學人愛好者