靜音抑制

Tech giants like Google and Microsoft have taken notice of exciting new AI research such as GPT-3, which can write articles, website markup, and even software code. But will their bottom lines stifle any real progress?

像Google和Microsoft這樣的科技巨頭已經注意到了激動人心的AI新研究，例如GPT-3，它可以編寫文章，網站標記甚至軟件代碼。 但是，他們的底線會扼殺任何實際進展嗎？

By Ben Dickson

通過本迪克森

A recent article in The Guardian stirred up a lot of excitement-and a little fear-on social media. The reason: The initial draft was reportedly written by GPT-3, OpenAI’s new text generator.

《衛報》上最近的一篇文章激起了社會媒體的興奮和恐懼。 原因：據報道，最初的草案是由OpenAI的新文本生成器GPT-3編寫的。

Since its beta release, GPT-3, an artificial intelligence system that takes a cue and generates text, has captivated the tech community and the media. Developers and computer scientists have been using it to write articles, website markup, and even software code. Some entrepreneurs are contemplating creating new products on GPT-3.

自測試版發布以來，具有提示功能并生成文本的人工智能系統GPT-3吸引了技術界和媒體。 開發人員和計算機科學家一直在使用它編寫文章，網站標記甚至軟件代碼。 一些企業家正在考慮在GPT-3上開發新產品。

While flawed in fundamental ways, GPT-3 still shows how far advances in natural language processing have come. This is by far the largest and most coherent text-generation algorithm ever created.

盡管GPT-3在根本上存在缺陷，但它仍然顯示出自然語言處理技術已經取得了多大進步。 這是迄今為止創建的最大，最一致的文本生成算法。

But it also highlights some of the problems the AI research community faces, including its growing dependence on the wealth of large tech companies. This is a problem that could endanger the scientific mission for which OpenAI and other AI research labs were founded.

但這也凸顯了AI研究界面臨的一些問題，包括其對大型科技公司財富的日益依賴。 這個問題可能會危及OpenAI和其他AI研究實驗室建立的科學使命。

GPT-3的費用 (The Cost of GPT-3)

GPT-3 is a massive deep-learning model. Deep learning is a type of AI system that develops its behavior through experience. Every deep learning model is composed of many layers of parameters that start at random values and gradually tune themselves as the model is trained on examples.

GPT-3是一個大規模的深度學習模型。 深度學習是一種通過經驗來發展其行為的AI系統。 每個深度學習模型均由多層參數組成，這些參數層從隨機值開始，并隨著在實例上進行訓練而逐漸進行自我調整。

Before deep learning, programmers and domain experts had to manually write the commands that defined the logic and rules to parse and make sense of text. With deep learning, you provide a model with a large corpus of text-say, Wikipedia articles-and it adjusts its parameters to capture the relations between the different words. You can then use the model for a variety of language tasks such as answering questions, automatic email-reply suggestions, and advanced search.

在進行深度學習之前，程序員和領域專家必須手動編寫定義邏輯和規則的命令，以解析和理解文本。 通過深度學習，您可以為模型提供大量的語料庫，例如Wikipedia文章，并且可以調整其參數以捕獲不同單詞之間的關系。 然后，您可以將該模型用于各種語言任務，例如回答問題，自動電子郵件回復建議和高級搜索。

Research and development in the past few years has shown that in general, the performance of deep-learning models improves as they are given larger numbers of parameters and trained on bigger data sets.

過去幾年的研究和開發表明，一般而言，深度學習模型的性能會得到提高，因為它們會被賦予更多的參數并在更大的數據集上進行訓練。

In this respect, GPT-3 has broken all records: It is composed of 175 billion parameters, which makes it more than a hundred times larger than its predecessor, GPT-2. And the data set used to train the AI is at least 10 times larger than GPT-2’s 40-gigabyte training corpus. Although there’s much debate about whether larger neural networks will solve the fundamental problem of understanding the context of language, GPT-3 has outperformed all of its predecessors in language-related tasks.

在這方面，GPT-3打破了所有記錄：它由1,750億個參數組成，這使其比其前身GPT-2大100倍以上。 用于訓練AI的數據集至少比GPT-2的40 GB訓練語料庫大10倍。 盡管關于大型神經網絡是否能夠解決理解語言上下文的根本問題仍有很多爭議，但GPT-3在與語言有關的任務中勝過了所有其前任。

But the benefits of larger neural networks come with trade-offs. The more parameters and layers you add to a neural network, the more expensive its training becomes. According to an estimate by Chuan Li, the Chief Science Officer of Lambda, a provider of hardware and cloud resources for deep learning, it could take up to 355 years and $4.6 million to train GPT-3 on a server with a V100 graphics card.

但是，更大的神經網絡的好處在于權衡取舍。 添加到神經網絡的參數和層越多，其訓練費用就越高。 根據用于深度學習的硬件和云資源提供商Lambda的首席科學官Chuan Li的估計，在配備V100顯卡的服務器上訓練GPT-3可能需要長達355年的時間和460萬美元。

“Our calculation with a V100 GPU is extremely simplified. In practice, you can’t train GPT-3 on a single GPU, but with a distributed system with many GPUs like the one OpenAI used,” Li says. “One will never get perfect scaling in a large distributed system due to the overhead of device-to-device communication. So in practice, it will take more than $4.6 million to finish the training cycle.”

“我們使用V100 GPU的計算得到了極大簡化。 在實踐中，您不能在單個GPU上訓練GPT-3，但要在具有許多GPU的分布式系統(例如使用的一個OpenAI)上進行訓練，” Li說。 “由于設備到設備通信的開銷，在大型分布式系統中永遠無法獲得完美的擴展。 因此，在實踐中，完成培訓周期將需要超過460萬美元。”

This estimate is still simplified. Training a neural network is hardly a one-shot process. It involves a lot of trial and error, and engineers must often change the settings and retrain the network to obtain optimal performance.

該估計仍被簡化。 訓練神經網絡幾乎不是一shot而就的過程。 它涉及很多試驗和錯誤，工程師必須經常更改設置并重新培訓網絡以獲得最佳性能。

“There are certainly behind-the-scenes costs as well: parameter tuning, the prototyping that it takes to get a finished model, the cost of researchers, so it certainly was expensive to create GPT-3,” says Nick Walton, the co-founder of Latitude and the creator of AI dungeon, a text-based game created on GPT-2.

“當然還有幕后的成本：參數調整，獲得完整模型所需的原型，研究人員的成本，因此創建GPT-3肯定是昂貴的，”合作伙伴Nick Walton說道。 -Latitude的創始人和AI地下城的創造者，這是一款基于GPT-2創建的基于文本的游戲。

Walton said that the real cost of the research behind GPT-3 could be anywhere between 1.5 to 5 times the cost of training the final model, but he added, “It’s really hard to say without knowing what their process looks like internally.”

沃爾頓說，GPT-3背后的研究的實際成本可能是訓練最終模型的成本的1.5到5倍之間，但他補充說：“在不知道內部流程是什么樣的情況下，很難說。”

追求盈利模式 (Going to a For-Profit Model)

OpenAI was founded in late 2015 as a nonprofit research lab with the mission to develop human-level AI for the benefit of all humanity. Among its founders were Tesla CEO Elon Musk and Sam Altman, former Y Combinator president, who collectively donated $1 billion to the lab’s research. Altman later became the CEO of OpenAI.

OpenAI成立于2015年末，是一個非營利性研究實驗室，其使命是開發造福全人類的人類級AI 。 其創始人包括特斯拉首席執行官埃隆·馬斯克(Elon Musk)和前Y Combinator總裁薩姆·奧特曼(Sam Altman)，他們共同為該實驗室的研究捐款10億美元。 奧特曼后來成為OpenAI的首席執行官。

But given the huge costs of training deep-learning models and hiring AI talent, $1 billion would cover only a few years’ worth of OpenAI’s research. It was clear from the beginning that the lab would run into cash problems long before it reached its goal.

但是考慮到培訓深度學習模型和雇用AI人才的巨額成本，10億美元將僅涵蓋OpenAI研究的幾年價值。 從一開始就很明顯，實驗室在達到目標之前就陷入了現金問題。

“We’ll need to invest billions of dollars in upcoming years into large-scale cloud compute, attracting and retaining talented people, and building AI supercomputers,” the lab declared in 2019, when it renamed itself OpenAI LP and restructured to a “capped-profit” company. The change allowed venture capital firms and large tech companies to invest in OpenAI for returns “capped” at a hundred times their initial investment.

該實驗室在2019年宣布時將其命名為OpenAI LP，并改組為“封閉式”實驗室，該實驗室于2019年宣布：“未來幾年，我們需要投資數十億美元用于大規模云計算，吸引和留住人才，并建造AI超級計算機。” -利潤”公司。 這一變化使風險投資公司和大型科技公司可以在OpenAI上進行投資，以將其“最高”回報限制為初始投資的100倍。

Shortly after the announcement, Microsoft invested $1 billion in OpenAI. The infusion of cash allowed the company to continue to work on GPT-3 and other expensive deep-learning projects. But investor money always comes with strings attached.

宣布后不久，微軟向OpenAI投資了10億美元。 注入的現金使公司得以繼續從事GPT-3和其他昂貴的深度學習項目。 但是投資者的錢總是附帶條件。

向模糊過渡 (Shifting Toward Obscurity)

In June, when it announced GPT-3, the company did not release its AI model to the public, as is the norm in scientific research. Instead, it released an application programming interface (API) that allows developers to give GPT-3 input and obtain the results. In the future, the company will commercialize GPT-3 by renting out access to the API.

6月，當它宣布GPT-3時，該公司并未像科學研究的規范那樣向公眾發布其AI模型。 相反，它發布了一個應用程序編程接口(API)，允許開發人員提供GPT-3輸入并獲取結果。 將來，該公司將通過出租API的訪問權來實現GPT-3的商業化。

“Commercializing the technology helps us pay for our ongoing AI research, safety, and policy efforts,” OpenAI wrote in a blog post announcing the GPT-3 API.

OpenAI在宣布GPT-3 API的博客文章中寫道：“將該技術商業化有助于我們為正在進行的AI研究，安全和政策工作付費。”

But to make GPT-3 profitable, OpenAI will have to make sure other companies can’t replicate it, which is why it is not making the source code and trained model public. Organizations and individuals can request access to the GPT-3 API-but not every request is approved.

但是，要使GPT-3盈利，OpenAI必須確保其他公司不能復制它，這就是為什么它不公開源代碼和經過訓練的模型。 組織和個人可以請求訪問GPT-3 API，但并非所有請求都得到批準。

Among those who weren’t given access to GPT-3 API are Gary Marcus, cognitive scientist and AI researcher, and Ernest Davis, computer science professor at New York University, who were interested in testing the capabilities and limits of GPT-3.

未被授予使用GPT-3 API權限的人包括認知科學家和AI研究人員Gary Marcus和紐約大學計算機科學教授Ernest Davis，他們對測試GPT-3的功能和限制感興趣。

“OpenAI has thus far not allowed us research access to GPT-3, despite both the company’s name and the nonprofit status of its oversight organization. Instead, OpenAI put us off indefinitely despite repeated requests-even as it made access widely available to the media,” Marcus and Davis wrote in an article published in MIT Technology Review. “OpenAI’s striking lack of openness seems to us to be a serious breach of scientific ethics, and a distortion of the goals of the associated nonprofit.”

“盡管公司的名稱和其監督組織的非營利組織身份，OpenAI迄今仍不允許我們對GPT-3進行研究訪問。 相反，盡管反復提出要求，OpenAI仍然無限期地推遲了我們的工作，即使它使媒體可以廣泛使用它。” Marcus和Davis在《麻省理工科技評論》上發表的一篇文章中寫道。 “ OpenAI明顯缺乏開放性，在我們看來，這嚴重違反了科學道德規范，并扭曲了相關非營利組織的目標。”

The two scientists managed to run the experiments through a colleague who had access to the API, but their research was limited to a small number of tests. Marcus had been a vocal critic of the hype surrounding GPT-3’s predecessor.

兩位科學家設法通過可以訪問API的同事進行了實驗，但是他們的研究僅限于少量測試。 馬庫斯曾大聲批評圍繞GPT-3的前身大肆宣傳。

可以保存AI研究嗎？ (Can AI Research Be Saved?)

GPT-3 shows the growing challenges of scientific AI research. The focus on creating larger and larger neural networks is increasing the costs of research. And, for the moment, the only organizations that can dispense that kind of money are large tech companies such as Google, Microsoft, and SoftBank.

GPT-3顯示了AI科學研究日益嚴峻的挑戰。 對創建越來越大的神經網絡的關注增加了研究成本。 而且，目前，唯一可以分配這類資金的組織是大型科技公司，例如Google，Microsoft和SoftBank。

But those companies are interested in short-term returns on investment, not long-term goals that benefit humanity in its entirety.

但是那些公司對短期投資回報感興趣，而不是對整個人類有利的長期目標。

OpenAI now has a commitment to Microsoft and other potential investors, and it must show proof that it is a profitable company to ensure future funding. At the same time, it wants to pursue its scientific mission of creating beneficial AGI (artificial general intelligence, essentially human-level AI), which does not have short-term returns and is at least decades away.

OpenAI現在對微軟和其他潛在投資者有承諾，并且必須證明它是確保未來資金投入的盈利公司。 同時，它希望履行其科學使命，即創建有益的AGI(人工通用情報，本質上是人類水平的AI)，該投資沒有短期回報，而且至少需要幾十年的時間。

Those two goals conflict in other ways. Scientific research is predicated on transparency and information sharing among different communities of scientists. In contrast, creating profitable products requires hiding research and hoarding company secrets to keep the edge over competitors.

這兩個目標在其他方面存在沖突。 科學研究基于不同科學家社區之間的透明度和信息共享。 相反，創造有利潤的產品需要隱藏研究和ho積公司機密，以保持領先于競爭對手的優勢。

Finding the right balance between the nonprofit mission and the for-profit commitment will be extremely difficult. And OpenAI’s situation is not an isolated example. DeepMind, the UK-based research lab that is considered one of OpenAI’s peers, faced similar problems after it was acquired by Google in 2014.

在非營利組織的使命與營利性承諾之間找到適當的平衡將非常困難。 而且OpenAI的情況并非孤立的例子。 DeepMind是位于英國的研究實驗室，被認為是OpenAI的同行之一，它在2014年被Google收購后也面臨類似的問題。

Many scientists believe that AGI-if ever achieved-will be one of the most impactful inventions of humanity. If this is true, then achieving AGI will require the concerted efforts and contributions of the international community, not merely the deep pockets of companies whose main focus is their bottom line.

許多科學家認為，如果能夠實現AGI，它將是人類最有影響力的發明之一。 如果這是真的，那么實現AGI將需要國際社會的共同努力和貢獻，而不僅僅是主要關注其底線的公司的財大氣粗。

A good model might be the Large Hadron Collider project, which obtained a $9 billion budget from funding agencies in CERN’s member and non-member states. While member states will eventually benefit from the results of CERN’s work, they don’t expect the organization to turn in profits in the short term.

一個很好的模型可能是大型強子對撞機項目，該項目從CERN成員國和非成員國的供資機構獲得了90億美元的預算。 盡管成員國最終將從CERN的工作成果中受益，但他們并不希望該組織在短期內實現盈利。

A similar initiative might help OpenAI and other research labs to continue chasing the dream of human-level AI without having to worry about returning investor money.

類似的舉措可能會幫助OpenAI和其他研究實驗室繼續追逐人類級AI的夢想，而不必擔心會退還投資者的錢。

Originally published at https://www.pcmag.com.

最初發布在https://www.pcmag.com 。

翻譯自: https://medium.com/pcmag-access/is-research-for-profit-holding-back-ai-innovation-c335a8dc44f7

靜音抑制

總結

以上是生活随笔為你收集整理的静音抑制_正在研究利润以抑制创新的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。