Meta views Llama and its family of AI software as being comparable to Linux, the open source OS which provides a strong competitor for Microsoft's Windows. Zuckerberg's corporation is giving considerable backing to metaverse technologies whilst at the same time putting much of its energy into artificial intelligence. Its negative image in the country's capital, Washington D.C., has caused some agencies such as the GAO to be reluctant to make use of its language AI model.
At Meta's annual Connect conference last month, virtual reality fans assembled to learn about Mark Zuckerberg's considerable investment in the metaverse, the technology which is intended to shape the company's future. However, at this year's event, VR developers were saturated with talks about an area becoming increasingly pertinent in the present: artificial intelligence. Joseph Spisak, who joined the company as director of product development for generative AI two months earlier, joked during his session at Connect: "Don't tell Mark, but it seems less like mixed reality and more AI these days. Kind of like an AI conference, which is kind of in my wheelhouse." Panels about Meta's newest Quest 3 VR headset and augmented reality developer software were wedged between conversations about Llama, Meta's large language model (LLM), which has achieved enormity since OpenAI's ChatGPT chatbot was released in November, leading big tech corporations to rush to market with comparable offerings. Mark re-affirmed attendees of Connect that Llama was the strength behind the company's most recent digital assistants revealed at the conference. Even though Zuckerberg states the expansion of the nascent metaverse is essential to his company's triumph, AI has established itself as the market he's targeting right now. Meta perceives Llama and its suite of generative AI software as the open source alternative to GPT, OpenAI's LLM, and Google's PaLM 2, which fuels the search giant's Bard AI technology. Market specialists equate Llama's placement in generative AI to Linux, the open source opponent to Microsoft Windows, in the PC operating system sector. In the same manner Linux software popularized corporate servers globally and became an important component of the contemporary internet, Meta views Llama as the eventual digital framework holding up the forthcoming age of AI applications.
Llama has proved difficult to evaluate and complex for many investors to understand. For that reason, Meta has invested heavily in developing Llama 2, the version that came out in July, and related generative AI programs. Yann LeCun, the AI specialist hired by Zuckerberg in 2013, voiced his approval of the July launch on Twitter, referring to it as something that "will revolutionize the LLM market." Although Meta has given away the software for free to tech innovators, it does earn money from cloud-computing companies such as Microsoft and Amazon, who offer access to Llama 2 as one of their own AI services. Although Mark Zuckerberg believes that only in the long run will Llama 2 be able to produce significant revenue, he has declared that it won't be a substantial source of income in the near future.
Meta is hoping to gain advantages from Llama in other ways. During a July analysts meeting, Zuckerberg suggested that third-party developers could potentially produce "efficiency gains" which would make it more cost effective for Meta to operate its artificial intelligence (AI) systems. Financing head Susan Li declared the figure for 2023 is likely to be between $27 billion and $30 billion, a drop from the $32 billion of last year, but it may raise again in 2024 due to relevant investments in AI and data centers.In addition, the usage of the Llama project has the potential to draw powerful technologists who are acquainted with Meta's development plans. Facebook has previously used open source ventures, such as its PyTorch coding framework for machine learning applications, as a recruitment method, attracting talented specialists interested in being part of a cutting-edge program.Spisak was involved in the open source AI projects when he worked for Meta from 2018 to January 2023. He then left and joined Google after which he came back to Meta in July. Meta believes that developers will steadily improve Llama 2 and other connected AI software to enhance their efficiency, thus permitting Meta to outsource research and development activities to volunteers.Cai GoGwilt, the chief architect at legal tech startup Ironclad, said that the open source group had already been working on the original version of Llama to "make it quicker and make it run on a cellular phone." GoGwilt shared that his firm is awaiting the outcome of the committed developers on the development of Llama 2.“Part of the reason we're not using it right away is that we're more interested in what the open source community will do with it,” GoGwilt said.
Meta released the LLM Llama in February with variants ranging from 7 to 65 billion parameters, which in essence are variables that control the model's size and data processing. This is comparable to OpenAI's GPT; both are transformer neural networks, a form of AI software created by a Google team. To handle the expensive process of training massive AI models, such as Llama, Meta employed its Research SuperCluster computer, equipped with 16,000 Nvidia A100 GPUs --the AI world's go-to computer chips. Initially designed in Meta's Fundamental AI Research group, the Llama initiative has since been relocated to the generative AI group, headed by Ahmad Al-Dahle, an Apple veteran of over 16 years. Mark Zuckerberg announced the unit in February.
In January, the team began with the task of training Llama 2, supposedly using only public online data, distinct from all Facebook user data. It is still uncertain as to whether Meta will continue to integrate user data into Llama 3. In order to optimize efficiency, Nvidia's multi-billion dollar AI chips are attracting attention from Mark Zuckerberg and other prominent figures. Meta is already a big consumer of Nvidia, and given that it likely cost $20 million to train Llama 2 --a sum considerably higher than the estimated $2.4 million for its predecessor-- investing in Meta-endorsed applications is a smart move for Nvidia to reduce AI training expenses. To supplement this, Meta has its own developing AI chip projects, providing them with a possible differentiator from Nvidia's chips. Log10 CEO Arjun Bansal, a prior AI chip chief, suggested "It gives them some price negotiating room. Nvidia wants to charge a lot and they can be like, 'Hey, we got our own thing.'"
Nathan Lambert recalls the enthusiasm of his coworkers at AI startup Hugging Face when Meta unveiled its highly anticipated Llama 2. Lambert and his team worked tirelessly to make sure their infrastructure was ready for the influx of coders who wanted to take Llama 2 for a spin. Hugging Face, alongside Microsoft Azure and Amazon Web Services, was one of Meta's partners for the launch of Llama 2 and was arguably the most significant, as developers, AI researchers, and numerous companies utilize their platform to share code, data sets, and models, making it one of the foremost communities in the field. Though there is an array of open-source LLMs available, Lambert said Llama 2 is the most popular by far. “It’s the model that most people are playing with and most startups are playing with,” commented Lambert, who announced on October 4th that he was leaving Hugging Face, though he did not specify his destination.
The project has fruitfully created some dispute. Some in the industry deem Meta’s licensing agreement to use Llama 2 to be restrictive, clashing with the spirit of collaborative improvement and imagination. For instance, third-party developers must acquire Meta’s approval to employ Llama 2 if they incorporate the software into any products or services that had a minimum of 700 million monthly active users in the month before its July release. Detractors have argued that this clause was a strategy of barring competitors such as Snap or TikTok from utilizing Llama 2 for their own services. “It’s quite stringent,” commented Umesh Padval, a venture partner at Thomvest Ventures and investor in AI startup Cohere. “It seems as though Meta wants all the advantages of open source for their business while fending off competition.”
Lambert said Meta could better please the open-source community by providing more info on the distinct, hidden data sets used to train Llama 2, so developers could pick apart the training process more comfortably. Open-source adherents and privacy experts have been pushing for greater transparency into the kinds of data used to train LLMs, but businesses have not yet revealed many details. In a statement, a Meta spokesperson explained, “We believe in open innovation and don't desire to place undue limitations on how others can employ our model. However, we do wish people to use it correctly. This is a special commercial license that balances open access to the models with responsibility and safeguards to help address potential misuse.”
Despite its opponents, Meta's model has acquired a substantial amount of early traction. At Connect, the firm disclosed that “more than 30 million downloads of Llama-based models have been made through Hugging Face and more than 10 million of those came in the past 30 days alone.” Nvidia's Fan noted in his X post that Llama 2's new commercial license could entice additional businesses to experiment with the language model compared to the original Llama. He suggested that “AI researchers from big companies had been reticent of Llama-1 because of licensing issues, but now I think many of them will board the ship and contribute their influence.”
As of now, businesses investing in AI prefer to utilize commercially accessible LLMs, according to a recent TC Cowen survey of 680 firms in cloud computing. The survey revealed that 32% of those queried have used or plan to use commercially packaged LLMs like OpenAI's GPT-4 software, while 28% are focused on open source LLMs like Llama and Falcon, developed in the United Arab Emirates. Only 12% of respondents had planned on using in-house LLMs.
At the U.S. Government Accountability Office, Taka Ariga is exploring how the newest technologies like LLMs can aid the agency to carry out more efficient audits and investigations as part of the Innovation Lab. By the end of the year, Ariga and his team are planning to conclude their first experiment, determining if LLMs can be utilized to consolidate various GAO reports and data on a specific topic, along with other possibly related information from other agencies.Ariga commented, "The public or a member of Congress may ask what the GAO has done in the region of nuclear safety? Well, we've actually done much, but that's on a report-by-report basis; you can't conduct that kind of topical search."The GAO is currently employing AWS' Bedrock generative AI service to help the agency experiment with various popular LLMs, such as products made available by startups like Cohere and Anthropic. Even though AWS mentioned that Bedrock is soon going to support Llama 2, Ariga said the GAO is first testing Anthropic's Claude LLM and is unlikely to use Llama 2 because of Meta's questionable reputation in Washington.Meta has drawn the wrath of lawmakers for several reasons, like data privacy scandals, antitrust probes, and allegations that Facebook suppresses conservative voices, as Ariga pointed out, comparing Zuckerberg to Tesla and X boss Elon Musk. "Mark Zuckerberg is, just like Elon, a bit of a lightning rod when referring to political technology," he said.A spokesperson for Meta commented, "We are aware that AI has made many great contributions to society, but it also brings with it risks. We are devoted to constructing responsibly and we have numerous resources, such as our responsible use guide, to support those using Llama 2 to do so properly."Even among prospective customers that are unconcerned about reputational matters, Meta has to affirm that its LLM technology is superior. Nur Hamdan, a product manager at AI startup aiXplain, declared that OpenAI's GPT-4 performs better than Llama 2 at comprehending context for extended conversations. That implies GPT-4 is more inclined to generate conversations in a more realistic manner, according to Hamadan.Studies comparing GPT-4, Llama 2 and other LLMs have become commonplace. In one such test, researchers learned that GPT-4 was able to generate better software code than Llama 2. Meta has since launched a version of Llama 2 specifically for creating code. Here's why
Meta is up against some major competitors in today's market, including Amazon, Google, and well-funded startups like OpenAI and Cohere. All of them aim to establish themselves as the main source of next-generation apps. Meta has an ace up its sleeve though, turning to open-source software in contrast to other firms that are selling their technologies and pairing them with other services.
Guido Appenzeller, who has held senior roles at both VMware and Intel, remarked, "The likes of Google and Microsoft, they may have some conflicts of interest. Facebook, on the other hand, is putting their faith in open-source and democratizing it, granting broad access. It's something that packs a real powerful punch."
Microsoft's spokesperson emailed a statement which declared that the company will give customers the freedom of choice as to what model they want and that this could include both proprietary and open-source programs. They also added that these models all have their own advantages, and they are striving to make it easier for customers to select, adjust and safely deploy these tools to maximize the results.
Attempts to contact Amazon and Google for comment were unsuccessful.
It is expected that Llama, Meta's new platform, could have an effect similar to that of Kubernetes. This open-source program was introduced by Google in 2014 and completely shifted the game for many prominent startups, such as Docker and CoreOS, the latter of which Red Hat purchased in 2018. Meta, with Llama 2, is applying that same tactic to an industry that is predicted to be even bigger.
Padval of Thomvest, while being a fan of Facebook and acknowledging Mark's efforts in reinventing the company, conceded that "in this case, I don't know how it's going to evolve."
Meta is pitted against formidable opponents in the current market scene - Amazon, Google, OpenAI, Cohere - all working to secure their positions as the foundation of the next generation of apps. Meta's strategy of utilizing open source gives it an edge over those that focus on selling their products plus other services. Guido Appenzeller, who previously held senior roles at VMware and Intel, commented that the likes of Google and Microsoft may have some conflicts of interest, whereas Facebook is focusing on providing broad access to open source and thus democratizing it. Microsoft's spokesperson mentioned that they will give customers a range of options and let them choose what model they prefer. Amazon and Google, however, did not respond to requests for comment.
It is believed that Llama, Meta's new platform, can have a comparable influence to Kubernetes, an open-source program released by Google back in 2014. This software caused the business models of several hot startups, such as Docker and CoreOS, to transform dramatically, the latter of which was purchased by Red Hat in 2018. Meta, with Llama 2, is adopting a similar strategy but for a much bigger industry. Thomvest's Padval admires Mark's work in reinventing the company, though he is uncertain as to how this situation will play out.
Comments