Falcon 180B: Diffusion Design by Bogdan Iancu

Technology Innovation Institute (TII) releases FALCON 180B LLM

On September 6th, Falcon 180B was released on Hugging Face

Falcon 180B is the largest openly available language model, with 180 billion parameters. It was trained on a massive 3.5 trillion tokens using TII’s RefinedWeb dataset, which represents the longest single-epoch pretraining for an open model.

Falcon 180B is the best openly released LLM today, outperforming Llama 2 70B and OpenAI’s GPT-3.5 on MMLU and is on par with Google’s PaLM 2-Large on HellaSwag, LAMBADA, WebQuestions, Winogrande, PIQA, ARC, BoolQ, CB, COPA, RTE, WiC, WSC, ReCoRD. Falcon 180B typically sits somewhere between GPT 3.5 and GPT4 depending on the evaluation benchmark.

Falcon 180B is available in the Hugging Face ecosystem, starting with Transformers version 4.33.

Falcon 180B is a powerful tool that can be used to improve efficiency and productivity in a variety of industries. It is a valuable tool for researchers and businesses alike, and it has the potential to revolutionize the way that language is used and understood.

Here are some additional details about Falcon 180B:

  • It is a scaled-up version of Falcon 40B and builds on its innovations such as multi-query attention for improved scalability.
  • It was trained on 3.5 trillion tokens on up to 4096 GPUs simultaneously, using Amazon SageMaker for a total of ~7,000,000 GPU hours.
  • The dataset for Falcon 180B consists predominantly of web data from RefinedWeb (~85%).
  • It can be used for commercial purposes but under very restrictive conditions.

Key features:

  • 180 billion parameters: Falcon 180B is one of the largest language models ever created, with more parameters than any other publicly available model. This gives it a significant advantage in terms of performance and capabilities.
  • Trained on 3.5 trillion tokens: Falcon 180B was trained on a massive dataset of text and code, which gives it a deep understanding of language and how it can be used.
  • State-of-the-art performance: Falcon 180B achieves state-of-the-art results across natural language tasks, including question answering, code generation, translation, summarization, and creative writing.
  • Openly available: Falcon 180B is available for anyone to use, which makes it a valuable tool for researchers and businesses alike.

Benefits:

  • Improved efficiency and productivity: Falcon 180B can be used to automate tasks, such as question answering and code generation, which can save time and improve productivity.
  • New insights: Falcon 180B can be used to generate new insights from data, such as by summarizing large amounts of text or translating languages.
  • Better decision-making: Falcon 180B can be used to help make better decisions by providing information and insights that would not be available otherwise.
  • Enhanced creativity: Falcon 180B can be used to generate creative text formats, such as poems, code, scripts, musical pieces, emails, letters, etc., which can help people be more creative.
  • Increased understanding of language: Falcon 180B can be used to better understand language and how it is used, which can be beneficial for a variety of tasks, such as machine translation and natural language processing.

Other AI News

  1. China’s Tencent unveils large language AI model, announces availability for enterprise use. On Thursday, September 7th, Chinese tech giant Tencent Holdings unveiled its much-anticipated large language artificial intelligence (AI) model, “Hunyuan,” a significant move in China’s rapidly progressing AI sector. In a live demo at a Shenzhen conference, the firm revealed that over 50 of its products and services have incorporated the Hunyuan Foundation.

According to Tencent’s Vice President Jiang Jie, the competition in China’s AI industry is heating up with over 130 sizable language models available as of July, leading to “a war of a hundred models.” Hunyuan, boasting over 100 billion parameters and utilizing more than 2 trillion tokens for training, stands as a notable contender in the market where other prominent firms like Baidu Inc and SenseTime Group have recently showcased their AI models.

Tencent, which claims the top spot as China’s most valuable internet company, positioned Hunyuan as a powerful tool, capable of performing efficiently in both Chinese and English. The company asserted that Hunyuan excels in tasks such as generating extensive texts and solving particular math problems, even outperforming OpenAI’s ChatGPT in certain respects. Additionally, it reportedly has a 30% lower rate of “hallucination,” a term used to describe AI’s presentation of incorrect information as facts, compared to Meta Platform Inc’s Llama 2 model.

  1. AI company Brand Engagement Network to list publicly through SPAC deal

AI startup Brand Engagement Network (BEN) has agreed to go public through a $358 million merger with DHC Acquisition Corp, a special purpose acquisition company (SPAC). The deal, which confirms earlier reports by Reuters, will grant BEN around $40 million in gross proceeds.

DHC’s Co-CEO and CFO, Chris Gaertner, highlighted BEN’s advantageous position due to its existing partnerships, which reduce its capital requirements compared to other companies going public through a SPAC.

Based in Jackson, Wyoming, BEN specializes in developing AI-powered chatbots and conversational AI technologies utilized in various sectors including automotive, healthcare, and customer service. The firm stands out in the robust AI investment landscape, with AI and machine learning startups having amassed approximately $39.4 billion in funding globally this year, according to data from PitchBook.

Following the closure of the deal, the merged entity will adopt the name BEN and plans to list on the Nasdaq with the ticker “BNAI”. The transaction underscores the ongoing investor interest in AI-focused enterprises amid a vibrant yet challenging funding environment.

  1. Imbue secures $200 mln at over $1 bln valuation in AI funding Series B round

AI research lab Imbue has secured $200 million in a Series B fundraising round that included contributions from Astera Institute and Nvidia, catapulting the company’s valuation to over $1 billion, according to a blog post published by the company on Thursday. The injection of funds, inspired by the global sensation ChatGPT, is slated to fast-track Imbue’s endeavour to create AI systems capable of reasoning and coding.

  1. Pentagon considers massive AI fleet to counter China, Wall Street Journal reports

The Pentagon is contemplating the creation of a large network of AI-powered technology, drones, and autonomous systems in the next two years to counter Chinese threats, according to a Wall Street Journal report on Wednesday.

  1. Silicon Valley AI chip startup, d-Matrix, has secured $110 million in a Series B funding round, with notable investors including Microsoft Corp and Temasek. Despite a challenging fundraising environment for chip companies, partly due to Nvidia’s dominance in the AI chip sector, d-Matrix succeeded in attracting long-term investment for its energy-efficient chip technology designed to power generative AI applications, such as ChatGPT.

Led by Temasek and featuring contributions from Playground Global and Microsoft, the recent funding round comes as the Santa Clara firm gears up to launch its product next year, focusing on the “inference” segment of AI processing. The chips, known for low power requirements and high efficiency levels, are currently under Microsoft’s evaluation for potential use.

CEO Sid Sheth emphasized that the capital raised is from sources experienced in nurturing semiconductor businesses to success, hinting at a sustainable future for the startup that anticipates breaking even with annual revenue ranging between $70 million and $75 million within two years. The valuation remains undisclosed, with the company having previously raised $44 million.

  1. Chinese tech companies 360 Security Technology and iFlytek launched their artificial intelligence (AI) models to the public on Tuesday, following the necessary security assessments and approvals required in China. This step comes amid the Chinese government’s enhanced support for AI development, underlining the technology’s significant role in the competitive dynamics with the US. iFlytek introduced its “Spark” AI model specializing in voice recognition, while 360 Security Technology, known for antivirus software, unveiled its “Zhinao” AI model, as reported by state-backed Securities Times. This follows recent moves by Baidu Inc and SenseTime Group to release ChatGPT-style chatbots after obtaining governmental authorization.
  2. Morgan Stanley is preparing to deploy an AI chatbot developed in collaboration with OpenAI, creators of ChatGPT, to assist financial advisors in managing client interactions more efficiently. After several months of trials involving 1,000 advisors, the tool will officially launch this month, streamlining the process of locating research materials and forms among numerous documents.

The AI’s future capabilities could encompass generating meeting summaries, drafting follow-up emails, updating sales databases, and arranging subsequent appointments, all pending client approval. The technology could also guide financial advisors in effectively handling areas such as taxation, retirement planning, and inheritance issues. However, the responsibility for investment advice will continue to rest with human advisors.

Sal Cucchiara, Morgan Stanley’s chief information officer for wealth and investment management, who facilitated the partnership with OpenAI, emphasized that the AI tool is envisioned as an enhancement rather than a replacement for human advisors.

This strategic move is aligned with Morgan Stanley’s broader objectives to bolster its wealth division, aiming for $10 trillion in assets under management. While the venture represents a significant leap in the utilization of AI in the banking sector, other industry giants are also advancing in the deployment of AI for diverse applications including data analytics and customer service.

  1. Zoom is enhancing its service offering with a newly rebranded AI tool named “Zoom AI Companion,” which aims to streamline user experiences by providing a range of assistance features. Subscribers to Zoom’s paid service will be the first to access the functionalities.

The AI companion is designed to offer real-time help during meetings, including crafting chat responses, summarizing ongoing discussions for late attendees, and identifying crucial details to generate post-meeting summaries and action points. Future iterations will enable users to engage with the tool using natural language to receive aid in various tasks, such as meeting preparations and analysing communications.

To optimize the assistant’s performance, Zoom will leverage a federated approach, utilizing large language models including Meta Llama 2 and offerings from OpenAI and Anthropic. This method aims to save users from the hassle of selecting the appropriate model to access new features.

Following concerns over a recent update to Zoom’s terms of service pertaining to data usage for training AI models, the company clarified its position, stressing transparency, and user control over AI functionalities. The statement highlighted that neither Zoom’s nor third-party AI models are trained using user content such as audio, video, or chat data. In the upcoming release, AI features will be disabled by default, giving account administrators and hosts detailed control over the deployment of AI tools during meetings, with participants being informed about the active AI utilities.

The move comes after the company faced criticism in April 2022 for considering the introduction of emotional AI features, which was met with backlash concerning user trust and potential bias. Zoom maintains its commitment to user-centric transparency and control over new AI integrations.

  1. SAP, a leader in enterprise resource planning (ERP), is set to acquire the German startup LeanIX, known for offering enterprises a comprehensive overview of their software usage and aiding in business transformation strategies. Though the financial specifics remain undisclosed, it is speculated that SAP expended over $1 billion in the acquisition, with aims to finalize the deal by the end of 2023’s fourth quarter.

LeanIX, which facilitates a unified and data-driven inspection of a company’s IT landscape, has fostered a robust clientele since its 2012 inauguration, including over 10% of Fortune 500 companies and half of Germany’s DAX 40 firms. The startup, also boasting a generative AI assistant for efficient documentation and IT recommendations, will soon integrate with SAP’s expansive transformation suite.

SAP CEO Christian Klein emphasized that the merger aspires to establish a groundbreaking suite to holistically assist clients in their business evolution, promoting a culture of continual adaptability and enrichment, grounded on an intricate understanding of IT applications and business processes. This collaboration hints at a prospective launch of self-optimizing applications and processes through generative AI, although the specifics are yet to be unveiled.

  • LeanIX will sustain its services across non-SAP landscapes while enhancing SAP’s existing platforms like Signavio, RISE with SAP, and the Business Technology Platform, offering an amplified, integrated perception of IT and business processes for SAP’s users.

With a noteworthy global footprint and around $120 million raised in investments, LeanIX stands as a valuable addition to SAP’s growth strategy, promising a future of AI-enabled modernization.

  1. Salesforce, the owner of the messaging app Slack, has unveiled Slack AI, a development aimed at enhancing collaborative work by introducing AI-driven features into the platform. The announcement comes a month after Slack revamped its user experience, receiving mixed reviews.

Rob Seaman, Slack’s SVP of enterprise product, highlighted that the platform is transforming into an “intelligent productivity platform” with a focus on collaboration, knowledge repository, and work automation. The newly introduced features include Channel recaps and Thread summaries that grant users AI-generated overviews of channels and conversations, facilitating quicker insights into the most crucial discussions without the need to review all preceding messages. This promises to be a boon for individuals rejoining ongoing conversations or returning from leaves of absence, enabling them to pinpoint vital information promptly.

In addition, the Search Answers function integrates AI into the platform’s search system, where users can pose natural language questions and receive succinct responses based on the collective data present in Slack, without replacing the existing search experience. These features, powered by Slack’s proprietary language models (LLMs) and hosted within a secure private cloud, ensure user data remains protected and compliant with Slack’s security standards.

Alongside AI advancements, Slack introduced Lists for better work management and an enhanced workflow builder, which integrates with various tools to streamline task automation without requiring coding knowledge. The Lists feature incorporates work management into communication flows, enabling users to oversee projects efficiently from initiation to completion. Moreover, a forthcoming automation hub will offer templates to facilitate the quick establishment and sharing of workflows.

Slack intends to trial Slack AI and work management features in the coming winter, eyeing a full launch in 2024. Meanwhile, the updated automation builder is presently available, with its hub slated for release later this month. Looking ahead, the company plans to delve deeper into AI capabilities, foreseeing further innovation in productivity enhancements.

About The Author

Bogdan Iancu

Bogdan Iancu is a seasoned entrepreneur and strategic leader with over 25 years of experience in diverse industrial and commercial fields. His passion for AI, Machine Learning, and Generative AI is underpinned by a deep understanding of advanced calculus, enabling him to leverage these technologies to drive innovation and growth. As a Non-Executive Director, Bogdan brings a wealth of experience and a unique perspective to the boardroom, contributing to robust strategic decisions. With a proven track record of assisting clients worldwide, Bogdan is committed to harnessing the power of AI to transform businesses and create sustainable growth in the digital age.