Stability LM 2 1.6B :Multi-Modal Transformers Design using DALL-E3

Stability AI introduces Stable LM 2 1.6B Language Model

Introduction

  • Stability AI introduces Stable LM 2 1.6B, the first language model in the new Stable LM 2 series.
  • This model is a 1.6 billion parameter small language model, trained on multilingual data in English, Spanish, German, Italian, French, Portuguese, and Dutch.

Features

  • Multilingual Capability: Trained on approximately 2 trillion tokens for two epochs, incorporating data in seven languages.
  • Model Variants: Includes a base model and an instruction-tuned version.
  • Compact Size and Speed: Designed for fast experimentation and iteration with moderate resources.
  • Pre-Training Checkpoint Release: Provides the final pre-training checkpoint before the cooldown, including optimizer states, to facilitate fine-tuning and experimentation by developers.

Performance

  • Benchmark Performance: Outperforms models under 2 billion parameters on most tasks and even some larger models.
  • Multilingual Performance: Shows superior performance on translated versions of ARC Challenge, HellaSwag, TruthfulQA, MMLU, and LAMBADA compared to other models.
  • Few-Shot Performance: Demonstrates strong few-shot performance across general benchmarks outlined in the Open LLM Leaderboard.

Usage and Accessibility

  • Commercial and Non-Commercial Use: Available for use with a Stability AI Membership, catering to a wide range of users from individuals to enterprises.
  • Transparency in Training: Upcoming technical report to provide detailed data mix and training procedure for community reproduction.

Community and Support

  • Developer Support: Encourages responsible development and use of the model, acknowledging potential issues like high hallucination rates or toxic language.
  • Community Engagement: Offers updates and support through newsletters, social media, and a Discord community.

Conclusion

  • Stable LM 2 1.6B represents a significant advancement in small language models, offering high performance in a compact and efficient package.
  • Its multilingual capabilities and focus on speed and performance make it a valuable tool for developers and model creators in various fields.

Other AI News

  • Amazon Enhances Shopping Experience with AI Chatbot for Product Queries and Creative Interactions

Amazon is testing a new AI feature in its mobile apps for iOS and Android, allowing customers to ask specific questions about products. This AI tool can provide detailed information, such as the dimensions of a shelf or the lifespan of a battery, and even engage in more creative tasks like writing a Christmas carol about snow boots. Users can access this feature through the questions box on product pages, where the AI responds to both practical and whimsical queries. For instance, it can generate jokes about flash card readers, create bedtime stories about hard drives, and describe snow boots in a playful manner. However, the AI tool is programmed to maintain professionalism, declining to engage in flirtatious behavior when prompted.

Amazon’s introduction of this AI chatbot is part of a broader initiative to incorporate generative AI into its retail operations. The company has been using large language models to summarize product reviews, detect fake reviews, and recommend clothing sizes to customers. Additionally, Amazon’s enterprise cloud division, AWS, has been actively developing generative AI capabilities. In November, AWS launched the Titan Image Generator, a text-to-image prompt tool for generating pictures, and offers access to popular foundational AI models like Meta’s Llama 2, Anthropic’s Claude, and Stability AI’s Stable Diffusion. This expansion into AI-enhanced customer interaction reflects Amazon’s commitment to integrating advanced technology into its retail and cloud services.

  • Rabbit’s R1 AI Gadget to Use Perplexity for Real-Time, Up-to-Date Search Results

Rabbit, the company behind the popular AI gadget R1, has announced a partnership with Perplexity, integrating its conversational AI-powered answer engine into the R1 device. This collaboration aims to provide live, up-to-date answers without any knowledge cutoff, a significant advancement over traditional large language models (LLMs) that are limited to historical data. The first 100,000 Rabbit R1 purchases will come with a one-year subscription to Perplexity Pro, which normally costs $20 per month and includes access to newer LLMs like GPT-4.

The Rabbit R1, designed by Teenage Engineering and priced at $199, has already seen substantial interest with 50,000 preorders sold. It features a 2.88-inch touchscreen, scrolling navigation wheel, and a rotating camera, functioning as a universal controller for various apps. The device is capable of tasks ranging from sending messages to making online orders, powered by its “Large Action Model” AI. Perplexity AI, which combines search with LLMs like OpenAI’s GPT-3.5, will work behind the scenes on the R1 device alongside other unnamed leading LLMs, all without a subscription. This partnership between Rabbit and Perplexity represents a significant step in enhancing AI-powered devices with real-time, accurate information access.

  • Google Announces $1 Billion Investment in New London Data Center to Boost UK Tech and AI Growth

Google has announced a $1 billion investment to build a data center in Waltham Cross, near London, as part of its ongoing expansion in the UK. This investment, following the company’s acquisition of office buildings in central London and King’s Cross in 2022, reinforces its commitment to the region’s growing technology and artificial intelligence sector. The data center, situated on a 33-acre site acquired in 2020, aims to support the increasing demand for AI and cloud services in the UK. The British government, led by Prime Minister Rishi Sunak, has welcomed this investment as a significant vote of confidence in the UK’s tech industry.

Alphabet CFO Ruth Porat highlighted that the new facility will not only meet the rising need for Google’s services but also create construction and technical jobs. Additionally, the data center’s design includes energy conservation measures, with plans to utilize waste heat for the benefit of the local community. This development comes shortly after Microsoft’s announcement of a £2.5 billion ($3.2 billion) investment in the UK, emphasizing the growing importance of the region as a hub for technology and AI development. Google, which already employs over 7,000 people in Britain, continues to strengthen its presence and infrastructure in the country.

  • Databricks Customizes AI-Powered Data Intelligence Platform for Telecom Sector

Databricks has launched a specialized AI-powered data intelligence platform tailored for the telecommunications industry. Named the Data Intelligence Platform for Communications, this offering combines Databricks’ data lakehouse architecture with generative AI models from MosaicML and partner-powered solution accelerators. This platform enables communication service providers (CSPs) to efficiently utilize their datasets and enhance their business operations. The platform is designed to integrate structured, semi-structured, and unstructured data, facilitating the development of downstream AI/ML applications. With the acquisition of MosaicML last year, Databricks has strengthened its AI capabilities, leading to the rebranding of its Data Lakehouse as the Data Intelligence Platform.

This industry-specific platform allows telecom companies and network providers to consolidate all relevant business information, including unstructured data, from their customers. Using this data and various solution accelerators provided by partners, the platform accelerates workflows for common and high-value use cases. Some of the accelerators include large language models for building customer support chatbots, analytical dashboards for network reliability, geospatial analytics, and entity resolution dashboards for a comprehensive customer view. Databricks’ offering is already gaining traction, with enterprises like AT&T benefiting from it. This initiative is part of Databricks’ broader strategy to provide tailored solutions across various industries, including manufacturing and retail, similar to Snowflake’s approach with its industry-specific data clouds.

  • DataStax Launches New Data API to Streamline Building of Generative AI RAG Applications

DataStax, a leading commercial vendor of the open-source Apache Cassandra database, has introduced a new data API to simplify the creation of generative AI retrieval augmented generation (RAG) applications. This enhancement to AstraDB, DataStax’s cloud database-as-a-service, aims to bridge the gap between DataStax and purpose-built vector databases like Pinecone. The new API allows developers to use Python and JavaScript to access the database, making it easier to build applications and reducing the impedance mismatch between developers’ work and the database’s capabilities. Previously, developers had to use the Cassandra Query Language (CQL), which required more data modeling knowledge and was not as optimized for vector data.

The new data API automatically handles vectorization, presents a simpler interface in languages like Python and JavaScript, and optimizes performance by storing and indexing vector data more efficiently at the database level. This approach improves performance compared to building on top of existing Cassandra APIs and data models. DataStax’s vector database advancement includes the JVector search engine, part of AstraDB, which uses the DiskANN algorithm for better retrieval capabilities. JVector and the data API are open-sourced, benefiting the Cassandra community and AstraDB customers. This development by DataStax reflects a commitment to making cloud services more accessible to developers working with generative AI applications.

  • Pecan AI Introduces Predictive Generative AI to Simplify AI Predictions for Businesses

Pecan AI, an eight-year-old startup specializing in predictive analytics, has launched Predictive GenAI, a new tool that combines modern generative AI capabilities with predictive machine learning. This innovative tool aims to make predictive AI more accessible for business users, particularly those closer to the business side of companies. Predictive GenAI features two key components: Predictive Chat, a chatbot-style interface for natural language queries, and Predictive Notebook, a proprietary SQL-based notebook for building predictive models. The notebook automates the transformation of a company’s native data into an AI-ready dataset for predictive modeling, addressing the challenge of preparing data in the proper format required for predictive modeling.

Pecan AI’s Predictive GenAI addresses the limitations of generative AI in making predictions and the complexity of predictive machine learning techniques. The company is also advancing in automating data preparation and feature engineering, with innovations to improve issues like data leakage in machine learning. This development represents a significant step in democratizing AI predictions for businesses, making complex predictive modeling more accessible and user-friendly.

  • Vicarius Secures $30M in Series B Funding to Advance AI-Powered Vulnerability Remediation

Vicarius, a New York-based AI-powered cybersecurity startup, has raised $30 million in a Series B funding round led by Bright Pixel Capital. This investment, which brings the company’s total funding to $56 million, will be used to enhance its automated vulnerabilities management capabilities, increasingly powered by AI technologies. Vicarius is revolutionizing the vulnerability management market with its end-to-end platform, vRx, which uses AI to automate the discovery, prioritization, and remediation of vulnerabilities. The company boasts over 400 customers, including Fortune 500 companies like PepsiCo, Hewlett Packard Enterprise, and Equinix, and has an active community of researchers contributing to its vSociety community.

Last summer, Vicarius released vuln_GPT, a tool that uses generative AI to help find and remediate software vulnerabilities. Since its introduction, vuln_GPT has seen significant improvements, including enhanced accuracy of generated scripts and expanded scope of remediation to include Linux and Apple macOS vulnerabilities. Vicarius is also developing x_comply, an AI-driven compliance and benchmark tool set to launch in 2024, aiming to streamline compliance documents and scripts for easier identification and addressing of compliance discrepancies. This funding and development indicate Vicarius’s growing influence in the field of automated cybersecurity solutions.

  • Meta Announces Development of Open Source Artificial General Intelligence (AGI)

In a significant move, Meta CEO Mark Zuckerberg announced that the company is developing open-source artificial general intelligence (AGI). This initiative involves bringing together two of Meta’s AI research teams, FAIR and GenAI, with the goal of building full general intelligence and open-sourcing it as much as possible. Zuckerberg’s vision is to make general intelligence widely available for everyone’s benefit, emphasizing advances in various AI areas, from reasoning to coding. This announcement aligns with Meta’s ongoing efforts in AI and the metaverse, as Zuckerberg highlighted the role of AI in future devices like Ray-Ban smart glasses, which will integrate AI to assist users in their daily activities.

This development from Meta comes after OpenAI CEO Sam Altman’s recent comments on AGI at the World Economic Forum in Davos, Switzerland, and amidst ongoing debates about open-source versus closed-source AI. The decision to open-source future AGI projects by Meta could significantly influence the AI landscape, especially considering recent concerns about the potential risks of open models, such as the presence of deceptive ‘sleeper agents’ in AI systems. Zuckerberg’s announcement is a clear indication of Meta’s commitment to leading in the field of AGI and contributing to the broader AI community.

  • Stability AI Unveils Stable Code 3B for Enhanced AI-Powered Code Generation

Stability AI has released Stable Code 3B, a new 3-billion parameter model focused on code completion capabilities for software development. This model, part of Stability AI’s suite of AI tools, can run locally on laptops without dedicated GPUs while still offering competitive performance against larger models like Meta’s CodeLLaMA 7B. Stable Code 3B is built on Stability AI’s Stable LM 3B natural language model and has been further trained on software engineering data, covering 18 different programming languages. It demonstrates leading performance in benchmark tests across multiple languages, including Python, Java, JavaScript, Go, Ruby, and C++. Early benchmarks indicate that Stable Code 3B matches or exceeds the completion quality of models over twice its size. The model is available for commercial use as part of Stability AI’s new membership subscription service, alongside other AI tools like Stable Diffusion image generation tools, StableLM Zephyr 3B for text content generation, Stable Audio for audio generation, and Stable Video for video generation.

  • Cohere in Talks to Raise Up to $1 Billion for AI Business Expansion

Cohere, a Toronto-based AI startup, is reportedly in talks to raise between $500 million and $1 billion in capital, according to a source familiar with the matter. This fundraising effort comes amid a surge of interest in AI, sparked by the success of ChatGPT, which led to AI startups attracting a significant portion of venture capital in the U.S. last year. Cohere, valued at $2.2 billion following a $270 million funding round in June, competes with OpenAI (ChatGPT’s creator) and Anthropic, focusing on business applications of AI to enhance professional efficiency. The company has already made strides in simplifying access to infectious disease information for a specialist business client.

At the World Economic Forum’s annual meeting in Davos, executives expressed keen interest in monetizing AI technologies. Cohere’s planned capital raise aims to fund AI development, talent recruitment, and sales expansion. Founded in 2019, Cohere has been working to make its technology available across multiple cloud platforms and has strategically avoided financing in the form of cloud usage credits. While the exact valuation in the current funding discussions is unclear, Cohere’s CEO Aidan Gomez, a co-author of a key paper on AI architecture, has not yet commented on the matter.

  • Tokyo Startup Sakana AI Raises $30M for Compact AI Model Development Inspired by Nature

Sakana AI, a Tokyo-based startup co-founded by former Google engineers, has secured $30 million in seed funding to develop smaller, more efficient AI models. The funding round, led by Lux Capital and Khosla Ventures, also saw participation from major Japanese tech companies like Sony, NTT, and KDDI. Sakana, which means “fish” in Japanese, is inspired by the collective behaviors of animal groups and aims to create AI models that work together efficiently, in contrast to the trend of building single gigantic models. The startup’s approach is a response to the computing resource and environmental concerns associated with training large AI systems. Sakana’s founders, David Ha and Llion Jones, bring significant expertise to the company, with Jones co-authoring a landmark paper on the Transformer model. This early-stage backing from both U.S. and Japanese investors reflects confidence in Sakana’s potential to pioneer a new AI paradigm and position Japan as a key player in strategic technology.

  • Pinecone Launches Innovative Serverless Vector Database to Address AI Hallucinations

Pinecone, a New York City-based startup, has announced a revolutionary serverless vector database architecture, marking a significant breakthrough in reducing AI hallucinations. This new architecture, designed to build more knowledgeable and cost-efficient AI applications, promises up to 50x cost reductions and eliminates infrastructure hassles. Key innovations include the separation of reads, writes, and storage to reduce workload costs, a unique architecture with vector clustering on top of blob storage for low-latency, low-cost, fresh vector search over nearly unlimited data sizes, and a multi-tenant compute layer for on-demand retrieval. Pinecone CEO Edo Liberty believes this serverless architecture is significant for the industry, enabling a new generation of generative AI applications that were previously impossible.

The launch of Pinecone serverless is a sign of the maturing generative AI ecosystem and tech stack. The product includes integrations with leading AI companies like Anthropic, Anyscale, Cohere, Confluent, Langchain, Pulumi, and Vercel, indicating a collaborative approach in the AI industry. Companies like Notion, Blackstone, Canva, Domo, and Gong are already working with Pinecone serverless, benefiting from the heavy machinery behind the scenes that makes it easy and cheap for large-scale indexing and providing RAG and knowledge over content. This development is a testament to the evolving generative AI technology stack, offering more efficient and effective solutions for businesses.

  • Google DeepMind’s AlphaGeometry AI Solves Geometry Problems at Olympiad Level

Google DeepMind has developed a new AI system, AlphaGeometry, capable of solving complex high school geometry problems at a level comparable to a human gold medalist in the International Mathematical Olympiad (IMO). AlphaGeometry combines a neural language model for generating intuitive ideas with a symbolic deduction engine that verifies them using formal logic and rules. The system was tested on 30 geometry problems from the IMO, solving 25 within the standard time limit of 4.5 hours, matching the average score of human gold medalists. This achievement demonstrates AI’s ability to reason logically and discover new mathematical knowledge.

AlphaGeometry’s success in geometry, a field that requires both creativity and rigidity, highlights the potential of AI in mathematics. The researchers developed a novel neuro-symbolic approach, leveraging neural networks for pattern recognition and symbolic systems for formal logic. This method allows the AI to solve difficult mathematical problems and even discover new theorems. The researchers hope that AlphaGeometry, which they have open-sourced, will inspire further research in mathematics, science, and AI. The system’s ability to generalize to unseen problems and automate the discovery and verification of new knowledge could accelerate human understanding across various disciplines.

About The Author

Bogdan Iancu

Bogdan Iancu is a seasoned entrepreneur and strategic leader with over 25 years of experience in diverse industrial and commercial fields. His passion for AI, Machine Learning, and Generative AI is underpinned by a deep understanding of advanced calculus, enabling him to leverage these technologies to drive innovation and growth. As a Non-Executive Director, Bogdan brings a wealth of experience and a unique perspective to the boardroom, contributing to robust strategic decisions. With a proven track record of assisting clients worldwide, Bogdan is committed to harnessing the power of AI to transform businesses and create sustainable growth in the digital age.