Apple Inc. has released an opensource A.I model OpenELM (Open-source Efficient Language Models) on Hugging Face. Apple's OpenELM comprises a family of eight open-source large language models (LLMs), ranging from 270 million to 3 billion parameters. These extensive combination of parameters, weights and biases are credited for the model's performance.
In their research paper on OpenELM, Apple highlights its superiority over the Allen Institute for AI's OLMo Models by 2.36%. Even more impressive, OpenELM achieves this while requiring only half the number of pre-training tokens.
Understanding the composition of OpenELM is key. It consists of four pre-trained models and four instruct models. Pre-trained models are AI models trained on large datasets, capable of performing various tasks and further fine-tuned for specific purposes. On the other hand, instruct models are AI models fine-tuned to follow specific instructions.
The data used to train OpenELM models is drawn from a variety of publicly available datasets, including Wikipedia, Wikibooks, Reddit, Github, and Project Gutenberg. However, it's important to note that Apple acknowledges these models may not be without flaws. Due to the nature of publicly available datasets, there's a possibility of producing inaccurate, harmful, biased, or objectionable outputs in response to user prompts. Hence, Apple emphasizes the need for thorough safety testing and implementing appropriate filtering mechanisms tailored to specific requirements.
One significant aspect of Apple's OpenELM release is its level of openness. Unlike previous practices that often limited access to model weights and inference code, Apple provides the complete framework for training and evaluating the language model on publicly available datasets, including training logs, multiple checkpoints, and pre-training configurations.
The model is released under the Apple Sample Code License, allowing developers/users to use, reproduce, modify, and redistribute it with or without changes, as long as certain licensing and disclaimer requirements are met.
Comparing Apple's approach to open-source with that of its competitors, such as Meta, reveals a distinct difference. While Meta's open-source releases came with certain restrictions, Apple's OpenELM is more open and accessible. Despite this, the only "True Free and Open Source" licensed model is MistralAI with Apache Licence.
In the broader context of Apple's AI endeavors, OpenELM represents a significant milestone. Unlike its competitors, Apple has not ventured into commercial model releases like Google's Gemini or Microsoft's Phi-3. Instead, Apple has been steadily expanding its AI portfolio through acquisitions, including Darwin AI, a Canadian startup.
Furthermore, rumors suggest Apple may be exploring to potentially bringing new AI-powered features to Apple's flagship devices as seen with Samsung flaships GalaxyAI. This collaboration could signal a new phase in Apple's AI journey, blending cutting-edge technologies from different industry leaders to enhance user experiences.