Apple offers an open source reference PyTorch implementation of the Transformer architecture to help developers deploy Transformer models to Apple devices. In 2017, Google launched the Transformers models. Since then, it has become the model of choice for natural language processing (NLP) problems.
The transformers’ self-attention mechanism helps models focus on certain parts of the input and reason more effectively. Generative Preformed Transformer (GPT-3) and Bi-Directional Encoder Representations of Transformers (BERT) are some of the popular transformer designs.
Apple now leverages the Transformer architecture for a growing number of ML models. This architecture enables experiences such as panopticon segmentation in Camera with HyperDETR, on-device scene analysis in Photos, image captioning for accessibility, machine translation, and many more. .
Apple Neural Engines
Apple introduced its first Neural Engine in September 2017 as part of the Apple A11 ‘Bionic’ chip. In 2018, it released an API named Core ML to allow developers to take advantage of the Apple Neural Engine in the Apple A12.
In 2017, Neural Engine was only available on iPhone. Now it is available on iPad (from A12 chip) and Mac (from M1 chip).
At the recent Apple WorldWide Developers Conference (WWDC) 2022, Apple showcased the Apple M2 with 16 Neural Engine cores that could deliver over 40% faster performance than its predecessor.
Transformer’s architecture has impacted many fields, including NLP and computer vision. The reference implementation of PyTorch is specifically optimized for the Apple Neural Engine (ANE), which is a group of specialized cores functioning as a neural processing unit (NPU) to accelerate AI and ML workloads.
According to Apple, the implementation will help developers minimize the impact of their ML inference workloads on application memory, responsiveness, and device battery life. The growing adoption of on-device ML deployment will also go a long way to protecting user privacy, as data for inference workloads stays on-device.
Apple has shared four important principles behind the reference implementation to help developers optimize their models for ANE execution.
Principle 1: Choose the right data format
Principle 2: Grouping of large intermediate tensors
Principle 3: Minimization of memory copies
Principle 4: Bandwidth Management
What is the real motive?
Apple, in general, is not known for its contribution to AI and ML, even though the company has invested heavily in these technologies.
As a company, Apple behaves like a cult. Nobody knows what happens between the four walls of Apple. For the common man, Apple is a consumer electronics company unlike tech giants such as Google or Microsoft. Google, for example, is considered a leader in AI, with top AI talent working for the company and has published numerous research papers over the years. Google also owns Deepmind, another leading AI research company.
Apple is struggling to recruit top AI talent, and for good reason. “Apple, with its top-five employer branding, is currently struggling to recruit top AI talent. In fact, in order to allow potential recruits to see some of the exciting machine learning work going on at Apple, Apple recently had to change its incredibly secretive culture and offer a publicly viewable Apple Machine Learning Journal,” said Dr. John Sullivan, author. .
Over the past two years, Apple has increased its engagement with the AI/ML community.
In 2016, Apple announced that it would allow its AI and ML researchers to publish and share their work. Next year, the first Apple publicly released academic document won the Best Paper Award at the 2017 Computer Vision and Pattern Recognition Conference. Over the years, it pioneered AI/ML tools to accelerate machine learning on iPhones. For example, Apple started using deep learning for face detection in iOS 10. With the release of the Vision framework, developers can now use this technology and many other computer vision algorithms in their apps. “We faced significant challenges in developing the framework so that we could maintain user privacy and operate effectively on the device.” Apple also launched the “Apple Machine Learning Journal” website.
In 2020, the Cupertino-based tech giant announced a new residency program for AI and ML experts. The latest decision to open source a reference implementation of PyTorch for deploying the Transformer architecture on Apple Neural Engine also signals a change in Apple’s attitude toward open source.