Can local AI use the Apple Neural Engine?

Separate the hardware, the framework and the model

The Neural Engine is an accelerator on supported Apple hardware. Core ML is a framework that can schedule inference across available processors. MLX is a different machine-learning framework. A downloaded model is yet another layer: its format and implementation must be supported by the runtime you select.

These names describe different parts of the stack. Running locally establishes where computation happens, not which processor executes it. Likewise, unified memory means processors can access shared memory; it does not mean every framework automatically supports every compute engine. MLX's documentation explicitly names CPU and GPU as its currently supported device types [3].

Transformers can use the Neural Engine, but implementation matters

Apple's 2022 transformer article describes an implementation optimized for Neural Engine execution and demonstrates it with DistilBERT, a model used there for sequence classification [1]. That is direct evidence that transformers are not categorically excluded from the hardware. It is not proof that any downloaded chat model can be switched to the Neural Engine.

The implementation changes include tensor layout, smaller intermediate tensors and avoiding unnecessary memory copies. Apple also explains that Core ML can blend CPU, GPU and Neural Engine execution when a model cannot run entirely on one engine [1]. This is model-conversion and optimization work, not simply turning on a generic AI setting.

For an application developer, the question is whether the exact model representation and operations map effectively to the target device. For someone who just wants local chat, a runtime that already supports the chosen model is usually a better starting point than a hardware porting project.

Core ML does not mean Neural Engine execution

MLComputeUnits controls which processors a Core ML model is allowed to use. The all option lets the operating system choose among available units. cpuAndGPU excludes the Neural Engine; cpuAndNeuralEngine allows the CPU and Neural Engine but excludes the GPU [2]. Notice that the latter still permits the CPU.

A useful counterexample comes from Apple itself: its 2024 Llama 3.1 deployment article uses Core ML but specifically targets the M1 Max GPU [4]. The authors explain their choice in terms of that model's memory demands and the device's combination of compute and bandwidth. It would be wrong to cite this as a Neural Engine benchmark.

That example does not establish that the GPU wins on every Apple chip or workload. It establishes the narrower and more useful point: framework branding cannot substitute for checking the chosen execution backend.

What this means if you use MLX

The current MLX documentation lists CPU and GPU support [3]. Do not assume that an MLX-based LLM application is secretly using the Neural Engine because it is designed for Apple silicon. Check whether the application also uses other frameworks for separate features; different parts of one app can have different execution paths.

An idle Neural Engine is therefore not by itself evidence that your MLX setup is broken. The model may be using the documented GPU path exactly as intended. Conversely, a tool claiming new ANE support needs its own evidence: supported models, software requirements, the actual backend and measured behavior on your device.

RepoRadar recommendation: do not replace a working runtime solely to make an accelerator utilization graph move. Compare the end-to-end task you care about, including output quality and sustained responsiveness. This is a decision rule, not a measured ranking of runtimes.

How to verify the backend and the benefit

Start by recording the application version, model identifier, model format and device. Read the runtime's backend documentation and logs. For a Core ML integration, inspect the compute-unit configuration and use Apple's performance-report and Core ML Instrument workflow to examine the model's execution [2][5]. Allowed units and observed activity are different evidence.

If you compare backends, keep the model, prompts and output settings consistent. Measure model-load time separately from repeated inference; for a generative model, distinguish time to first output from continued generation. Record memory use and the power or thermal conditions that could explain variation. Do not present numbers from a different model or device as your own.

• Confirm what the runtime supports before looking for a missing switch.
• Record actual execution evidence rather than a marketing label.
• Compare repeated runs under the same conditions and keep failed runs visible.
• Check output correctness after conversion or compression, not just latency.
• Treat energy savings as unverified unless you measured energy with a documented method.

Boundaries that remain even when inference is local

Choosing the Neural Engine does not decide whether an app sends telemetry, downloads models or calls a remote service for another feature. Apple's on-device deployment discussion concerns local model inference [4]; it is not a privacy audit of every application built on the framework. Review the full application data path before feeding it confidential material.

Experimental runtimes and reverse-engineered drivers deserve separate compatibility and security review. The same-day community discussion motivated this answer, but its claims about hardware generations are not used here as a specification. RepoRadar has not run these models on a Mac or reproduced a Neural Engine speedup in this review.

Next action: identify the backend your current app actually uses. If it is MLX, a CPU/GPU path matches the documented design. If you are building with Core ML, profile one supported model before investing in ANE-specific optimization. Keep it only if the measured result improves the real task.