iOS Guides | Samples

CoreML Namespace

The CoreML namespace provides relatively high-level access to machine learning model runtimes.

Remarks

The CoreML namespace, introduced in iOS 11, allows runtime querying of a broad variety of machine-learning models produced by frameworks such as scipy-learn, TensorFlow, and Azure Custom Vision cognitive services. CoreML does not support on-device modification of models or weights, however it does support loading a model from a URL, so developers could use that to download updated models.

CoreML relies on a "model" that is distributed as a single .mlmodel file. This model is compiled into a usable form by either using integrated tools in Xcode, Xamarin Studio, or at the command-line. At the command line, an .mlmodel file can be compiled with xcrun coremlcompiler compile model.mlmodel outputfolder . A compiled model takes the form of a directory called modelname.mlmodelc . The model is then loaded at runtime with code similar to:

C# Example

var bundle = NSBundle.MainBundle;
var assetPath = bundle.GetUrlForResource("MarsHabitatPricer", "mlmodelc");
NSError mlErr;
model = MLModel.Create(assetPath, out mlErr);

Models in CoreML are fairly "black box" and do not have an extensive API. Rather, the iOS developer must know the input and output names and types the model expects. For instance, an image-recognition model might expect a CVPixelBuffer of size 227x227 identified as "image" and might have two outputs: a string identified as "classLabel" and a NSDictionary with NSString keys and double values in the range [0,1] representing the confidence of the prediction.

Developers must convert their native representations into CoreML-compatible instances of IMLFeatureProvider . The easiest way to do this is with a MLDictionaryFeatureProvider :

C# Example

CVPixelBuffer pixelBuffer = // from image or video stream
var imageValue = MLFeatureValue.Create(pixelBuffer);

var inputs = new NSDictionary<NSString, NSObject>(new NSString("image"), imageValue);

NSError error, error2;
var inputFp = new MLDictionaryFeatureProvider(inputs, out error);
if(error != null)
{
	ErrorOccurred(this, new EventArgsT<string>(error.ToString()));
	return;
}
var outFeatures = model.GetPrediction(inputFp, out error2);
if(error2 != null)
{
	ErrorOccurred(this, new EventArgsT<string>(error2.ToString()));
	return;
}

var predictionsDictionary = outFeatures.GetFeatureValue("classLabelProbs").DictionaryValue;

A more complex, but more flexible, way is to implement IMLFeatureProvider :

C# Example

public class MarsHabitatPricerInput : NSObject, IMLFeatureProvider
{
	public double SolarPanels { get; set; }
	public double Greenhouses { get; set; }
	public double Size { get; set; }

	public NSSet<NSString> FeatureNames => new NSSet<NSString>(new NSString("solarPanels"), new NSString("greenhouses"), new NSString("size"));

	public MLFeatureValue GetFeatureValue(string featureName)
	{
		switch (featureName)
		{
			case "solarPanels":
				return MLFeatureValue.Create(SolarPanels);
			case "greenhouses":
				return MLFeatureValue.Create(Greenhouses);
			case "size":
				return MLFeatureValue.Create(Size);
			default:
				return MLFeatureValue.Create(0);
		}
	}
}

Getting a prediction occurs synchronously, with a call to MLModel.GetPrediction :

C# Example

NSError prErr;
IMLFeatureProvider outFeatures = model.GetPrediction(pricerInput, out prErr);
double result = outFeatures.GetFeatureValue("price").DoubleValue;

CoreML currently supports:

TypeVariantsProduced by:
Neural networksConvolutional, feed-forward, recurrent Caffe, Keras, Azure Custom Vision
Tree ensemblesRandom forests, boosted trees, decision trees scikit-learn, XGBoost
SVMsScalar and multiclass scikit-learn, LIBSVM
Generalized linear modelsLinear and logistic regressionscikit-learn
Pipeline modelsSequentially chained models scikit-learn

Apple has open-sourced (3-clause BSD License) python tools to create CoreML models: https://pypi.python.org/pypi/coremltools

Classes

TypeReason
IMLFeatureProviderAn interface that defines an input or output feature and its value.
MLDictionaryConstraintContains a value that constrains the type of dictionary keys.
MLDictionaryFeatureProviderAn implementation of IMLFeatureProvider that is backed by a NSDictionary.
MLFeatureDescriptionA developer-meaningful description of a moder.
MLFeatureTypeEnumerates the kinds of features supported by CoreML.
MLFeatureValueAn immutable value and MLFeatureType for a feature.
MLImageConstraintContains constraints for an image feature.
MLModelEncapsulates a trained machine-learning model.
MLModelDescriptionA developer-meaningful description of the MLModel.
MLModelErrorEnumerates errors that may occur in the use of Core ML.
MLModelErrorExtensionsExtension method for MLModelError.
MLModelMetadataA DictionaryContainer that holds metadata related to a MLModel.
MLMultiArrayRepresents an efficient multi-dimensional array.
MLMultiArrayConstraintContains constraints for a multidimensional array feature.
MLMultiArrayDataTypeEnumerates the types of values stored in a MLMultiArray.
MLPredictionOptionsContains a value that indicates whether to restrict prediction computations to the CPU.