iOS Guides | Samples

Vision Namespace

The Vision namespace provides high-level image-recognition and registration facilities.

Remarks

The Vision namespace, introduced in iOS 11, provides a common interface for high-level image recognition, segmentation, and machine-learned tasks.

The Vision namespace provides a number of built-in image processing functions:

TaskRequest types
Face detection and geometry VNDetectFaceRectanglesRequest, VNDetectFaceLandmarksRequest
Barcode recognition VNDetectBarcodesRequest
Image registration VNTranslationalImageRegistrationRequest, VNHomographicImageRegistrationRequest
Text detection VNDetectTextRectanglesRequest
Horizon detection and straightening VNDetectHorizonRequest
Object detection and tracking VNDetectRectanglesRequest, VNTrackRectanglesRequest, VNTrackObjectRequest

In addition to the built-in functions, Vision supports flexible image-based queries to Monotouch.CoreMLMLModel objects. In contrast with the precise input requirements of CoreML, implementers of IVNTargetedImageRequest can accept a variety of image formats: CIImage, CGImage, and CVPixelBuffer objects. The system scaled and converts the image to the input format required by the MLModel.

In all cases, Vision requests and an image are passed to a VNImageRequestHandler, whose VNImageRequestHandler.Perform method executes a callback, passing one or more VNObservation objects of a request-appropriate type. For example:

C# Example

var rectangleRequest = new VNDetectRectanglesRequest(HandleRectangles);
var handler = new VNImageRequestHandler(img, orientation, new VNImageOptions());
DispatchQueue.DefaultGlobalQueue.DispatchAsync(()=>{
    NSError error;
    handler.Perform(new VNRequest[] {rectangleRequest}, out error);
    if (error != null)
    {
       ErrorOccurred(error); 
    }
});

void HandleRectangles(VNRequest request, NSError error){
	VNRectangleObservation[] observations = request.GetResults<VNRectangleObservation>();
  // ... etc ...

Classes

TypeReason
IVNFaceObservationAcceptingInterface for VNImageBasedRequest objects that may detect faces (e.g., VNDetectFaceLandmarksRequest).
VNBarcodeObservationA VNRectangleObservation in which a barcode was recognized.
VNBarcodeSymbologyEnumerated supported barcode standards.
VNBarcodeSymbologyExtensionsExtension methods relating to VNBarcodeSymbology objects.
VNClassificationObservationSubclass of VNObservation for CoreML models that predict a single feature.
VNCoreMLFeatureValueObservationA VNObservation that contains a feature detected by a Core ML model.
VNCoreMLModelWraps a CoreML model for use within the Vision namespace.
VNCoreMLRequestA subclass of VNImageBasedRequest that uses a Core ML model for processing.
VNDetectBarcodesRequestA subclass of VNImageBasedRequest that detects barcodes.
VNDetectedObjectObservationA VNObservation that includes the detection of a rectangular object.
VNDetectFaceLandmarksRequestA VNImageBasedRequest for recognizing face components.
VNDetectFaceRectanglesRequestA VNImageBasedRequest to retrieve the bounding boxes of any faces detected in the source.
VNDetectHorizonRequestA VNImageBasedRequest to retrieve the horizon in the source.
VNDetectRectanglesRequestA VNImageBasedRequest to retrieve rectangular regions detected in the source.
VNDetectTextRectanglesRequestA VNImageBasedRequest to retrieve the bounding boxes of any text blocks detected in the source.
VNErrorCodeEnumerates errors associated with Vision requests.
VNErrorCodeExtensionsExtension method for the VNErrorCode class.
VNFaceLandmarkRegionAbstract base class for classes holding information about facial landmarks.
VNFaceLandmarkRegion2DA VNFaceLandmarkRegion that holds two-dimensional information about a recognized facial landmark.
VNFaceLandmarksAbstract base class for classes that hold information about recognized facial landmarks.
VNFaceLandmarks2DA VNFaceLandmarks object that hold information about landmarks in 2D space.
VNFaceObservationA VNDetectedObjectObservation that includes a face.
VNHomographicImageRegistrationRequestA VNImageRegistrationRequest that attempts to use homography to align source images.
VNHorizonObservationA VNObservation that includes detection of a horizon line.
VNImageAlignmentObservationAbstract base class of VNObservation that hold information about aligning images.
VNImageBasedRequestAbstract base-class for VNRequests that operate on images.
VNImageCropAndScaleOptionEnumerates how an image should be cropped and scaled during processing.
VNImageHomographicAlignmentObservationAn observation whose results are a 3D warp transform for aligning two images .
VNImageOptionsA DictionaryContainer holding options to be used in Vision queries.
VNImageRegistrationRequestAbstract sub-class of VNTargetedImageRequest that attemps to align an image with a reference image.
VNImageRequestHandlerCoordinates the vision requests of a single image.
VNImageTranslationAlignmentObservationA VNImageAlignmentObservation in which the alignment is restricted to an affine transform.
VNObservationAbstract base class for vision processing results.
VNPixelBufferObservationA VNObservation that works on CVPixelBuffer object(s).
VNRectangleObservationA VNDetectedObjectObservation of a rectangular region.
VNRequestAbstract base-class for vision-processing request.
VNRequestCompletionHandlerA delegate that is called once for each feature detected in a VNRequest.
VNRequestTrackingLevelEnumerates the emphasis of the tracking algorithm.
VNSequenceRequestHandlerCoordinates the vision requests of a sequence of images (such as a video stream).
VNTargetedImageRequestAbstract VNImageBasedRequest for requests that operate on two images.
VNTextObservationA VNDetectedObjectObservation of a text block.
VNTrackingRequestAbstract VNImageBasedRequest that is the base class for requests that track an object between multiple images.
VNTrackObjectRequestA VNTrackingRequest that tracks a previously recognized object across multiple frames.
VNTrackRectangleRequestA VNTrackingRequest that tracks a rectangular region over multiple frames.
VNTranslationalImageRegistrationRequestA VNImageRegistrationRequest that restricts the registration result to an affine transform.
VNUtilsA set of utility functions for working with images.