iOS Guides | Samples

Vision Namespace

The Vision namespace provides high-level image-recognition and registration facilities.

Remarks

The Vision namespace, introduced in iOS 11, provides a common interface for high-level image recognition, segmentation, and machine-learned tasks.

The Vision namespace provides a number of built-in image processing functions:

TaskRequest types
Face detection and geometry VNDetectFaceRectanglesRequest, VNDetectFaceLandmarksRequest
Barcode recognition VNDetectBarcodesRequest
Image registration VNTranslationalImageRegistrationRequest, VNHomographicImageRegistrationRequest
Text detection VNDetectTextRectanglesRequest
Horizon detection and straightening VNDetectHorizonRequest
Object detection and tracking VNDetectRectanglesRequest, VNTrackRectanglesRequest, VNTrackObjectRequest

In addition to the built-in functions, Vision supports flexible image-based queries to CoreMLMLModel objects. In contrast with the precise input requirements of CoreML, implementers of IVNTargetedImageRequest can accept a variety of image formats: CIImage, CGImage, and CVPixelBuffer objects. The system scaled and converts the image to the input format required by the MLModel.

In all cases, Vision requests and an image are passed to a VNImageRequestHandler, whose VNImageRequestHandler.Perform method executes a callback, passing one or more VNObservation objects of a request-appropriate type. For example:

C# Example

var rectangleRequest = new VNDetectRectanglesRequest(HandleRectangles);
var handler = new VNImageRequestHandler(img, orientation, new VNImageOptions());
DispatchQueue.DefaultGlobalQueue.DispatchAsync(()=>{
    NSError error;
    handler.Perform(new VNRequest[] {rectangleRequest}, out error);
    if (error != null)
    {
       ErrorOccurred(error); 
    }
});

void HandleRectangles(VNRequest request, NSError error){
	VNRectangleObservation[] observations = request.GetResults<VNRectangleObservation>();
  // ... etc ...

Classes

TypeReason
IVNFaceObservationAcceptingInterface for VNImageBasedRequest objects that may detect faces (e.g., VNDetectFaceLandmarksRequest).
IVNRequestRevisionProviding
VNBarcodeObservationA VNRectangleObservation in which a barcode was recognized.
VNBarcodeObservationRequestRevision
VNBarcodeSymbologyEnumerated supported barcode standards.
VNBarcodeSymbologyExtensionsExtension methods relating to VNBarcodeSymbology objects.
VNClassificationObservationSubclass of VNObservation for CoreML models that predict a single feature.
VNCoreMLFeatureValueObservationA VNObservation that contains a feature detected by a Core ML model.
VNCoreMLModelWraps a CoreML model for use within the Vision namespace.
VNCoreMLRequestA subclass of VNImageBasedRequest that uses a Core ML model for processing.
VNCoreMLRequestRevision
VNDetectBarcodesRequestA subclass of VNImageBasedRequest that detects barcodes.
VNDetectBarcodesRequestRevision
VNDetectedObjectObservationA VNObservation that includes the detection of a rectangular object.
VNDetectedObjectObservationRequestRevision
VNDetectFaceLandmarksRequestA VNImageBasedRequest for recognizing face components.
VNDetectFaceLandmarksRequestRevision
VNDetectFaceRectanglesRequestA VNImageBasedRequest to retrieve the bounding boxes of any faces detected in the source.
VNDetectFaceRectanglesRequestRevision
VNDetectHorizonRequestA VNImageBasedRequest to retrieve the horizon in the source.
VNDetectHorizonRequestRevision
VNDetectRectanglesRequestA VNImageBasedRequest to retrieve rectangular regions detected in the source.
VNDetectRectanglesRequestRevision
VNDetectTextRectanglesRequestA VNImageBasedRequest to retrieve the bounding boxes of any text blocks detected in the source.
VNDetectTextRectanglesRequestRevision
VNErrorCodeEnumerates errors associated with Vision requests.
VNErrorCodeExtensionsExtension methods for the Vision.VNErrorCode enumeration.
VNFaceLandmarkRegionAbstract base class for classes holding information about facial landmarks.
VNFaceLandmarkRegion2DA VNFaceLandmarkRegion that holds two-dimensional information about a recognized facial landmark.
VNFaceLandmarksAbstract base class for classes that hold information about recognized facial landmarks.
VNFaceLandmarks2DA VNFaceLandmarks object that hold information about landmarks in 2D space.
VNFaceObservationA VNDetectedObjectObservation that includes a face.
VNFaceObservationRequestRevision
VNHomographicImageRegistrationRequestA T:VNImageRegistrationRequest that attempts to use homography to align source images.
VNHomographicImageRegistrationRequestRevision
VNHorizonObservationA VNObservation that includes detection of a horizon line.
VNImageAlignmentObservationAbstract base class of VNObservation that hold information about aligning images.
VNImageBasedRequestAbstract base-class for VNRequests that operate on images.
VNImageCropAndScaleOptionEnumerates how an image should be cropped and scaled during processing.
VNImageHomographicAlignmentObservationAn observation whose results are a 3D warp transform for aligning two images .
VNImageOptionsA DictionaryContainer holding options to be used in Vision queries.
VNImageRegistrationRequestAbstract sub-class of VNTargetedImageRequest that attemps to align an image with a reference image.
VNImageRequestHandlerCoordinates the vision requests of a single image.
VNImageTranslationAlignmentObservationA VNImageAlignmentObservation in which the alignment is restricted to an affine transform.
VNObservationAbstract base class for vision processing results.
VNPixelBufferObservationA VNObservation that works on CVPixelBuffer object(s).
VNRecognizedObjectObservation
VNRecognizedObjectObservationRequestRevision
VNRectangleObservationA VNDetectedObjectObservation of a rectangular region.
VNRectangleObservationRequestRevision
VNRequestAbstract base-class for vision-processing request.
VNRequestCompletionHandlerA delegate that is called once for each feature detected in a VNRequest.
VNRequestRevision
VNRequestTrackingLevelEnumerates the emphasis of the tracking algorithm.
VNSequenceRequestHandlerCoordinates the vision requests of a sequence of images (such as a video stream).
VNTargetedImageRequestAbstract VNImageBasedRequest for requests that operate on two images.
VNTextObservationA VNDetectedObjectObservation of a text block.
VNTextObservationRequestRevision
VNTrackingRequestAbstract VNImageBasedRequest that is the base class for requests that track an object between multiple images.
VNTrackObjectRequestA VNTrackingRequest that tracks a previously recognized object across multiple frames.
VNTrackObjectRequestRevision
VNTrackRectangleRequestA VNTrackingRequest that tracks a rectangular region over multiple frames.
VNTrackRectangleRequestRevision
VNTranslationalImageRegistrationRequestA VNImageRegistrationRequest that restricts the registration result to an affine transform.
VNTranslationalImageRegistrationRequestRevision
VNUtilsA set of utility functions for working with images.