iOS Guides | Samples

MonoTouch.ARKit Namespace

The ARKit namespace provides support for augmented-reality sessions, including both high- and low-level APIs for projecting computer-generated imagery into a video stream.

Remarks

ARKit was added in iOS 11 and provides for mixed-reality sessions that combines camera input with computer-generated imagery that appears "attached" to the real-world.

ARKit is only available on devices running A9 and more-powerful processors: essentially iPhone 6S and newer, iPad Pros, and iPads released no earlier than 2017.

ARKit apps do not run in the Simulator.

Developers have three choices for rendering AR scenes:

ClassUse-case
ARSCNView Combine SceneKit 3D geometry with video
ARSCKView Combine SpriteKit 2D imagery with video
Export "renderer:updateAtTime:" from their IARSCNViewDelegate.Allows complete custom rendering.

ARKit coordinate systems and transforms

ARKit uses device motion and "visual odometry" to create a model of the device's camera and real-world "feature points" in relation to a virtual coordinate system. The coordinate system uses meters as its units. The virtual coordinate system has an origin calculated to be the camera's location at the time that the ARSession was started. Location and orientation within ARKit are primarily represented using NMatrix4 "native matrices". In the case of ARKit, these are column-major transforms:

Position or translation is in Matrix4.M14, Matrix4.M24, and Matrix4.M34. The 3x3 matrix defined by Matrix4.M11 to Matrix4.M33 is the rotation matrix.

C# Example

SCNVector3 Position(NMatrix4 m) => new SCNVector3(m.M14, m.M24, m.M34);

Initialization

The ARSession object manages the overall augmented-reality process. The ARSession.Run method takes a ARConfiguration and an ARSessionRunOptions object, as shown below:

C# Example

ARSCNView SceneView = ... // initialized in Storyboard, `ViewDidLoad`, etc.

// Create a session configuration
var configuration = new ARWorldTrackingConfiguration {
	PlaneDetection = ARPlaneDetection.Horizontal,
	LightEstimationEnabled = true
};

// Run the view's session
SceneView.Session.Run(configuration, ARSessionRunOptions.ResetTracking);

Once a ARSession is running, it's ARSession.CurrentFrame property holds the active ARFrame. Because the system attempts to run ARKit at 60 frames per second, developers who reference the ARSession.CurrentFrame must be sure to Dispose the frame after they have lost it.

The system tracks high-contrast "feature points" in the camera's view. These are available to the developer as a ARPointCloud object that can be read at ARFrame.RawFeaturePoints. Generally, however, developers rely on the system to identify higher-level features, such as planes or human faces. When the system identifies these higher-level features, it adds ARAnchor objects whose ARAnchor.Position properties are in the world-coordinate system. Developers can use the ARSCNViewDelegate.DidAddNode, ARSCNViewDelegate.DidUpdateNode, and ARSCNViewDelegate.DidRemoveNode methods to react to such events and to attach their custom geometry to real-world features.

The augmented-reality coordinates are maintained using visual odometry and the device's motion manager. Experimentally, the tracking seems very solid over distances of at least tens of meters in a continuous session.

Classes

TypeReason
ARAnchorA position, orientation, and scale that is located in the real world to which augmented reality objects can be attached.
ARCameraInformation about the location and orientation of the camera that captured an augmented-reality frame.
ARConfigurationConfiguration information for augmented reality sessions.
ARErrorCodeEnumerate causes for an ARSession failure.
ARErrorCodeExtensionsExtension method for ARSession.
ARFrameA frame in an augmented-reality session.
ARHitTestResultA result generated by the ARFrame.HitTest method.
ARHitTestResultTypeEnumerates the kinds of objects detected by the ARFrame.HitTest method.
ARLightEstimateAn estimate of the real-world lighting environment.
AROrientationTrackingConfigurationAn ARConfiguration that only tracks the device orientation and uses the device's rear-facing cameras.
ARPlaneAnchorA subclass of ARAnchor used to represent real-world flat surfaces.
ARPlaneAnchorAlignmentThe orientation of an ARPlaneAnchor (Currently restricted to horizontal).
ARPlaneDetectionEnumerates the valid orientations for detected planes (currently, only horizontal).
ARPointCloudA set of 3-dimensional points, indicating image-processing's belief in a fixed real-world point on a physical surface.
ARSCNDebugOptionsVisualization options for use with the SCNView.DebugOptions property of ARSCNView.
ARSCNViewA subclass of SCNView that supports augmented-reality content.
ARSCNView+ARSCNViewAppearance
ARSCNViewDelegateDelegate object for ARSCNView objects.
ARSCNViewDelegate_ExtensionsOptional methods of the IARSCNViewDelegate interface.
ARSessionManages the camera capture, motion processing, and image analysis necessary to create a mixed-reality experience.
ARSessionDelegateDelegate object for the ARSession object, allowing the developer to respond to events relating to the augmented-reality session.
ARSessionDelegate_ExtensionsOptional extension methods for the IARSessionDelegate interface.
ARSessionObserver_ExtensionsOptional methods of the IARSessionObserver interface.
ARSessionRunOptionsEnumerates options in calls to ARSession.Run.
ARSKViewA subclass of SKView that places Sprite Kit objects in an augmented-reality session.
ARSKView+ARSKViewAppearance
ARSKViewDelegateDelegate object allowing the developer to respond to events relating to a ARSKView.
ARSKViewDelegate_ExtensionsOptional methods of the IARSKViewDelegate interface.
ARTrackingStateEnumerates the quality of real-world tracking in an augmented-reality ARSession.
ARTrackingStateReasonEnumerates the causes of ARTrackingState.Limited.
ARWorldAlignmentEnumerates options for how the world coordinate system is created.
ARWorldTrackingConfigurationConfiguration for a session that tracks the device position and orientation, and optionally detects horizontal surfaces.
IARSCNViewDelegateInterface defining the ARSCNViewDelegate protocol.
IARSessionDelegateExtends the IARSessionObserver interface.
IARSessionObserverInterface defining methods that respond to events in an ARSession.
IARSKViewDelegateDelegate-object interface for the SKView sprite-kit view.