A G3D::Widget that manages world-space G3D::MarkerEntitys based on polling a low-level XR instance. More...

Inherits G3D::Widget.

Classes
class	TrackedEntity

class	VoluntaryBlinkEvent
	Blinks used for gaze-directed user interfaces. More...

Public Member Functions
virtual Rect2D	bounds () const override
	Inherited from Surface2D. More...

const shared_ptr< Camera > &	cameraForView (int viewIndex) const

virtual float	depth () const override
	Inherited from Surface2D. More...

const shared_ptr< Camera > &	eyeCamera (int index) const

virtual void	fireEvent (const GEvent &event)
	Fire an event on the containing window. More...

virtual Ray	gazeRay (int eyeIndex) const
	World-space gaze ray for the given eye, reported directly from the GazeTracker. More...

WidgetManager *	manager () const

virtual void	onAfterEvents ()
	Invoked after all onEvent handlers for the current frame and before onUserInput for any other Widget. More...

virtual void	onAI ()

virtual void	onBeforeGraphics () override
	Updates the HMD cameras based on the current GApp::activeCamera()'s parameters, as well as the gaze data. More...

virtual bool	onEvent (const GEvent &event)
	Returning true consumes the event and prevents other GModules from seeing it. More...

virtual void	onNetwork ()

virtual void	onPose (Array< shared_ptr< Surface > > &surfaceArray, Array< shared_ptr< Surface2D > > &surface2DArray)
	Appends a posed model for this object to the array, if it has a graphic representation. More...

virtual void	onSimulation (RealTime rdt, SimTime sdt, SimTime idt) override
	Samples from the XR and then creates or updates the MarkerEntitys in the Scene as needed. More...

virtual void	onUserInput (UserInput *ui)

virtual bool	pollBlinkEvent (VoluntaryBlinkEvent &event)

virtual float	positionalEventZ (const Point2 &pixel) const
	The camera-space z position that this Widget considers this pixel to be at. More...

virtual Point2	projectedMonocularGazePoint (int eyeIndex, int viewIndex, const Rect2D &viewport) const
	2D screen-space view point for one eye. More...

virtual void	render (RenderDevice *rd) const override
	Inherited from Surface2D. More...

virtual Point3	robustBinocularGazePoint () const
	World space 3D gaze point inferred from the scene and the gaze rays. More...

virtual void	setDepth (float d)
	Called by the WidgetManager. More...

virtual void	setManager (WidgetManager *m)
	Called by the WidgetManager when this module is added to it. More...

float	standingHeadHeight () const
	Returns the height above ground of the top of the first player's head. More...

virtual OSWindow *	window () const
	Returns the operating system window that is currently rendering this Widget. More...

Static Public Member Functions
static shared_ptr< XRWidget >	create (const shared_ptr< XR > &xr, GApp *app)

static void	sort (Array< shared_ptr< Surface2D >> &array)
	Sorts from farthest to nearest. More...

static void	sortAndRender (RenderDevice *rd, Array< shared_ptr< Surface2D >> &array)
	Calls sort, RenderDevice::push2D, and then render on all elements. More...

Protected Member Functions
	XRWidget (const shared_ptr< XR > &xr, GApp *app)

void	processGazeTracker ()

Point3	robustBinocularGazePoint (const Ray gazeRay[2]) const

void	simulateBodyFrame (RealTime rdt)

Static Protected Member Functions
template<class T , class ... ArgTypes>
static shared_ptr< T >	createShared (ArgTypes &&... args)
	Like std::make_shared, but works for protected constructors. More...

Protected Attributes
class GApp *	m_app

Queue< VoluntaryBlinkEvent >	m_blinkEventQueue

RealTime	m_blinkStartTime = nan()
	If currently tracking a LONG_BLINK, this is the time at which it began. More...

shared_ptr< MarkerEntity >	m_body

shared_ptr< Entity >	m_controllerEntity [2]

float	m_depth

Table< String, shared_ptr< TrackedEntity > >	m_entityTable

Point3	m_gazePoint
	This is NaN when it needs to be updated. More...

Ray	m_gazeRay [2]
	In world space. More...

RealTime	m_lastFastHandMovementTime = 0.0f
	The last time either hand moved quickly. More...

Vector3	m_latchedFacingFromHands = Vector3::nan()
	What the facing direction was according to the hands the last time that the hands were moving slowly. More...

WidgetManager *	m_manager
	The manager, set by setManager(). More...

Ray	m_preBlinkGazeRay [2]
	The gaze rays before the user blinked when m_blinkStartTime != nan. More...

shared_ptr< MarkerEntity >	m_trackedVolume

shared_ptr< Camera >	m_vrEyeCamera [2]
	Camera not visible to the normal scene graph that is updated every frame based on m_eyeFrame. More...

shared_ptr< XR >	m_xrSystem

Static Protected Attributes
static const int	MAX_BLINK_QUEUE_SIZE = 4

Friends
class	VRApp

Detailed Description

A G3D::Widget that manages world-space G3D::MarkerEntitys based on polling a low-level XR instance.

Manages MarkerEntities for:

"XR Tracked Volume"
"XR Left Hand"
"XR Left Controller"
"XR Right Hand"
"XR Right Controller"
"XR Head"
"XR Body"
Other tracked objects as "XR Tracked Object ##"

The body entity is currently synthesized from the head and hand positions. It is approximately the center of mass of the torso.

Attach the camera to the head entity and geometry to the hand entities to work with GApp. To put the player in a moving reference frame, you can attach the "XR Tracked Volume" entity to a vehicle.

Everything in XR is relative to the tracked volume, so to teleport, move the "XR Tracked Volume", not the player.

Constructor & Destructor Documentation

◆ XRWidget()

G3D::XRWidget::XRWidget	(	const shared_ptr< XR > &	xr,
		GApp *	app
	)

protected

Member Function Documentation

◆ bounds()

virtual Rect2D G3D::Widget::bounds ( ) const

inlineoverridevirtualinherited

Inherited from Surface2D.

Implements G3D::Surface2D.

Reimplemented in G3D::GuiWindow, G3D::GConsole, and G3D::FullScreenWidget.

◆ cameraForView()

const shared_ptr<Camera>& G3D::XRWidget::cameraForView ( int viewIndex ) const

inline

Referenced by eyeCamera().

◆ create()

static shared_ptr<XRWidget> G3D::XRWidget::create	(	const shared_ptr< XR > &	xr,
		GApp *	app
	)

static

◆ createShared()

template<class T , class ... ArgTypes>

static shared_ptr<T> G3D::ReferenceCountedObject::createShared ( ArgTypes &&... args )

inlinestaticprotectedinherited

Like std::make_shared, but works for protected constructors.

Call as createShared<myclass>.

◆ depth()

virtual float G3D::Widget::depth ( ) const

inlineoverridevirtualinherited

Inherited from Surface2D.

Controls the depth of objects when rendering. Subclasses may override this but it can interfere with the normal handling of rendering and event delivery. depth = 0 is usually the "top" widget and depth = 1 is usually the "bottom" widget.

Implements G3D::Surface2D.

Reimplemented in G3D::GuiWindow, G3D::GConsole, and G3D::FullScreenWidget.

◆ eyeCamera()

const shared_ptr<Camera>& G3D::XRWidget::eyeCamera ( int index ) const

inline

Deprecated:: use cameraForView

◆ fireEvent()

virtual void G3D::Widget::fireEvent ( const GEvent & event )

virtualinherited

Fire an event on the containing window.

Reimplemented in G3D::WidgetManager.

◆ gazeRay()

virtual Ray G3D::XRWidget::gazeRay ( int eyeIndex ) const

virtual

World-space gaze ray for the given eye, reported directly from the GazeTracker.

The result is latched immediately before onGraphics runs.

See also: GApp::gazeTracker, XRWidget::projectedMonocularGazePoint, XRWidget::robustBinocularGazePoint, GazeTracker

◆ manager()

WidgetManager* G3D::Widget::manager ( ) const

inlineinherited

◆ onAfterEvents()

virtual void G3D::Widget::onAfterEvents ( )

inlinevirtualinherited

Invoked after all onEvent handlers for the current frame and before onUserInput for any other Widget.

Reimplemented in G3D::WidgetManager, and G3D::GameController.

◆ onAI()

virtual void G3D::Widget::onAI ( )

inlinevirtualinherited

Reimplemented in G3D::GuiWindow, G3D::WidgetManager, G3D::ThirdPersonManipulator, G3D::GConsole, G3D::FirstPersonManipulator, G3D::VideoRecordDialog, and G3D::UprightSplineManipulator.

◆ onBeforeGraphics()

virtual void G3D::XRWidget::onBeforeGraphics ( )

overridevirtual

Updates the HMD cameras based on the current GApp::activeCamera()'s parameters, as well as the gaze data.

Reimplemented from G3D::Widget.

◆ onEvent()

virtual bool G3D::Widget::onEvent ( const GEvent & event )

inlinevirtualinherited

Returning true consumes the event and prevents other GModules from seeing it.

Motion events (GEventType::MOUSEMOTION, GEventType::JOYHATMOTION, JGEventType::OYBALLMOTION, and GEventType::JOYAXISMOTION) cannot be cancelled.

Reimplemented in G3D::GuiWindow, G3D::WidgetManager, G3D::ThirdPersonManipulator, G3D::GConsole, G3D::Discovery::Client, G3D::SceneEditorWindow, G3D::FirstPersonManipulator, G3D::VideoRecordDialog, G3D::UprightSplineManipulator, G3D::CameraControlWindow, G3D::GuiMenu, G3D::ControlPointEditor, G3D::DeveloperWindow, and G3D::ArticulatedModelSpecificationEditorDialog.

◆ onNetwork()

virtual void G3D::Widget::onNetwork ( )

inlinevirtualinherited

Reimplemented in G3D::GuiWindow, G3D::WidgetManager, G3D::ThirdPersonManipulator, G3D::Discovery::Server, G3D::GConsole, G3D::FirstPersonManipulator, G3D::Discovery::Client, and G3D::UprightSplineManipulator.

◆ onPose()

virtual void G3D::Widget::onPose	(	Array< shared_ptr< Surface > > &	surfaceArray,
		Array< shared_ptr< Surface2D > > &	surface2DArray
	)

inlinevirtualinherited

Appends a posed model for this object to the array, if it has a graphic representation.

The posed model appended is allowed to reference the agent and is allowed to mutate if the agent is mutated.

Reimplemented in G3D::GuiWindow, G3D::WidgetManager, G3D::Discovery::Client, G3D::SceneEditorWindow, G3D::CallbackWidget, G3D::FullScreenWidget, G3D::UprightSplineManipulator, and G3D::ControlPointEditor.

◆ onSimulation()

virtual void G3D::XRWidget::onSimulation	(	RealTime	rdt,
		SimTime	sdt,
		SimTime	idt
	)

overridevirtual

Samples from the XR and then creates or updates the MarkerEntitys in the Scene as needed.

Reimplemented from G3D::Widget.

◆ onUserInput()

virtual void G3D::Widget::onUserInput ( UserInput * ui )

inlinevirtualinherited

Reimplemented in G3D::GuiWindow, G3D::WidgetManager, G3D::ThirdPersonManipulator, G3D::GConsole, G3D::FirstPersonManipulator, G3D::UprightSplineManipulator, and G3D::CameraControlWindow.

◆ pollBlinkEvent()

virtual bool G3D::XRWidget::pollBlinkEvent ( VoluntaryBlinkEvent & event )

virtual

BETA API Likely to receive incompatible changes in future releases. Gets the next blink event from the queue, returning false if there was not one.: The queue saturates at a small number of entries, since many gaze tracking programs might ignore these events.

◆ positionalEventZ()

virtual float G3D::Widget::positionalEventZ ( const Point2 & pixel ) const

inlinevirtualinherited

The camera-space z position that this Widget considers this pixel to be at.

Used for positional event (e.g., mouse click) delivery. Higher means closer.

Large negative values are far away. 0 is the closest 3D object. Values greater than zero are used for 2D objects. Returning fnan() [the default] requests the WidgetManager to set hte positionalEventZ to the object's normal event zorder.

See the implementation of ControlPointEditor::positionalEventZ for an example of how to use this with a 3D object.

Reimplemented in G3D::ControlPointEditor.

◆ processGazeTracker()

void G3D::XRWidget::processGazeTracker ( )

protected

◆ projectedMonocularGazePoint()

virtual Point2 G3D::XRWidget::projectedMonocularGazePoint	(	int	eyeIndex,
		int	viewIndex,
		const Rect2D &	viewport
	)		const

virtual

2D screen-space view point for one eye.

Useful for visualizing the raw results from a GazeTracker. Projection of the ws gaze ray into the view of the given index by assuming that they have the same center of projection. This is the case (or at least, a good approximation!) for a HMD with two views when eyeIndex matches the primary display for that eye. This will not necessarily work for wide field of view HMDs that use multiple projections. This is not the projection of robustGazePoint. The result is latched immediately before onGraphics runs.

See also: GApp::gazeTracker, XRWidget::gazeRay, XRWidget::robustBinocularGazePoint, GazeTracker

◆ render()

virtual void G3D::Widget::render ( RenderDevice * rd ) const

inlineoverridevirtualinherited

Inherited from Surface2D.

Implements G3D::Surface2D.

Reimplemented in G3D::GConsole, G3D::GuiWindow, G3D::GuiMenu, and G3D::DebugTextWidget.

◆ robustBinocularGazePoint() [1/2]

Point3 G3D::XRWidget::robustBinocularGazePoint ( const Ray gazeRay[2] ) const

protected

◆ robustBinocularGazePoint() [2/2]

virtual Point3 G3D::XRWidget::robustBinocularGazePoint ( ) const

virtual

World space 3D gaze point inferred from the scene and the gaze rays.

This is is probably what the user is actually looking at in 3D. The result is semantically latched immediately before onGraphics runs, but is only computed on demand because of the ray casts involved. This may be slow if not using VRApp or Model::useOptimizedIntersect() == true.

See also: GApp::gazeTracker, XRWidget::projectedMonocularGazePoint, XRWidget::gazeRay, GazeTracker

◆ setDepth()

virtual void G3D::Widget::setDepth ( float d )

inlinevirtualinherited

Called by the WidgetManager.

This is the depth that the Widget is expected to use when posed as a Surface2D. Subclasses may override or ignore this but it can interfere with the normal handling of rendering.

◆ setManager()

virtual void G3D::Widget::setManager ( WidgetManager * m )

inlinevirtualinherited

Called by the WidgetManager when this module is added to it.

The argument may be nullptr

Reimplemented in G3D::SceneEditorWindow, G3D::ControlPointEditor, G3D::CameraControlWindow, G3D::GameController, G3D::DeveloperWindow, G3D::ProfilerWindow, G3D::TextureBrowserWindow, and G3D::SettingsWindow.

◆ simulateBodyFrame()

void G3D::XRWidget::simulateBodyFrame ( RealTime rdt )

protected

◆ sort()

static void G3D::Surface2D::sort ( Array< shared_ptr< Surface2D >> & array )

staticinherited

Sorts from farthest to nearest.

◆ sortAndRender()

static void G3D::Surface2D::sortAndRender	(	RenderDevice *	rd,
		Array< shared_ptr< Surface2D >> &	array
	)

staticinherited

Calls sort, RenderDevice::push2D, and then render on all elements.

◆ standingHeadHeight()

float G3D::XRWidget::standingHeadHeight ( ) const

Returns the height above ground of the top of the first player's head.

◆ window()

virtual OSWindow* G3D::Widget::window ( ) const

virtualinherited

Returns the operating system window that is currently rendering this Widget.

Reimplemented in G3D::WidgetManager.

Referenced by G3D::FullScreenWidget::bounds().

Friends And Related Function Documentation

◆ VRApp

friend class VRApp

friend

Member Data Documentation

◆ m_app

class GApp* G3D::XRWidget::m_app

protected

◆ m_blinkEventQueue

Queue<VoluntaryBlinkEvent> G3D::XRWidget::m_blinkEventQueue

protected

◆ m_blinkStartTime

RealTime G3D::XRWidget::m_blinkStartTime = nan()

protected

If currently tracking a LONG_BLINK, this is the time at which it began.

◆ m_body

shared_ptr<MarkerEntity> G3D::XRWidget::m_body

protected

◆ m_controllerEntity

shared_ptr<Entity> G3D::XRWidget::m_controllerEntity[2]

protected

◆ m_depth

float G3D::Widget::m_depth

protectedinherited

Referenced by G3D::Widget::depth(), and G3D::Widget::setDepth().

◆ m_entityTable

Table<String, shared_ptr<TrackedEntity> > G3D::XRWidget::m_entityTable

protected

◆ m_gazePoint

Point3 G3D::XRWidget::m_gazePoint

mutableprotected

This is NaN when it needs to be updated.

Mutable because it us updated on the first call to the const accessor.

◆ m_gazeRay

Ray G3D::XRWidget::m_gazeRay[2]

protected

In world space.

Updated by processGazeTracker

◆ m_lastFastHandMovementTime

RealTime G3D::XRWidget::m_lastFastHandMovementTime = 0.0f

protected

The last time either hand moved quickly.

◆ m_latchedFacingFromHands

Vector3 G3D::XRWidget::m_latchedFacingFromHands = Vector3::nan()

protected

What the facing direction was according to the hands the last time that the hands were moving slowly.

◆ m_manager

WidgetManager* G3D::Widget::m_manager

protectedinherited

The manager, set by setManager().

This cannot be a reference counted pointer because that would create a cycle between the Widget and its manager.

Referenced by G3D::Widget::manager(), G3D::GuiWindow::manager(), G3D::Widget::setManager(), and G3D::GuiWindow::setVisible().

◆ m_preBlinkGazeRay

Ray G3D::XRWidget::m_preBlinkGazeRay[2]

protected

The gaze rays before the user blinked when m_blinkStartTime != nan.

◆ m_trackedVolume

shared_ptr<MarkerEntity> G3D::XRWidget::m_trackedVolume

protected

◆ m_vrEyeCamera

shared_ptr<Camera> G3D::XRWidget::m_vrEyeCamera[2]

protected

Camera not visible to the normal scene graph that is updated every frame based on m_eyeFrame.

The GApp::activeCamera() is bound to one of these during GApp::onGraphics.

See also: m_eyeFrame

Referenced by cameraForView().

◆ m_xrSystem

shared_ptr<XR> G3D::XRWidget::m_xrSystem

protected

◆ MAX_BLINK_QUEUE_SIZE

const int G3D::XRWidget::MAX_BLINK_QUEUE_SIZE = 4

staticprotected

Classes

Public Member Functions

Static Public Member Functions

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Static Protected Attributes

Friends

Detailed Description

Constructor & Destructor Documentation

◆ XRWidget()

Member Function Documentation

◆ bounds()

◆ cameraForView()

◆ create()

◆ createShared()

◆ depth()

◆ eyeCamera()

◆ fireEvent()

◆ gazeRay()

◆ manager()

◆ onAfterEvents()

◆ onAI()

◆ onBeforeGraphics()

◆ onEvent()

◆ onNetwork()

◆ onPose()

◆ onSimulation()

◆ onUserInput()

◆ pollBlinkEvent()

◆ positionalEventZ()

◆ processGazeTracker()

◆ projectedMonocularGazePoint()

◆ render()

◆ robustBinocularGazePoint() [1/2]

◆ robustBinocularGazePoint() [2/2]

◆ setDepth()

◆ setManager()

◆ simulateBodyFrame()

◆ sort()

◆ sortAndRender()

◆ standingHeadHeight()

◆ window()

Friends And Related Function Documentation

◆ VRApp

Member Data Documentation

◆ m_app

◆ m_blinkEventQueue

◆ m_blinkStartTime

◆ m_body

◆ m_controllerEntity

◆ m_depth

◆ m_entityTable

◆ m_gazePoint

◆ m_gazeRay

◆ m_lastFastHandMovementTime

◆ m_latchedFacingFromHands

◆ m_manager

◆ m_preBlinkGazeRay

◆ m_trackedVolume

◆ m_vrEyeCamera

◆ m_xrSystem

◆ MAX_BLINK_QUEUE_SIZE