Skip to content

Latest commit

 

History

History
145 lines (106 loc) · 5.65 KB

README.md

File metadata and controls

145 lines (106 loc) · 5.65 KB
Raw

Handy Work

A module for doing efficient real-time pose detection from WebXR Hand Tracking using Web Workers to ensure it doesn't interfere with the main thread

The pose tracking module is Framework Agnostic, it doesn't rely on any particular library it should work just as well with THREE as BabylonJS or Play Canvas.

Poses can be found in the /poses/ folder and additional poses are welcome

There is also an AFrame module which handles pose tracking and displaying hand models.

Hand Pose Module Usage Example

import {
  update as handyWorkUpdate,
  loadPose
} from "../build/esm/handy-work.js";

loadPose('relax', '../poses/relax.handpose');
loadPose('fist', '../poses/fist.handpose');
loadPose('flat', '../poses/flat.handpose');
loadPose('point', '../poses/point.handpose');

// In RAF
handyWorkUpdate([controller1, controller2], referenceSpace, frame, callback);

Using via a CDN

If you use it via a CDN because it uses a Worker you need to use the standalone version which has the Worker encoded as a string.

import('https://cdn.jsdelivr.net/npm/[email protected]/build/esm/handy-work.standalone.js')
.then(function ({
  update,
  loadPose,
  dumpHands
}) {
  const handyWorkUpdate = update;
  const dumpHands = dumpHands;
  const loadPose = loadPose;

  loadPose('relax', POSE_FOLDER + 'relax.handpose');
  loadPose('fist', POSE_FOLDER + 'fist.handpose');
  loadPose('flat', POSE_FOLDER + 'flat.handpose');
  loadPose('point', POSE_FOLDER + 'point.handpose');
  loadPose('horns', POSE_FOLDER + 'horns.handpose');
  loadPose('shaka', POSE_FOLDER + 'shaka.handpose');
  loadPose('vulcan', POSE_FOLDER + 'vulcan.handpose');
}.bind(this));

handy-work

Documentation for using build/esm/handy-work.js or build/esm/handy-work.standalone.js

handy-work.loadPose(name, url) ⇒ Promise.<void>

Loads a pose from the Web the recommended way of using this API

Kind: static method of handy-work
Returns: Promise.<void> - Resolves once it has been completed

Param Type Description
name string Name of the pose
url string URL of the pose to download and add to the registry

handy-work.getPose(name) ⇒ Promise.<Float32Array>

Get a pose from the registry

Kind: static method of handy-work
Returns: Promise.<Float32Array> - A copy of the pose from the registry in the worker

Param Type Description
name string of the pose to fetch

handy-work.setPose(name, pose) ⇒ Promise.<void>

Get a pose from the registry

Kind: static method of handy-work
Returns: Promise.<void> - Resolves once the pose has been uploaded to the worker

Param Type Description
name string of the pose to set
pose Float32Array Array buffer with the information about the current pose

handy-work.resetHands()

Reset the current tracking performed automatically when the device returns from sleep

Kind: static method of handy-work

handy-work.dumpHands()

On the next frame save the current hand pose to a file

Kind: static method of handy-work

handy-work.generatePose(inputSources, referenceSpace, frame) ⇒ void | Float32Array

Get a pose from the current hand position

Kind: static method of handy-work
Returns: void | Float32Array - The generated pose buffer for the pose.

Param Type Description
inputSources XRInputSource Array of inputs you want to generate inputs for, requires a left AND right hand
referenceSpace XRReferenceSpace Current reference space
frame XRFrame Current active frame

handy-work.update(inputSources, referenceSpace, frame, callback)

Kind: static method of handy-work

Param Type Description
inputSources Array.<XRInputSource> The inputs you want to do pose tracking for
referenceSpace XRReferenceSpace The reference space for your scene
frame XRFrame The current active frame
callback function This gets called with an Array of Arrays with the poses and their distances