Spatial Repair Tutorials
During equipment repair or assembly, users follow an expert’s recorded hand paths and tool orientation overlaid directly on the physical object.
When a repair technician, lab instructor, or craftsperson performs a standard procedure, they wear Vision Pro to record their hand paths, points of gaze, and tool orientation. Afterward, the product splits the continuous recording into short actions—such as locating a retaining screw, switching sockets, or turning a part into position—and lets the author remove unnecessary clips and add a brief safety warning.
A later user faces the same model of equipment and first aligns it by identifying three visible features. The tutorial then overlays arrows, hand paths, and tool orientation onto the physical object in front of them. It reveals the next step only after the current one is complete, so users do not have to compare a floating video with the parts in their hands. They can freeze the view when something is unclear or switch back to the recorder’s perspective.
Each step retains the required tools, the expected action, and a photo of the physical item from the recording. Learners can mark a step as “completed,” “position differs,” or “stuck,” showing maintainers where the process most often breaks down. For equipment with minor physical variations, they can reassign anchors and continue using the same workflow.
The first version focuses on repeatable processes such as desktop equipment assembly and laboratory instrument maintenance, and supports only tutorials reviewed by an assigned lead. It does not replace electrical safety verification or automatically generate repair actions for unfamiliar equipment.
Why now
On July 29, an article showing a practical Vision Pro use case reached Hacker News; as observed on July 30, the post ranked fifth with 330 points and 160 comments. S1 The discussion has renewed attention on whether headsets can support real work, and may prompt teams that already own the hardware to revisit training gaps in hands-on tasks.
Target user
Best suited to equipment labs that already have Vision Pro, campus training labs, and small repair teams. The key moment is before an experienced worker leaves, or when a novice performs a repeatable procedure independently for the first time. On site, both hands are often occupied and parts can be difficult to identify from flat images. Team leads also need to see where learners get stuck so they can keep improving the tutorials.
Minimal entry point
Build recording and playback as a native visionOS app. Capture hand paths with ARKit hand tracking and render spatial content in RealityKit. Start device alignment with three manually selected feature points, adding image anchors and world anchors where needed. S3 Recording the real-world scene requires access to the main camera. That capability is available to enterprise apps and requires authorization. S2 Save points of gaze only when users confirm them with gaze-and-pinch; do not access raw eye-tracking data. S3 Have the recorder confirm action boundaries, using pauses in movement only to suggest possible cuts. The first release supports one device at a time, one reviewer, and sequential steps.
Punching above its weight
Find initial users through local university labs, makerspaces, and equipment distributors. They typically have fixed equipment, repeatable procedures, and clearly designated reviewers. Run an on-site pilot with one real machine, delivering reusable tutorials and a report on where learners get stuck. Public demos should use only non-sensitive action clips, with hands-on before-and-after videos as acquisition material.
Competitors & gaps
- PTC Vuforia Expert CaptureGoogle
- PTC Vuforia Expert Capture already lets experts record hands-free. Its assets include first-person video and photos, which can then be edited in the cloud into step-by-step procedures. It also supports location-aware steps and publishing to glasses, phones, tablets, and desktops. S4 It is more mature for large manufacturers in approvals and cross-device distribution. The opening here is to turn the expert’s movements themselves into a spatial tutorial: hand paths, tool orientation, and confirmed points of attention are overlaid on the same model of physical equipment. The product also collects “position differs” and “stuck” feedback by step, helping small teams identify where a tutorial fails. Early customers need not build a full industrial platform; they only need to maintain a small set of repeat-use machines. If recording, calibration, and review remain time-consuming, however, it will be difficult to win budget from established platforms.
How it makes money
Charge a team subscription that includes an equipment tutorial library, reviewer seats, and learner accounts. Add implementation fees for private deployment, support with enterprise permission applications, or new equipment templates.
The case against
The recording workflow is constrained first by platform permissions. Main camera access is available only to approved enterprise apps, making it difficult for individual developers to ship a complete version directly. S2 visionOS also does not directly provide where a user is looking; gaze trails can only be captured as deliberate confirmation points. S3 Hand paths can be disrupted by occlusion, reflective tools, and tight spaces. Even slight anchor drift can send an arrow to the wrong part. A repair tutorial that gives incorrect guidance can damage equipment or put people at risk. Every model requires recording, review, and ongoing maintenance, so content costs can quickly exceed app development costs.