swiftui - Custom snap gesture in visionOS - Stack Overflow

admin管理员组

文章数量:1123919

I’ve been developing a snap gesture for my app. While it works, it’s inconsistent and activates mistakenly.

Per Wikipedia, a snap occurs when tension is created with the index, middle, or ring finger against the thumb and releases with force in 7 milliseconds, producing sound.

To replicate this, I tried tracking contact between the finger and thumb, then check if the finger is close to points 1 and 5 in the diagram below upon release.

The current implementation "kind of works" but lacks consistency for seamless user interaction. For example, if the .thumbTip and .indexFingerTip touch while the hand unintentionally closes, the gesture activates incorrectly, which isn’t ideal. In which way can I create something robust and consistent? Below is some code for context and background understanding.

This is all part of a @MainActor @Observable final class GestureModel: Sendable I am creating Here are the variables I was using

fileprivate var lastContactTime: TimeInterval = 0
fileprivate let session = ARKitSession()
fileprivate var handTracking = HandTrackingProvider()
fileprivate var latestHandTracking: HandsUpdates = .init(left: nil, right: nil)
fileprivate var startingContact: Bool = false
fileprivate var activeHandSide: String? // Tracks the active hand ("left" or "right")
internal var isSnapGestureActivated: Bool = false
    
internal struct HandsUpdates {
    var left: HandAnchor?
    var right: HandAnchor?
}

Then, the start functions begins the hand tracking session. The update checks for hand updates (called outside in the views)

/// Start the hand tracking session.
internal func startTrackingSession() async {
    do {
        if HandTrackingProvider.isSupported {
            try await session.run([handTracking])
            print("ARKitSession starting.")
        }
    } catch {
        print("ARKitSession error:", error)
    }
}

/// Updates the hand tracking session differentiating the cases of updates.
/// This function ignores every state except the update.
internal func updateTracking() async {
    for await update in handTracking.anchorUpdates {
        switch update.event {
        case .updated:
            let anchor = update.anchor
            guard anchor.isTracked else { continue }
            
            // Update the latest hand tracking state
            await MainActor.run {
                if anchor.chirality == .left {
                    latestHandTracking.left = anchor
                } else if anchor.chirality == .right {
                    latestHandTracking.right = anchor
                }
            }
            
            // Detect if a snap occurred
            Task.detached { [self] in
                let leftSnapMiddle = await snapGestureActivated(for: "left", finger: .middleingerTip)
                let leftSnapRing = await snapGestureActivated(for: "left", finger: .ringFingerTip)
                let leftSnapIndex = await snapGestureActivated(for: "left", finger: .indexFingerTip)
                    
                let rightSnapMiddle = await snapGestureActivated(for: "right", finger: .middleFingerTip)
                let rightSnapRing = await snapGestureActivated(for: "right", finger: .ringFingerTip)
                let rightSnapIndex = await snapGestureActivated(for: "right", finger: .indexFingerTip)
                    
                // Check if any snap occurred
                let anySnap = leftSnapMiddle || leftSnapRing || leftSnapIndex || rightSnapMiddle || rightSnapRing || rightSnapIndex
                    
                 if anySnap {
                    await MainActor.run {
                        isSnapGestureActivated = true
                    }
                    try? await Task.sleep(nanoseconds: 500_000_000)
                    await MainActor.run {
                        isSnapGestureActivated = false
                    }
                }
            }

        default:
            break
        }
    }
}

Finally, the function for the gesture:

/// Detects a snapping gesture starting from the thumb and a specified finger (index, middle, or ring) of the specified hand.
/// - Parameters:
///   - handSide: Specify "left" or "right" to detect snap gestures for the respective hand.
///   - finger: The specific finger to check for the snap gesture.
/// - Returns: True if the user snapped with the specified finger of the specified hand.
fileprivate func snapGestureActivated(for handSide: String, finger: HandSkeleton.JointName) -> Bool {
    // If another hand is active, skip processing for this hand
    if let activeHand = activeHandSide, activeHand != handSide {
        print("\(handSide.capitalized) hand ignored because \(activeHand.capitalized) hand is active.")
        return false
    }
        
    // Detect the hand
    guard let handAnchor = (handSide == "left" ? latestHandTracking.left : latestHandTracking.right), handAnchor.isTracked else {
        print("\(handSide.capitalized) hand anchor is not tracked.")
        return false
    }
        
    // Define the skeleton anchor
    guard let handSkeleton = handAnchor.handSkeleton else {
        print("\(handSide.capitalized) hand skeleton not available.")
        return false
    }
        
    // Define world positions
    let thumbPosition = matrix_multiply(
        handAnchor.originFromAnchorTransform,
        handSkeleton.joint(.thumbTip).anchorFromJointTransform
    ).columns.3.xyz
        
    let fingerPosition = matrix_multiply(
        handAnchor.originFromAnchorTransform,
        handSkeleton.joint(finger).anchorFromJointTransform
    ).columns.3.xyz
        
    let thumbKnucklePosition = matrix_multiply(
        handAnchor.originFromAnchorTransform,
        handSkeleton.joint(.thumbKnuckle).anchorFromJointTransform
    ).columns.3.xyz
        
    let indexFingerMetacarpalPosition = matrix_multiply(
        handAnchor.originFromAnchorTransform,
        handSkeleton.joint(.indexFingerMetacarpal).anchorFromJointTransform
    ).columns.3.xyz

    // Here goes the logic for snap gesture that I cannot make consistent :(
        
    return false
    }
}

I know it's a long question, but I really hope that developers will help me. Thank you in advance.

本文标签： swiftuiCustom snap gesture in visionOSStack Overflow