【AR Motion-Sensing Challenge】 Ridge of Taiwan Adventure | 2019 ITF Taiwan Pavilion - Green-Screen Photos and Multi-Player Motion-Sensing Interaction

Project Design Concept: In support of the Tourism Bureau's "Ridge of Taiwan Tourism Year" campaign, we created two highly interactive experiences at the 2019 ITF Taiwan Pavilion, letting visitors feel the appeal of Taiwan's mountain forests on the show floor. Using green-screen compositing technology, visitors could easily take a photo as if they had reached a summit. We also designed an AR motion-sensing challenge game for up to 4 players. The camera captures each player's silhouette in real time and places it inside a virtual mountain landscape. Players use walking and hand gestures to create a fully immersive adventure that blends the real and virtual worlds.

Interactive Experience Flow:

1. Green-Screen Background Removal Photo (Chroma Key Photo)

• Scene Selection: Visitors view four stunning Taiwan mountain scenes (such as Yushan and Xueshan) on a 43-inch display and tell a staff member which background they want.
• Real-Virtual Compositing: Standing in front of the green screen, the display shows the composited image in real time. Visitors adjust their pose as if they are standing on a mountaintop.
• Scan and Save: After the photo is taken, the system generates a unique QR Code. Visitors scan it to download their "summit souvenir photo" to their phone.

2. Multi-Player AR Motion-Sensing Challenge (AR Somatosensory Game)

• AR Real-Virtual Co-Performance: Supports up to 4 players simultaneously. Each player's silhouette appears in real time on a virtual mountain trail shown on screen.
• Movement Mechanic: Players march in place, and the trees and mountain scenery on screen scroll backward, creating the sensation of walking along a globe or a side-scrolling path.
• Three-Stage Level Challenges:
  1. Snowman Sweep: Snowmen blocking the path appear on screen. Players wave their hands to touch and clear them away.
  2. Parting the Clouds: Mist and cloud layers drift through the mountain forest. Players make wide sweeping motions with both hands to push the clouds aside and reveal the scenery.
  3. Harvest Catch: The harvest moment begins and apples and fruit fall from above. Players bring both hands together and the system detects the gesture to generate a virtual bag, letting players catch the fruit and earn points.
• Score and Reward: The total score is displayed at the end of the game. Players who meet the target score receive a gift on the spot.