Light gun compatible with modern displays

Light gun is a niche computer gaming controller that gives the player a convincing illusion of shooting targets on the screen. This input device was especially prevalent in the arcade era and early home console era.

With the resurgence of retro games in location-based and home environments, the interest in this kind of controller has also grown. Unfortunately, the fundamental principle of operation of the original lightgun is tied to its contemporary CRT display technology, and it is becoming increasingly difficult for operators to source compatible monitors.

The market has offered several modern alternatives in this space, with the base principle of the solutions falling into one of three categories:

• Mechanical positioning: the controller is fixed to a gimbal that measures the pan and tilt angles and computes the aim point. This is the traditional solution and is still usable for some arcade cabinets.
• IR light: the target (or some other anchor in the environment) is marked by IR beacons, while the controller has sensors to detect its orientation in relationship to these beacons. These systems allow for a freely moving controller but require visible hardware markers.
• Visible light/computer vision: The most advanced solution is to use a camera sensor in the gun and image recognition techniques to identify the display visually and then compute the target point coordinates. There are very few solutions in this space, and certainly no wireless devices.

The problem

Create a wireless light-gun controller that works with any display technology and doesn’t require the operator to visibly modify the display or the environment. For home use cases, the wireless option is a bonus.

Core requirements:

• High precision, low latency
• Rugged construction,
• Cheap, off-the-shelf components
• Compatibility with industry standards for both displays as well as input controllers

The solution

The prototype controller was constructed out of the shell of an existing lightgun, with Raspberry Pi Zero 2W at its core, a standard MIPI camera module for image acquisition, and a mixture of custom and modified input controls offered by the original shell.

Since the controller operation is based on image recognition, there needs to be some well-known marker on the screen (or near it) for the OpenCV-based algorithm to localize the reticle. We accomplished this by injecting a special pattern into the game’s video output using resident software on the host machine.

In order to drive the game, the host machine only needed to pair with the lightgun, just as it would with any other HID Bluetooth gamepad. The controller offered complete gamepad emulation, including:

• Analog stick 1, driven by the camera
• Analog stick 2, emulated via a thumbstick mounted on the case and passed through I2C ADCs
• Full set of gamepad buttons, including trigger, reload, ABXY, and d-pad. All mapped to the buttons that came with the donor controller.

The original shell that came from an old PS2 lightgun was modified to feature a side I2C display to aid with calibration and to run custom GUI controlled via combinations of physical buttons.

The movement latency was measured at an average of 45ms from the motion of the case to the update of the HID joystick on the host machine. Unfortunately, the effective latency is higher, as it is a sum of display latency and lightgun’s own 45ms.