Write your first VR web application using React and WebXR

This is a tutorial for those who have a VR device on your hands and want to try building something for VR or for those who are just curious how VR with React works. For this post, I’m going to assume that you’re only familiar with React and never tried any 3D or VR libraries before.

Virtual Reality in the browser?

How can we display something in a VR headset from the browser? Usually, virtual reality applications/games are made in game engines like Unity or Unreal but browsers also have the capability to display graphics to your headset! Thanks to the introduction of the API called WebXR.

Why though?

Why would you, as a developer, choose the web as a platform for your application? In my opinion, the biggest advantages are a huge target audience (every headset can open a browser) and incredibly short iteration cycle i.e. hot-reloading, no compile times, instant startup.

As for disadvantages, the biggest one is performance. You are stuck with JS and WebGL. But I don’t think it’s such a big problem because you can still create great experiences that are not graphically intensive and will still be as exciting. The game I’ve spent the most time in, Beat Saber, has relatively simple graphics but still provides a great experience. I know I’m oversimplifying and there’s a bit more going on but you get the idea.

WebXR

So what is WebXR? It’s a set of APIs that adds support for accessing virtual reality and augmented reality devices. It is still a working draft but browsers like Chrome, Firefox and Oculus Browser already support it. It allows you to render to the head-mounted display, get device’s position, access inputs like controllers or even hands.

The Stack

To make a VR React application we’ll use the following stack:

Three.js is a library for 3D graphics, react-three-fiber is react renderer for Three.js, drei is a collection of reusable components for r3f and react-xr is a collection of hooks to help you build XR experiences in react-three-fiber applications.

react-three-fiber

react-three-fiber is an amazing library that allows React devs to get into 3D on the web in a matter of minutes. It is not some kind of high-level abstraction over THREE.js, it is THREE.js made available to use in the React. With it displaying interactive 3D object is as easy as:

<Canvas>
  {/* Scene lighting */}
  <ambientLight />
  <spotLight />

  {/* Controll camera with mouse/touch */}
  <OrbitControls />

  {/* Some geometry */}
  <Box>
    <meshStandardMaterial color="#e23" />
  </Box>
</Canvas>

If you’re familiar with react you should be able to get started in no time.

With react-three-fiber we can use THREE.js primitives like ambientLight the same way we can use DOM primitives like div or image.

Furthermore, we can use drei which contains easy to use react components for react-three-fiber like Box in the example above.

react-xr

Now that we have a 3D scene, let’s make it available in VR. For that, the only thing you need to do is to replace <Canvas> component with <VRCanvas> from react-xr package. It’s still the same canvas component but with all additional wiring necessary for VR to function.

You’ll notice that you now have Enter VR button available at the bottom of the screen that should start VR experience.

Adding controllers

Now you can see and move around in the virtual world but one crucial thing is missing - controllers. To add controllers you can use a component from react-xr package called <DefaultXRControllers/>. It will load appropriate controller models and put them in a scene.

To try it out without headset you can use WebXR extension. It will allow you to enter VR mode and manipulate the position of the headset, controllers and perform actions like squeeze or select. This is very useful for development and debugging as you don’t need to put your headset on every time you make a change.

Interactivity

Let’s add some basic interactivity to the scene. We’ll use a simple Hover event and change the color of the cube when any of the controllers is pointing at the object. To accomplish that we can use component <Hover> from the react-xr.

It works by wrapping any object in the scene and providing callbacks like this:

<Hover onChange={isHovered => {/* handle hover event */}}>
  <Cube />
</Hover>

Codesandbox with the final code

Learn more

We barely scratched the surface of what is possible with libraries like react-three-fiber and react-xr, I encourage you to check out more examples in GitHub repositories here and here. Remember, every r3f scene can be easily adjusted to be available in VR.

Blog by Oleksandr Dubenko