WebGL WebVR Meetup at Google

[0700.. 0800] Lingfei Meng (MuraVision) - Muravision makes a device that does scan-based PBR texturing. He will be there to give a brief talk on PBR textures and also to give a live demonstration.

[0800..0830] Josh Carpenter and Iker Jamardo (Google) - Experiments in AR for the Web

[0830..0900] Gareth Morgan (Axum Graphics) - Axum Graphics is developing, Auteurist. It is an entirely WebGL-based 3D content creation tool for filmmakers, that renders photo-realistic video (both conventional and 360-degree).

[0900..0930] Jamieson Brettle (Google) - Overview of the Draco project which is focusing on compressing 3D assets for delivery on the web.  They are working towards having Draca be part of the glTF2.0 specification.

[0920..0940] Will Eastcott (PlayCanvas) - PlavCanvas has some exciting updates to share with the group having to do with AR....

[] Tony Parisi (Unity) - Tony will give us an update on the state of glTF. *IF* he's not traveling to New York on the 28th.

MURA vision

• Lingfei Meng
• pbr based texture scanner
• http://www.muravision.com/
• BRDF - Physically based rendering
• scanner can scan 20cm x 2cm
• Big one can scan 50cm x 50cm
• demo_sektop_scan
  • scanner to get -> color, diffuse, specular, opacity, roughness, depth, glossiness, normal,
  • mamoset toolbag 3.03 -> very simple renderer
  • business model: sell the scanner($90k) or sell the scanned material ($130 per scan)
  • application: subscantance designer
• PBR : glossiness, mathematical representation of texture
• packages in mdl.
• depth 16 bit, others 8 bit

The Argumented Reality Web

• Iker Jamardo (daydream webxr tech lead) @judax & Josh Carpenter (ux lead, daydream webxr, canadian) @joshcarpenter

• Argon Browser v4, AR browser for mobile

• AR.js https://github.com/jeromeetienne/AR.js/blob/master/README.md

• AR + ARCore, uses webasembly

• hundreads of millions of devices will soon have basic AR capabilities

• WebAR at I/O, Tango project

• http://webar-wayfair-prototype.appspot.com

• these builds were build on top of WebVR 1.1 and enabled

  • accurate motion tracking
  • way to composite the pass through camera feed and the virtual scene
  • a way to understand the world

• ReactVR

• Room scan => from tango device

• Space Puppy,

• point cloud

• from these builds, we learned

  • that being able to synchronize the motion tracked pose and the rendering of the pass through camera feed in Chrominum is not trivial, as WebGL’s command buffer makes rendering asynchronous..
  • That the initial names for the new features were not ideal!

• Fast forward to august

• WebARonARcore,

• WebARonARKit

• expose an experimental API extension to WebVR 1.1. to privide AR capabilities on top of IOS ARKit and Android ARCore:

  • 6 DOF device motion tracking: through the webvr 1.1 api

• Goal:

  • Learning from ARKit and ARCore APIs
  • Explorer AR integration in the browser stack
  • enabling delveopers to experiment early

• https://www.wired.com/story/google-joins-the-augmented-reality-party-with-arcore/

• http://three.ar.js

• community experiments

• webgl based graffiti

• expose plannes -> surfaces

• UJI primitives

• three.js + css only -> made an AR, google doodles!

• We also seen similar hybird apps

  • active theory thee.js-powered arkit app
  • sketch fab arkit app
  • mobizlla webxr for iso: github.com/mozilla/webxr-ios
  • stink stuidos’ arkit-web’ :github.com/stinkstudios/arkit-web

• What’s next

  • explanier for AR on the web
  • coorporation between borser vendors
  • protyping use cases and design principles

• all will be opensource in 2 weeks

• https://github.com/google-ar/three.ar.js

• building with webgl + special version of chronmnium

AAutheurist

• Gareth Morgan (Axum Graphics)
• WebGL based contents creation tool
• announcement: W3C november 6th, session at 1:30pm declarative web vr
• https://axumgfx.com/
• 360 degree video
  • conventional video
  • stereoscopic 360 video
• browser based => rendering on the cloud
• frontend: three.js, backend: python flask, mongodb
• render farm with radeon pro -> streaming services
• final output render to vimeo
• https://vimeo.com/ => vr video, sample 260 degree video
• https://www.axum.graphics
• raedon renderer pro??
• using renderstorm for backend

Draco 3D Compression

• draco -> 3d compression codec for web, opensourced since jan 2017
• https://codelabs.developers.google.com/codelabs/draco-3d/#0

• super fast on the web

• Jamieson Brettle (Google)
• compare gzip vs draco
• encoder.decoder just released encoder
• draco 1.x release
  • webassembly decoder, 2x faster decode speed,
  • js encoder
  • npm package
• draco extension to glTF 2.0
  • glTF 2.0 renders with draco
  • three.js
  • babylon.js
  • sample models
  • https://goo.gl/s8Acmd
• accessing draco
  • https://github.com/google/draco
• Songbird
  • https://github.com/google/songbird
  • creating and rendering ambisonic signals
  • written in js
  • raytracing in sounds
  • recalcualte the sound source based on the postion of the actor in the scene
• adding a source

audioElement.play();

• paranode? compatitor of songbird

PlayCanvas

• the web-first game engine https://playcanvas.com/

• https://github.com/playcanvas Will Eastcott (PlayCanvas), since 2011

• platform for making interactive contents

• like unity, runs on browser

• playcanvas ar ->

getUserMedia/Stream API

• you can now make browser based AR with JS
• capturing the camera stream

var constraints = {}

var promise = navigator.mediaDevices.getUserMedia(const)
promise.???

var vide = document.createElement(;video')
video.setAttributes(''); # for full screen
video.srcObject = stream;

• playcanvas editor and code
• creating AR marker on the playcanvas editor with ui or code

var hiro = new pc.Entity("Hiro Marker")
hiro.addComponent("script")
hiro.script.create("arMarker", {attributes: {}});
app.root.addChild(hiro)

• problem with ARtoolkit
  • requires printing of markers
  • sensitive to lighting conditions
  • sensitive to camera hardware
  • cpu intensive
• hiring WebGL ninjas
• animation FBX, -> converted the json. keyframe
• 150kb library