Tomorrow's Web and Future Technologies - WDC2011

TOMORROW’S WEB

Future technology is awesome

Rob Hawkes

Hi, I’m Rob Hawkes and I’m here to talk a little about tomorrow’s Web and the awesome technology that’s coming with it.

If you don’t already know, I work at Mozilla.

My official job title is Technical Evangelist, but I prefer what it says on my business card.

Part of my job is to engage with developers like yourselves about cool new technologies on the Web.

And for those of you with no idea of what a rawket is, I made a multiplayer game called Rawkets in which players fly around in little rockets and shoot each other in the face with the latest Web technologies. It’s quite addictive!

http://rawkets.com

Please don’t hesitate to ask a question as we go along, but leave it to the end if it can wait.

All I ask is that you raise your hand and I’ll get to you as soon as I can.

It’s also worth noting that these slides will be online on my blog and the Mozilla Hacks website very soon.

I’ll include all the links and notes mentioned throughout the talk.

So let’s start on our journey towards tomorrow’s Web…

Created by Phil Banks (@emirpprime)

Our first stop is HTML5.

If you’ve met me before then you probably already know about my slight addiction to HTML5 canvas and visual programming.

HTML5 & JavaScript

What is all the fuss about?

But why is it important?

Well to put it simply, HTML5 & JavaScript are the underlying technologies behind everything related to the future of the Web.

Pretty much every new technology that is coming out within the browser-space is connected to HTML5 and JavaScript in some way.

And what’s great is that every major browser has invested in them, so they won’t be going anywhere any time soon.

But there’s more to it than that.

Open technologies

Anyone can view the source code

They are open technologies.

Anyone can get involved in their creation; through browser developers like Mozilla, or through standards organisations like the W3C.

A few weeks ago I was in a W3C meeting to explore what is needed for making games with these technologies. What was cool was that anyone was allowed to take part; which included everyday developers, employees of major browsers, and games companies.

Also, these technologies are open in that anyone can view the resulting code that is used within Web pages, which is a fantastic way of learning.

Free technologies

Free to use. Free to develop with

They are free technologies.

Anyone can use these technologies without having to pay anything, both for using the technology and developing with it.

This is unlike closed environments like Flash where you have to pay to use official code editors and production environments.

Plugin-less

No more reliance on third-party software

They are technologies baked directly into the browser, which means no more plugins!

No longer do you have to rely on users having third-party software installed to use rich media.

No compilation

Save time with development and testing

HTML5 and JavaScript doesn't require compiling, so development and testing can be extremely rapid.

These technologies allow for quickly hacking stuff together to experiment then tidying things up later.

It’s this rapid nature of JavaScript and HTML5 that make them so fun to develop with.

Interoperable

They are built to work across platforms

These technologies are built to work across platforms; whether that’s desktop, mobile, TV, or anything else!

This makes it great to develop this way because you can be sure that it will work on any platform that has support.

Cross-browser support

Most major fe

atures are supported

As with any technology on the Web it’s important that you can use it across all the major browsers.

Fortunately the bigger features of HTML5 like video are supported by all the major browsers, with some of the newer and smaller features getting better support as time goes on.

The situation isn’t perfect but we’re definitely in a position where these technologies can now be used in production.

There’s a fantastic website called Can I Use? which lets you know what browsers support each technology - http://caniuse.com

Before we move on I have a little disclaimer…

Tomorrow means tomorrow

Do not expect to use these right now

I’ve painted a pretty rosy picture about HTML5 and JavaScript, but I should be honest with you.

The other technologies that I’m going to talk about today are called future technologies for a reason.

While tonnes of HTML5 and JavaScript technologies already exist and are ready to be used, these other future technologies aren’t fully developed and supported yet.

Some are usable today, but for others it may be months, even years before they mature. Some may never reach a cross-browser state.

Consider this talk as inspiration about what the future may hold. An advanced warning about some possible directions the Web is heading.

So with that out of the way, let’s continue on our journey…

WebSockets

Bi-directional real-time communication

WebSockets is particular favourite of mine.

It a JavaScript API that allows you to provide bi-directional real-time communication between a browser and a server.

This means you can instantly push data to and from the user as soon as it becomes available.

No more AJAX

Quicker and cheaper

Before WebSockets you would probably have used AJAX to constantly check for changes on the server.

This is overkill in a lot of situations and uses an incredible amount of bandwidth in comparison to WebSockets, and well all know bandwidth doesn’t grow on trees.

Using WebSockets

It is great fo

r all sorts of things

But forget about bandwidth and just think about what a truly real-time experience can allow you to do on the Web.

WebSockets are being used in a whole variety of situations, particularly where speed is important.

Multiplayer gaming

Communicating between players

WebSockets is perfect for multiplayer gaming.

Rawkets is the game I mentioned at the beginning that allows you to shoot your friends in the face with the latest Web technologies.

It’s still not really at a beta release level yet hence the bugs you might notice in this video, but it’s a good demo of a game created using WebSockets for multiplayer communication.

This performance would have been impossible with AJAX.

http://rawkets.com

Real-time analytics

Keeping your finger on the pulse

Another use of WebSockets that is becoming popular is real-time website analytics.

Gaug.es uses WebSockets to power their real-time website analytics dashboard.

Who doesn’t want to know exactly where everyone is on their website at any point in time? It’s totally not creepy.

But in all seriousness, knowing analytics data like this in real-time can make all the difference in some situations.

http://gaug.es

Live streaming content

Instant updates

WebSockets is also perfect for subscribing to and distributing a live feed of data.

In another project of mine I used WebSockets to produce a real-time graph of sentiment on Twitter.

The way this is done is by using Node.js to scrape tweets using the Twitter Streaming API, calculating the sentiment, then transmitting those values to the browser with WebSockets. This whole process takes just a few milliseconds.

Use it today

It is in a state of flu

x, but it is usable

You can definitely use WebSockets today.

It’s supported by all the major desktop browsers, which includes IE from version 10, and is supported on iOS as well as Firefox and Opera mobile.

However, the support for WebSockets is inconsistent across some browsers as there are many versions of its specification in existence.

It’s very much in a state of flux at the moment and it will probably be a little while until it settles down and matures.

http://caniuse.com/websockets

In the meantime you can use things like Socket.IO, which is a Node.js module that uses WebSockets when possible and falls back to Flash when it’s not supported.

http://socket.io

WebRTC

Audio and video communication

Related to WebSockets but inherently different is WebRTC, which stands for Real-time Communications on the Web.

It differs from WebSockets because its focus is on video and audio communication, much like you see in applications like Skype.

Mozilla is working on WebRTC, alongside Google and Opera.

It’s pretty early days.

http://www.webrtc.org

But it’s not just a pipe dream, it’s actively being worked on in various permutations by various companies.

For example, Mozilla have been working on a project called Rainbow, which is a Firefox addon that allows you to access the webcam and microphone through nothing more than JavaScript.

http://mozillalabs.com/rainbow

What’s cool is that Rainbow is now shifting development to focus on implementing the WebRTC API, with the aim of not requiring the addon at all.

http://mozillalabs.com/rainbow/2011/08/04/whats-next-rainbow-and-webrtc/http://jboriss.wordpress.com/2011/10/19/enabling-real-time-communication-on-the-web-platform/

Even Ericsson have been experimenting with WebRTC.

What’s cool about Ericsson is that they got as far as releasing a working prototype that uses Webkit and requires no plugins. It’s pretty amazing!

https://labs.ericsson.com/apis/web-real-time-communication/documentation

Peer to peer communication

A nice by-product of WebRTC

A great thing about WebRTC is that it inherently requires a peer to peer infrastructure.

This further differentiates it from WebSockets which requires a server in the middle that acts as a mediator for messages sent between browsers.

I won’t get into the nitty gritty but the basic reasoning behind why WebRTC needs peer to peer is that you just won’t get the required performance for voice and video communication if you have to send data via a server. Every millisecond counts.

This is great for developers because the peer to peer technology can be used in other situations outside of those considered by WebRTC.

Things like peer to peer gaming (like most RTS games), file transfers between browsers, amongst a whole range of other possibilities.

Our next stop is just a short distance from WebRTC.

Device APIs

Accessing hardware with JavaScript

Device APIs is a catch-all term for the technologies that allow developers to access hardware and operating system APIs using JavaScript.

At Mozilla we’re working on the WebAPI effort which is our attempt at solving this problem.

In other camps it might be referred to as DAP, which is the Device APIs Working Group who are producing a W3C specification for accessing various parts of a device through JavaScript.

The approach we’re taking is to try and replicate the functionality of a mobile phone with JavaScript.

What APIs would you need to do that?

http://hacks.mozilla.org/2011/08/introducing-webapi/http://arewemobileyet.com/

My favourites

Device APIs I cannot wait fo

r

There are so many Device APIs in the pipeline when you combine all the efforts by the various browser manufacturers.

Here are just a few of my favourites.

Media Capture API

Accessing the camera and microphone

The Media Capture API will let you access the camera and microphone on a device.

Being able to do this with JavaScript will make a whole world of difference in so many areas.

For example, instead of requiring a user to upload a profile image you could take one using the webcam and use that instead. Simple but effective.

You could also do things like using the microphone input for voice recognition in for input areas. I know Google are already experimenting heavily in this area.

WebNFC API

RFID-like wireless communication

The WebNFC API is pretty cool.

It gives you the ability to transmit and receive data within distances no larger than a few centimetres.

The idea is that it can be used in phones for things like payment (like the new Barclaycard), travel (think Oyster Card), and file transfer.

https://bugzilla.mozilla.org/show_bug.cgi?id=674741

WebSMS API

Sending texts with JavaScript. ‘Nuff s

aid

The WebSMS API isn’t the most glamorous, but the idea of sending text messages with JavaScript is quite appealing.


WebTelephony API

Make and receive calls

The WebTelephony API allows you to make and receive phone calls using JavaScript.

I can just imagine something like this being used to hook into a Web-based version of Siri that answers your calls for you if you’re busy.


WebVibrator API

Vibrating you with open technologies

But my new favourite by far is the WebVibrator API, it just makes me laugh every time I hear it.


And just to show how fresh the development of these Device APIs is, just this Wednesday Opera released an experimental build with a webcam API built in.

http://labs.opera.com/news/2011/10/19/

Boot to Gecko*

An operating system for the Web

* Working title

Boot to Gecko will be the result of Mozilla’s work in this area.

It’s an operating system for the Web, focussed particularly on mobile and tablet devices.

It’s similar in concept to Google’s Chrome OS, except we’re taking things a little further than that with the JavaScript APIs hooked directly into all the capabilities of the device.

It’s still early days at the moment but you can expect to hear more about it in the near future.

https://wiki.mozilla.org/B2G

Full Screen API

Simple, yet powerful

The Full Screen API allows you to expand any HTML element to fill the users screen, even if the browser isn’t running full screen itself.

It’s a really simple API that is useful for immersive visual media like video and games.

The Mozilla implementation is not perfect yet because you can’t use the keyboard in full-screen mode, but it’s in the latest Nightly builds and works in all other respects.

https://bugzilla.mozilla.org/show_bug.cgi?id=545812http://blog.pearce.org.nz/2011/09/mozilla-full-screen-api-progress-update.htmlhttps://wiki.mozilla.org/Platform/Features/Full_Screen_APIs

Better input

Break free fro

m the keyboard and mouse

Another simple thing to look forward to is improved input.

The ramifications of improvements in this area are profound for games and Web applications.

For too long we’ve put up with simple keyboard and mouse controls for the browser.

Joystick API

Bringing the console to the Web

The Joystick API is one of the major improvement to input that is coming.

Both Mozilla and Google are working an an implementation of this and there is actually an experimental build of Firefox that has it working.

What I find most interesting about the Joystick API is that it might be just the thing we need to finally justify HTML5 gaming on a TV or console.

Who wants to use a keyboard and mouse while sitting on the sofa?

https://wiki.mozilla.org/JoystickAPI

Mouse Lock API

Locking the cursor in one place

The Mouse Lock API is an attempt at improving the mouse as an input device.

It would be used in situations like games and 3D visualisations where the mouse position rotates or moves you around a 3D space.

As it stands there’d still be a cursor moving around the screen causing all sorts of trouble when you want to click on something in your game.

With the new API you can lock your mouse position and stop it from getting in the way and being a nuisance.

Both Google and Mozilla are working on an implementation of this right now.


WebGL

3D graphics platform

Next up is WebGL, which brings the ability to provide advanced 3D graphics and accelerated 2D graphic directly within the browser.

It’s great for gaming and visualisations.

I debated including this as it’s not massively new, but the browser support is pretty lacking and it’s constantly being improved.

Rome is a music video created with WebGL. It’s an amazing example of what the technology can achieve in a real-world situation given a large team.

http://ro.me

This is something I made especially for the ASSEMBLY event in Finland.

It’s an audio visualiser that uses WebGL and the HTML5 Audio Data API.

http://robhawkes.github.com/webgl-html5-audio-visualiser/

HelloRacer is a little game that lets you drive a Formula One car around your browser. It’s a beautiful example of WebGL in action.

http://helloracer.com/webgl/

This is a rather freaky example of how realistic WebGL can be.

It’s a demo that shows how well materials like skin can be rendered. This isn’t much unlike the quality of modern games consoles!

http://www.everyday3d.com/j3d/demo/014_Head.html

Tinkercad is probably the best use of WebGL that I’ve seen in a production situation.

It’s a Web app that allows you to create 3D objects in your browser using WebGL, then get them printed, in 3D, and sent to your doorstep in just a few simple clicks. It’s seriously awesome.

https://tinkercad.com

Web applications

Not just a fancy website

The concept of Web apps is something that is gaining a lot of traction at the moment.

No doubt this is as a result of the success of native applications on the desktop and mobile, particularly with iOS and Android.

Google are spending a lot of time on Web apps with the Chrome platform.

https://chrome.google.com/webstore

It’s something we’re also spending a lot of time on at Mozilla.

Although we’re approaching things a little differently.

We envisage Web apps to run on any device, any browser, and to be distributed through any store or website.

https://apps.mozillalabs.comhttps://developer.mozilla.org/en/OpenWebApps

Offline assets

Who needs the Internet anyway

One of the main differences between Web apps and native apps is that native apps can be run offline.

New technologies like the application cache allow for a website or Web app to cache necessary assets to that it can still run while offline.

This includes things like JavaScript files, CSS and images.

Combining this technique with intelligent use of things like local storage will allow your application to continue working even if the Internet connection goes down. You just sync up all the changes when it gets connected again.

https://developer.mozilla.org/en/Offline_resources_in_Firefox

App-like experience

Run from the desktop or home screen

Something that needs to be tackled with Web apps is how to make them feel like real applications rather than websites.

One way that is being considered is completely removing the browser chrome and running the application in it’s own window.

This will effectively mean that you have full control of the app UI and it won’t look like it’s being run in a browser.

This kind of approach isn’t anything new but it will be the first time it will be baked into browser themselves.

For now you can use things like Fluid, which is a Mac app that lets you turn a website into an app that you can run from an icon on the desktop.

It uses a browser engine behind the scenes but it hides away all the unnecessary chrome so it looks native.

I currently use this approach for music streaming websites like Grooveshark to turn them into an app.

That way I don’t have to remember which tab they’re open in my browser.

Web Intents

Finding the right tool for th

e job

Web Intents is inherently attached to Web apps.

It allows services on the Web to tell the browser that they’re capable of performing a specific action; like uploading a photo, or posting a message on Twitter.

Subsequent Web apps can then refer to this list of actions, or intents as they’re referred to, whenever you want to upload a photo or post a message to Twitter.

This means that the browser can intelligently suggest which services are capable of performing the required task, rather than displaying them all.

It’s going to be massively useful for situations like the sharing of content, performing payments, etc.

http://webintents.org

And that brings us to the end of our journey.

Threshold of something cool

The time is now

Since I began developing on the Web I really can’t remember a time when all the browsers were working together like this on amazing technologies.

We can now create rich and immersive applications, visualisations, and games that blur the boundary between the Web and the desktop.

All of this with nothing more than the technologies normally used to build a website.

We can even build an entire operating system for a phone using these technologies. That still blows my mind.

We really are on the threshold of something awesome here.

Rob Hawkes

Rawkets.comHTML5 & WebSockets game

Twitter sentiment analysisDelving into your soul

RECENT PROJECTS

Rawkes.comPersonal website and blog

MORE COOL STUFF

jsCraftMinecraft port to JavaScript

Rawket ScientistTechnical Evangelist at Mozilla

@robhawkes

Get in touch with me on Twitter: @robhawkes

Follow my blog (Rawkes) to keep up to date with stuff that I’m working on: http://rawkes.com

I’ve recently worked on a project that analysis sentiment on Twitter: http://rawkes.com/blog/2011/05/05/people-love-a-good-smooch-on-a-balcony

Rawkets is my multiplayer HTML5 and JavaScript game. Play it, it’s fun: http://rawkets.com

I’m currently experimenting with WebGL to see how feasible a JavaScript port of Minecraft would be. The working title is jsCraft.

Become a canvas master

RAWKES.COM/FOUNDATIONCANVAS

Out now

Paperback and digital formats

Learn how to animate

Make two cool space games

Foundation HTML5 Canvas

Foundation HTML5 Canvas is out now on Amazon and other reputable book stores.

http://rawkes.com/foundationcanvas

Hand-picked experts

@ASKMDN & #ASKMDN ON TWITTER

One hour every fortnight

Web development topics

Great discussions

Ask MDN

ASKMDN

And lastly, I’d like to quickly mention Ask MDN which is a project that I’m working on at Mozilla.

The concept is simple. One hour every fortnight we gather a bunch of experts on Twitter to answer your questions about a particular topic.

We’ve had 4 sessions to date and it’s going down really well. If you follow @AskMDN on Twitter you’ll be sure not to miss the next session.

http://twitter.com/AskMDN

THANK YOU

Any questions?

Rob Hawkes

@robhawkes

Thank you.

If you have any questions feel free to grab me on Twitter (@robhawkes), or email [email protected]

Tomorrow's Web and Future Technologies - WDC2011

Technology

Transcript of Tomorrow's Web and Future Technologies - WDC2011