How Browsers Work ☛

As a web developer, learning the internals of browser operations helps you make better decisions and know the justifications behind development best practices. While this is a rather lengthy document, we recommend you spend some time digging in; we guarantee you’ll be glad you did.

All you have ever wanted to know about the internals of Web browsers and more…

I link to the document on HTML5 rocks which is nicely presented but the original is available on Tali Garsiel web site.

(via FunctionSource)

⚑

HornetQ Talks at Marseille & Nice JUGs

Next week, I will talk about HornetQ and Messaging at the French mediterranean JUGs.

The first talk will be at Marseille on Thursday 2011/03/10 and the second at Nice on Friday 2011/03/11. On the same days, Arnaud Simon from Red Hat will also present AMQP and Qpid.

My talks will be about HornetQ & the Web, how HornetQ embraces the Web and offers messaging features on top of Web technologies (REST, HTML5, etc.). It will also have a (brief) introduction to Messaging and JMS for developers who want to leverage messages for their application (Web-based or not).

See you there!

URL Design ☛

Kyle Neath:

URLs are universal. They work in Firefox, Chrome, Safari, Internet Explorer, cURL, wget, your iPhone, Android and even written down on sticky notes. They are the one universal syntax of the web. Don’t take that for granted.

GitHub URLs are a good example to imitate. I often edit them directly to switch from one project to another.

It is painful to see how some Web sites are not bothered by their ugly URLs.
Don’t get me started on the URLs generated by some Web frameworks. I suspect they are ugly on purpose so that you can know which framework is used by a Web site just by looking at its URLs.

In the beginning was the command line, and then came the address bar…

⚑

WebSocket disabled in Firefox 4 ☛

This is a serious threat to the Internet and Websocket and not a browser specific issue. The protocol vulnerabilities also affect Java and Flash solutions. In a web environment that could for example mean that a widely used JavaScript file – like Google analytics – could be replaced on a cache you go through with a malware file.

I think Mozilla decision makes sense and they are right to be cautious.

I am interested by Web Sockets (I implemented a JavaScript library to use STOMP over Web Sockets for messaging) but we need to be very careful about opening a new communication channel between Web clients and servers. Web Sockets must integrate with the whole Web infrastructure (including caches and proxies) and not open a whole can of security issues.

The HTML5 Web Socket API has a lot of potental, especially now that it is available on iOS-based devices (I wrote a node.js-based application which uses it) but it is critical to get the protocol right before it is widespread.

Once Web Sockets are out in the wild, we will have to live with them for a long time, deal with their issues on our own and face ugly consequences for rushing it out.

⚑

James Clark: XML vs the Web ☛

James Clark:

So what’s the way forward? I think the Web community has spoken, and it’s clear that what it wants is HTML5, JavaScript and JSON. XML isn’t going away but I see it being less and less a Web technology; it won’t be something that you send over the wire on the public Web, but just one of many technologies that are used on the server to manage and generate what you do send over the wire.

My background is in “enterprisey” Java and I use XML in Maven, Ant, configuration files, etc. All the Java projects I worked had lots of code to map between XML and Java data structures. I tried to use XML schema and bindings but it replaces some noisy boilerplate code by other noisy boilerplate code. Using schema did help to enforce a structure but not all constraints can be expressed with it and I ended up having an additional programmatic validation phase in Java.

In comparison, working with JavaScript and JSON on the Web is much more simpler. I create JavaScript objects, send them over the wire with the string representation returned by JSON.stringify() and use again JavaScript objects on the server (with JSON.parse() in node.js or using a JSON to Java library for Java servers).
I don’t have schema and I don’t need it. A good documentation is enough in most cases (in the same way, I learn about a XML document by looking at its structure and documentation and rarely its schema).

I believe than XML (and its associated technologies) will end up as an enterprise technology and a opaque content model (e.g. to save OpenOffice documents) and be less and less prevalent on the Web where HTML (with microformats) and JSON are better suited to represent data.

(via John Gruber)

⚑

JSApp.US: a node.js Hosting Provider ☛

I was looking for a provider to host the HTML5 / node.js Web application that I am blogging about here and there and JSApp.US is the first one that I found.

It is straightforward to use and I have deployed my Web application on it. You can try it by yourself on your iPhone or iPad by going to http://board.jsapp.us/.

At first, it complained that __dirname was not defined when server.js is trying to read files based on the URL path. I modified server.js to remove the leading / from the path and it works fine:

// won't work on JSApp.us
fs.readFile(__dirname + path, function(err, data) {
   ...
}
// but this works
fs.readFile(path.substring(1, path.length), function(err, data) {
   ...
}

The offline support does not work: somehow this provider does not return the Content-Type HTTP header set by server.js. Since the cache manifest is not served with the expected text/cache-manifest, I suppose that Safari does not take it into account and bypass the offline configuration.

update: I was wrong the cache.manifest file is returned with the correct Content-Type:

$ curl -Gi http://board.jsapp.us/cache.manifest
HTTP/1.1 200 OK
Server: nginx/0.8.53
Date: Sat, 27 Nov 2010 00:06:45 GMT
Content-Type: text/cache-manifest
Transfer-Encoding: chunked
Connection: keep-alive
 
CACHE MANIFEST
# 2010-11-27
 
CACHE:
/screen.css
/client.js
/jquery.min.js

I don’t know why offline support is not working… The rest of the application seems to work fine though.

It will make the next version of the application more interesting. I want to add Web Sockets support to display the pieces of the other users on the Web application (in addition to its own piece). With this hosting provider, I’ll be able to release a version that everyone will be able to try.

As an aside, it uses SkyWriter (né Bespin) to edit the code and the integration is pretty slick.

⚑

Offline Support and Standalone Mode for HTML5 Web Application

Let’s continue working on the HTML5 Web Application for the iPhone with node.js

Last time, I created the first version of the Web application to move a piece when the user moves the iPhone and coded a simple Web server on top of node.js.

The Web application works fine but it is not pretty and there are a few things we can improve. Before looking at Web Sockets and other fancy code, let’s make the Web application prettier.

iPhone Standalone Web Application

The major annoyance is that Safari chrome takes a lot of space and hides the bottom of the board:

To display a Web application without Safari chrome, we can make it a standalone Web application.

In index.html, we add some <meta> information to the <head> part:

<head>
  ...
  <meta name="viewport" content="width=device-width, user-scalable=0" />
  <meta name="apple-mobile-web-app-capable" content="yes" />
  <meta name="apple-mobile-web-app-status-bar-style" content="black">
</head>

The viewport meta ensures that the Web page will be sized to the device width and prevents the user to change the page scale (by pinching it).

The apple-mobile-web-app-capable will launch the Web application in full-screen mode (without Safari chrome) to look like a native application. The Web application must be added to the iPhone home screen and start from there to appear in full-screen mode.

Last thing is to render the iPhone status bar in black by setting the apple-mobile-web-app-status-bar-style to black.

With these 3 meta information, the Web application looks better and display the whole board

Offline Support

At the moment, node.js serves only static pages. When the iPhone has loaded the Web application, it no longer connects to the Web server.

Let’s add offline support so that we can run the Web application even when the server is down (or unreachable).

HTML5 supports offline applications through the cache manifest. The best introduction on the subject is the Offline chapter in Dive into HTML5.

For this Web application this means that we must write a cache.manifest file:

CACHE MANIFEST
 
CACHE:
/screen.css
/client.js
/jquery.min.js

This cache manifest tells the Web browsers to call all these files to run the Web application offline.

This cache manifest must be declared in the entry point of the Web application (index.html in this case) with the manifest attribute of the html element:

<!doctype html>
<html manifest="/cache.manifest">
...

Finally, we need to update server.js to send the cache.manifest file with the correct HTTP Content-Type. Only the getContentType()function needs to be modified:

function contentType(path) {
   if (path.match('.js$')) {
      return "text/javascript";
   } else if (path.match('.css$')) {
      return  "text/css";           
   } else if (path.match('.manifest$')) {
      return  "text/cache-manifest";
   }  else {
      return "text/html";
   }
}

With this modification, cache.manifest will be sent to the Web browser with the correct text/cache-manifest Content-Type.

We must restart the server to take into account the server.js modification:

$ node.js server.js
HTTP server running at htpp://0.0.0.0:8080

I can then reload the Web application and add it to the iPhone home screen. With offline support, I will be able to use the Web application even if the server is not running or unreachable.

Here is a small video of the result with the updated Web application:

Note the black status bar and the full screen mode after I started the Web application from the iPhone home screen. It is indistinguishable from a native application but runs on the Web (it’s not obvious from the video but the camera was recording from above: the iPhone was held horizontally)

Source Code

I have pushed the application on Github. You can clone it with Git:

git clone git://github.com/jmesnil/board-node.git

Conclusion

By adding a few meta information to the Web application, it is possible to make it look better integrated to the iPhone.
Offline support is trickier than this simple example make it look like but it gives the right idea to start dealing with it.

Next step is to add Web Sockets support to the node.js to display the moving pieces of other users on the Web application.

HTML5 Web Application for iPhone and iPad With node.js

Update: I have written another post about adding offline support to the Web application and making it look like a native iPhone application.

On the server, I want to learn more about node.js which is an interesting way to develop server-side applications. On the client-side, I also wanted to check what can be done with HTML5 on iPhone and iPad with the release of iOS 4.2.1.

I will write a simple web application to do both at the same time. The idea is simple enough but will allow me to dive into node.js and HTML5 quite extensively.

The idea is to write a web application that display a piece moving on a board when the user moves its device. Later on, I will expand it so that the user will also see the pieces of all others users connected to the server.

But the first iteration will be enough to familiarize with:

node.js (to serve static content at first)
Canvas API (to draw the board and the piece)
DeviceOrientation Event API (to detect the orientation of the mobile, it was added in iOS 4.2.1)

Web Server

I will use node.js as my Web server (I installed it from Git by following the build instructions on its web site).

The first version of the server needs to serve static files with the correct HTTP Content-Type. At first, it will serve only HTML pages, CSS stylesheets and JavaScript files.

The whole server.js code is:

var http = require('http');
var url = require('url');
var fs = require('fs');
var sys = require('sys');
 
// the HTTP server
var server;
// the HTTP port
var port = 8080;
 
server = http.createServer(function(req, res){
  var path = url.parse(req.url).pathname;
  if (path == '/') {
     path = '/index.html'
  }
  console.log("serving " + path);
  fs.readFile(__dirname + path, function(err, data){
     if (err) {
        res.writeHead(404);
        res.end();        
     } else {
        res.writeHead(200, {'Content-Type': contentType(path)});
        res.write(data, 'utf8');
        res.end();
     }
  });
});
 
function contentType(path) {
   if (path.match('.js$')) {
      return "text/javascript";
   } else if (path.match('.css$')) {
      return  "text/css";           
   }  else {
      return "text/html";
   }
}
 
server.listen(port);
console.log("HTTP server running at htpp://0.0.0.0:" + port );

The code is straightforward: when a request is handled by the server, it looks in the current directory (where server.js is) for a file with the given path and writes its content in the HTTP response body.
The contentType(path) function checks the file suffix to use the correct HTTP Content-Type for the response.

It will serve files from http://<server-name>:8080/.
For the rest of the example, I will use the name of my machine, blackbook.local: to access the Web application from my iPhone, I am using http://blackbook.local:8080.

In future iterations, I will add more interesting code to node.js (to use WebSockets for example) but for the moment this simple server is enough.

Web Application

Let’s now focus on the client-side part of the Web application.

index.html

The Web application is loaded from a single HTML5 page, index.html:

<!doctype html>
<html>
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, user-scalable=0" />
  <title>Board</title>
  <link rel=stylesheet href=screen.css>
</head>
<body>
  <canvas id="board" width="320" height="460"></canvas>
 
  <script src='jquery.min.js'></script>  
  <script src="client.js"></script>
</body>
</html>

The page contains a single <canvas> element named board. All the code to draw on the canvas is in client.js JavaScript file.

client.js

client.js contains code to:

get orientation information from the device (using the DeviceOrientation Event API)
draw the board and the piece based on the orientation information (using the Canvas API)

First, it defines a few constants used to draw the board and the piece:

var kBoardWidth = 320;
var kBoardHeight = 460;
var kCircleRadius = 32;

The piece object is defined literally:

var piece = {
   center : {
      x: kBoardWidth / 2,
      y: kBoardHeight / 2,
      xShift : 0,
      yShift : 0
   },
   color: "#000"
};

The piece has a center position with its coordinates (initially at the center of the board) and its acceleration on the x and y axes, and its color is black.

I use jQuery to bootstrap the application when the document is ready:

$(document).ready(function() {
   var board = document.getElementById("board");
   var context = board.getContext("2d");
   window.addEventListener("devicemotion", function(event) {
      var accel = event.accelerationIncludingGravity;
      piece.center = computeCenter(piece.center, accel);
      drawGrid(context);
      drawPiece(context, piece);
   }, true);
});

First we keep a reference on the board canvas that was declared in the HTML page and its associated 2D context. The context will be used to draw the piece and the board on the canvas.

We use the DeviceOrientation API (supported by the iPhone and the iPad since iOS 4.2.1) to detect the acceleration of the device. Periodically, the browser will call the handler associated to the devicemotion event.
When that happens, the handler retrieves the accelerationIncludingGravity property, compute the new center position of the piece and draw the board and the piece on the canvas.

The method to draw the board is using the 2D context (the board is displayed as a grid):

function drawBoard(context) {
   context.clearRect(0, 0, kBoardWidth, kBoardHeight);
   for (var x = 0.5; x < kBoardWidth; x += 10) {
       context.moveTo(x, 0);
       context.lineTo(x, kBoardHeight);
   }
   for (var y = 0.5; y < kBoardHeight; y += 10) {
      context.moveTo(0, y);
      context.lineTo(kBoardWidth, y);
   }
   context.strokeStyle = "#eee";
   context.stroke();
}

The code to draw the piece is also using the 2D context to draw a circle from the piece’s center and its color:

function drawPiece(context, piece) {
   context.fillStyle = piece.color;
   context.beginPath();
   context.arc(piece.center.x, piece.center.y, kCircleRadius, 0, Math.PI * 2, false);
   context.closePath();
   context.fill();
}

Finally, we need to compute the updated position of the piece’s center based on its current position and the acceleration information from the browser:

function computeCenter (oldCenter, acceleration) {
   newCenter = {};
   newCenter.xShift = oldCenter.xShift * 0.8 + acceleration.x * 2.0;
   newCenter.yShift = oldCenter.yShift * 0.8 + acceleration.y * 2.0;
   newCenter.x = oldCenter.x + oldCenter.xShift;
   // use *minus* to compute the center's new y
   newCenter.y = oldCenter.y - oldCenter.yShift;
   // do not go outside the boundaries of the canvas
   if (newCenter.x < kCircleRadius) {
      newCenter.x = kCircleRadius;
   }
   if (newCenter.x > kBoardWidth - kCircleRadius) {
      newCenter.x = kBoardWidth - kCircleRadius;
   }
   if (newCenter.y < kCircleRadius) {
      newCenter.y = kCircleRadius;
   }
   if (newCenter.y > kBoardHeight - kCircleRadius) {
      newCenter.y = kBoardHeight - kCircleRadius;
   }
   return newCenter;
}

To smooth the movement and increase the acceleration when the mobile is moved, I applied a low-pass filter:

newCenter.xShift = oldCenter.xShift * 0.8 + acceleration.x * 2.0;
newCenter.yShift = oldCenter.yShift * 0.8 + acceleration.y * 2.0;

The rest of the method ensures that the piece will remain in the boundaries of the board.

screen.css

Last thing is to make sure that the canvas fills the whole Web page by removing the margin and padding on the body element. This is done through CSS in the screen.css file:

body {
   margin: 0px;
   padding: 0px;
}

All these files are located in the same directory than server.js.

Run the example

Let’s start the node.js server:

$ node server.js
HTTP server running at htpp://0.0.0.0:8080

Now, from my iPhone, I go to the Web page on http://blackbook.local:8080 (if you go on the Web application from your machine, nothing will happen since the devicemotion event is not available on desktop browsers).

As I have not found a way to prevent the iPhone to change the screen orientation when it is moved, I have locked the iPhone in portrait mode before running the Web application.

Here is a small video of the result:

It’s not obvious from the video but the camera was recording from above: the iPhone was held horizontally.

Source Code

I have pushed the application on Github. You can clone it with Git:

git clone git://github.com/jmesnil/board-node.git

The code is slightly different as I have already started the next iteration but the main idea remains the same.

Conclusion

With a few lines of JavaScript on both the client and server sides, it is possible to:

use node.js to write a Web server returning files
use the Canvas API to draw things on a Web page
use the DeviceOrientation Event API orientation to have interaction between the device and the Web page

Next steps will be to improve the user experience:

make the application appear as a standalone application on the iPhone home screen and remove Safari chrome
support offline mode to use the Web application even when the server is not running

Jeff Mesnil

How Browsers Work ☛

HornetQ Talks at Marseille & Nice JUGs

URL Design ☛

WebSocket disabled in Firefox 4 ☛

James Clark: XML vs the Web ☛

JSApp.US: a node.js Hosting Provider ☛

Offline Support and Standalone Mode for HTML5 Web Application

iPhone Standalone Web Application

Offline Support

Source Code

Conclusion

Further Reading

HTML5 Web Application for iPhone and iPad With node.js

Web Server

Web Application

index.html

client.js

screen.css

Run the example

Source Code

Conclusion

Further Reading

“Read Lature”: A Web Application to Save Articles and Read Them Later

HTML Glyphs Cheatsheet