Javascript Blog Archive

Breaking Up Your Asset Pipeline

The Rails Asset Pipeline to me is kind of like Bundler. At first I was very nervous about it and thought that it would be very troublesome. But after a while of using it, I realized the wisdom behind it and my life got a lot easier. The asset pipeline is fabulous for putting all your assets into a single file, compressing them, and serving them in your site without cluttering everything up. Remember these days?

<%= javascript_include_tag 'jquery.validate.min.js','jquery.watermark.min.js','jquery.address-1.4.min','jquery.ba-resize.min','postmessage','jquery.cookie','jquery.tmpl.min','underscore','rails','knockout-1.3.0beta','knockout.mapping-latest' %>

A basic component of the Asset Pipeline is the manifest file. A manifest looks something like this

// Rails asset pipeline manifest file
// app/assets/javascript/application.js

//= require jquery
//= require jquery_ujs
//= require_tree .

For a quick rundown on how the Asset Pipeline works, I highly recommend taking a few minutes to watch Railscast episode 279. But there are a few things that I'd like to point out here.

First, notice that the name of the file is application.js. This means that all the javascript specified in the manifest file will be compiled together into a single file named application.js. In production, if specified, it will also be compressed and uglified. So rather than specifying a really long and ugly javascript_include_tag, you just need one:

<%= javascript_include_tag "application.js" %>

Second, this means that if you only have one manifest file in your application, then all of your javascript will be loaded on every page, assuming that your javascript_include_tag is loading in the head of your layout. For small applications with very little javascript, this is fine, but for large projects where there are large client-side applications, this could be a problem for performance. For example, do you really need to load all of ember.js or backbone.js or knockout.js in the admin portion of your app when it isn't used at all? Granted, these libraries are pretty small, but the applications that you build that go along with them don't need to be loaded on every page.

So I thought there should be a way to break up the manifest file so that only javascript we need is loaded. The need for this came when I was upgrading the main functionality of a website to use the Backbone.js framework. The app was large and complex and the "single-page-application" was only a part of it. I didn't want my application to load on the portion of the site where it wasn't need. I looked high and low on the web for a solution to this but only found obscure references so I thought I would take some time to put the solution out there cleanly hoping to save some of you from trying to figure it out on your own.

The solution lies in the first point raised earlier about the asset pipeline. The fact is, you can create a manifest file anywhere in your assets directory and use the same directives to load the javascript. For example, in the app/assets/javascripts/ directory the application.js file is used by default. But it can be named anything and placed anywhere. For my problem, I only wanted my javascript application loaded when I explicitly called it. My directory structure looked like this:

|~app/
| |~assets/
| | |+fonts/
| | |+images/
| | |~javascripts/
| | | |+admin/
| | | |+lib/
| | | |+my_single_page_app/
| | | |-application.js

My application.js file looks just like the one above but I removed the require_tree directive. This is important because now I don't want all of my javascript to load from the application.js. I just want some of the big stuff that I use all over like jQuery. (Ok, I also added the underscore library in there too because it's just so darn useful!) This file is loaded in the head of my layout just as I described above.

Then, I created another manifest file named load_my_app.js in the root of my_single_page_app/. It looks like this:

//= require modernizr
//= require backbone-min
//= require Backbone.ModelBinder
//= require my_single_page_app/my_app
//= require_tree ./my_single_page_app/templates
//= require_tree ./my_single_page_app/models
//= require_tree ./my_single_page_app/collections
//= require_tree ./my_single_page_app/views
//= require_tree ./my_single_page_app/utils

Then in my view that displays the single page app, I have these lines:

<% content_for :head do %>
  <%= javascript_include_tag "my_single_page_app/load_my_app.js" %>
<% end %>

Now my single page application is loaded into the page as my_single_page_app/load_my_app.js by the Asset Pipeline and it's only loaded when needed. And a big bonus is that I don't need to worry about any of the code that I wrote or libraries I want to use interfering with the rest of the site.

Using Modernizr with the Rails Asset Pipeline

Like many web developers, I use Google Chrome to develop my front-end user interface. I do this because the Chrome Developer Tools are very nice at letting me fix CSS styles and debug JavaScript. One downfall to using Google Chrome as my only view of my website while developing, however, is that I often find that I've used some feature of HTML or CSS that isn't supported by other browsers. While this problem seems to come less often these days, I still find the occasional glitch. Most notably, this seems to happen with Microsoft's Internet Explorer (IE) more than any other browser.

During a recent project, I finished up the UI so that everything looked and felt great. Then I popped open IE to see what sort of things I would need to fix. One feature of HTML that I didn't know wasn't supported by IE was that of placeholder text for form inputs. For example:

<input type="text" name="user_name" placeholder="Enter your user name" />

This HTML generates a nice looking input box with the words "Enter your user name" already entered into the input field. When a user clicks on the field, the placeholder text disappears so that the user can begin to add their custom text. This is a popular solution for many web-sites so I was shocked to learn that IE up to version 10 doesn't support this feature. My mind turned to horrid thoughts of writing custom javascript to handle placeholder text in input fields. It didn't take long for me to think, surely someone has to have already fixed this.

Thankfully, someone had.

I found placeholders.js on GitHub. It was a clean solution and it solved my problem by just adding it to my application. Since I'm using Rails 3.2, I just added the Placeholders.js file to the vendor/assets/javascript directory and then added it to the bottom of my app/assets/javascript/application.js manifest file like so:

//= require Placeholders.js

Boom, suddenly all my placeholders were working flawlessly. But I was a little irritated with this solution. Why should all my non-IE users have to load this javascript file? And if I have more issues, there could be many of these little javascript solutions I need for things like gradients and rounded borders. And what about custom CSS that I had to write just for these features to work? Surely there must be a better way.

Enter Modernizr.

From Modernizr's web site, "Modernizr is a JavaScript library that detects HTML5 and CSS3 features in the user’s browser." In short, Modernizer detects if certain features are available and then gives you a way to load custom assets to deal with those features, or the lack thereof. It introduced to me the concept of polyfills. A polyfill is simply some code that helps you deal with browser incompatibilities. The nice JavaScript Placeholders.js file mentioned earlier is a polyfill.

Modernizr does two things to help you detect these compatibilities and deal with them. First, it adds some custom classes to your <html> tag that indicate what features are enabled and disabled in your browser. A <html> tag in IE might look something like this:

<html class="js no-borderradius no-cssgradients">

While in Chrome it would look like this:

<html class="js borderradius cssgradients">

This enables you to put together some CSS code to add extra styles for incompatible browsers. For example, some SASS to deal with a css gradient issues may look like this:

.no-cssgradients gradient {
  background-image: blue url('assets/ie_gradients/blue_gradient.png') no-repeat;
}

This would then load a custom gradient image rather than using the CSS gradient for incompatible browsers.

The second thing Modernizr does for you is let you custom load assets based on compatibility. It does this using the yepnope.js library. You simply load your polyfills based on what compatibility is detected. This is exactly what I needed for my Placeholders.js file. I only wanted it to load for browsers that didn't support the placehoder input attribute. I found a great blog article on how to implement the Placeholders.js library using Modernizr and started following the directions. And here there be dragons.

YepNope.js is essentially an assets loader. You load assets in Modernizr using the Modernizer.load function that wraps the YepNope functionality; like this:

Modernizr.load({
  test: Modernizr.input.placeholder,
  nope: ['Placeholder.js'],
  complete: function(){Placeholders.init();}
});

Well, that doesn't quite sit well with the other asset loader in my app, the Rails asset pipeline. So, to cut to the chase, here's how you get the two to play nicely with each other.

First, place the Modernizr library (modernizr.js in this case) in the vendor/assets/javascript directory.

Second, place the following line at the top of your asset pipeline manifest js file (such as app/assets/javascripts/application.js):

//= require modernizr

Third, DO NOT, put your polyfill in the asset pipeline manifest. If you do, it will load for all browsers thus defeating the whole point. I moved my Placeholders.js file from the vendor directory to the app/assets/javascripts/polyfills directory and made sure that I wasn't loading this director in my manifest with a require_tree directive.

Fourth, add the following line to your config/application.rb if you want the polyfill to be compiled like the rest of your assets:

config.assets.precompile += ['polyfills/Placeholders.js']

Fifth, you are now ready to use Modernizr to load your polyfill. I did so in my JavaScript app using CoffeeScript like so:

# Polyfills
Modernizr.load
  test: Modernizr.input.placeholder
  nope: ['/assets/polyfills/Placeholders.js']
  complete: -> Placeholders.enable() if Placeholders?

Now my Placeholders.js library only loads when the browser doesn't support input placeholders. Hooray!

Note: One thing that tripped me up a little is that you can either use the development Modernizr library or you can build your own with just the functionality you want to detect. This corresponds directly to the test line in the load function options. Make sure if you build your own modernizr.js file that you include all the features you test. In my case, I had to include the "Input Attributes" option.

Paper Source: Case Study with Google Maps API

Basic Google map with location markers.

Recently, I've been working with the Google Maps API on Paper Source. Paper Source is one of our Interchange clients who has over 40 physical stores throughout the US. On their website, they had previously been managing static HTML pages for these 40 physical stores to share store information, location, and hours. They wanted to move in the direction of something more dynamic with interactive maps. After doing a bit of research on search options out there, I decided to go with the Google Maps API. This article discusses basic implementation of map rendering, search functionality, as well as interesting edge case behavior.

Basic Map Implementation

In it's most simple form, the markup required for adding a basic map with markers is the shown below. Read more at Google Maps Documentation.

HTML

<div id="map"></div>

CSS

#map {
  height: 500px;
  width: 500px;
}

JavaScript

//mapOptions defined here
var mapOptions = {
  center: new google.maps.LatLng(40, -98),
  zoom: 3,
  mapTypeId: google.maps.MapTypeId.ROADMAP
};

//map is the HTML DOM element ID where it will be rendered
var map = new google.maps.Map(document.getElementById("map"), mapOptions);

//all locations is a JSON object representing locations,
//where each location has a latitude and longitude
$.each(all_locations, function(i, loc) {
  var marker = new google.maps.Marker({
    map: map,
    position: new google.maps.LatLng(loc.latitude, loc.longitude)
  });
})

Building Search Functionality

Search interface. Search results are listed on the left, and map with markers is shown on the right.

Next up, I needed to build out search functionality. Google has its own geocoder to allow address searches. Here is the basic markup for running a search:

var geocoder = new google.maps.Geocoder();
//search is a variable representing the user search, such as a zip code, city name, or state name
geocoder.geocode({ 'address' : search }, function(results, status) {
  var search_center = results[0].geometry.bounds.getCenter();

  var mapOptions = {
    center: search_center,
    zoom: 10,
    mapTypeId: google.maps.MapTypeId.ROADMAP
  };
  var map = new google.maps.Map(document.getElementById("map"), mapOptions);

  $.each(all_locations, function(i, loc) {
    var marker = new google.maps.Marker({
      map: map,
      position: new google.maps.LatLng(loc.latitude, loc.longitude)
    });
  })
}

In the above code, the search term is passed into the Geocoder object and a map with all locations marked is rendered. To determine which markers are in the visible map boundaries, the following map.getBounds().contains() method would be leveraged:

var visible_locations = [];
$.each(all_locations, function(i, loc) {
  if(map.getBounds().contains(new google.maps.LatLng(loc.latitude, loc.longitude))) {
    visible_locations.push(loc);
  }
});
//render visible locations to the left of the map

One final step here is to add a listener to the map, so that visible locations are updated when the user zooms in and out. This is accomplished with the following listener:

google.maps.event.addListener(map, 'zoom_changed', function() {
  //call method to rerender visible locations
});

Handling Zero Results

What happens if your Geocoder object can't find the address? A simple conditional can be used:

geocoder.geocode({ 'address' : search }, function(results, status) {
  if(status == "ZERO_RESULTS") {
    //notify customer that no results have been found
  } else {
    //got results, render location
  }
}

Calculate and Sort by Distance

The next layer of logic I needed to add was the ability to determine the distance between the search address and sort the results by distance. To calculate distance, I did some research and settled on the following code:

var R = 6371;
$.each(all_locations, function(i, loc) {
  var loc_position = new google.maps.LatLng(loc.latitude, loc.longitude);
  var dLat  = locations.rad(loc.latitude - search_center.lat());
  var dLong = locations.rad(loc.longitude - search_center.lng());

  //calculate spherical distance between search position and location
  var a = Math.sin(dLat/2) * Math.sin(dLat/2) +
  Math.cos(locations.rad(search_center.lat())) *
    Math.cos(locations.rad(search_center.lat())) * 
    Math.sin(dLong/2) * Math.sin(dLong/2);
  var c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1-a));
  var d = R * c;
  loc.distance = d;

  //convert distance to miles
  loc.readable_distance = 
    (google.maps.geometry.spherical.computeDistanceBetween(search_center, loc_position) * 
    0.000621371).toFixed(2);
});

To sort the locations by distance, I leverage jQuery sort:

var sort_by_distance = function(obj) {
  return obj.sort(function(a, b) {
    if(a.distance > b.distance) {
      return 1;
    } else {
      return -1;
    }
  })
};
var sorted_locations = sort_by_distance(all_locations);

Adjust Map Boundaries to Include Specific Markers

Another interesting use case I needed to handle was forcing the map to zoom out to include stores within 100 miles if there was nothing in the initial map boundaries, e.g.:

The search for "27103" doesn't return any nearby stores, so the map is extended to include stores within 100 miles.

To accomplish this functionality, I added a bit of code to extend the map boundaries:

geocoder.geocode({ 'address' : search }, function(results, status) {
  var search_center = results[0].geometry.bounds.getCenter();

  var mapOptions = {
    center: search_center,
    zoom: 10,
    mapTypeId: google.maps.MapTypeId.ROADMAP
  };
  var map = new google.maps.Map(document.getElementById("map"), mapOptions);
  var current_bounds = results[0].geometry.bounds;

  $.each(all_locations, function(i, loc) {
    var loc_position = new google.maps.LatLng(loc.latitude, loc.longitude);
    var dLat  = locations.rad(loc.latitude - search_center.lat());
    var dLong = locations.rad(loc.longitude - search_center.lng());

    //calculate spherical distance between search position and location
    var a = Math.sin(dLat/2) * Math.sin(dLat/2) +
    Math.cos(locations.rad(search_center.lat())) *
      Math.cos(locations.rad(search_center.lat())) * 
      Math.sin(dLong/2) * Math.sin(dLong/2);
    var c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1-a));
    var d = R * c;
    loc.distance = d;

    //convert distance to miles
    loc.readable_distance = 
      (google.maps.geometry.spherical.computeDistanceBetween(search_center, loc_position) * 
      0.000621371).toFixed(2);

    var marker = new google.maps.Marker({
      map: map,
      position: new google.maps.LatLng(loc.latitude, loc.longitude)
    });

    if(loc.readable_distance < 100) {
      current_bounds.extend(loc_position);
    }
  });

  //Google map method to fit map boundaries to desired boundaries
  map.fitBounds(current_bounds);
}

Disable Scroll and Zoom on Mobile-Sized Devices

One final behavior needed was to disable map zooming and scrolling on mobile devices, to improve the usability on mobile/touch interfaces. Here's how this was accomplished:

var options_listener = google.maps.event.addListener(map, "idle", function() {
  if($(window).width() < 656) {
    map.setOptions({
      draggable: false,
      zoomControl: false,
      scrollwheel: false,
      disableDoubleClickZoom: true,
      streetViewControl: false
    });
  }
  google.maps.event.removeListener(options_listener);
});

Conclusion

With all this code, the final location search functionality at Paper Source includes:

• Basic United States map rendering to display all physical store locations.
• Search by location which shows stores within 100 miles, and allows users to zoom in and out to adjust their search. Search lists results sorted by distance.
• "Saved" or "Quick" searches by states, which displays all physical stores by state.
• Adjustment of mobile display map options.

Generating PDF documents in the browser

What you will learn.

• How to generate PDF documents in the browser with JavaScript
• How to generate them out of normal HTML
• How to open those PDFs in new windows
• How to render them inline inside the DOM

Introduction

Every once in a while as web developers we face the challenge of providing PDF documents for the data we have persisted in the database.

The usual approach is to:

• generate the document with a library / DSL in the language of the backend
• or - generate normal html view and use a utility like wkhtmltopdf

That works nice, but what if you’re developing an SPA app which only consumes JSON data from its backend? Imagine a scenario when the backend isn’t capable of producing responses other than JSON data. What could you do there?

The solution

Thanks to some very bright folks behind the jsPDF library we have both above mentioned options right inside the browser.

I strongly encourage you to visit their website. There is a nice live coding editor set up which reflects in real time the PDF your code is producing.

Using the DSL

The whole process looks like this:

# 1. create jsPDF object:
doc = new jsPDF()

# 2. put something interesting in there:
doc.setFontSize(22)
doc.text(20, 20, 'This is a title')
doc.setFontSize(16)
doc.text(20, 30, 'This is some normal sized text underneath.')

# 3. choose some useful outlet:
doc.output('dataurlnewwindow', {})

Where is the documentation?

One thing I didn’t like about the experience which was really great otherwise, was the fact that you could browse the Internet for hours without luck, looking for some useful docs for the library.

For all of you who don’t know the shortcut yet, here it is:

git clone https://github.com/MrRio/jsPDF.git

And in the doc directory you will find nicely generated docs out of the project sources - very neat!

Can I generate it just from HTML?

Yes you can. On the project’s website the authors are warning us that this feature is still experimental. But I have to say that last time I used it, all went well and the code produces nice PDF to this day (yay!).

doc.fromHTML $('.report_web').get(0), 10, 10,
  'width': 170,
  'elementHandlers': 
    'LI': (el, renderer) =>
      if renderer.y > 250
        doc.addPage()
        renderer.y = 10
      else
        renderer.y += 10
      false
    'H1': (el, renderer) =>
      doc.setFontSize(14)
      doc.setFontStyle('bold')
      doc.text($(el).text(), 10, renderer.y)
      doc.setFontSize(12)
      doc.setFontStyle('normal')
      true
doc.output 'dataurlnewwindow', {}

Take a look at the code above. #fromHtml method takes the DOM element, margins, and a hash of options. From my experience the elementHandlers attribute seems to be mandatory (I got errors without it).

Every element handler is just a pair of capitalized tag name and the function which takes DOM element and the renderer context.

The renderer variable contains data of the current paragraph, the pdf internal object, settings object and x and y. You have to remember to change x and y variables yourself after creating some content.

One last thing to remember is that the handler can return either true or false. If it returns true, then the engine will not create content out of the given DOM element on its own. Otherwise, after your handler finishes its execution, the engine will generate the content for you. So you can use those handlers to munge a few things in the output document or you can take over the whole control and do it your way.

Open PDF output in a new window

You may have notice in one of above examples, that we were simply creating new windows with PDF documents there.

To recap, opening PDF documents in new window:

doc.output 'dataurlnewwindow', {}

Render PDFs in the DOM

Nesting the PDF document inside the DOM seems way cooler. To do so:

data = doc.output 'dataurlstring', {}
$('#report').html "<iframe src='#{data}'></iframe>"

You will notice that the #output method takes a string which tells it how you’d like to generate the PDF. If you specify the ‘dataurlstring’ it will return a string containing the url data you use in your iframe. That is how the last example works.

There is much more to jsPDF than shown in this short post. If you’re interested, you can browse the docs included in the projects repository. There are a lot of useful methods for creating almost any type of document you would want.

Data binding in web applications

Ever since JavaScript was introduced - the world of web programming was constantly been visited by creative minds, ready to solve burning problems web developers were experiencing.

Navigating through some of the oldest web-dev articles, we still can feel the pain of manipulating the DOM, the-vanilla-way and creating ever so clever hacks to make the code work on all of the web browsers.

Then, the JS-frameworks epidemy started to disseminate. As web developers - we started to have some powerful tools. The productivity of average nerdy Joe raised immediately as he started using Prototype.js, Mootools or jQuery.

Evolution of UI programming for browsers

And so was the start of our global evolution, towards making the Web - our main means of interacting with users. Few with big enough imagination were already seeing HTML, CSS & JS as the future standard of creating user interfaces for data-rich applications. Of course, there were toolkits like ExtJS which aimed at nothing but this - but who would have known back then, that we will be able to build apps for smartphones the way we are building ones for the web?

I have my little observation in life — all hot stuff in the world of technology, has their phases.. That is:

• they don’t exist, but the problems they solve do and they are painful
• they show up, and there is massive excitement
• people discover new pains brought on by these solutions
• next pain and the beginning of the "next big thing"

At the beginning it was painful to write all the vanilla JS code without shooting yourself in your own foot (well let’s just say it - JS is Pandora’s Box of pain). Then you realize you’ve got your sweet & cool jQuery with which you can rapidly impress your boss/clients/friends.

Your next surprise is that jQuery isn’t enough - ever wondered why frameworks like Backbone.js were created?

Data binding

Here is the challenge for you: write GMail web interface clone using just jQuery. Here is the spoiler: you’d get lost in managing your app’s state - and syncing UI to reflect it. In fact - even small todo app in vanilla jQuery could end up being quite complicated. That’s just not the way other folks are dealing with UI programming. A long time ago they were - but they got smarter. We can learn from their mistakes.

When I was beginning my journey into the world of professional programming, I was given a job of creating small business app with C# and Windows Forms. Despite the fact that in the world of desktop apps you have to be mindful of much more things than in the web/browsers world (like updating your UI only on the main thread), the whole experience was pretty much the same as with vanilla jQuery:

1. create inputs
2. subscribe to its events
3. write the code that updates inner state based on the state of those inputs/controls

This quickly gets very cumbersome as the needs grows. You could have a requirement that one set of inputs is dependent on another, which results in hours of coding and more hours in debugging.

The way smart devs are tackling this issue is called data binding. This simply means that you tell specific portion of UI to be bound to some value from your business layer.

One nice example of the framework that employs data binding is Apple’s Cocoa. The XCode even has nice visual tools to bind controls values with the data model. Another example of the framework that uses data binding extensively is WPF (Windows Presentation Foundation). The concept widely used there is the MVVM pattern (Model-View-ViewModel). Which basically says that you have your view model object and UI which is bound to the view model and all you have to do is to manage the view model.

That’s the core concept here: managing objects is much easier than managing UI

Data binding on web

As the need came with more and more amazing web apps - we now have quite a nice choice when it comes to employing data binding:

Angular.js
Angular.js is one of the most impressive front-end JS frameworks there are. It has view templates mechanism allowing you to specify your binding statements directly there.

Ember.js
Ember.js gains popularity every day. Just like Angular.js, it has its own nice templating language (handlebars) which allows you to have your binding statements in the markup too.

Knockout.js
If you don’t want to use any of those frameworks, that's a great library that implements just the data binding part.
It uses the concept I described before: the MVVM pattern. You are simply defining your view model and telling your UI where it has to get values from and that's it.

Backbone.js
But what if you’d like to use Backbone.js? It’s true that it’s amazing and is one neat piece of technology you can use to build your next ‘cool app’. There is one caveat though: you have to set up all the niceties that comes along with Angular.js or Ember.js yourself, including all the sweetness of data binding.

Backbone.js + Knockout.js

Here is a simple way of using Knockout.js with Backbone.js:

• for every Backbone.js view, define complementary view model
• put ‘ko.applyBindings’ at the end of your ‘render’ method
• do not use any logic in your view templates - rely solely on data binding

So for example:

# app/views/photos.coffee

module.exports = class PhotosView extends Backbone.View
   template: require "./templates/photos/index"

   initialize: () ->
       @view = new PhotosViewModel()
       @render()

   render: () ->
       $(@el).html(@template())
       ko.applyBindings(@view, @el)

   setData: (photos) =>
       @view.photos(photos)

class PhotosViewModel
   constructor: ->
       @photos = ko.observableArray []

# app/views/templates/photos/index.eco

Now - with this, you can manipulate the photos array however you like, and the UI will get auto-updated on its own. But this idea shines the most when it comes to implementing forms with it. Take a look at the following example:

# app/views/company.coffee

module.exports = class EditCompanyView extends Backbone.View
   template: require "./templates/companies/edit"

   initialize: () ->
       @view = new EditCompanyViewModel()
       @render()

   render: () ->
       $(@el).html(@template())
       ko.applyBindings(@view, @el)

   setData: (company) =>
       @view.company(company)

class EditCompanyViewModel
   constructor: ->
       # let’s instantiate it with default company for bindings to work
       @company = ko.observable(new Company())
       @companyName = ko.computed
           read: => @company.get('name')
           write: (value) => @company.set('name': value)
       @companyState = ko.computed
           read: => @company.get('state')
           write: (value) => @company.set('state': value)
       @companyWebsite = ko.computed
           read: => @company.get('url')
           write: (value) => @company.set('url': value)
       @saveCompany = () =>
           @company.save()

# app/views/templates/companies/edit.eco

Name:
   
   State:
   
   Website:
   

   Submit

Other solutions

These are quite contrived examples to give you an idea of how it can be done. The beauty of Backbone is that it allows you to structure your code however you’d like. And there are plenty of other solutions for this. Most notable:

• http://kmalakoff.github.com/knockback/
• http://rivetsjs.com/
• https://github.com/DreamTheater/Backbone.DataBinding

Crossed siting; or How to Debug iOS Flash issues with Chrome

This situation had all the elements of a programming war story: unfamiliar code, an absent author, a failure that only happens in production, and a platform inaccessible to the person in charge of fixing this: namely, me.

Some time ago, an engineer wrote some Javascript code to replace a Flash element on a page with an HTML5 snippet, for browsers that don't support Flash (looking at you, iOS). For various reasons, said code didn't make it to production. Fast forward many months, and that engineer has left for another position, so I'm asked to test it, and get it into production.

Of course, it works fine. My only test platform is an iPod, but it looks great here. Roll it out, and ker-thunk: it doesn't work. Of course, debugging Javascript on an iPod is less than optimal, so I enlisted others with Apple devices and found that it mostly failed, but maybe worked a few times, depending on [SOMETHING].

To make matters a bit worse, the Apache configurations for the test and production environments differed, just enough to raise my suspicions and convince me that was worth investigating. Once I went down that path, it was tough to jar myself loose from that suspicion.

I tried disabling Flash in Firefox to trigger the substitution, but that didn't seem to have the desired effect (as the replacement didn't happen, which was a different error than the replacement failing). I tried a browser emulation site (which shall remain nameless for this post, as I don't think they are bad at what they do, but they don't emulate iOS browsers in this capacity).

Eventually we disabled flash in Chrome (by visiting the chrome://plugins page). That unveiled the hidden error:

XMLHttpRequest cannot load http://www.somewhere.com/ajax/newstuff.html. Origin http://somewhere.com is not allowed by Access-Control-Allow-Origin.

There's the crux of it: the browser was sitting on an address which appeared different than that of the AJAX target.

The site involved is an Interchange site, and page constructed a URL using the [area] tag, which makes a fully-qualified URL from a fragment like "ajax/newstuff". That URL was being seen by the iOS browsers as a cross-site scripting attempt, as it didn't precisely match where the browser found the page. The error was not visible in the Safari browser on my iPod, and browsers I had access to which could have displayed the error, weren't suffering it.

I replaced the [area] tag with a plain relative URL and the problem disappeared.

TL;DR:

               $.ajax({
-                  url: "[area href=|ajax/newstuff|]",
+                  url: "ajax/newstuff.html",
                   type: 'html',

The original code caused a cross-site scripting failure.

Ghost Table Cells in IE9

What's this about ghosts?

I recently came across an arcane layout issue in my work on the new RJ Matthews site. The problem was specific to Internet Explorer 9 (IE9). The related CSS styles had been well tested and rendered consistently across a variety of browsers including IE7 and 8. Everything was fine and dandy until some new content was introduced into the page for a "Quickview" feature. While all of the other browsers continued to behave and render the page correctly, the layout would break in random and confusing ways in IE9. The following screenshots compare the correct layout with an example of the broken layout in IE9.

Correct grid layout:

Broken layout in IE9:

The Stage

The following is a list of the factors at work on the page in question:

• Internet Explorer 9
• Browser mode: IE9, Document mode: IE9 standards
• Some content manipulation performed via JavaScript (and jQuery in this case)
• Lots of table cells

Debugging

The page included a list of products. The first "page" of twelve results was shown initially while JavaScript split the rest of the list into several additional pages. Once this JavaScript pagination function was complete, users could cycle through products in bite-sized pieces.

My first thought was that the issue may be related to CSS or Javascript. I tested and debugged the styles thoroughly, tweaked styles and edited the underlying HTML structure to see if that might resolve the problem. I also tested the JavaScript and compared the original HTML with the parts which had been paginated via JavaScript. No dice.

When changes improved the paginated HTML, the bug appeared in the initial HTML. Other changes resolved the issue in the original HTML but it appeared in the paginated HTML.

I inspected the table with the Chrome Developer Tools console and also with the Developer Tools in IE9. There did not appear to be any differences between the rows which rendered properly and those which were skewed.

Bugging Out

At this point I began to research the issue and discovered it was a bug in the IE9 browser. This Microsoft forum post describes the issue and includes responses from Microsoft stating that it will not be fixed. It also includes a sample application which demonstrates the issue. I tested and verified that the problem has been addressed and fixed in Internet Explorer 10 thankfully.

This explained the many, seemingly random ways I had seen the grid break. At times the cells were squished and pushed to the left. This was because the ghost cell had been added at the end of a row. Other times the cells were shifted to the right (as seen in the screenshot above). In this case, the ghost cell had been added to the middle of a row.

The Fix

Further digging revealed the the issue was related to whitespace between table cells. The solution was fairly simple: use a regular expression to remove all whitespace between the table elements:

$('#problem-table').html(function(i, el) {
  return el.replace(/>\s*</g, '><');
});

With all of the whitespace removed from the affected <table>, IE9 rendered the page correctly.

jQuery Performance Tips: Slice, Filter, parentsUntil

I recently wrote about working with an intensive jQuery UI interface to emulate highlighting text. During this work, I experimented with and worked with jQuery optimization quite a bit. In the previous blog article, I mentioned that in some cases, the number of DOM elements that I was traversing at times exceeded 44,000, which caused significant performance issues in all browsers. Here are a few things I was reminded of, or learned throughout the project.

• console.profile, console.time, and the Chrome timeline are all tools that I used during the project to some extent. I typically used console.time the most to identify which methods were taking the most time.
• Caching elements is a valuable performance tool, as it's typically faster to run jQuery calls on a cached jQuery selector rather than reselecting the elements. Here's an example:
  

  Slower	Faster
  //Later in the code $('.items').do_something();	//On page load var cached_items = $('.items'); //Later in the code cached_items.do_something();

• The jQuery .filter operator came in handy, and gave a bit of a performance bump in some cases.
  

  Slower	Faster
  $('.highlighted');	cached_items.filter('.highlighted');

• jQuery slicing from a cached selection was typically much faster than reselecting or selecting those elements by class. If retrieving slice boundaries is inexpensive and there are a lot of elements, slice was extremely valuable.
  

  Slower	Faster
  cached_items.filter('.highlighted');	cached_items.slice(10, 100);

• Storing data on elements was typically a faster alternative than parsing the id from the HTML markup (class or id). If it's inexpensive to add data values to the HTML markup, this was a valuable performance tool. Note that it's important to test if the jQuery version in use automatically parses the data value to an integer.
  

  Slower	Faster
  //given var slice_start = parseInt($('tt#r123') .attr('id') .replace(/^r/, ''));	//given var slice_start = $('tt#r123') .data('id');

• Advanced jQuery selectors offered performance gain as opposed to jQuery iterators. In the example below, it's faster to use selectors :has and :not combined rather than iterating through each parent.
  

  Slower	Faster
  $.each($('p.parent'), function(i, el) { //if el does not have any visible spans // do_something() });	$('p.parent:not(:has(span:visible))') .do_something();

• The jQuery method parentsUntil was a valuable tool instead of looking at the entire document or a large set of elements. In the cases where the children were already defined in a subset selection, I used the parentsUntil method to select all parents until a specific DOM element.
  

  Slower	Faster
  $('#some_div p.parent:not(:has(span:visible))') .do_something();	subset_of_items .parentsUntil('#some_div') .filter(':not(:has(span:visible))') .do_something();

The best takeaway I can offer here is that it was almost always more efficient to work with as precise set of selected elements as possible, rather than reselecting from the whole document. The various methods such as filter, slice, and parentsUntil helped define the precise set of elements.

Verify Addresses the Easy Way with SmartyStreets

Adding an address form is a pretty common activity in web apps and even more so with ecommerce web apps. Validations on forms allow us to guide the user to filling out all required fields and to make sure the fields conform to basic formats. Up until now going further with addresses to verify they actually exist in the real world was a difficult enough task that most developers wouldn't bother with it. Imagine though the cost to the merchant who ships something to the wrong state because the customer accidently selected "SD" (South Dakota) when they thought they were selecting "SC" (South Carolina), a simple enough mistake to make and one that wouldn't be caught by most address forms. In today's ecommerce world customers expect deliveries to be fast and reliable, and in this case the customer would have to wait until the package is returned to the merchant with "Address Unknown" only to have to wait even longer for the reshipment. Even worse for the merchant, maybe the package never gets returned.

SmartyStreets is a new API web app that I implemented for our client Mobixa, a web app that allows people to sell their used mobile phones. Mobixa sends shipping labels and payment checks to customers so that they can send their phones to Mobixa and get paid for it. Wrong addresses can delay the whole process and Mobixa wanted to reduce the number of bad addresses that were being keyed in by the customers. SmartyStreets provides an easy way for developers to allow Address verification to their web forms so that addresses are verified against the US Postal Service's valid addresses. SmartyStreets is CASS certified meaning that they meet the USPS accuracy requirements.

The big advantage of Smarty Streets is that adding address verification to a form can be as easy as adding a link to their jQuery based plugin and then a script tag with your SmartyStreets API key. The plugin autodetects address fields and when a minimum of 3 fields are entered (address, city, state), it will display an indicator sprite to the user and send an async request to the API for verification. The verification has three possible outcomes:

1. The address is verified. A valid address will display a verified image to the side of the zip code and all of the address fields will be modified with a more "correct" address, with correct being defined as what matches the USPS official definition for the matched address. Zip codes will be modified with the carrier route code, so "92806" becomes "92806-3433". An address modification would for example change "1735 pheasant" to "1731 N Pheasant St.". Proper casing and spelling errors will also be enacted.

2. The address is ambiguous. An ambiguous address is one that returns multiple matches. Let's say for example that "1 Rosedale St.", could be "1 N Rosedale Street" or "1 S Rosedale Street". In this case it displays a popup which allows the user to select the correct address or to override the suggestions and continue with the address they entered.

3. The address is invalid. An invalid address informs the user that it's invalid and both the invalid address and the ambiguous address offer the user two additional choices. "Double checking the address" will rerun the address validation after the user has modified the address. "I certify that what I type is correct", is a second choice which allows the user to continue with the address they typed. This last choice is important because it allows the user the power to continue with what they want instead of forcing them to conform to the address validation.

Checks are performed when SmartyStreets senses that it has enough address information to run a search. During this time the submit button to the form is disabled until the check is completed. Once a check is performed once, it will not perform again unless the user elects to "Double check" the address, this is a good design choice to prevent the user from getting stuck in an infinite loop of sorts.

Our implementation of SmartyStreets into Mobixa included customizing it to do things a little bit differently than the out of the box defaults. The jQuery plugin comes with many events and hooks for adding customization and if you want to go your own way you can implement everything on the frontend save for the API call yourself. Documentation on the website is useful, and the developers of SmartyStreets conveniently answered my questions via an Olark chat window.

The costs of SmartyStreets is that you have to spend time to implement it in your app, a monthly fee based on number of API calls, and also that your UI flow will change slightly in that the user will need to wait for the API call to complete before submitting the form. I don't always implement validation when I have an address form in an app, but when I do, I like to use SmartyStreets.

Lazy AJAX

Don't do this, at least not without a good reason. It's not the way to design AJAX interfaces from scratch, but it serves well in a pinch, where you have an existing CGI-based page and you don't want to spend a lot of time rewriting it.

I was in a hurry, and the page involved was a seldom-used administration page. I was attempting to convert it into an AJAX-enabled setup, wherein the page would stand still, but various parts of it could be updated with form controls, each of which would fire off an AJAX request, and use the data returned to update the page.

However, one part of it just wasn't amenable to this approach, or at least not quick-and-dirty. This part had a relatively large amount of inline interpolated (Interchange) data (if you don't know what Interchange is, you can substitute "PHP" in that last sentence and you'll be close enough.) I wanted to run the page back through the server-side processing, but only cared about (and would discard all but) one element of the page.

My lazy-programmer's approach was to submit the page itself as an AJAX request:

$.ajax({
    url: '/@_MV_PAGE_@',
    data: {
        'order_date': order_date,
        'shipmode' : shipmode
    },
    method: 'GET',
    async: true,
    success: function(data, status){
        $('table#attraction_booklet_order').replaceWith(
            $(data).find('#attraction_booklet_order').get(0)
        );
        $('table#attraction_booklet_order').show();
    }
}); 

In this excerpt, "MV_PAGE" is a server-side macro that evaluates to the current page's path. The element I care about is a rather complex HTML table containing all sorts of interpolated data. So I'm basically reloading the page, or at least that restricted piece of it. The tricky bit, unfamiliar to jQuery newcomers, lets you parse out something from the returned document much as you would from your current document.

Again, don't do this without a reason. When I have more time, I'll revisit this and improve it, but for now it's good enough for the current needs.

Interactive Piggybak Demo Tour

A new interactive tour of Piggybak and the Piggybak demo has been released at piggybak.org. Piggybak is an open source Ruby on Rails ecommerce framework built as a Rails 3 engine and intended to be mounted on existing Rails applications.

The tour leverages jTour (a jQuery plugin) and guides you through the homepage, navigation page, product page, cart and checkout pages, gift certificate page, advanced product option page, and WYSIWYG driven page.The tour also highlights several of the Piggybak plugins available and installed into the demo such as plugins that introduce advanced product navigation, advanced product optioning, and gift certificate functionality. Below are a few screenshots from the demo.

An interesting side note of developing this tour is that while I found many nice jQuery-driven tour plugins available for free or at a small cost, this jQuery plugin was the only plugin offering decent multi-page tour functionality.

Here is the starting point of Piggybak tour.	The Piggybak tour adds an item to the cart during the tour.
The Piggybak tour highlights advanced product navigation in the demo.	The Piggybak tour highlights features and functionality on the one-page checkout.

If you'd like to check out the interactive tour, visit the Piggybak demo page here and click on the link to begin the tour! Or contact End Point right now to get started on your next Ruby on Rails ecommerce project!

Web service integration in PHP, jQuery, Perl and Interchange

Jeff Boes presented on one of his latest projects.

CityPass.com decided on a project to convert their checkout from being served by Interchange to have the interface served by PHP, but continue to interact with Interchange for the checkout process through a web service.

The original site was entirely served by Interchange, the client then took on a project to convert the frontend to PHP while leveraging Interchange for frontend logic such as pricing and shipping as well as for backend administration for order fulfillment.

Technologies used in the frontend rewrite:

• PHP
• jQuery for jStorage, back-button support and checkout business logic
• AJAX web services for prices, discounts, click-tracking

The Interchange handler is conduit.am that handles the processing of the URL. From this ActionMap the URLs are decoded and passed to a Perl module, Data.pm, which handles processing the input and returning the results.

An order is just a JSON object so testing of the web service is easy. We have a known hash, we post to the proper URL and compare the results and verify they are the same. New test cases are also easy, we can capture any order (JSON) to a log file and add it as a test case.

UTOSC 2012 talks of interest

It's been two weeks now since the Utah Open Source Conference for 2012. My fellow End Pointers wrote previously about it: Josh Ausborne about the mini Liquid Galaxy we set up there for everyone to play with, and Josh Tolley with a write-up of his talks on database constraints and KML for geographic mapping markup.

There were a lot of interesting talks planned, and I could only attend some of them. I really enjoyed these:

• Rob Taylor on AngularJS
• Brandon Johnson on Red Hat's virtualization with oVirt, Spacewalk, Katello, and Aeolus
• Clint Savage about RPM packaging with Mock & Koji
• Daniel Evans on testing web applications with Capybara, embedded WebKit, and Selenium (which End Pointer Mike Farmer wrote about here back in December)
• Aaron Toponce on breaking full-disk encryption (I missed this talk, but learned about it from Aaron in the hallway track and his slides afterwards)
• Matt Harrison's tutorial Hands-on intermediate Python, covering doctest, function parameters and introspection, closures, function and class decorators, and more.

I gave a talk on GNU Screen vs. tmux, which was fun (and ends with a live demo that predictably fell apart, and audience questions that you can't hear on the recording). Here's the video:

Follow this Direct YouTube link in case the embedded version doesn't work for you. And here are the presentation slides.

I have a bit more to cover from the conference later!

Keeping Your Apps Neat & Tidy With RequireJS

RequireJS is a very handy tool for loading files and modules in JavaScript. A short time ago I used it to add a feature to Whiskey Militia that promoted a new section of the site. By developing the feature as a RequireJS module, I was able to keep all of its JavaScript, HTML and CSS files neatly organized. Another benefit to this approach was the ability to turn the new feature "on" or "off" on the site by editing a single line of code. In this post I'll run through a similar example to demonstrate how you could use RequireJS to improve your next project.

File Structure

├── index.html
└── scripts
    ├── main.js
    ├── my
    │   ├── module.js
    │   ├── styles.css
    │   └── template.html
    ├── require-jquery.js
    ├── requirejs.mustache.js
    └── text.js

The dependencies included RequireJS bundled together with jQuery, mustache.js for templates and the RequireJS text plugin to include my HTML template file.

Configuration

RequireJS is included in the page with a script tag and the data-main attribute is used to specify additional files to load. In this case "scripts/main" tells RequireJS to load the main.js file that resides in the scripts directory. Require will load the specified files asynchronously. This is what index.html looks like:

<!DOCTYPE html>
<html>
<head>
    <title>RequireJS Example</title>
</head>
<body>
    <h1>RequireJS Example</h1>
    <!-- This is a special version of jQuery with RequireJS built-in -->
    <script data-main="scripts/main" src="scripts/require-jquery.js"></script>
</body>
</html>

I was a little skeptical of this approach working on older versions of Internet Explorer so I tested it quickly with IE6 and confirmed that it did indeed work just fine.

Creating a Module

With this in place, we can create our module. The module definition begins with an array of dependencies:

define([
  "require",
  "jquery",
  "requirejs.mustache",
  "text!my/template.html"
  ],

This module depends on require, jQuery, mustache, and our mustache template. Next is the function declaration where our module's code will live. The arguments specified allow us to map variable names to the dependencies listed earlier:

  function(require, $, mustache, html) { ... }

In this case we're mapping the $ to jQuery, mustache to requirejs.mustache and, html to our template file.

Inside the module we're using Require's .toUrl() function to grab a URL for our stylesheet. While it is possible to load CSS files asynchronously just like the other dependencies, there are some issues that arise that are specific to CSS files. For our purposes it will be safer to just add a <link> element to the document like so:

  var cssUrl = require.toUrl("./styles.css");
  $('head').append($('',
    { rel: "stylesheet", media: "all", type: "text/css", href: cssUrl }));

Next, we define a view with some data for our Mustache template and render it.

  var view = {
    products: [
      { name: "Apples", price: 1.29, unit: 'lb' },
      { name: "Oranges", price: 1.49, unit: 'lb'},
      { name: "Kiwis", price: 0.33, unit: 'each' }
    ],
    soldByPound: function(){
      return (this['unit'] === 'lb') ? true : false;
    },
    soldByEach: function() {
      return (this['unit'] === 'each') ? true : false;
    }
  }

  // render the Mustache template
  var output = mustache.render(html, view);

  // append to the HTML document
  $('body').append(output);
});

The Template

I really like this approach because it allows me to keep my HTML, CSS and JavaScript separate and also lets me write my templates in HTML instead of long, messy JavaScript strings. This is what our template looks like:

<ul class="hot-products">
  {{#products}}
  <li class="product">
    {{name}}: ${{price}} {{#soldByEach}}each{{/soldByEach}}{{#soldByPound}}per lb{{/soldByPound}}
  </li>
  {{/products}}
</ul>

Including the Module

To include our new module in the page, we simply add it to our main.js file:

require(["jquery", "my/module"], function($, module) {
    // jQuery and my/module have been loaded.
    $(function() {

    });
});

When we view our page, we see that the template was was rendered and appended to the document:

Optimizing Your Code With The r.js Optimizer

One disadvantage of keeping everything separate and using modules in this way is that it adds to the number of HTTP requests on the page. We can combat this by using the the RequireJS Optimizer. The r.js script can be used a part of a build process and runs on both node.js and Rhino. The Optimizer script can minify some or all of your dependencies with UglifyJS or Google's Closure Compiler and will concatenate everything into a single JavaScript file to improve performance. By following the documentation I was able to create a simple build script for my project and build the project with the following command:

node ../../r.js -o app.build.js

This executes the app.build.js script with Node. We can compare the development and built versions of the project with the Network tab in Chrome's excellent Developer Tools.

Development Version:

Optimized with the RequireJS r.js optmizer:

It's great to be able to go from 8 HTTP requests and 360 KB in development mode to 4 HTTP requests and ~118 KB after by running a simple command with Node! I hope this post has been helpful and that you'll check out RequireJS on your next project.

RailsConf 2012: Day One

Today kicks off the official first day of RailsConf 2012, in Austin, Texas (yesterday RailsConf unofficially kicked off with Ignite Rails). This is my fourth RailsConf, and I've found that I get increasingly more out of each one as my time working with Rails has accumulated.

The morning started with a keynote by DHH. DHH talked about progress and keeping an interest in something because it continues to make progress. Initially, the Rails community had a negative reaction to progress like Ruby 1.9, Rails 3, the Asset Pipeline and CoffeeScript, but after the initial learning curve, the new tools and progress was appreciated by the community. DHH talked about how there have been many studies on what prevents people from progressing, and one of those causes is loss aversion, meaning that people hate losing or failing enough that it prevents them from advancing. To me, the point of his talk was to prepare everyone for Rails 4 and how it will challenge and break things. Rails 4 is admittedly not coming out anytime soon, so he didn't touch on any specifics.

Using Backbone.js with Rails

One session talk I attended was Using Backbone.js with Rails by Sarah Mei of Pivotal Labs. Backbone is not an MVC framework in JavaScript, but it is lumped into traditional JavaScript MVC frameworks. It serves as a lightweight infrastructure for organizing code like templates, interactive elements, and events. Because Backbone is a relatively immature framework, like other JavaScript frameworks, there isn't a standard way of writing code, so the implementation of Backbone varies widely.

Sarah talked about the love story between JS and Rails:

She touched on the basics of Backbone from a Rails perspective:

• Models: Backbone and Rails models are very similar. An application will likely have Backbone models that mirror Rails models.
• Templates: Backbone templates are akin to Rails views or a mustache template.
• Views: Views are similar to Rails controllers. They set a template, class name, and define events.

Next, Sarah talked about two methods of implementation using Backbone.js and went through some code examples:

• Greenfield App
  • This approach tends to be more API driven and makes sense for an API-driven application that also requires buildout of native mobile apps.
  • In this approach, the server side returns [JSON, etc.] data and doesn't render HTML.
  • Sarah noted that the amount of code required here can be substantial because the framework is lightweight, so she would recommended considering alternatives (such as ember.js).
• "Backbone as Frosting" Implementation
  • In this implementation, there is a mixture of server-side and client-side rendering.
  • Sarah noted here that the lightweightedness of Backbone works nicely.

I haven't used Backbone.js in any client projects, but I've looked into it recently out of interest. I like the general idea of improving JavaScript code organization via the frosting approach described by Sarah. A few code examples were shared during the presentation — check out the slides here.

CoffeeScript for the Rubyist

Another talk I attended today was CoffeeScript for the Rubyist by Mark Bates. CoffeeScript is a JS meta language that compiles into JS and can easily integrate with your current JavaScript. It's easy to read, write, and maintain. I took the CoffeeScript lesson over at Code School recently (highly recommended!). Mark went through a brief history of JavaScript and explained that like the evolution to early assembly languages to C to Ruby (with a few missing steps), CoffeeScript's aim is to be an improved tool for writing readable and maintainable code. The syntax of CoffeeScript is a hybrid of Ruby and Python which is simple, uses no semicolons, typically no curly braces, no "function" keyword, relaxed parenthesis, and significant whitespace.

Mark then went into a review of Ruby-like CoffeeScript behavior. Below are some of the important topics in CoffeeScript that he covered, as well as links to the documentation:

• conditionals similarity (inline, postfix)
• objects and hashes are similar (can't screw up last comma)
• ranges
• string interpolation
• heredocs, (""" works)
• functions Ruby 1.9 Lamda syntax is similar to CoffeeScript version
• default arguments
• splats (e.g. first, second, others...)
• loops and comprehensions (for number in numbers)
• combination of loops and comprehensions and conditionals
• classes
• inheritence (e.g. class Manager extends Employee)
• Bound functions (with a fat arrow)
• Existential operator (e.g. if foo?, if console?.log "foo")

In addition to the talks on Backbone.js and CoffeeScript, I'll post the links to slides and video for the other talks I attended when they become available.

What's Next?

It's funny (but not surprising) how many of the talks I attended on Day One weren't actually about Rails or the specifics of Rails. Perhaps I'll try to pick a few more Rails-centric talks in the next couple of days of the conference and report on those. Stay tuned!

Rails 3 remote delete link handlers with Unobtrusive Javascript

I recently encountered a bug in a Rails 3 application that used a remote link_to tag to create a Facebook-style "delete comment" link using unobtrusive javascript. I had never worked with remote delete links like this before, so I figured I’d run through how I debugged the issue.

Here are the relevant parts of the models we're dealing with:

class StoredFile < ActiveRecord::Base
  has_many :comments, :dependent => :destroy
end
class Comment < ActiveRecord::Base
  belongs_to :user
  belongs_to :stored_file
end

Here's the partial that renders a single Comment (from the show.html.erb view for a StoredFile) along with a delete link if the current_user owns that single Comment:

<%= comment.content %> -<%= comment.user.first_name %>
<% if comment.user == current_user >
  <%= link_to 'X', stored_file_comment_path(@stored_file, comment), :remote => true, :method => :delete, :class => 'delete-comment' >
<% end ->

Here’s a mockup of the view with 3 comments:

At first, the bug seemed to be that the "X" wasn’t actually a link, and therefore, didn't do anything. Clicking the "X" with Firebug enabled told a different story. There was a link there (hidden by sneaky CSS,) and the Firebug console showed that it was sending the appropriate request to the correct url: /stored_files/78/comments/25

The development.log file on the server corraborated the story and showed a successful delete:

Started DELETE "/stored_files/78/comments/25"
  Processing by CommentsController#destroy as JS
  Parameters: {"stored_file_id"=>"78", "id"=>"25"}
  SQL (0.6ms)  DELETE FROM "comments" WHERE "comments"."id" = 25
  Completed 200 OK in 156ms

So far, so good. I know that our client code is making the correct request and the Rails app is handling it appropriately. I knew that the existing code "worked," but still didn’t provide UI feedback to inform the user. I needed to write some jQuery to handle the successful (HTTP 200) server response to our unobtrusive javascript call.

Normally, when writing my own handler (e.g. to handle a button click) to initiate an Ajax call with jQuery, I’d use $.ajax or $.post and use its built-in success handler. Something like this:

$.ajax({
    type: 'POST',
    url: 'my_url',
    data: { param_name1: 'param_value1'},
    success: function(data){ 
        alert('Successfully did the needful!');
    }
});

It turns out that you still define a event handler when handling server responses to unobtrusive javascript, it’s just that the syntax is very different. I needed to bind the ‘ajax:success’ event when it’s fired by any of my comment delete links (where class=”delete-comment”, as specified in my link_to call).

$(document).on('ajax:success', '.delete-comment', function() {
    // .parent() is the div containing this "X" delete link
    $(this).parent().slideUp();
    }
);

(Note that I happen to be using the newer jQuery 1.7+ .on() method and argument list instead of the older, more common .live() style. In this case, they are functionally equivalent, but the methods that .on() replaces are deprecated in jQuery 1.7.)

Now, when the user clicks the "X" to delete one of their own comments, the successful unobtrusive javascript call is detected and the div containing that single comment is neatly hidden with the help of jQuery's slideUp() method. 

Much better!

Using Disqus and Ruby on Rails

Recently, I posted about how to import comments from a Ruby on Rails app to Disqus. This is a follow up to that post where I outline the implementation of Disqus in a Ruby on Rails site. Disqus provides what it calls Universal Code which can be added to any site. This universal code is just JavaScript, which asynchronously loads the Disqus thread based on one of two unique identifiers Disqus uses.

Disqus in a development environment

Before we get started, I'd recommend that you have two Disqus "sites"; one for development and one for production. This will allow you to see real content and experiment with how things will really behave once you're in production. Ideally, your development server would be publicly accessible to allow you to fully use the Disqus moderation interface, but it isn't required. Simply register another Disqus site, and make sure that you have your shortname configured by environment. Feel free to use whatever method you prefer for defining these kinds of application preferences. If you're looking for an easy way, considering checking out my article on Working with Constants in Ruby. It might look something like this:

# app/models/article.rb

DISQUS_SHORTNAME = Rails.env == "development" ? "dev_shortname".freeze : "production_shortname".freeze

Disqus Identifiers

Each time you load the universal code, you need to specify a few configuration variables so that the correct thread is loaded:

• disqus_shortname: tells Disqus which website account (called a forum on Disqus) this system belongs to.
• disqus_identifier: tells Disqus how to uniquely identify the current page.
• disqus_url: tells Disqus the location of the page for permalinking purposes.

# app/views/disqus/_thread.html.erb
# assumes you've passed in the local variable 'article' into this partial
# from http://docs.disqus.com/developers/universal/

<div id="disqus_thread"></div>
<script type="text/javascript">

    var disqus_shortname = '<%= Article::DISQUS_SHORTNAME %>';
    var disqus_identifier = '<%= article.id %>';
    var disqus_url = '<%= url_for(article, :only_path => false) %>';

    /* * * DON'T EDIT BELOW THIS LINE * * */
    (function() {
        var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
        dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
        (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
    })();
</script>

The above code will populate the div#disqus_thread with the correct content based on your disqus_identifier. By setting up a single partial that will always render your threads, it becomes very easy to adjust this code if needed.

Disqus Identifier Gotcha

We found during our testing a surprising and unexpected behavior in how Disqus associates a thread to a URL. In our application, the landing page was designed to show the newest article as well as the Disqus comments thread. We found that once a new article was posted, the comments from the previous article were still shown! It seems Disqus ignored the unique disqus_identifier we had specified and instead associated the thread with the landing page URL. In our case, a simple routing change allowed us to forward the user to the unique URL for that content and thread. In your case, there may not be such an easy work around, so be certain you include both the disqus_identifier and disqus_url JavaScript configuration variables above to minimize the assumptions Disqus will make. When at all possible, always use unique URLs for displaying Disqus comments.

Comment Counters

Often an index page will want to display a count of how many comments are in a particular thread. Disqus uses the same asynchronous approach to loading comment counts. Comment counts are shown by adding code such as the following where you want to display your count:

# HTML
<a href="http://example.com/article1.html#disqus_thread" 
   data-disqus-identifier="<%=@article.id%>">
This will be replaced by the comment count
</a>

# Rails helper
<%= link_to "This will be replaced by the comment count", 
    article_path(@article, :anchor => "disqus_thread"), 
    :"data-disqus-identifer" => @article.id %>

At first this seemed strange, but it is the exact same pattern used to display the thread. It would likely be best to remove the link text so nothing is shown until the comment count is loaded, but I felt for my example, having some meaning to the test would help understanding. Additionally, you'll need to add the following JavaScript to your page.

# app/view/disqus/_comment_count_javascript.html.erb
# from http://docs.disqus.com/developers/universal/
# add once per page, just above </body>

<script type="text/javascript">
   
    var disqus_shortname = '<%= Article::DISQUS_SHORTNAME %>';

    /* * * DON'T EDIT BELOW THIS LINE * * */
    (function () {
        var s = document.createElement('script'); s.async = true;
        s.type = 'text/javascript';
        s.src = 'http://' + disqus_shortname + '.disqus.com/count.js';
        (document.getElementsByTagName('HEAD')[0] || document.getElementsByTagName('BODY')[0]).appendChild(s);
    }());
</script>

Disqus recommends adding it just before the closing </body> tag. You only need to add this code ONCE per page, even if you're planning on showing multiple comment counts on a page. You will need this code on any page with a comment count, so I do recommend putting it in a partial. If you wanted, you could even include it in a layout.

Styling Comment Counts

Disqus provides extensive CSS documentation for its threads, but NONE for its comment counters. In our application, we had some very particular style requirements for these comment counts. I found that in Settings > Appearance, I could add HTML tags around the output of the comments.

This allowed me to style my comments as needed, although these fields are pretty small, so make sure to compress your HTML as much as possible.

Labeling input boxes including passwords

I'm currently working on a new site and one of the design aspects of the site is many of the form fields do not have labels near the input boxes, they utilize labels that are inside the input box and fade away when text is entered. The label is also supposed to reappear if the box is cleared out. Originally I thought this was a pretty easy problem and wrote out some jQuery to do this quickly. The path I went down first was to set the textbox to the value we wanted displayed and then clear it on focus. This worked fine, however I reached a stumbling block when it came to password input boxes. My solution did not work properly because text in a password box is hidden and the label would be hidden as well. Most people would probably understand what went in each box, but I didn't want to risk confusing anyone, so I needed to find a better solution

I did some searching for jQuery and labels for password inputs and turned up several solutions. The first one actually put another text box on top of the password input, but that seemed prone to issue. The solution I decided to ultimately use is called In-Fields Labels, a jQuery plugin by Doug Neiner. In this solution Doug has floating labels that appear over the top of the textbox, and they dim slightly when focus is gained and then disappear completely when typing begins. The plugin does not mess with the value in the input box at all.

It was fairly easy to get up and running. I added the plugin to the page, created some styling for the labels, added label tags with the class of 'overlay' for each input box and called $('label.overlay').inFieldLabels();. This was all that was needed to get us going.

Normal view

Focus in the password box

Typing in the password box

The effect is pretty cool and it provides a good interface for the user as they are reminded up until the time they type in the box what they are supposed to enter.

Nifty In-Button Confirmation

I've been working on a personal email client after work, called Warm Sunrise, that forces myself to keep a manageable inbox. One of the goals of the project was to get to a zero-inbox everyday, so I needed a 'Delete All' button that was easy-to-use without running the risk of accidentally deleting emails. I took a look at JavaScript's confirm, which is jarring, and jQuery's dblClick, which doesn't provide any feedback to the user after the first click, leaving the user to wonder why their emails weren't deleted.

Given these options, I built my own button using Rails 3.1, jQuery, and CoffeeScript, that better fit the goals I set out with. It requires a double click, but gives the user a confirmation in the button itself, without any sort of timeout. You can see a video of it in action here:

Starting with app/views/letters/index.html.erb, I generated the buttons using Rails helpers and Twitter's Bootstrap classes:

<%= link_to 'Write letter', new_letter_path, :class => "btn primary pull-right far-right" %>
<%= link_to 'Delete all', '#', :class => "btn pull-right no_danger", :id => "delete_all" %>
<%= link_to 'Are you sure?', delete_all_letters_path, :method => :destroy, :class =>"btn pull-right danger confirm", :id => "delete_all", :style => "display:none;" %>

Notice that the 'Delete all' button doesn't actually specify a url and the 'Are you sure?' link's style is set to "display:none"

Here's the relationship I set up in my models:

app/models/letter.rb

belongs to :user

app/models/user.rb

has_many :letters, :dependent => :destroy

I set up config/routes.rb to work with the explicit path I set in:

post 'delete_all_letters' => 'letters#delete_all'

Finally, I finished this lot by adding the delete_all action to my app/controllers/letters_controller.rb:

def delete_all 
    current_user.letters.delete_all

    respond_to do |format|
        format.html { redirect_to letters_url, notice: 'Successfully deleted all letters.' }
        format.json { head :ok }
    end 
end 

CoffeeScript is a beautiful language that compiles to JavaScript, which I prefer to JavaScript itself. You can read more about it here. Let's take a look at the CoffeeScript that makes this button work:

$('a#delete_all.no_danger').hover( ->
    $(this).addClass('danger')
    $(this).click( ->
        $('a#delete_all.no_danger').hide()
        $('a#delete_all.confirm').show()
    )   
)
$('a#delete_all.no_danger').mouseleave( ->
    $(this).removeClass('danger')
)
$('a#delete_all.danger').mouseleave( ->
    $(this).hide()
    $('a#delete_all.no_danger').show()
)

Since the button's text changes to a confirmation on the first click, makes it better for my purposes than Javascript's dblClick method. Check the video to see what it looks like in action.

Let's take a look at what this compiles to in plain JavaScript, too, since this is the only thing the browser sees:

$('a#delete_all.no_danger').hover(function() {
    $(this).addClass('danger');
    return $(this).click(function() {
        $('a#delete_all.no_danger').hide();
        return $('a#delete_all.confirm').show();
    });
});
$('a#delete_all.no_danger').mouseleave(function() {
    return $(this).removeClass('danger');
});
$('a#delete_all.danger').mouseleave(function() {
    $(this).hide();
    return $('a#delete_all.no_danger').show();
});

Not shown in the video, but I modified index.html.erb to only show the 'Delete all' button when the user has a zero-inbox.

<%= link_to 'Write letter', new_letter_path, :class => "btn primary pull-right far-right" %>
<% if !@letters.empty? %>
    <%= link_to 'Delete all', '#', :class => "btn pull-right no_danger", :id => "delete_all" %>
    <%= link_to 'Are you sure?', delete_all_letters_path, :method => :destroy, :class =>"btn pull-right danger confirm", :id => "delete_all", :style => "display:none;" %>
<% end %>

Preventing Global Variables in JavaScript

JavaScript's biggest problem is its dependence on global variables
--Douglas Crockford, JavaScript: The Good Parts

Recently I built out support for affiliate management into LocateExpress.com’s Sinatra app using JavaScript and YUI.

I used a working page from the admin, Service Providers, as a starting point to get something up and running for affiliates quickly. By the time I finished, the Affiliates page worked great, but forms on the Service Provider page no longer populated with data.

Identifying a misbehaving global variable

There were no errors in the console, and the forms on the Service Providers page remained broken even after restoring an old copy of service_providers.js. As it turns out, a global variable, edit_map, was defined within service_providers.js, and again in the copied affiliates.js. Credit for spotting the problem goes to Brian Miller.

The fix was as simple as moving edit_map's declaration into the file's YUI sandbox, so that version of edit_map wouldn't be visible to any other pages in the admin.

Preventing global variables

As projects grow and complexity increases, it becomes easier and easier to overlook global variables and thus run into this tough-to-debug problem. Douglas Crockford’s Javascript: The Good Parts covers several workarounds to using global variables.

Rather than declaring variables globally, like this:

var edit_map = { 'business[name]' : 'business_name' };

the author recommends declaring them at the beginning of functions whenever possible:

YUI().use("node", "io", "json",
function(Y) {
    var edit_map = { 'business[name]' : 'business_name' };
    ...
});

In all other cases, he suggests using Global Abatement, which prevents your global variables from affecting other libraries. For example,

var LocateExpress = {};
LocateExpress.edit_map = { 'business[name]' : 'business_name' };

YUI().use("node", "io", "json",
function(Y) {
    ...
    return LocateExpress.edit_map;
});

I highly recommend JavaScript: The Good Parts to learn about the best JavaScript has to offer and workarounds for its ugly side. The author also wrote a very popular code-checker, JSLint, which could help debug this nasty problem by highlighting implicit global variables.

Rails 3.1: Upgrading a Simple App - Part 1

Here at End Point, I've worked with a few Rails 3 applications in production and a couple of Rails 3.1 apps in development, so I've become familiar with the new features and functionality including the Rails 3.1 Asset Pipeline that I mentioned earlier this year. I thought it was a good time to upgrade our website to Rails 3.1 and share the experience.

To start, here's a quick summary of our website:

• Simple Rails application running on Rails 2.1.2 with no database
• Static pages throughout the site, fully cached
• Rake tasks to generate partials throughout the site to display dynamic blog content
• Site uses a moderate amount of jQuery and jQuery plugins.
• Site is optimized in terms of asset serving (ETags, Expires headers, CSS sprites, etc.)

While I've worked with a few Rails 3 apps, I haven't been involved in the actual upgrade process myself. There are plenty of resources out there with upgrade advice, including a few RailsCasts (one, two, and three). My favorite resource was the rails_upgrade gem, a gem that is now officially supported by Rails to help with the upgrade process. I followed the instructions to install the gem (script/plugin install git://github.com/rails/rails_upgrade.git) to install it as a plugin in our site's application in a fresh git branch (on a camp, of course!).

The rails_upgrade provides a few new rake tasks for checking compatibility, upgrading the routes, creating a Gemfile, and upgrading configuration. For me, the most valuable task was the rake rails:upgrade:check task. Here's what the output looked like for this app:

Deprecated session secret setting
Previously, session secret was set directly on ActionController::Base; it's now config.secret_token.
More information: http://lindsaar.net/2010/4/7/rails_3_session_secret_and_session_store

The culprits: 
 - config/initializers/session_store.rb

Old router API
The router API has totally changed.
More information: http://yehudakatz.com/2009/12/26/the-rails-3-router-rack-it-up/

The culprits: 
 - config/routes.rb

New file needed: config/application.rb
You need to add a config/application.rb.
More information: http://omgbloglol.com/post/353978923/the-path-to-rails-3-approaching-the-upgrade

The culprits: 
 - config/application.rb

Deprecated constant(s)
Constants like RAILS_ENV, RAILS_ROOT, and RAILS_DEFAULT_LOGGER are now deprecated.
More information: http://litanyagainstfear.com/blog/2010/02/03/the-rails-module/

The culprits: 
 - app/views/layouts/application.rhtml
 - ...

Soon-to-be-deprecated ActiveRecord calls
Methods such as find(:all), find(:first), finds with conditions, and the :joins option will soon be deprecated.
More information: http://m.onkey.org/2010/1/22/active-record-query-interface

The culprits: 
 - app/views/blog_archive/_ruby_on_rails.html.erb
        - ...

Deprecated AJAX helper calls
AJAX javascript helpers have been switched to be unobtrusive and use :remote => true instead of having a seperate function to handle remote requests.
More information: http://www.themodestrubyist.com/2010/02/24/rails-3-ujs-and-csrf-meta-tags/

The culprits: 
 - app/views/blog_archive/_company.html.erb
        - ...

Deprecated ActionMailer API
You're using the old ActionMailer API to send e-mails in a controller, model, or observer.
More information: http://lindsaar.net/2010/1/26/new-actionmailer-api-in-rails-3

The culprits: 
 - app/controllers/contact_controller.rb

Old ActionMailer class API
You're using the old API in a mailer class.
More information: http://lindsaar.net/2010/1/26/new-actionmailer-api-in-rails-3

The culprits: 
 - app/models/contact_form.rb

As you can see, the upgrade check spits out a list of necessary and recommended upgrades and the corresponding *culprits*. It's also nice that the task provides documentation in the form of a link for each message. Studying the source of the plugin, I found additional examples of upgrade messages: named_scope updates, validate_on_* syntax, test_help path updates, gem bundling configuration, Rails generator API syntax updates, messaging on known broken plugins (e.g. searchlogic, cucumber, nifty-generators), and depracation on ERb helper and AJAX calls.

I went through and applied my updates, according to the checklist. Notable updates were:

Before

ActionController::Routing::Routes.draw do |map|
  map.root :controller => 'home', :action => 'index'
  map.connect 'contact/submit', :controller => 'contact', :action => 'submit'
  map.connect ':controller/:id'
  map.connect '*path', :controller => 'redirect' 
end

After

Endpoint::Application.routes.draw do
  root :to => 'home#index'
  match 'contact/submit' => 'contact#submit'
  match ':controller(/:id)', :action => :index
  match '*path' => 'redirect#index'
end

source 'http://rubygems.org'

gem 'rails', '3.1.0'
gem 'json'

# Gems used only for assets and not required
# in production environments by default.
group :assets do
  gem 'sass-rails', "  ~> 3.1.0"
  gem 'coffee-rails', "~> 3.1.0"
  gem 'uglifier'
end

gem 'jquery-rails'
gem 'fastercsv'
gem 'execjs'
gem 'therubyracer'
gem 'rake', '0.8.7'

Something that didn't come up in the rails upgrade check that is required to have a working app is renaming all rhtml files to html.erb, briefly described here.

To get the basic app working, I moved the public/stylesheets and public/javascripts to the new app/assets directories to start. I did not move the images out of the public/ directory because several of the images in the application are referenced by blog articles.

I followed the directions here combined with a bit of troubleshooting to configure a Rails 3.1 app that does not require a database.

The upgrade was a relatively painless process, although it still took a few hours for even the most basic application with only a handful of controllers, routes, and one mailer. My experience suggests that with a more complex application, the upgrade will take at least a few hours, if not much more. This simple app doesn't do much with remote forms and links, so I didn't spend any time upgrading the app to work with the jquery-ujs gem. Also, I obviously didn't mess around with Rails 3.1 ActiveRecord issues since the application is database-less. Both of these items may add significant overhead to the upgrade process.

I spent a significant amount of time working with the new asset pipeline and restructuring the assets, which I plan to describe in Part 2 of the upgrade. Stay tuned!

CSS Fixed, Static Position Toggle

In a recent Rails project, I had to implement a simple but nifty CSS trick. A request came in to give a DOM element fixed positioning, meaning as the user navigates throughout the page, the DOM element stays in one place while the rest of the page updates. This is pretty common behavior for menu bars that show up along one of the borders of a window while a user navigates throughout the site. However, this situation was a bit trickier because the menu bar that needed fixed positioning was already a few hundred pixels down the page below header and navigation content:

I came up with a nifty way of using jQuery to toggle the menu bar CSS between fixed and static positioning. The code uses jQuery's scroll event handler to adjust the CSS position setting of the menu bar as users scroll through the page. If the window scroll position is below it's original top offset, the menu has fixed positioning at the top of the window. If the window scroll position is above it's original top offset, the menu has static positioning. Here's what the code looks like:

var head_offset = jQuery('#fixed_header').offset();
jQuery(window).scroll(function() {
    if(jQuery(window).scrollTop() < head_offset.top) {
        jQuery('#fixed_header').css({ position: "static"}); 
    } else {
        jQuery('#fixed_header').css({ position: "fixed", top: "0px" }); 
    }   
}); 

And perhaps the most effective demonstration of this behavior comes in the form of a video, created with Screencast-O-Matic. I also tried capturing with Jing, which is another handy tool for quick Screenshots and Screencasts. Note that the header content has CSS adjustments for demo purposes only:

jQuery and hidden elements

Out of sight, not out of mind.

While extending some jQuery functionality on a page, I noted that a form element's "change" handler wasn't being invoked, which meant some of the page initialization code would be left out. So I helpfully added it:

$('input[name=foobar]').change(...).change();

What I failed to contemplate was how this would impact another page, which just happened to have some (but not all) of the same form elements, and referenced the same JS code at page load. Specifically, my page's sibling had:

<input name="foobar" type="hidden">

And of course this broke in production.

Well, that's interesting. A change handler on a hidden form input field isn't usually all that useful, so I figured out I really needed:

$('input[name=foobar]').filter(':visible').change(...).change();

It happens that the ".filter()" step is actually more efficient than doing it all in one selector ("input[name=foobar]:visible"), because of some obscurities within jQuery. That little discovery was of value to an End Point co-worker, who realized she could shave a little time off a page load elsewhere, so my minor page malfunction will redeem itself.

Rails Optimization: Digging Deeper

I recently wrote about raw caching performance in Rails and advanced Rails performance techniques. In the latter article, I explained how to use a Rails low-level cache to store lists of things during the index or list request. This technique works well for list pages, but it doesn't necessarily apply to requests to an individual thing, or what is commonly referred to as the "show" action in Rails applications.

In my application, the "show" action loaded at ~200 ms/request with low concurrency, with the use of Rails fragment caching. And with high concurrency, the requests shot up to around 2000 ms/request. This wasn't cutting it! So, I pursued implementing full-page caching with a follow-up AJAX request, outlined by this diagram:

First, the fully-cached is loaded (quickly). Next, an AJAX request is made to retrieve access information. The access information returns a JSON object with information on whether or not there is a user, and if that user has edit access to that thing. If there is no user, the page stays as is. If there is a user, but he does not have edit permissions, the log out button is shown and the username is populated. If there is a user and he has edit permissions, the log out button is shown, the username is populated, and additional buttons requiring edit access are shown.

The Code

To cache the full page, I use the caches_page method, and cache only on requests of HTML format (other formats are not cached):

class ThingsController < ApplicationController
 caches_page :show, :if => Proc.new { |c| c.request.format.html? }
 ...
end

My access level request looks something like this:

def accessibility
  respond_to do |format|
    format.json do
      render :json => {
        :logged_in => current_user ? current_user.to_json(:only => [:id, :username]) : false,
        :can_edit => current_user ? Thing.find(params[:id]).can_edit?(current_user) : false }
    end
  end 
end 

My HTML has some bits of code sprinkled throughout it:

...
<a href="#" id="edit_thing" class="requires_editability">Edit</a>
...
<a href="#" id="my_account" class="requires_logged_in"><!-- no username yet --></a>
...

My jQuery AJAX request looks something like the code shown below. Note that I remove elements that do not apply to the current request:

$.ajax({
  type: 'GET',
  cache: false,
  url: editability_path,  //editability_path is defined in the HTML (a JavaScript variable)
  dataType: "JSON",
  error: function(xhr){
    $('.require_editability,.require_loggged_in').remove();
  },
  success: function(results) {
    if(results.logged_in) {
      $('.require_logged_in').show();
      $('#my_account').html(results.logged_in.username);
      if(results.can_edit) {
        $('.require_editability').show();
      } else {
        $('.require_editability').remove();
      }
    } else {
      $('.require_editability,.require_loggged_in').remove();
    }
  }
});

And don't forget the sweeper to clear the fully cached page after edits (or other ActiveRecord callbacks):

class ThingSweeper < ActionController::Caching::Sweeper

  observe Thing

  def after_save(record)
    expire_page :controller => :things, :action => :show, :id => record.id
  end
end

Additional Notes

There are some additional notes to mention:

• If a user were to hack the AJAX or JavaScript, server-side validation is still being performed when an "edit" action is submitted. In other words, if a hacker somehow enabled an edit button to show up and post an edit, a server-side response would prohibit the update because the hacker does not have appropriate accessibility.
• HTML changes were made to accommodate this caching behavior, which was a bit tricky. HTML has to handle all potential use cases (no user, user & no edit access, user & edit access). jQuery itself can also be used to introduce new elements per use case.
• The access level AJAX request is also hitting more low-level Rails caches: For example, the array of things that a user has edit permissions is cached and the cache is cleared with standard Rails sweepers. With this additional caching component, the access level AJAX request is hitting the database minimally.
• Performance optimization scenarios such as this make an argument against inline editing of resources. If there were a backend admin interface to allow editing of things, full-page caching would be more straight-forward to implement.

Conclusion

With this functionality, fully cached pages are served with an average of less than 5 ms/request, and the AJAX accessibility request appears to be around 20 ms/request (although this is harder to test with simple command line tools). This is an improvement over the 200 ms/request initially implemented. Additionally, requests at a high concurrency don't bog down the system as much.

Company Presentation: jQuery and Rails

Yesterday, I gave a company presentation on jQuery and Rails. The talk covered details on how jQuery and Rails work together to build rich web applications, with a considerable amount of focus on AJAX methods. Check out the slides here:

One piece of knowledge I took away from the talk is how different the Rails 3 approach is for unobtrusive AJAX behavior using helpers like link_to_remote and remote_form_for. Mike Farmer made a recommendation to read the rails.js source here to see how onclick behavior is handled in Rails 3.

JavaScript and APIs at RailsConf 2011

A big trend of RailsConf 2011 has been the presence of JavaScript, whether it be talk of CoffeeScript, node.js, Sproutcore, backbone.js, etc. I attended Yehuda Katz's talk on Building Rails Apps for the Rich Client on Day 4 of the conference.

Part 1: Rails versus Sinatra Smackdown

Yehuda gave a two part talk about developing API-heavy applications. The first bit addressed why to develop in Rails rather than Sinatra if your application is API-heavy and doesn't appear to be utilizing valuable parts of Rails? This is fairly applicable to a couple of Sinatra projects that I've done at End Point — I like Sinatra a lot, but at some point you begin to replicate parts of Rails. Yehuda explained that because it's easy to develop web applciations efficiently using Rails conventions, developers can become forgetful/ignorant of the underlying functionality of Rails that doesn't ship with something like Sinatra, much like how working with an ORM can breed developers who aren't familiar with the underlying database interactions. The checklist for things that Rails manages we might forget about includes:

• ActionDispatch internals
• Session deserialization
• Browser standards mode
• Cookie abstraction and management
• Deserialization of content types like JSON, XML
• Reloading (by comparison Sinatra doesn't reload automagically unless you use a tool like shotgun)
• Handling IP Spoofing
• Routing / Complex Routing
• Browser Caching (ETags, Last-Modified)
• Content negotiation to automatically support different MIMEs
• ActionMailer

This list is is a nice checklist of items you might review when you are considering building in Sinatra to be reminded of things that won't necessarily be trivial to implement.

Part 2: Consistent APIs

The second part of Yehuda's talk covered his experience with building client rich applications with APIs. His general observation is that while Rails follows universal conventions like the ActiveRecord convention that requires foreign keys to be named {table_name}_id, there isn't a convention for APIs. While there has been decent support for building APIs in Rails since 2006, there's been missing documentation on what to generate which has producted APIs with limited consistency. He gave the following pro tips for developing an API:

• Return a JSON object with one or more keys, that is iterable. Rather than returning

  { product_title: "Some Awesome Product." }
  

  return something that has one or more keys that is iterable, such as:

  {
      "products": [
          {
              id: 1,
              title: "Some Awesome Product."
          }
      ]
  }
  

• Avoid nested resources like /users/1/orders. Instead, always make the request do a GET on /orders or a PUT on /orders/1. This will help maintain consistency (and maintainability) with a simple asset reqeusts.
• In general, create a convention and follow conventions. Essentially, apply the principles of Rails here to produce maintainable and consistent code.

Finally, Yehuda presented the bulk_api gem to address the use case where you need to get a set of items, but want to avoid making single requests for multiple items and avoid getting all items and parse through them on the client side. An ecommerce example of this might be applicable in inventory management: Let's say you have ~30 unrelated listed inventory units that need to be updated to an on_hand state. From the admin screen, you may click on checkboxes next to those 30 items and then "Update". Rather than sending a single request to update each item, the bulk_api gem will make a single request to update the inventory unit status for all of the objects. Similarly, you can make a GET request to the bulk api to retrieve a set of objects, such as looking at a set of products on page 2 of a product listing. Yehuda went into more details on authentication and authorization with this gem, so I'd recommend reading the documentation here.

Conclusion

Yehuda concludes that Rails is great for JSON APIs and that we should continue to be building abstractions similar to ActiveRecord abstractions that follow conventions in a sensible way. Hooray for convention over configuration.

Locally served YUI3

For the vast majority of sites serving static JS and CSS files such as those required by YUI, or jQuery, etc. from a CDN makes great sense, improved performance through caching and geography, reduced load, improved uptime, leveraging some large corporations' resources, etc. Unfortunately as soon as you hit an SSL secured page you can't use a CDN's resources securely, a common thing with e-commerce sites and administration panels. In the YUI case that means doing at least a little bit of extra configuration and maintenance of an installed library base (an all too common task in typical server side development that's becoming more common as libraries are maintained for client side usage as well). Toss in "combo loading" and all of a sudden it feels like the software development cycle can't just be discarded cause you are working on the web, maybe this is really what they meant by web 2.0. But I digress...

Since I'm familiar with and working on YUI3, here is how we have it working for non-CDN based serving for an administration panel developed for a newer generation of an Interchange site. Our custom application uses YUI3 (core), modules from the YUI3 Gallery, and a few modules that are pulled in using the YUI 2in3 version compatibility layer. Now down to business...

Each of the libraries is provided by the YUI team through github.com. Since our project has git under it (you are using git, right?) we can set up submodules to help us with tracking new versions as they are released (or really be on the bleeding edge ahead of releases). To start we choose a location to serve the files, this is fairly arbitrary I happened to choose /vendor. Then set up new submodules for each of the libraries in use,

• git submodule add http://github.com/yui/yui3 vendor/yui3
• git submodule add http://github.com/yui/yui3-gallery vendor/yui3-gallery
• git submodule add http://github.com/yui/2in3 vendor/yui-2in3

Follow that with the normal git submodule init, git submodule update cycle to pull down the HEAD version of each library. At this point the library files are available for local serving, and the current version is pegged against the superproject.

With the files locally stored they can be served locally. To do so we need to adjust the "seed" file for the YUI core, the loader, and any CSS files loaded on the page. For instance,

(For now ignore the "/combo" portion of this, more on that later.)

With the yui.js and loader.js files loading from a local resource YUI will automatically know where to look for other YUI3 core files. But for non-core files (such as gallery modules or 2in3 loaded modules) we need to provide more information to the loader through the YUI_config global variable.

The above configuration has parameters for both combo and non-combo loading. Lines 4-7 tell the loader where to find core modules explicitly, though the loader should default to these based on the location of the seed file. Starting on line 9 there are definitions for the additional groups of files that we now want locally served. The first group, lines 11-20, are needed to load our local gallery files. Lines 25-44, as the comment above them indicates, are for loading specific CSS files that were not properly handled by the first group, though I've yet to track down why (more robust skin support may fix this issue). Lines 46-62 bring in YUI 2-in-3, note the specific minor version of 2 is pegged in line 47 and 50. With this configuration in place, and properly located files on the filesystem you should be able to use any core, gallery, or 2in3 modules loaded from your local system.

But what about "combo loading." "Combo loading" allows us to reduce the number of roundtrip requests made to pull down the dependencies by concatenating a set of files together (either JS or CSS but not together), when serving from the CDN you get this because Yahoo! has provided a combo loading service. You can see a PHP based loading service provided by the YUI team in the phploader project, https://github.com/yui/phploader/ . Because we are working with Interchange and aren't using PHP I've written a similar script to be used as plain ole' boring CGI, it can be found at github. In the example code this script would be setup to run at the root of our URL space, specifically as "/combo" and the script itself will need to know where to find the local root, "vendor", on the physical file system. (It is important to note that combo loading CSS files takes some special parsing to make images and other referenced files work correctly. Also note that my version does not work with includes in the CSS.)

Finally one of the biggest benefits to this set up is being able to test literally any commit in any of the modules while leaving all else the same. This can allow you to determine when a bug first entered the picture (see git bisect) or test against development branches ahead of releases. To test against the latest and greatest between releases, you simply have to go to one of the submodule paths, fetch any recent updates, then check out the master (or any) branch, merge from the origin branch, and test. If all your tests pass, then you git add the submodule path in the superproject, git commit, and know that your stack is now up to date. Hopefully some day the gallery modules themselves will be git submodules so that maintaining a local gallery will not involve pulling down all modules and an application can peg (or update) a specific module as needed by the application, essentially you'd have your own gallery.

Virtual Page Tracking and Goals with Google Analytics

Sometimes I come across websites that don't use RESTful URLs or have too unique (with an order number) URLs during checkout and I need to implement Goal Tracking in Google Analytics on these user interactions. I've also had to implement Goal Tracking in a non-ecommerce web application where tabbed on-page browsing guides users through a 3-step process. Examples of situations that pose challenges to traditional page tracking in Google Analytics include:

• Throughout Interchange's checkout, URLs are posts to "/process", which makes the user interactions difficult to distinguish.
• Throughout Spree's checkout, URLs are posts to "/order/:id/edit", which are distinct and can be difficult to aggregate.
• In a Sinatra application we developed recently, the single page URL is "/locate.html", but tabbed browsing occurs through three unique steps.

Google Analytics Tagging

To add Goal Tracking by URL, pages must first be tagged as "virtual pages". To implement virtual page tracking in Google, it's as simple as including a new virtual page URL in the _trackPageview action:

<script type="text/javascript">

  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-XXXXXXXX-1']);
- _gaq.push(['_trackPageview']);
+ _gaq.push(['_trackPageview', '/cart/basket']);

  (function() {
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
  })();

</script>

In the cases I described above, here are the tags used for each:

Goal Definition

After tagging is complete, you'll need to define a Goal in Google Analytics to begin tracking it.

First, you'll want to name your goal. Here I've named the goal "Purchase". In this case, the Goal is a URL Destination based on our new tags. The Goal URL is set to "/checkout/complete" for the Spree example described above. In this case, I have it set to a Regular Expression Match, but an Exact Match will work as well. I haven't assigned a Goal Value and I've left the match case insensitive for now. Next, I add the steps involved leading up to the goal URL. This will allow you to visualize a goal funnel and see where users may be abandoning checkout. In this example, each of the steps of the funnel correspond to a checkout step (shopping basket, optional register page, individual checkout steps, receipt/complete page). Note that this screenshot was taken from the new Google Analytics interface. The old Google Analytics interface may appear slightly different, but contains similar content.

Looking at the Data

With virtual page tracking and goal tracking, you can see how many goals have been completed during a certain time frame:

You can also compare regular segments of traffic: e.g. users that found your site through search versus referrals:

You can compare advanced segments of visitors: e.g. marketing campaign #1 versus marketing campaign #2. This type of comparison may give you more actionable metrics to determine which marketing campaign to put more resources into.

Last but not least, you can visualize the funnel of conversion to see where you might be losing customers. This might also be actionable in that you may be able to A/B test checkout process (or another type of conversion) changes to improve conversion.

Lazy Image Loading in JavaScript with jQuery

This week I had a duh moment while working on a little jQuery-driven interface for a side hobby.

I've occasionally used or attempted to do image preloading in JavaScript, jQuery, and YUI. Preloading images happens after a page is loaded: follow-up image requests are made for images that may be needed, such as hover images, larger sizes of thumbnail images on the page, or images below the fold that do not need to load at page request time. Adobe Fireworks spits out this code for preloading images, which is a bit gross because the JavaScript is typically inline and it doesn't take advantage of common JavaScript libraries. But this is probably acceptable for standalone HTML files that get moved between various locations during design iterations.

<body onload="MM_preloadImages('/images/some_image.png','/images/some_other_image.png')">

I found many examples of preloading images with jQuery that look something like this:

jQuery.preloadImages = function()
{
  for(var i = 0; i<arguments.length; i++)
  {
    jQuery("<img>").attr("src", arguments[i]);
  }
}

I implemented this method, but in my code the preloading was happening asynchronously and I needed to find something that would execute some other behavior after the image was loaded. Before I found the solution I wanted, I tried using jQuery's get method and tested jQuery's ready method, but neither was suitable for the desired behavior. I came across jQuery's load event, which binds an event handler to the "load" JavaScript event and can be used on images. So, I came up with the following bit of code to lazily load images:

 var img = $('<img>')
  .attr('src', some_image_source);
 $(element).append(img);
 if($(img).height() > 0) {
  // do something awesome
 } else {
  var loader = $('<img>')
   .attr('src', 'images/ajax_loader.gif')
   .addClass('loader');
  $(element).append(loader); 
  $(img).load(function() {
   // do something awesome (the same awesome thing as above)
   loader.remove();
  });
 }

So my bit of code creates a new image element. If the image's height is greater than 0 because it's already been requested, it does some awesome method. If its height is 0, it displays an ajax loader image, then does the same awesome method and removes the ajax loader image. See the screenshot below to get an idea of how this is used.

The image on the left has been loaded and resized to fit its frame. The image to be displayed on the right is loading.

Interestingly enough, the code above works in IE 8, Chrome, and Firefox, but it appears that IE handles image loading a bit differently than the other two browsers — I haven't investigated this further. This lazy-image loading reduces unnecessary requests made to pre-load images that may or may not be accessed by users and the added touch of an ajax loader image communicates to the user that the image is loading. I haven't added a response for image load failure, which might be important, but for now the code makes the assumption that the images exist.

I found a few jQuery plugins for lazy image loading, but I think they might overkill in this situation. One of the jQuery plugins I found is based on YUI's ImageLoader, a utility that similarly delays loading of images.

jQuery and Long-Running Web App Processes: A Case Study

I was recently approached by a client's system administrator with a small but interesting training/development project. The sys-admin, named Rod, had built a simple intranet web application that used a few PHP pages, running on an Amazon EC2 instance, to accept some user input and kick off multiple long-running server side QA deployment processes. He wanted to use Ajax to start the process as well as incrementally display its output, line-by-line, on the same web page. However, waiting for the entire process to finish to display its output was a poor user experience, and he wasn't able to get an Ajax call to return any output incrementally over the lifetime of the request.

Rod asked me to help him get his web app working and train him on what I did to get it working. He admitted that this project was a good excuse for him to learn a bit of jQuery (a good example of keeping your tools sharp) even if it wasn't necessarily the best solution in this case. I have always enjoy training others, so we fired up Skype, got into an IRC channel, and dove right in.

1. Install the Firebug add-on for Firefox
   
2. Use Firebug's Console tab to watch for javascript errors and warnings
   
3. Use Firebug's Net tab to monitor what ajax calls your app is making, and what responses they are getting from the server
   
4. Replace all those hateful debug alert() calls with console.log() calls, especially for ajax work

A special note about console.log(): Some browsers (including any Firefox that doesn't have Firebug installed) do not natively supply a console object. To work around this, we defined the following console.log stub at the very top of our single javascript file:

if (typeof console === 'undefined') {
    console = { log: function() {} };
}

Rod's javascript was a mix of old-school javascript that he had remembered from years ago, and new-school jQuery he had recently pulled from various tutorials. His basic design was this: When the user clicked the "Deploy" button, it should kick off two separate Ajax requests; "Request A" would initiate a POST request to deploy.php. Deploy.php made a number of system calls to call slow-running external scripts, and logged their output to a temporary logfile on the server. "Request B" would make a GET request to getoutput.php every 2 seconds (which simply displayed the output of said logfile) and display its output in a scrollable div element on the page.

Hearing Rod describe it to me, I wondered if he might be headed down the wrong path with his design. But, he already had put time into getting the server-side code working and did not want to change direction at this point. Discussing it with him further, it became clear that he did not want to re-write the server-side code and that we could in fact make his current design produce working code with teachable concepts along the way.

To start, Rod told me that his "ajax POST request (Request A) wasn't firing." As the Russian proverb says, "Trust, but verify." So, we opened Firebug's Net tab, clicked the web app's Deploy button (actually its only button - Steve Jobs look out) and saw that the ajax request was in fact firing. However, it was not getting back a successful HTTP 200 status code and as such, was not getting handled by jQuery as Rod expected. Expanding the ajax request in the Net tab let us see exactly what name/value data that was getting POSTed. We spotted a typo in one of his form input names and fixed it. Now Request A was clearly firing, POSTing the correct data to the correct URL, and getting recognized as successful by jQuery. (More on this in a bit.)

Rod's code was making Request A from within a jQuery event handler defined for his form's Deploy button. But, he was making Request B via an HTML onClick attribute within that same HTML tag. He was getting all sorts of strange results with that setup based on which request was returning first, if Request B's function call was correctly returning false to prevent the entire form from getting POSTed to itself, etc. Consolidating logic and control into event handlers that are defined in one place is preferable to peppering a web page with HTML onClick, onChange, etc. javascript calls. So, we refactored his original jQuery event handler and onClick javascript call into the following code snippet:

//global variable for display_output() interval ID
var poll_loop;

$(".deploy_button").click(function() {
    $.ajax({
        beforeSend: function() {
            $('#statusbox').html("Running deployment...");
        },
        type: "POST",
        url: "deploy.php",
        data: build_payload(),
        success: function() {
            console.log('Qa-run OK');
            //previously called via an onClick
            poll_loop = setInterval(function() {
                display_output("#statusbox", 'getoutput.php');
            }, 2000);
        },
        error: function() {
            console.log('Qa-run failed.');
        }
    });
});

That $.ajax(...) call is our jQuery code that initiates the Request A ajax call and defines anonymous functions to call based on the HTTP status code of Request A. If Request A returns an HTTP 200 status code from the server, the anonymous function defined for the 'success:' key will be executed. If any other HTTP code is returned, the anonymous function defined for the (optional) 'error:' key is executed. We refactored the onClick's call to display_output() into the 'success:' function above. Now, it only gets called if Request A is successful, which is the only time we'd want it to execute.

The body of the 'success:' anonymous function calls setInterval() to create an asynchronous (in that it does not block other javascript execution) javascript loop that calls display_output() every 2 seconds. The setInterval() function returns an "interval ID" that is essentially a reference to that interval. We save that interval ID to the 'poll_loop' variable that we intentionally make global (by declaring it with 'var' outside any containing block) so we can cancel the interval later.

Here is the display_output() function that makes Request B and gets called every 2 seconds:

function display_output(elementSelector, sourceUrl) {
    $(elementSelector).load(sourceUrl);
    var html = $(elementSelector).html();
    if (html.search("EODEPLOY") > 0) {
        window.clearInterval(poll_loop);
        alert('Deployment Finished.');
    }
    if (html.search("DEPLOY_ERROR") > 0) {
        window.clearInterval(poll_loop);
        alert('Deployment FAILED.');
    }
}

That .load() method is jQuery shorthand for making an ajax GET request and assigning the returned HTML/text into the element object on which it's called. Because the display_output() function is responsible for terminating the interval that calls it, we need to define our end cases. If either "EODEPLOY" (for a successful deployment) or "DEPLOY_ERROR" (for a partially failed deployment) appear as strings within the resulting HTML, we call clearInterval() to stop the infinite loop, and alert the user accordingly. If neither of our end cases are encountered, display_output() will be executed again in 2 seconds.

As it stands, the poll_loop interval will run indefinitely if the server-side code somehow fails to ever return the two strings we're looking for. I left that end case as an exercise up to Rod, but suggested he add a global variable that could be used to measure the number of display_output() calls or the elapsed time since the Deploy button was clicked, and end the loop once an upper limit was hit.

1. Client-side input validation using javascript regular expressions
   
2. Matching server-side input validation because sometimes the call is coming from inside the house
   
3. Adding a unique identifier that is passed as part of both Request A and Request B to better identify requests and to prevent temp file naming conflicts from multiple concurrent users.
   
4. Packaging display_output()'s "Deployment FAILED" output and providing a button to easily send the output to Rod's team

I'm sure there are a ton of other possible solutions for a project like this. For example, I know that Jon and Sonny developed a more advanced polling solution for another client, www.locateexpress.com, using YUI's AsyncQueue. Without getting to deeply into the server-side design, I'm curious to hear how other people might approach this problem. What do you think?

YUI Extensions and Post Initialization

When using YUI3's provided extension mechanism to enhance (composite, mix in, role, whatever you like to call it, etc.) a Y.Base inherited base class, it is often helpful to have "post initialization" code run after the attributes' values have been set. The following code provides an easy way to hook onto a Y.Base provided attribute change event to run any post initialization code easily.

Using nginx to transparently modify/debug third-party content

In tracking down a recent front-end bug for one of our client sites, I found myself needing to use the browser's JavaScript debugger for stepping through some JavaScript code that lived in a mix of domains; this included a third-party framework as well as locally-hosted code which interfaced with -- and potentially interfered with -- said third-party code. (We'll call said code foo.min.js for the purposes of this article.) The third-party code was a feature that was integrated into the client site using a custom domain name and was hosted and controlled by the third-party service with no ability for us to change directly. The custom domain name was part of a chain of CNAMEs which eventually pointed to the underlying *actual* IP of the third-party service, so their infrastructure obviously relied on getting the Host header correctly in the request to select which among many clients was being served.

It appeared as if there was a conflict between code on our site and that imported by the third party service. As part of the debugging process, I was stepping through the JavaScript in order to determine what if any conflicts there were, as well as their nature (e.g., conflicting library definitions, etc.). Stepping through our code was fine, however the third-party's JS code was (a) unfamiliar, and (b) minified, so this had the effect of putting all of the JavaScript code more-or-less on one line, which made tracing through the code in the debugger much less useful than I had hoped.

My first instinct was to use a JavaScript beautifier to reverse the minification process, but since I had no control over the code being included from the third-party service, this did not seem to be directly feasible. The third-party code was deployed only on our production site and relied on hard-coded domains which would make integrating it into one of our development instances challenging since we had no control over the contents of the returned resources. Since the relevant feature (and subsequent bugs) was only on the production site, making extensive modifications to how things were done and potentially breaking that or other features for users while I was debugging was obviously out as an option.

Enter nginx. I've been doing a lot with nginx lately as far as using it as a reverse proxy cache, so it's been on my mind lately. So I came up with this technique:

1. Look up the IP address for the third-party's domain name (used for later purposes).
2. Install nginx on localhost, listening to port 80.
3. Modify /etc/hosts to point the third-party's domain name to the nginx server's IP (also localhost in this case).
4. Configure a new virtual host with the following logical constraints:
   • We want to serve specific files (the beautified JavaScript) from our local server.
   • We want any other request going through that domain to be passed-through transparently, so neither the browser nor the third-party server treat it differently.

Given these constraints, this is the minimal configuration that I came up with (the interesting parts are located in the server block):

/etc/hosts:

example.domain.com 127.0.0.1

nginx.conf:

worker_processes 1;

events {
    worker_connections 10;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    
    server {
        server_name example.domain.com;
        root /path/to/local_root;

        try_files $uri @proxied;

        location @proxied {
            proxy_set_header Host $http_host;
            proxy_pass http://1.2.3.4;
        }
    }
}

Once I had the above configured/setup, I downloaded/saved the foo.min.js file from the third-party service, ran it through a JS beautifier, and saved it in the local nginx's cache root so it would be served up instead of the actual file from the third-party service. Any other requests for static resources (images, other scripts, etc) would pass-through to the third-party server, so I had my nicely-formatted JavaScript code to step through, the production site worked as normal for anyone else despite potential local changes to the file on my end (i.e., adding JavaScript alert() calls to the file, and no one was the wiser.

A few notes

The try_files directive instructs nginx to first look for a file named after the current URI (foo.min.js in our example) in our local cache, and if this is not found, then fallback to the proxied location block; i.e., relay the request to the original upstream server. We explicitly set the Host header on the proxy request because we want the request to behave normally with respect to name-based hosting, and provide the saved IP address to contact the server in question.

We only needed to preserve/lookup the upstream server's IP address because we're running the nginx server on localhost, so if we used a domain name the lookup would return the same IP defined in /etc/hosts; if the nginx server was running on a different machine, you would be able to just use the domain name as both the server_name and the proxy_pass parameters and set the entry for the host in your local /etc/hosts file to the IP of the nginx server.

A possible extension would be to detect when an upstream request matched a minified URL (via a location ~ \.min\..*\.js$ block) and automatically beautify/cache the content in our local cache. This could be accomplished via the use of an external FastCGI script to retrieve, post-process, and cache the content.

This technique can also be used when dealing with testing changes to a production site on which you are unable or unwilling to make potentially disruptive changes for the purposes of testing static resources. (JavaScript seems the most obvious application here, but this could apply to serving up images or other static content which would be resolvable by the local cache.)

I always need to remind myself to undo changes to /etc/hosts as soon as I'm done testing when using tricks like these. Particularly in something like this which is more-or-less transparent, the behavior would be functionaly identical as long as code/scripts on the third-party site stayed the same, but could easily introduce subtle bugs if the third-party services made changes to their codebase. Since our local copies would mask any remote changes for those non-proxied resources, this could be very confusing if you forget that things are set up this way.

JSON pretty-printer

The other day Sonny and I were troubleshooting some YUI JavaScript code and looking at some fairly complex JSON. It would obviously be a lot easier to read if each nested data structure were indented, and spacing standardized.

I threw together a little Perl program based on the JSON man page:

#!/usr/bin/env perl

use JSON;

my $json = JSON->new;
undef $/;
while (<>) {
    print $json->pretty->encode($json->decode($_));
}

It took all of 2 or 3 minutes and I even left out strictures and warnings. Living on the edge!

It turns a mess like this (sample from json.org):

{"glossary":{"title":"example glossary","GlossDiv":{"title":"S","GlossList":
{"GlossEntry":{"ID":"SGML","SortAs":"SGML","GlossTerm":"Standard Generalized Markup Language",
"Acronym":"SGML","Abbrev":"ISO 8879:1986","GlossDef":{"para":
"A meta-markup language,used to create markup languages such as DocBook.",
"GlossSeeAlso":["GML","XML"]},"GlossSee":"markup"}}}}}

into this much more readable specimen:

{
   "glossary" : {
      "GlossDiv" : {
         "GlossList" : {
            "GlossEntry" : {
               "GlossDef" : {
                  "para" : "A meta-markup language,used to create markup languages such as DocBook.",
                  "GlossSeeAlso" : [
                     "GML",
                     "XML"
                  ]
               },
               "GlossTerm" : "Standard Generalized Markup Language",
               "ID" : "SGML",
               "SortAs" : "SGML",
               "Acronym" : "SGML",
               "Abbrev" : "ISO 8879:1986",
               "GlossSee" : "markup"
            }
         },
         "title" : "S"
      },
      "title" : "example glossary"
   }
}

But today I thought back to that and figured surely something like that must already be at hand if I'd just looked for it. Sure enough, there are many easy options that work conveniently from the shell, similarly to that script:

• json_xs (Perl JSON::XS)
• python -mjson.tool (Python 2.6+)
• prettify_json.rb (Ruby json gem)

And those were just the ones that were likely already on the machine I was using! Hooray for convenience.

jQuery Tips and an Ecommerce Demo

I've recently been jumping back and forth between YUI and jQuery on several different client projects. I prefer working with jQuery, but whenever I work with another framework, I realize what's out there and how I should continue to improve my skills in my preferred framework. I read up on jQuery tips and put together a summary of common tips I need to follow more explained here in an ecommerce demo.

The Setup

Before we get started, some notes on the ecommerce demo and performance testing: The fake product images come from DryIcons A JSON array contains product information (price, image, and title). The code runs on a quick and dirty sinatra app. console.time(‘task’) and console.timeEnd(‘task’) are used to debug task runtime for performance measurement The performance numbers provided in the article were measured in Chrome (Ubuntu) where the average of 10 tests is reported. In all the tests, an additional for loop was added as part of the test to see measurable performance differences. See the notes at the bottom of the article on performance differences between Chrome and Firefox.

A screenshot from the demo app.

1. The first tip I came across was a recommendation to use a for loop instead of jQuery's each method. I start off with some un-optimized code to test that loops through our products JSON object and renders the images to the page. In this case, the use of the for loop instead of each doesn't give us a significant performance difference.

for(var k=0;k<10;k++) { $('div.products').html(''); console.time('test ' + k); for(var j=0;j<10;j++) { $.each(products, function(index, e) { var ihtml = $('div.products').html(); ihtml += '<a href="#">' + '<img class="product" src="/images/' + e.image + '" /></a>'; $('div.products').html(ihtml); }); } console.timeEnd('test ' + k); }	for(var k=0;k<10;k++) { $('div.products').html(''); console.time('test ' + k); for(var j=0;j<10;j++) { for(var i=0;i<products.length; i++) { var ihtml = $('div.products').html(); ihtml += '<a href="#">' + '<img class="product" src="/images/' + products[i].image + '" /></a>'; $('div.products').html(ihtml); } } console.timeEnd('test ' + k); }
10 tests average: 505ms	10 tests average: 515ms

Products displayed with a for loop instead of jQuery's each method.

2. The next "I Can't Believe I'm Not Using this jQuery Technique" I found was a recommendation to use data tag. Although I've read about the data tag, I haven't worked with it enough to use it consistently. With this code, we assign product data (name, price) to each product link as it’s added to the DOM. Upon clicking a product, instead of traversing through the products array, we render the "featured_product" contents based on it's data. The data tag is recommended over assigning values to arbitrary HTML tag attributes such as assigning our product <a> and <img> the name and price values to title or alt attributes.

for(var i=0;i<products.length; i++) { var link = $(document.createElement('a')) .attr(‘href’, ‘#’) .html('<img class="product" src="/images/' + products[i].image + '" />') .data('name', products[i].name) .data('price', products[i].price); if(products[i].featured) { link.addClass('featured'); } if(products[i].sale) { link.addClass('sale'); } $('div.products').append(link); }; $('div.products a').click(function() { $('div.featured_product') .html('<h1>' + $(this).data('name') + '</h1>'); $(this).find('img').clone().appendTo('.featured_product'); });

jQuery's data tag is used to populate the right side of the page as a product is clicked.

3. The next tip I came across frequently was a recommendation to cache your selectors, shown in the example below. A selector $('div.products a.featured') is created and used when users like to view "Featured" Items. This gave me a 33% performance gain.

$('a.featured').click(function() { for(var k=0;k<10; k++) { console.time('test ' + k); for(var j=0;j<100;j++) { $('div.products a').css('background', '#FFF'); $('div.products a.featured').css('background', '#999'); } console.timeEnd('test ' + k); } });	var all_products = $('div.products a'); var featured_products = $('div.products a.featured'); $('a.featured').click(function() { for(var k=0;k<10; k++) { console.time('test ' + k); for(var j=0;j<100;j++) { all_products.css('background', '#FFF'); featured_products.css('background', '#999'); } console.timeEnd('test ' + k); } });
10 tests average: 31ms	10 tests average: 21ms

Multiple products added to test onclick for identifying "featured" products.

4. Another tip I came across was the recommendation on using context in jQuery selectors, described here in the jQuery documentation. In one example, I try updating the $('div.products') selector to set the context as $(‘div.wrapper’), but saw performance worsen here. In another example, I added "this" as a context for populating the featured product, but again saw performance worsen slightly here. In this case, performance gain will depend on your original selector, but it's worth testing.

for(var k=0;k<10;k++) { $('div.products').html(''); console.time('test ' + k); for(var j=0;j<10;j++) { for(var i=0;i<products.length; i++) { ... $('div.products').append(link); } } console.timeEnd('test ' + k); }	for(var k=0;k<10;k++) { $('div.products').html(''); console.time('test ' + k); for(var j=0;j<10;j++) { for(var i=0;i<products.length; i++) { ... $('div.products', $('div.wrapper')).append(link); } } console.timeEnd('test ' + k); }
10 tests average: 64ms	10 tests average: 76ms

5. Another common tip I came across that "I can't believe I don't follow" is to use an id selector instead of a class. I’ve admittedly read this several times, but again it's a practice that I sometimes forget about. In my ecommerce setup, I add a loop to add the products 10x to our DOM, and with a change of selector from 'div.products' to 'div#products', I saw a small performance improvement.

for(var k=0;k<10;k++) { $('div.products').html(''); console.time('test ' + k); for(var j=0;j<10;j++) { for(var i=0;i<products.length; i++) { ... $('div.products').append(link); } } console.timeEnd('test ' + k); }	for(var k=0;k<10;k++) { $('div.products').html(''); console.time('test ' + k); for(var j=0;j<10;j++) { for(var i=0;i<products.length; i++) { ... $('div#products').append(link); } } console.timeEnd('test ' + k); }
10 tests average: 64ms	10 tests average: 60ms

6. I found a recommendation to minimize the DOM minimally. This is a tip that I typically follow, but another one that's easily forgotten. In our ecommerce setup, I create a single point of appending to the div#products selector after generating my product links and data. This gave a ~25% performance gain.

for(var k=0;k<10;k++) { $('div.products').html(''); console.time('test ' + k); for(var j=0;j<10;j++) { for(var i=0;i<products.length; i++) { ... $('div#products').append(link); } } console.timeEnd('test ' + k); }	for(var k=0;k<10;k++) { $('div#products').html(); console.time('test ' + k); var collection = $(document.createElement('div')); for(var j=0;j<10;j++) { for(var i=0;i<products.length; i++) { ... collection.append(link); } } $('div#products').append(collection); console.timeEnd('test ' + k); }
10 tests average: 60ms	10 tests average: 45ms

7. Another tip I came across was to use event delegation in jQuery. The idea is that your event has more context to work with than general selectors. You can access the event and manipulate it’s parent. I found that $(e.target).parent() is the same as manipulating $(this) performance-wise. It’s likely that using one or the other is much faster than using a general DOM selector such as $(div.product' + id).

var featured_product = $('div#featured_product'); $('div#products a').click(function(e) { var product = $(e.target).parent(); // same as $(this) featured_product .html('<h1>' + product.data('name') + '</h1>') .append(product.find('img').clone()); });

8. One tip that I've never seen before is to use ".end()" in chaining. Instead of reselecting the $('div.products') region, I use "find()" to apply a css style, then traverse up the chain to find another set of products to apply a css style, and repeat. This change gave a small performance bump, but you might tend to use the cached selectors described in Tip #3 instead of the following.

$('a#special').click(function() { for(var k=0;k<10;k++) { console.time('test ' + k); for(var j=0;j<100; j++) { $('div#products').find('a').css('background', '#FFF'); $('div#products').find('.featured').css('background', '#999'); $('div#products').find('.sale').css('background', '#999'); } console.timeEnd('test ' + k); } });	$('a#special').click(function() { for(var k=0;k<10;k++) { console.time('test ' + k); for(var j=0;j<100; j++) { $('div#products') .find('a') .css('background', '#FFF') .end() .find('.featured') .css('background', '#999') .end() .find('sale') .css('background', '#999'); } console.timeEnd('test ' + k); } });
10 tests average: 47ms	10 tests average: 37ms

Featured and Sale products are highlighted here.

9. I found several examples of writing your own selectors and how easy it is! I wrote two selectors with the following to identify products under $20.00 and products over $1000.00.

$(function() { $('a#under20').click(function() { all_products.css('background', '#FFF'); $('div#products a:under20').css('background', '#999'); }); $('a#over1000').click(function() { all_products.css('background', '#FFF'); $('div#products a:over1000').css('background', '#999'); }); }); $.extend($.expr[':'], { under20: function(a) { if($(a).data('price') && $(a).data('price') < 20) { return true; } return false; }, over1000: function(a) { if($(a).data('price') && $(a).data('price') > 1000) { return true; } return false; } });

Products priced over $1000 are highlighted with a custom selector.

10. I also found several examples of how to write your own chain methods. In this example, I create two chain methods to set the product background to white or grey and update my under20 & over1000 methods to use this new chain method. The nice thing about creating your own chain methods is that these methods can be easily modified in the future with minimal code changes because it follows the DRY principle. I'm not sure if it's intended, but the use of my custom chain method did not work with the end() chain method described in Tip #8.

$.fn.unhighlight_product = function() { return $(this).css('background', '#FFF'); } $.fn.highlght_product = function() { return $(this).css('background', '#999'); } $(function() { $('a#under20').click(function() { all_products.unhighlight_product(); $('div#products a:under20').highlight_product(); }); $('a#over1000').click(function() { all_products.unhighlight_product(); $('div#products a:over1000').highlight_product(); }); });

Products priced under $20 are highlighted with a custom selector and custom chain method.

Again, the product images for this demo app are from DryIcons.com and the final application code can be found here. The application was also deployed on Heroku to verify that the code works in IE [8], Chrome, and Firefox.

In Case You are Interested

During development of the demo, I found a significant performance differences between Chrome and Firefox. See the following tests:

• Tip #1: 6200ms (FF) vs 515ms (Chrome)
• Tip #3: 106ms (FF) vs 21ms (Chrome)
• Tip #4: 258ms (FF) vs 60ms (Chrome)
• Tip #6: 169ms (FF) vs 45ms (Chrome)
• Tip #8: 154ms (FF) vs 37ms (Chrome)

Or in visual form:

New Year Bug Bites

Happy New Year! And what would a new year be without a new year bug bite? This year we had one where figuring out the species wasn't easy.

On January 2nd one of our ecommerce clients reported that starting with the new year a number of customers weren't able to complete their web orders because of credit card security code failures. Looking in the Interchange server error logs we indeed found a significant spike in the number of CVV2 code verification failures (Payflow Pro gateway error code "114") starting January 1st.

We hadn't made any programming or configuration changes on the system in the recent days. We double-checked to make sure: nope, no code changes. So it had to be a New Year's bug and presumably something with the Payflow Pro gateway or banks further upstream. We checked error logs for other customers to see if they were being similarly impacted, but they weren't. Our client contacted PayPal (the vendor for Payflow Pro) and they reported there were no problems with their system. The failures must indeed be card failures or a problem with the website according to them. We further checked our code looking for what we could possibly have done that might be the cause, double-checking our Git repository (which showed no recent changes) and reexamining our checkout code for possible year-based logic flaws.

Our client's top-notch customer service group got on the phone with a customer who'd gotten a security code failure and got PayPal tech support on another line. The customer service rep tried to place the customer's order on the website using the customer's credit card info and once again got the CVV2 error. She then did the credit card transaction using the swipe machine in the office, and lo and behold the order went through! What was going on??!

It turned out that despite the Payflow Pro gateway returning CVV2 verification errors what was really happening was that the year of the credit card was coming into the Payflow Pro gateway as "2012"—not as "2011" as entered into the checkout form. We knew all along that it was possible that the 114 error code responses were possibly misleading because payment gateway error codes are notorious this way. (Payment gateways blame the banks, saying they can only pass along what the banks give them. Some banks' credit card validations don't actually even care about the years being correct, but just that they not be in the past; but I digress...)

Previously we'd reviewed the checkout pages and the dropdown menus to verify that the dropdown menus weren't off, but nevertheless it very much sounded like this rather stupid problem could very well be the culprit. So we checked and checked again. What we found is that sometimes on the checkout form the year dropdown menu was mangled such that the values associated with the displayed years were YYYY+1.

The oddly intermittent behavior of the problem, the process of elimination and the all around hair pulling this loss of business was causing made somebody in the marketing group at our client realize that they are in fact still running an Omniture Test & Target A/B test on the checkout pages that they thought had been discontinued. To quote David Christensen (thanks, David!): "The Omniture system works by replacing select content for randomly chosen users in an effort to track user behavior/response to proposed site changes. Alternate site content is created and dynamically replaced for these users as they use the site, such as the specific content on the checkout page in this instance."

We verified that the Omniture A/B test's JavaScript replacement code was alternately mangling and not mangling the year dropdown on the checkout form as mentioned. Our client took out the A/B test and the "security code errors" dropped back to a normal low level.

This was a difficult and expensive problem—not only was there business lost because of the problem, but there were a lot of resources put into troubleshooting it. We've come away from this episode with some lessons learned and with plenty of food for thought. I'll leave it to commentators to opine away on this, including the End Point folks who scratched this itch: David Christensen, Jeff Boes, Mark Johnson, and Jon Jensen.

jQuery code for making a block level element clickable while maintaining left/middle/right clicking

While working on a recent redesign for a client we needed the ability to click on a div and have it function as a link to a product page. The initial implementation used the jQuery plugin BigTarget.js. The plugin searches within the div for a link and when the div is clicked changes the location to the link that is found. This plugin worked fine and was fairly easy to setup, however there was one drawback that we found once it was released in the wild. Most people expect to be able to right click, middle click, shift click, and control click to get the context-sensitive menu, open in a new window, or open in a new tab.

Enter Superlink.js, a jQuery plugin that uses a cool trick of moving the location of a link to the location of the mouse within the block level element (such as a div, li, tr, td, etc.). With this implementation the mouse is actually over a link so that the various ways of clicking function as expected. Initially I started moving the clickable area within a table, as shown in the example, but then quickly realized there was no reason this shouldn't work within a div or li. One other neat thing with this plugin is that the events attached to the block will continue to function, they are passed to the event handlers for the link being moved within the block.

Utah Open Source Conference 2010 part 1

It's been about a little over a month since the 2010 Utah Open Source Conference, and I decided to take a few minutes to review talks I enjoyed and link to my own talk slides.

Magento: Mac Newbold of Code Greene spoke on the Magento ecommerce framework for PHP. I've somewhat familiar with Magento, but a few things stood out:

• He finds the Magento Enterprise edition kind of problematic because Varien won't support you if you have any unsupported extensions. Some of his customers had problems with Varien support and went back to the community edition.
• Magento is now up to around 30 MB of PHP files!
• As I've heard elsewhere, serious customization has a steep learning curve.
• The Magento data model is an EAV (Entity-Attribute-Value) model. To get 50 columns of output requires 50+ joins between 8 tables (one EAV table for each value datatype).
• There are 120 tables total in default install -- many core features don't use the EAV tables for performance reasons.
• Another observation I've heard in pretty much every conversation about Magento: It is very resource intensive. Shared hosting is not recommended. Virtual servers should have a minimum of 1/2 to 1 GB RAM. Fast disk & database help most. APC cache recommended with at least 128 MB.
• A lot of front-end things are highly adjustable from simple back-end admin options.
• Saved credit cards are stored in the database, and the key is on the server. I didn't get a chance to ask for more details about this. I hope it's only the public part of a public/secret keypair!

It was a good overview for someone wanting to go beyond marketing feature lists.

Node.js: Shane Hansen of Backcountry.com spoke on Node, comparing it to Tornado and Twisted in Python. He calls JavaScript "Lisp in C's clothing", and says its culture of asynchronous, callback-driven code patterns makes Node a natural fit.

Performance and parallel processing are clearly strong incentives to look into Node. The echo server does 20K requests/sec. There are 2000+ Node projects on GitHub and 500+ packages in npm (Node Package Manager), including database drivers, web frameworks, parsers, testing frameworks, payment gateway integrations, and web analytics.

A few packages worth looking into further:

• express - web microframework like Sinatra
• Socket-IO - Web Sockets now; falls back to other things if no Web Sockets available
• hummingbird - web analytics, used by Gilt.com
• bespin - "cloud JavaScript editor"
• yui3 - build HTML via DOM, eventbus, etc.
• connect - like Ruby's Rack

I haven't played with Node at all yet, and this got me much more interested.

Metasploit: Jason Wood spoke on Metasploit, a penetration testing (or just plain penetrating!) tool. It was originally in Perl, and now is in Ruby. It comes with 590 exploits and has a text-based interactive control console.

Metasploit uses several external apps: nmap, Maltego (proprietary reconnaissance tool), Nessus (no longer open source, but GPL version and OpenVAS fork still available), Nexpose, Ratproxy, Karma.

The reconnaissance modules include DNS enumeration, and an email address collector that uses the big search engines.

It can backdoor PuTTY, PDFs, audio, and more.

This is clearly something you've got to experiment with to appreciate. Jason posted his Metasploit talk slides which have more detail.

So Many Choices: Web App Deployment with Perl, Python, and Ruby: This was my talk, and it was a lot of fun to prepare for, as I got to take time to see some new happenings I'd missed in these three languages communities' web server and framework space over the past several years.

The slides give pointers to a lot of interesting projects and topics to check out.

My summary was this. We have an embarrassment of riches in the open source web application world. Perl, Python, and Ruby all have very nice modern frameworks for developing web applications. They also have several equivalent solid options for deploying web applications. If you haven't tried the following, check them out:

• Perl: Dancer, Starman
• Python: Flask
• Ruby: Sinatra, Unicorn

That's about half of my notes on talks, but all I have time for now. I'll cover more in a later post.

Cross Browser Development: A Few CSS and JS Issues

Coding cross browser friendly JavaScript and CSS got you down? In a recent project, Ron, David, and I worked through some painful cross browser issues. Ron noted that he even banged his head against the wall over a couple of them :) Three of these issues come up frequently in my other projects full of CSS and JS development, so I wanted to share.

Variable Declaration in JS

In several cases, I noticed that excluding variable declaration ("var") resulted in broken JavaScript-based functionality in IE only. I typically include variable declaration when I'm writing JavaScript. In our project, we were working with legacy code and conflicting variable names may have be introduced, resulting in broken functionality. Examples of before and after:

var display_cart_popup = function() {
    popup_id = '#addNewCartbox';
    left = (parseInt($(window).width()) - 772) / 2;
    ...
};

var display_cart_popup = function() {
    var popup_id = '#addNewCartbox';
    var left = (parseInt($(window).width()) - 772) / 2;
    ...
};

...
address_display = '';

country = $(type+'_country').value;
address = $(type+'_address').value;
address2 = $(type+'_address2').value;
city = $(type+'_city').value;
state = $(type+'_state').value;
zip = $(type+'_zip').value;
...

...
var address_display = '';

var country = $(type+'_country').value;
var address = $(type+'_address').value;
var address2 = $(type+'_address2').value;
var city = $(type+'_city').value;
var state = $(type+'_state').value;
var zip = $(type+'_zip').value;
...

I researched this to gain more insight, but I didn't find much except a reiteration that when you create variables without the "var" declaration, they become global variables which may have resulted in conflicts. However, all the "learning JavaScript" documentation I browsed through includes variable declaration and there's no reason to leave it out for these lexically scoped variables.

Trailing Commas in JSON objects

According to JSON specifications, trailing commas are not permitted (e.g obj = { "1" : 2, }). From my experience, JSON objects with trailing commas might work in Firefox and WebKit browsers, but it dies silently in IE. Some recent examples:

{
  "response_message"    : '<?= $response_message ?>',
  "subtotal"            : <?= $subtotal ?>, 
  "shipping_cost"       : <?= $shipping ?>, 
  "carttotal"           : <?= $carttotal ?>, 
<?php if($add_taxes) { ?>
  "taxes"               : <?= $taxes ?>
<?php } ?>
}

{
  "response_message"    : '<?= $response_message ?>',
  "subtotal"            : <?= $subtotal ?>, 
  "shipping_cost"       : <?= $shipping ?>,  
<?php if($add_taxes) { ?>
  "taxes"               : <?= $taxes ?>,
<?php } ?>
  "carttotal"           : <?= $carttotal ?>
}

var fonts = {
[loop list=`$Scratch->{fonts}`]
    '[loop-param name]' : {
      'bold' : "[loop-param bold]",
      'italic' : "[loop-param italic]"
    },[/loop]
};

var fonts = {
[loop list=`$Scratch->{fonts}`]
    '[loop-param name]' : {
      'bold' : "[loop-param bold]",
      'italic' : "[loop-param italic]"
     },[/loop]
    'dummy' : {}
};

Additional solutions to avoid the trailing comma include using join (Perl, Ruby) or implode (PHP), conditionally excluding the comma on the last element of the array, or using library methods to serialize data to JSON.

Floating Elements in IE

Often times, you'll get a design like the one shown below. There will be a static width and repeating components to span the entire width. You may programmatically determine how many repeating elements will be displayed, but using CSS floating elements yields the cleanest code.

Example of a given design with repeating elements to span a static width.

You start working in Chrome or Firefox and apply the following CSS rules:

CSS rules for repeating floating elements.

When you think you're finished, you load the page in IE and see the following. Bummer!

Floating elements wrap incorrectly in IE.

This is a pretty common scenario. In IE, if the combined widths of consecutive floating elements is greater than or equal to 100% of the available width, the latter floating element will jump down based on the IE float model. Instead of using floating elements, you might consider using tables or CSS position rules, but my preference is to use tables only for elements that need vertical align settings and to stay away from absolute positioning completely. And I try to stay away from absolute positioning in general.

The simplest and minimalist change I've found to work can be described in a few steps. Let's say your floating elements are <div>'s inside a <div> with an id of "products":

<div id="products">
  <div class="product">product 1</div>
  <div class="product">product 2</div>
  <div class="product" class="last">product 3</div>
  <div class="product">product 4</div>
  <div class="product">product 5</div>
  <div class="product" class="last">product 6</div>
</div>

And let's assume we have the following CSS:

<style>
div#products { width: 960px; }
div.product { float: left; width: 310px; margin-right: 15px; height: 100px; }
div.last { margin-right: 0px; }
</style>

Complete these steps:

• First, add another div to wrap around the #products div, with an id of "outer_products"
• Next, update the 'div#products' width to be greater than 960 pixels by several pixels.
• Next, add a style rule for 'div#outer_products' to have a width of "960px" and overflow equal to "hidden".

Yielding:

<div id="outer_products">
  <div id="products">
    <div class="product">product 1</div>
    <div class="product">product 2</div>
    <div class="product" class="last">product 3</div>
    <div class="product">product 4</div>
    <div class="product">product 5</div>
    <div class="product" class="last">product 6</div>
  </div>
</div>

And:

<style>
div#outer_products { width: 960px; overflow: hidden; }
div#products { width: 980px; }
div.product { float: left; width: 310px; margin-right: 15px; height: 100px; }
div.last { margin-right: 0px; }
</style>

The solution is essentially creating a "display window" (outer_products), where overflow is hidden, but the contents are allowed to span a greater width in the inside <div> (products).

The white border outlines the outer_products "display window".

Some other issues that I see less frequently include the double-margin IE6 bug, chaining CSS in IE, and using '#' vs. 'javascript:void(0);'.

Simple audio playback with Yahoo Mediaplayer

Recently I had need to show a list of MP3 files with a click-to-play interface.

I came upon a very simple self-contained audio player:

<script type="text/javascript" src="http://mediaplayer.yahoo.com/js"></script>

The code to set up my links for playing was dirt-simple:

<script type="text/javascript">
var player = document.getElementById('player');
function add_to_player() {
    var link = this;
    player.src.replace(/audioUrl=.*/,'audioUrl=' + link.src);
    return false;
}
var links = document.getElementsByTagName('A');
for (var i = 0; i < links.length; i++) {
    if (links[i].src.match(/\.mp3$/)) {
        links.onclick = add_to_player;
    }
}
</script>

You could use various ways to identify the links to be player-ized, but I chose to just associate the links with a class, "mp3":

<a class="mp3" href="/path/to/file.mp3">Audio File 1</a>

Obviously, if jQuery is in use for your page, you can reduce the code to an even smaller snippet.

Debugging jQuery

A recent reskin project for a client requires that we convert some of their old Prototype code to jQuery as well as create new jQuery code. I have not done much with converting Prototype to jQuery and I felt like my debugging tools for JavaScript were under par. So this morning I set out to find what was available for jQuery and I found this article on the subject.

I've used Firebug for some time now, but was unaware of some of the supporting plugins that would certainly help with debugging JavaScript. Some of the plugins and methods found in the article that I found immediately helpful were:

• FireFinder: Makes it quite easy to verify that the selector values in your code are correct and that the proper elements are returned. I was able to immediately pinpoint problems with my selectors and this brought to light why certain events were not firing.
• Firebug Console: Using the console.log function allowed me to check values without littering my code with alert statements.
• FireQuery: At a glance this plugin for Firebug shows which elements have event handlers bound to them.
• Firebug Breakpoints: Setting breakpoints and watch statements in your code makes it easier to see what is happening in the JavaScript code as it is executed instead of trying to figure out what happened after the code has run its course.

Thanks to the author of the article, Elijah Manor, for the in-depth information on debugging jQuery code.

jQuery Auto-Complete in Interchange

"When all you have is a hammer, everything looks like a nail."

Recently, I've taken some intermediate steps in using jQuery for web work, in conjunction with Interchange and non-Interchange pages. (I'd done some beginner stuff, but now I'm starting to see nails, nails, and more nails.)

Here's how easy it was to add an auto-complete field to an IC admin page. In this particular application, a <select> box would have been rather unwieldy, as there were 400+ values that could be displayed.

That's the requisite header stuff. Then you set up the internal list of autocomplete terms:

The [perl] block just needs to emit a space-delimited list of the autocomplete terms. For instance,