Over the Christmas holidays, I sat down to sort out my photo library and I
wanted to make a web photo gallery to share the images with family and friends.
Being a mainly Python programmer, I wrote a Python script that will go through
all of my photos and create smaller versions and thumbnails of the images that
are suitable for the web. I think I have around 1500 images at the moment and
it took a good 20 minutes to convert everything using Python Imaging Library.

I was rather frustrated with the performance. I think the reason for the
slowness was the fact that the images were processed sequentially, one by one.
What if I could modify my program to process more than one image at a time.
This would certainly speed by the process. Then I thought about using threads
in Python and I think I threw up in my mouth a little.

Then I remembered Node.js. It’s supposed to be all asynchronous and
non-blocking. Perfect match. So I wrote a thumbnailer in node.

Essentially, it’s a worker queue that resizes images. You give it a source and
destination directories and a number of workers and run it. The total time went
from 20 minutes to 2 using about 5 workers. Much better, eh?

I decided to open source the project because I couldn’t find any node.js
projects that do something even remotely similar. It’s called node-thumbnail
and you can find it on both Github and npmjs.

The SheepDog Deployment team show consistent results above the average. Some specific highlights include:

Overall client satisfaction with their reseller
SheepDog 92%
Average 89%

Clients likelihood to renew
SheepDog 96%
Average 91%

Professionalism
SheepDog 97%
Average 93%

Responsiveness
SheepDog 94%
Average 90%

Technical ability
SheepDog 93%
Average 88%

Congratulations to the SheepDog Deployment team and their continued focus on customer satisfaction.

Deploying Django

Despite the Django official documentation’s recommendation to use Apache and mod_wsgi, most people seem to deploy Django with nginx and gunicorn. This was a pleasant surprise to us because we like the simplicity of this combination but thought it was less robust than Apache.

Provisioning servers

It seemed that everybody was talking about Chef and automatic deployments. Instead of manually connecting to a server via ssh, you can run one command which will install all the necessary packages for your application (nginx, postgresql, etc.) and configure them. It makes the whole process less error-prone. Also, you’re more likely to spin up a new VM on your development machine to try things out on a production environment because you don’t have to think about setting it up. Especially with tools like vagrant, which automates the creation and provisioning of virtual machines, it’s dead easy.

Pronouncing things

When you read names of technical products, you can’t always be sure how they’re supposed to be pronounced. At a conference like DjangoCon, it becomes clear:

• PyPI: pie-pee-eye
• PyPy: pie-pie
• wsgi: wizz-gy
• nginx: engine-x
• gunicorn: gun-i-corn

Did you go to DjangoCon US 2011? What are your thoughts?

Recently, one of our client domains came under a substantial Directory Harvest Attack (DHA). Spammers increasingly are probing domain email systems in an attempt to identify and “harvest” legitimate email addresses from your organization. When they obtain lists of valid email addresses, they then compile them into lists that are sold  or distributed to other spammers. Postini has indicated that the number of directory harvest attacks has nearly doubled in the past six months.

If your mail server is MS Exchange or Domino Lotus Notes, a harvest attack, can literally slow your mail server down to a crawl or in some cases, result in server crashes and database corruption.

Fortunately our client is on Google Apps and has Postini Message Security in place. The sustained harvest attack did not stop their mail flow and they were able to continue on with their daily work flow. Postini’s exclusive Preemptive Email Protection technology (preEMPT) is able to identify and block DHAs at the SMTP connection level, prior to content filtering, and ensures that none of these types of attacks reaches the enterprise email gateway.

Postini effectively prevents the directory harvest attack from overloading your mail servers and stealing addresses from your email directory. It also stops spammers from using addresses that would have been harvested to further flood your email system with unwanted messages. It saves your email infrastructure from wasting cycles and bandwidth responding to directory harvest attacks and probes and reduces the costs and workload of your IT staff since it requires to integration with existing email infrastructure, no software and no hardware.
If you are not using Postini Message Security we encourage you to review a Postini White Paper entitled “The Silent Killer: How spammers are stealing your email directory”

The problem

When you are developing a large Javascript application and you are using jQuery, your code tends to get disorganized very quickly. The script files are full of spaghetti code, endless callbacks and unrelated helper functions. The Backbone documentation puts it this way:

When working on a web application that involves a lot of JavaScript, one of the first things you learn is to stop tying your data to the DOM. It’s all too easy to create JavaScript applications that end up as tangled piles of jQuery selectors and callbacks, all trying frantically to keep data in sync between the HTML UI, your JavaScript logic, and the database on your server. For rich client-side applications, a more structured approach is helpful.

Backbone is trying to be that more helpful approach. Backbone is a Model-View-Controller framework that will hopefully add some structure to your project.

Models

In Backbone, Models are used to hold units of data. For example, here we create a simple Post class that will hold information about a blog post.

var Post = Backbone.Model.extend({
  initialize: function() {
    if (!this.get('title'))
      this.set('title', 'Untitled');
  }
});

Models have a constructor method and some basic helper methods. They can perform validation, load data from a remote server, save changes to a database and more. Behind the scenes, Backbone serializes your models to and from JSON.

Views

Backbone uses Views to represent a Model instance in the DOM. A view usually contains a model and provides hooks to interact with the model through standard browser events such as click,, focus or hover.

var PostView = Backbone.View.extend({
  events: {
    "h1 click": "titleClick"
  },

  initialize: function() {
    _.bindAll(this, 'render');
  },

  titleClick: function() {
  },
  render: function() {
  }

});

Each view instance knows how to render itself and it knows how to respond to predefined events. Here, we listen for clicks on the H1 tag.

Collections

Collections are arrays of model instances. They allow you to quickly perform operations on a group of model instances. This is useful for things like filtering, sorting or batch deleting of items.

Syncing

Backbone exposes a syncing interface. This allows you to completely customize the way that your data is transmitted between the server and the client. Perhaps, you prefer XML over JSON. Or, instead of saving your data on a server, you can save it in HTML5 localStorage.

From experience

When I first looked at Backbone, it seemed easy and straight-forward. However, when I went to build something with it I quickly realized it wasn’t as easy. Once you manage to put all the pieces together, it makes perfect sense. Reading through the examples and looking at some existing applications will help you a lot. Having wrapped my head around the key concepts, I was ready to build a complex app with no problems.

Next?

If you are interested in learning more about Backbone, be sure to check out the official documentation. I have also put together a small, proof-of-concept application for you to check out. It’s a simple expense manager and the source is on Github.

My 10-year old daughter, Lila, came up with this puzzle.  A grade school class is given a problem of the form

__ op __ op __

where there are two operations (ops) and three integers in the blanks.  They are asked to compute the answer.  Some of the students got the correct answer of 9.  However, some of the students did not remember their order of operations and consequently got the incorrect answer of 12.  What was the question?  I.e., what numbers were in the blanks and what were the operations?

There is more than one solution to the puzzle.  Can you find all of them?

(Spoiler) The last puzzle was:

What do the following to two very distinctive sentences have in common?

• THE WORKERS SUPPLY POLISH FURNITURE.
• BOB’S EVENING PROCEEDS FAIRLY.

They actually have quite a lot in common.  Both sentences can be pronounced two different ways, yielding two completely different parses of the sentence.  There are two words in each sentence whose pronunciation and grammatical parts of speech change.  For the first, supply is an alternate spelling of supplely.  For the second, Bob could be divying up the ante fairly.

Tools like Graphite are great when you are in total control of the hosting environment — which unfortunately is not the case with Google AppEngine. So, to this end we built our own tool which emulates the parts of Graphite that are important to us. We’ve been using it in production on gTrax for a while now, and its finally time for it to be open-sourced.

The tool is gae-event-logger and is available for download on our GitHub page. Event Logger has minimal set up (no database configuration, etc.) and uses memcache to handle its data. Using memcache has its advantages because of the speed that we can log at without seeing any performance degradation. The last thing you want to do is have developers afraid to actually use logging for fear that it could slow down your site! However, being backed by memcache means that we have no guarantees that data actually sticks around — if Google flushes memcache then the statistics are dropped. In practice however, this is pretty rare.

A live example of its output is below, and captures the success/failure rate of certain RPC calls made from the gTrax application. If we start to see spikes of rpc.failure events we know something wrong has happend! We have dozens (hundreds!?) of graphs like this that we monitor — everything from login success rates to user signups.

Let us know if you use Event Logger in a project!

Get Satisfaction has a RESTful API that can be used, however developers will immediately realize that calling an AJAX JSON feed from the client-side browser will fail because of Same Origin Policy restrictions in the browser. Fortunately, Get Satisfaction has an undocumented feature which will help us bypass the restriction in a clever, cross-browser way.

The idea is to make use of a callback function inside the feed. The usual feed without the callback (here) looks pretty standard:

{
    "data": [
        {
            "at_sfn": "http://getsatisfaction.com/sheepdoginc/topics/weve_moved-1io2po", 
            "author": {
                "at_sfn": "http://getsatisfaction.com/people/john_2286285", 
                "avatar": "http://s3.amazonaws.com/satisfaction-production/public/uploaded_images/7552101/sprinkles_medium.jpg", 
                "avatar_url_large": "http://s3.amazonaws.com/satisfaction-production/public/uploaded_images/7552101/sprinkles_large.jpg", 
                "avatar_url_medium": "http://s3.amazonaws.com/satisfaction-production/public/uploaded_images/7552101/sprinkles_medium.jpg", 
                "avatar_url_mini": "http://s3.amazonaws.com/satisfaction-production/public/uploaded_images/7552101/sprinkles_mini.jpg", 
                "avatar_url_small": "http://s3.amazonaws.com/satisfaction-production/public/uploaded_images/7552101/sprinkles_small.jpg", 
                "canonical_name": "john_2286285", 
                "champion": false, 
                "employee": true, 
                "id": 2569575, 
                "name": "John Everick", 
                "url": "http://api.getsatisfaction.com/people/2569575"
            }, 
            "champion": false, 
            "company_id": 81150, 
            "content": "We've moved the community for gTrax to this new location. Please continue to give your feedback, ask questions and help one another out as we pop in and out to gather your ideas on how to improve gTrax and answer any questions you have.\r<br /><br />\nCheers!", 
            "created_at": "2011/06/16 14:52:20 +0000", 
            "emotitag": {
                "face": "happy", 
                "feeling": "excited", 
                "intensity": 1
            }, 
            "employee": true, 
            "flag_count": 0, 
            "follower_count": 1, 
            "has_promoted_replies": false, 
            "id": 2544294, 
            "last_active_at": "2011/06/16 14:52:20 +0000", 
            "me_too_count": 1, 
            "most_recent_activity": "create", 
            "reply_count": 0, 
            "slug": "weve_moved-1io2po", 
            "status": "complete", 
            "style": "update", 
            "subject": "We've Moved!", 
            "url": "http://api.getsatisfaction.com/topics/2544294", 
            "user_defined_code": false
        }
    ], 
    "total": 1
}

Of course without a callback we won’t actually be able to use these results if they were fetched with AJAX. The argument we want to add is callback=gsUpdate. Adding the callback (here) argument to the GET string we get something a little more useful:

gsUpdate({"data":[{"me_too_count":1,"status":"complete","employee":true,"follower_count":1,"style":"update","reply_count":0,"slug":"weve_moved-1io2po","last_active_at":"2011/06/16 14:52:20 +0000","champion":false,"has_promoted_replies":false,"at_sfn":"http://getsatisfaction.com/sheepdoginc/topics/weve_moved-1io2po","created_at":"2011/06/16 14:52:20 +0000","most_recent_activity":"create","subject":"We've Moved!","emotitag":{"feeling":"excited","face":"happy","intensity":1},"company_id":81150,"author":{"avatar_url_large":"http://s3.amazonaws.com/satisfaction-production/public/uploaded_images/7552101/sprinkles_large.jpg","employee":true,"avatar_url_small":"http://s3.amazonaws.com/satisfaction-production/public/uploaded_images/7552101/sprinkles_small.jpg","avatar_url_medium":"http://s3.amazonaws.com/satisfaction-production/public/uploaded_images/7552101/sprinkles_medium.jpg","avatar_url_mini":"http://s3.amazonaws.com/satisfaction-production/public/uploaded_images/7552101/sprinkles_mini.jpg","champion":false,"at_sfn":"http://getsatisfaction.com/people/john_2286285","canonical_name":"john_2286285","name":"John Everick","avatar":"http://s3.amazonaws.com/satisfaction-production/public/uploaded_images/7552101/sprinkles_medium.jpg","url":"http://api.getsatisfaction.com/people/2569575","id":2569575},"flag_count":0,"content":"We've moved the community for gTrax to this new location. Please continue to give your feedback, ask questions and help one another out as we pop in and out to gather your ideas on how to improve gTrax and answer any questions you have.\r<br /><br />\nCheers!","url":"http://api.getsatisfaction.com/topics/2544294","id":2544294,"user_defined_code":false}],"total":1})

Way better! The external GSFN API is now co-operating with us to provide something we can hook into to actually use the results inside our GWT application. We’ll need to use a bit of JSNI to provide the gsUpdate hook from the GWT application, and then post-process the results so that they can be displayed via GWT widgets.

In our widget we’ll do three things. First, we need to export the gsUpdate callback to the top-level window object so that the Get Satisfaction call can find it. We then want to grab the data out of the stream and pass it back into native GWT for processing. When we pass back into native GWT, it’d be nice to have a JavaScriptObject wrapper around the JS objects, so that we can use them without coupling too tightly to the JSON response object.

import com.google.gwt.core.client.JavaScriptObject;

public class GSFNUpdateObject extends JavaScriptObject {
  protected GSFNUpdateObject() {}
  
  public final native String getDate() /*-{
    return this.created_at;
  }-*/;
  
  public final native String getTitle() /*-{
    return this.subject;
  }-*/;
  
  public final native String getContent() /*-{
    return this.content;
  }-*/;
  
  public final native String getLink() /*-{
    return this.at_sfn;
  }-*/;
  
  public final native String getAuthorName() /*-{
    return this.author.name;
  }-*/;
  
  public final native String getAuthorAvatar() /*-{
    return this.author.avatar_url_medium;
  }-*/;
}

class FancyWidget extends Composite {
  public FancyWidget() {

    gsfnUpdate();

    ScriptElement el = Document.get().createScriptElement();
    el.setLang("javascript");
    
    el.setSrc("http://api.getsatisfaction.com/products/67315/topics.json?style=update&callback=gsUpdate&limit=5&product=sheepdoginc_gtrax&sort=last_active_at");

    HeadElement element = Document.get().getElementsByTagName("head").getItem(0).cast();
    element.appendChild(el);

    // ...
  }

  public void recieveUpdateList(JsArray<GSFNUpdateObject> items) {
    updates.clear();
    
    for (int i = 0; i < items.length(); i++) {
      GSFNUpdateObject item = items.get(i);
      updates.add(new GSFNUpdateWidget(item));
    }
  }

  public native void gsfnUpdate() /*-{
		var reference = this;
		$wnd.gsUpdate = function(items) {
			reference.@your.namespace.FancyWidget::recieveUpdateList(Lcom/google/gwt/core/client/JsArray;)(items.data);
		}
  }-*/;

}

As you can see, the <script> tag is created and injected into the page load upon construction of the widget — from that point on there is a callback dance where results are fetched, processed and displayed.

The result is that the 5 most recent updates from Get Satisfaction are displayed — along with links back to Get Satisfaction for users to post questions, comments and ideas. You can format the results however you’d like, without relying on Get Satisfaction’s default widgets.

Hopefully this will help you in integrating with Get Satisfaction when using GWT — and maybe provide strategies and ideas for other APIs that support callback arguments as well!

Posted by Wilson Verge, Director IT Services Department, NSCC

Editors note: NSCC is at the forefront of education innovation in Canada with a committed focus on Learning, Capacity and the Future. Spread across 15 full-service campuses, NSCC is the largest and most diverse institution in Nova Scotia. Driving sustainable building and technological practices, NSCC brings together trades and technologies to create change in the environments of the future.

Ten years ago, the Nova Scotia Community College had a reputation for being a trade school with a stale brand and a lack of forward thinking. This type of brand has a hard time attracting a new generation of learners. There was a need for change and change they did. A decade later, NSCC is the most innovative and forward thinking institution in eastern Canada with a new roster of courses and programs focusing on producing the thought and technical leaders of the future. So, what better place to access the innovation of Google Apps for Education…enter Wilson Verge, NSCCs Director of IT Services Department and project Champion.

Wilson heads up the NSCC IT department which supports 55,000 users divided into student, faculty, staff and alumni user groups spread across campuses in fifteen towns and cities in Nova Scotia, Canada. To add to the complexity, NSCC’s legacy IT infrastructure was supporting a hybrid IT solution made up of several different versions of Microsoft Outlook. Seeing the need to mirror the College’s commitment to innovation in their IT environment, Wilson worked with Google’s Premier Canadian Partner, SheepDogInc to evaluate and eventually deploy Google Apps for Education as their perfect match. The maturity and robustness of the Google Apps for Education products were a deciding factor for the College who had many stakeholders to convince when making the decision to shift to a new messaging and collaboration platform.

Identifying change management as a critical point of failure, NSCC and SheepDog instituted a steering committee who conducted focus groups and user surveys as the first phase of the project allowing the College to gain acceptance in their stakeholder groups. Access to the marketing and change management support offered by Google helped seal the deal and lead the implementation of the solution to 22,000 users institution wide (with the additional 33,000 alumni to follow).

In addition to the many features that the Google Apps for Education offers users and administrators, there are two main differentiating points from the legacy Microsoft systems. The first is the seamless document collaboration of users, particularly staff in various campuses, allowing for up-to-date documentation and an end to version control. The second is the integration of a single sign on solution with the NSCC student portal and the active directory linked to Google Apps, which provides a successful solution for identity management not available in the past. Now up and running, NSCC is benefiting from the features the students and staff access in their lives outside the EDU space and productivity is at an all time high while the IT department can finally effectively track and manage user IDs.

The next step of innovation for the NSCC is anyone’s guess but their eyes have been opened to the potential of the Google Apps and the platform beneath. Discussions have started between the SheepDog software engineering division and the NSCC Geographic Information Systems (GIS) department. Stay tuned to see how these leaders of the future will leverage Google in one of the fastest growing technology sectors in the world.