Backbone at Dataminr

I started working at Dataminr a couple of months ago and I've had to make the switch across from using Closure and PlastronJS to jQuery and Backbone with Backbone.LayoutManager. The switch was frustrating as there was a lot of functionality missing, just rendering a list of items was a pain and the code showed it too. So I set about changing it with Backbone plugins which work extremely well for our situation.

The first change was to change where the logic was executing, there were lists that had all the logic for each child it contained and views were generating their own collections rather than allowing them to be passed in, so to customize there were large parts of code that was essentially a copy of what it extended but for a single line. That was the first thing to change, to push the logic down to where it should go and inject in dependancies. That already cut out a huge amount of code and cleaned things up, but there was still functionality that was copied over across multiple view. The solution? Why not create generalized mixins and have them applied to views, and so Backbone.Advice was born.

Backbone.Advice

Backbone.Advice is based on the advice functional mixin work of Angus Croll. It's made working with views a pleasure and allowed me to cut off huge chunks of code. By having the mixins as functions we could then call more mixins from within a mixin, basically setting up a chain of inheritance (more like a requires) with mixins. So for example we have a mixin that makes a view selectable:

  /**
   * MIXIN
   * adds on a selected paramenter for a view as well as methods to deal with
   * selected state and changes to the state.
   */
  Mixin.view.makeSelectable = function() {

    this.setDefaults({
      selectable: true,
      selected: false,
      select: function(sel) {
        if (!this.isSelectable())
          return;
        var temp = this.selected;
        this.selected = sel === undefined || !!sel;
        if (temp != this.selected) {
          if (this.selected)
            this.onSelect();
          else
            this.onDeselect();
        }
      },
      deselect: function() {
        this.select(false);
      },
      toggleSelect: function() {
        this.select(!this.selected);
      },
      isSelected: function() {
        return this.selected && this.isSelectable();
      },
      isSelectable: function() {
        return this.selectable;
      },
      onSelect: function() {},
      onDeselect: function() {}
    });
  };

Now we can actually build mixins on top of this. For example I might want a click to Select mixin:

  /**
   * MIXIN
   * toggle selection state on click.
   */
  Mixin.view.clickSelect = function() {

    this.mixin(Mixin.view.makeSelectable);

    this.addToObj({
      events: {
        'click': 'onClick'
      }
    });

    this.after('onClick', function(event) {
      if (!document.getSelection().isCollapsed)
        return;
      if (this.selectableElement(event.target))
        this.toggleSelect();
    });

    this.around('selectableElement', function(fn, el) {
      return fn(el) ||
        el == this.el ||
        this.$el.contains(el);
    });
  };

Now all I have to do is add the click to select to a view to make it selectable by clicking:

var ClickableView = Backbone.View.extend().mixin(Mixin.view.clickSelect);

The great thing about these mixins is that we don't have to recode our old code base to add them in, I can just mix them in on existing code! Want to see something even more excellent? well there are mixins to make a list navigatable by keyboard. So I can just do something like this:

var NavigatableList = Backbone.View.extend().mixin(Mixin.view.keyboardNavigatableList);

And that's it! I now have a list that I can navigate with the keyboard and it's going to select the appropriate child. The keyboardNavigatableList mixin actually uses other mixins that you could use on their own which include getChildren (allows you to return the child views of a view - its made to work with layoutManager and componentView but can easily be swapped out for whatever view manager you use),selectableChildren (keep a record of the selected children), singleSelectChild (only allow one child to be selected at a time) and keyboardNavigation (which actually extends focusable and gives you functions like onKeydown and onKeyup that you can mixin with instead of creating new events and potentially overwriting functionality as you extend your view).

So this was all pretty sweet but my biggest annoyance was that there was no way to just have a list, so I made a mixin for that as well. It was made to work with layoutManager but I was soon finding problems with layoutManager so I made another plugin...

Backbone.ComponentView

The main issues I ran in to with LayoutManager was around it's asynchronous code. In the code there were functions everywhere and deferreds being passed around that would be used to modify views in completely different parts of the code base. There was no way to know what would be happening to a view just by looking in the constructor, you'd have to trace all the code at run time, and even then it was hard to track down. One of the things that Google's Closure Library does well is with components, so naturally I wrote a port to use underscore instead of the Closure Library utilities.

Components have a lifecycle and a very definite way they should be used. Within a component you have an enterDocument and that is where you put the code to run after rendering. After writing a couple of extra methods to allow ComponentView to be used with setView and insertView (LayoutManager methods) I found all I really had to do was grab everything in beforeRender and afterRender and put them in enterDocument. To my surprise it took very little tweaking to get it running with this and allowed a good structure to clean up any asynchronous code, because now I could put it in to run synchronously.

The one thing it doesn't do is asynchronous loading of templates, but we use requireJS, so I just put them up there and we were away. In fact doing that was a good thing as now we can have the templates in the build system rather than what we had before where they were all compiled in to a single file and brought in first (this way we only have to bring in what we need).

So with ComponentView I also made an autolist mixin which I converted over from PlastronJS and now to get a full list I only have to do:

var List = Backbone.ComponentView.extend({
  itemView: SelectableItem
}).mixin([
  Mixin.view.keyboardNavigatableList,
  Mixin.ComponentView.autolist
]);

But we're still not done. In the interface we had a list split up in two, one for endorsed items and one for normal items. They both make separate server requests and are both lists, so they were both collections. The issue though is when an endorsed item comes in we want the item in the other list to know it's now endorsed and to change. So there was spaghetti code in the view to run through the models and match items across the two collections - it was a mess. The solution? we only want to use one instance of the model:

Backbone.ModelRegistry

Models in our application only last a certain amount of time, once they are out of all the collections we don't need them anymore. I wanted to share a model instance across collections, but I didn't want the register to hold a reference because then I would have to write logic to test whether a model should be removed periodically. The answer was pretty simple. On our page we always show these two collections so there isn't a problem keeping a handle to them, so why not just register the collections and then search through them for an existing model instance. So Backbone.ModelRegistry was born.

To make this work though I would need to override the Backbone.Collection.prototype._prepareModels to first check for the existence of a model and pass that back instead. So to do this I just wrote a mixin that would register a collection then augment it's _prepareModel method to check the registry for an existing model. So it became as simple as this:

var registery = new Backbone.ModelRegistry();
var Coll = Backbone.Collection.extend().mixin([
    Mixin.collection.modelStore
], {
    register: registry
});

Then I can extend coll, or mixin more things but any new instances will be registered with the registry and share model instances. I could even put that mixin on different constructors and as long as I pass the same register object through they will all share model instances. In fact we have two panes in our product that have lists that need to share their models but we don't want to share the models between panes, so we just create two model registers and use one for the top pane and the other for the bottom. Now we have the same models between them, but we were told that now we want to hide the models in the second collection if they are endorsed. So basically we want to update our collection to remove models that are endorsed - the problem is that there may have been a mistake and we need to put them back in. What we really needed was a filtered collection to make things easy for us.

Backbone.CollectionFilter

The main issue was that I don't want to know that the collection I'm getting is a filtered version. I want to still be able to interact with it as if it's the original collection with all it's original function calls. So what we have to do is create a new collection that extends our old one and then bind all of it's functions to the original context. Then we need to do have a look at the events getting passed through and choose whether they should fire at all (a model being added to the original but not passing a filter should not fire any event). And so Backbone.CollectionFilter came in to being. To use it we just need to pass in the original collection which will be used as a basis for inheritance, a filter function and an optional comparator that will allow you to sort your collection in a different way if necessary. This meant that to get the collection we just need to do this:

var NonEndorsed = new Backbone.CollectionFilter(endorsed, function(mod) {
    return !mod.get('endorsed');
});

And that's it, we can just pass along that collection and use it as normal. And that's it, we've got a great scalable system that we can mix and match to get what we need.

The Future

We're not stopping there though, there is heaps in the pipeline.

Nerve

which is a mediator that allows communication at the application level for Backbone. We're not using it yet but it's there when we need it.

Underscore.Extras

is still in the early stages. It'll be extra functionality that we can use to write clean functional code. The plan is it'll replace code that does a lot of existential testing.

HandlebarRivets

At the moment we're using Handlebars templates but we haven't got a way to hook up the data binding to that. Rivets.js allows good two way data binding and allows custom generators so work has begun on a handlebars to rivets converter which should give us the data binding from rivets and the ease of use of handlebars. Still very early (less than a day of work) but big plans for this one.

a quick note on CI

Going forward the plan will be to have all these as git submodules which will be hosted with JenkinsCI so we can develop them alongside our normal code. If you have a look in the above projects you can already see there is testing setup with Testacular (now Karma) and mocha with chai. It was an absolute pain getting it all running with requireJS so I'm hoping in the near future to switch over to Mantri