Closet Coder

I work in my closet. I code. Yep.

Feedback Loops and Estimation -or- What Rubik's Cubes Taught Me About Making Software

| Comments

Estimation is hard. It may not be listed as one of the top two problems in computer science, but it’s at least a close third. Over the years I’ve gotten to try all sorts of methodologies for estimating. Waterfall, Agile, Points, Stories, Requirements, Features, Epics, Pomodoros, Billable hours… you can go on forever with the Jargon of Management, trying to get information from a developer about how much longer The Client has to wait until The Feature is finished.

Rubik’s Cubes

About 4 years ago I started a job where a coworker was an avid Rubik’s cube solver.  I learned the steps and began competing with him to see who could solve a cube the fastest.  Eventually, my top time ended up around a minute and thirty seconds.  There are guys who can do it much faster–but I’m interested in consistency and improved speed.  Sometimes I can solve very quickly because some steps can be skipped (i.e. they are done for you) or because I’m using a very good cube, but in general, I can solve a cube in 1:30 - 3:00.

But I don’t do it blindfolded.

Blindfolded

The world record for the fastest solve is just under 10 seconds, but the world record for the fastest blindfold solve (i.e. from the time you pick it up, look it over, and then blindfold yourself to begin solving) is over a minute and a half–10 times as long.  The reason is simple–feedback.  Immediate and quick feedback allow you to take shortcuts and spend less time worrying about the next 200 moves and just worry about the next step or two.

So why do we still tend toward “blindfolded” approaches in software?

Waterfall vs. a Feedback loop

Call it Agile, call it scrum, call it feedback–it doesn’t matter.  What makes a team think they’re going to deliver better because they took the time to figure out all the steps, when you don’t know exactly how one step will affect the next?  It’s foolish when you consider it, but still, so many teams do it.  Even the supposed “agile” teams end up blindfolding releases and then trying to ‘feedback’ the sprints.  That’s not to say there should be no planning, but getting customer and developer feedback is crucial to accurate and fast solutions to problems.

What should we be doing instead?  In Rubik’s Cube solutions, you typically identify the next ‘step’ and iterate over it, examining the result and deciding the following step at that point.  You might be able to ‘blindfold’ for each step, but you’d still need to look at the end to figure out where to go next.  I think a similar paradigm works in software building.  Identify the key feature, and let the developer work on it, deploy it, and get feedback.  Iterate on it again if it’s not quite right.  If it is, then move along to the next step or feature.

Analogy Breakdown

Where does this analogy break down?  Architectural changes.  You can’t make sweeping changes underneath the system without a good deal of planning and “blindfolding” where you step through, unsure of the effect on the system because you can’t actually evaluate it. There are times this must occur–but as developers, we probably tend toward a desire to rearchitect more often than we should.  Always consider what the right solution is and evaluate the cost of technical debt vs. the potential cost of an underlying change that could have pervasive and unknown effects.  Sometimes it is necessary, but generally it’s not.  YAGNI is your friend.

Takeaway

The tighter the feedback loop, the more accurate and more quickly you’re able to deliver a solution, whether in Rubik’s world or in the Software space.  Focus on tight loops with customer feedback and developer feedback and watch your estimation troubles diminish.

Migrating Serialized Columns in Rails

| Comments

I recently switched a serialized column in Rails from one type (Hash) to another (OpenStruct) and ran into a little problem when I tried to migrate, namely, that loading the model threw a SerializationTypeMismatch error. Hmm… how am I going to get at the base YAML and translate all of these without being brittle?

The answer is to copy the column, nullify it, and the load the raw YAML manually:

1
2
3
4
5
6
7
8
add_column :my_table, :serialized_column_raw, :text
MyTable.update_all("serialized_column_raw = serialized_column")
MyTable.update_all("serialized_column = NULL")
MyTable.all.each do |mt|
  mt.serialized_column = OpenStruct.new(YAML::load(mt[:serialized_column_raw]))
  mt.save!
end
remove_column :my_table, :serialized_column_raw

How to Stay at Inbox Zero

| Comments

I saw a link recently on how to get to the ever elusive “Inbox Zero” and it seemed kinda lame–label everything “oldinbox” and archive everything in Gmail. Maybe that’s the only way to do it when you have 10K+ e-mails in your inbox, but it seems like the best way is to never get there to begin with.

Personally, I haven’t had more than 50 e-mail in my inbox at any given time in over 2 years. I ruthlessly archive and delete items and as soon as I’ve processed something into an action for “Things” I get it out of my inbox. For e-mails I just need to look at, reference or respond to, I use the stars feature, but I have only 8 starred items right now. I think at some point I had nearly 20… that’s about the max.

There are 3 very simple things you can do to stay at Inbox Zero:

  1. Use "Multiple Inboxes" - to keep things in front of you, use multiple inboxes (it's a Gmail Labs feature) and make your own version of the priority inbox.
  2. Use a catch-all and your own domain - I have a catch-all address getting checked by Gmail. I give out e-mail addresses to businesses like "amazon@my.domain.com" that way and have all of the mail that comes in on that domain marked with the label "bulk" -- and I have a "bulk" inbox using "Multiple Inboxes"
  3. Add "Send and Archive" in labs - Now, everytime you respond to somebody, send and archive, don't just send. You're done. Never worry about it again.

Okay, now you know how to keep a simple and clean inbox. Go forth and do likewise.

The Importance of Speed in Automation

| Comments

We are impatient people. This is something that we must work to fix in order to grow as individuals, but it is something that serves the automator well–or can be our downfall.

Joel Splosky wrote on the “Joel Test” that having anything less than the best tools money can buy is rediculous for a development team. The reasoning is this: If you’re paying developers what they are worth, then they are expensive, and wasting their time while they’re reading the Onion waiting for a build will kill your productivity–and your bottom line.

The same goes now for Test Driven Development. Having a great test suite is nearly essential, and anything more than a trivial application will have difficult and long running tests, but these tests must be managed well and there must be a way for a developer to quickly run through a cross section of the test suite as a sanity check before checking in. If your tests take 1 minute to run, then the developer is 6 times less likely to run them than if they take 10 seconds to run. If your tests take 30 minutes to run, then the developer is 60 times less likely to run them than if they take 30 seconds. Every additional test is great for coverage, but if it adds time, there is a point of diminishing returns.

Tagged tests are a must, and a solid set of fast tests that touch much of the code base is essential. It must also be easy to run individual tests–anything that hampers this reduces the liklihood of continued test driven development. Testing becomes a chore again and running the tests becomes an excuse for swordfighting in the hall.

So take the time to improve the speed of anything you wait on as developers–and reap the benefits tenfold.

Stop Googling // RailsTips by John Nunemaker

| Comments

Yesterday, one of my inter-web buddies IM’d me and asked if I had used Typhoeus before. I said yes, so he asked me if it was possible to follow redirects using it. He said he google’d it and nothing turned up.

I sharply responded, “LOOK AT THE CODE!”. We had some banter back and forth and a few minutes later he was automatically following redirects. It seems these days that developers often think if something does not turn up in a google search, it does not exist.

via Stop Googling // RailsTips by John Nunemaker.

Things and Pomodoro

| Comments

For the last two years, I’ve been a loose GTD-er, using Cultured Code’sThings” to dump my brain and keep track of tasks to do. I never got into the “project” side of GTD–planning out every project and asking why… I’m sure it has value, but that was never the problem I was trying to solve with GTD.

I just wanted to keep up with all the stuff everyone wanted me to do and I was tired of having things so easily slip through the cracks.

Well, 2+ years later and I’ve gotten very used to capturing, scheduling, prioritizing and crossing off items from my master list that isn’t in my brain. Things has been great for this, if only for the “Today” feature. For those of you looking for a great GTD app, I’d still recommend Things even though it doesn’t have OTA sync yet and there are many performance issues. It’s a simple app that does a simple thing.

Now, though, I’ve discovered that I have a problem getting motivated to do items on my list. Motivation is always difficult, but I tend to be paralyzed when I’m not sure how I will complete something. It makes me fail to even get started. Enter the Pomodoro Technique. I read this over the weekend and it made a huge difference on the first day. If you struggle with motivation and procrastination, this is the technique for you.

I’ve got 2 years worth of data in “Things” and my Pomodoro Technique is pretty new, so my first big question was how to use them TOGETHER. While this is neither pure GTD nor pure Pomodoro, I think I already have a great way to use Things better than I ever have and to stay motivated. If you’d like to try it, read on.

A Ruby Scoping Gotcha?

| Comments

Let’s take this basic class:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
class TestClass
  attr_accessor :one

  def my_method(branch=true)
    if branch
      puts "Do nothing to modify `one`"
    else
      puts "Modify `one` but it's a local variable"
      one = "test"
    end

    one # local variable
  end

  def my_non_modifying_method(branch=true)
    if branch
      puts "Do nothing to modify `one`"
    else
      puts "Do nothing to modify `one` either"
    end

    one #method call
  end
end
1
2
3
4
5
6
7
8
9
10
11
o = TestClass.new
o.one = "Value"

puts o.my_method
 => nil #might expect 'Value' if you're not paying attention
puts o.my_non_modifying_method #expects "Value"
 => "Value"
puts o.my_method(false)
 => "test"
puts o.my_non_modifying_method(false) #expects "Value"
 => "Value"

So remember, if you create any local variables anywhere in your method, even if they’re not called, they override the accessor methods and will give you results you’re not expecting. To get around it, make sure you always use self.accessor= to assign values when there is ambiguity.

Optional Heirarchal Checkbox Selection With Nested Attributes in Rails

| Comments

I had a process where I wanted users to fill out a survey which had hierarchal categories AND be able to specify some additional data for specific capabilities that the user had.

Now, you could easily do this for a small subset and hand-code every item, but I wanted a flexible survey system that allowed true hierarchy and generalized code.

Let’s start off with the basic survey and capabilities models and relationships:

1
./script/generate model Survey name:string
app/models/survey.rb
1
2
3
4
class Survey < ActiveRecord::Base
  has_many :survey_capabilities
  has_many :capabilities, :through => :survey_capabilities
end
1
./script/generate model Capability name:string parent_id:integer question:string
app/models/capability.rb
1
2
3
4
5
6
class Capability < ActiveRecord::Base
  has_many :survey_capabilities

  belongs_to :parent, :class_name => 'Capability'
  has_many :capabilities, :foreign_key => 'parent_id'
end
1
./script/generate model SurveyCapability survey:references capability:references answer:string
app/models/survey_capability.rb
1
2
3
4
class SurveyCapability < ActiveRecord::Base
  belongs_to :survey
  belongs_to :capability
end

Your first attempt at making a survey map to many capabilities will be something like this (formtastic):

1
<%= f.input :capabilities, :as => :check_boxes %>

But while that works on a basic level, it doesn’t work for capabilities that have a hierarchy and it doesn’t allow the user to specify additional data (i.e. answer a question about the capability).

So we’re going to need to accept nested resources. So we add this line to survey.rb:

1
accepts_nested_attributes_for :site_capabilities, :reject_if => lambda { |a| a[:capability_id].blank? || a[:capability_id].to_i == 0}, :allow_destroy => true

Now we need to recursively display hierarchal capabilities (If you show videos on your site, you might allow the user to invoke it, or require the user to invoke it, but if you don’t show videos, we don’t care about your invocation restrictions):

First, let’s make a quick way to show/hide enable/disable elements within a div:

public/javascripts/application.js
1
2
3
4
5
6
7
8
9
10
11
function toggle_fields(element_id, value) {
  if (!value) {
    Effect.SlideUp(element_id, { duration: 0.1 })
  }
  $(element_id).select('input').each(function(element) {if (value) { element.enable() } else { element.disable() }})
  $(element_id).select('select').each(function(element) {if (value) { element.enable() } else { element.disable() }})
  $(element_id).select('textarea').each(function(element) {if (value) { element.enable() } else { element.disable() }})
  if (value) {
    Effect.SlideDown(element_id, { duration: 0.1 })
  }
}

Now let’s create a helper that will set up the capabilities checkboxes and nested inputs:

app/helpers/survey_helper.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
def select_capabilities(f, collection)
    html = ""
    collection.each do |capability|
      survey_capability = f.object.survey_capabilities.select{|obj| obj.capability_id == capability.id}.first
      selected = !!survey_capability
      survey_capability ||= f.object.survey_capabilities.build(:capability_id => capability.id)
      f.fields_for :survey_capabilities, survey_capability  do |cap_form|
        html += cap_form.input :capability_id, :as => :boolean, :label => capability.name,
          :input_html => {:onclick => "toggle_fields('capability_#{capability.id}_details', this.checked);$('capability_#{capability.id}_delete').value = (!this.checked ? '1' : '0')"},
          :checked => selected, :checked_value => capability.id
        html += cap_form.input :_delete, :as => :hidden, :value => "0", :id => "capability_#{capability.id}_delete"
        html += content_tag :div, (capability.question.blank? ? "" : cap_form.input(:answer, :label => capability.question)) + (capability.capabilities.any? ? select_capabilities(f, capability.capabilities) : ""), {
            :id => "capability_#{capability.id}_details", :class => "details",
            :style => "display:#{selected ? "block" : "none"}"
        } if capability.capabilities.any? || !capability.question.blank?
      end
    end
    html
  end

There is a lot going on here. Let’s step through.

Keep in mind that we’re recursive, so first off, we’re passing in the collection of Capabilities we’re dealing with through the “collection” parameter, but that isn’t what we need to create in terms of nested form attributes–we need SurveyCapability objects for that, so we have to find or build them:

1
2
3
  survey_capability = f.object.survey_capabilities.select{|obj| obj.capability_id == capability.id}.first
  selected = !!survey_capability
  survey_capability ||= f.object.survey_capabilities.build(:capability_id => capability.id)

Then we create the fields_for section for nested form attributes and pass in the SurveyCapability we just created. Since we can specify that we want checkboxes here and specify the value, we make the checkbox the capability_id and make sure the ‘checked_value’ is the capability.id (it is ‘1’ by default).

1
2
  html += cap_form.input :capability_id, :as => :boolean, :label => capability.name,
                :checked => selected, :checked_value => capability.id

And while we’re at it, we’ll create a way to remove the relationship altogether if they uncheck the capability

1
   html += cap_form.input :_delete, :as => :hidden, :value => "0", :id => "capability_#{capability.id}_delete"

Finally, we build up the optional sub-question in the case of a click:

1
2
3
4
  html += content_tag :div, (capability.question.blank? ? "" : cap_form.input(:answer, :label => capability.question)) + (capability.capabilities.any? ? select_capabilities(f, capability.capabilities) : ""), {
            :id => "capability_#{capability.id}_details", :class => "details",
            :style => "display:#{selected ? "block" : "none"}"
        } if capability.capabilities.any? || !capability.question.blank?

And now we can add the :onclick option to the original checkbox so that appropriate inputs are toggled on click:

1
   :input_html => {:onclick => "toggle_fields('capability_#{capability.id}_details', this.checked);$('capability_#{capability.id}_delete').value = (!this.checked ? '1' : '0')"},

After we’ve got all that going on, we simply have to place it in the _form view:

app/views/surveys/_form.erb
1
2
3
<% semantic_form_for @survey do |f| %>
  <%= select_capabilities f, Capability.find(:all, :conditions => {:parent_id => nil}) %>
<% end %>

Instead of doing Capability.find…, let’s add a named scope to the Capability class:

app/models/capability.rb
1
2
3
4
class Capability
  named_scope :top_level, :conditions => {:parent_id => nil}
  ...
end

Yeah! Now you don’t have to change your controllers at all and you can have optional, hierarchal selection of checkboxes with nested attributes!

How and Why to Stop Multitasking - Peter Bregman - Harvard Business Review

| Comments

A study showed that people distracted by incoming email and phone calls saw a 10-point fall in their IQs. What&apos;s the impact of a 10-point drop? The same as losing a night of sleep. More than twice the effect of smoking marijuana.Doing several things at once is a trick we play on ourselves, thinking we&apos;re getting more done. In reality, our productivity goes down by as much as 40%. We don&apos;t actually multitask. We switch-task, rapidly shifting from one thing to another, interrupting ourselves unproductively, and losing time in the process.

via How and Why to Stop Multitasking - Peter Bregman - Harvard Business Review.