Big vs. Small in IT

Posted this to Hanselman over the weekend, and it's probably worth a redux in my own blog.

The underlying issues being discussed is that Joel Spolsky ripped in to Uncle Bob Martin. A couple of summaries are available.

On to the reply:
@Scott, I think a valuable read relative to this topic is Malcolm Gladwell's book "Outliers". Bob makes that comment that "it takes 5 years for a developer to really become experienced", which is a statement backed up by the research in this book (10,000 hour rule). A lot of this podcast felt like "dancing" around the subjects discussed in the book.

The core of your podcast is dancing around the question of "What is good enough?"

The definitions of enough vary quite dramatically from project to project and person to person.

In particular the definitions will vary by the size of the project / company. And this is where Joel and Bob really work at odds. They are coming from two completely different realms of project / company. Let's say there are 4 classifications:

1. Small / Micro / Start-up
2. Medium, established niche
3. Large, thousands of users, typically a legacy base
4. Web-scale, millions of users (Google, MS)

Now Joel clearly comes from world #1.
Bob is from world #3 or 4.
Scott, you're from world #3 (& now #4).

For most developers in world #1 or 2, Bob's SOLID principles are at best guiding lights for solving the "next big bottleneck". (And there's a lot to be said about first having bottlenecks worth solving) There is simply not enough value generated by writing interfaces for the large mass of internal layers that are never exposed. Lots of objects simply don't need a "single responsibility", because the cost to change is very low. In my simple DB app, I don't extend more than 1% of my classes, so why do I worry about the OCP? In "Fog Creek" world, there's a lot of leeway in the term "Quality".

The cumulative sum of generating unit test for vast swaths of Maintenance screens simply doesn't justify the time spent on this screens. In fact, most people in world #1 & #2 seek out tools (like CSLA) quite specifically to generate Maintenance screens b/c they just don't justify any custom functionality.

Let's flip this into a practical application. (with no offense to Jeff Atwood here)

I can tell you right now that StackOverflow will not scale to 100x as it is currently architected. If the number of StackOverflow users grows by 100x the entire architecture will need to change. No, I've never seen the codebase, I've just listened to the podcasts (Hanselminutes and their own) but just listening to the problems it's obvious that things would need to change.

For example, to achieve 100x:

• They would need multiple databases.
  
• They would need specialized services to push data back and forth.
  
• Your rep score would be updated every 60 minutes by a service (not in real time).
• There would be a server responsible for updating RSS feeds. Data about updates would be pushed to the DB and replicated to bulk insert files on the RSS server to manage the reads against the primary.

As I said above the SOLID principles would be a guiding light in solving these problems.

• They would need to break out interfaces for "high-communication" features and build standard interfaces for communication (Web Services, Bulk Insert files, etc.).
• They would need to reduce connectivity dependence. Web servers would read from local DB copies and update themselves periodically.
  
• LINQ-to-SQL would suffer as tables became stored on different DBs. They would need an IOC model for managing connections at a table level so that they could point connections at the right spots.

Of course, in the process of growing by 100x, they would move from world #1 into #3. And the nature of these SOLID principles is that they become more valuable as the scope of the project grows and the available resources to build that project also grow.

I work for an ad network, so I live in world #2 & #4.

In one world we serve 100s of millions of impressions in a day. In another world, we operate a user interface with 100s of users. The definition of "Quality" varies dramatically between these worlds. In one world, we can build drag & drop UIs with MS AJAX and leverage the servers for good response times. In another world we write these massively scalable and distributed apps with lots of interfaces and IOC and robust, error-resistant code.

When Joel talks about Bob having "never written much code", he's talking about Bob not having written very much of the type of code that Fog Creek writes.

When Bob lays out the framework for a new project he simply has too many tools that do not provide value in Joel's world.

When Joel lays out the framework for a new project he ignores many of the tools in Bob's framework because he's never really needed them before.

And that's OK... I mean look at StackOverflow, it already has every dev on its mailing list :). It doesn't actually have to grow 100x to cover its potential market.