Big vs. Small in IT

Posted this to Hanselman over the weekend, and it's probably worth a redux in my own blog.

The underlying issues being discussed is that Joel Spolsky ripped in to Uncle Bob Martin. A couple of summaries are available.

On to the reply:
@Scott, I think a valuable read relative to this topic is Malcolm Gladwell's book "Outliers". Bob makes that comment that "it takes 5 years for a developer to really become experienced", which is a statement backed up by the research in this book (10,000 hour rule). A lot of this podcast felt like "dancing" around the subjects discussed in the book.

The core of your podcast is dancing around the question of "What is good enough?"

The definitions of enough vary quite dramatically from project to project and person to person.

In particular the definitions will vary by the size of the project / company. And this is where Joel and Bob really work at odds. They are coming from two completely different realms of project / company. Let's say there are 4 classifications:

1. Small / Micro / Start-up
2. Medium, established niche
3. Large, thousands of users, typically a legacy base
4. Web-scale, millions of users (Google, MS)

Now Joel clearly comes from world #1.
Bob is from world #3 or 4.
Scott, you're from world #3 (& now #4).

For most developers in world #1 or 2, Bob's SOLID principles are at best guiding lights for solving the "next big bottleneck". (And there's a lot to be said about first having bottlenecks worth solving) There is simply not enough value generated by writing interfaces for the large mass of internal layers that are never exposed. Lots of objects simply don't need a "single responsibility", because the cost to change is very low. In my simple DB app, I don't extend more than 1% of my classes, so why do I worry about the OCP? In "Fog Creek" world, there's a lot of leeway in the term "Quality".

The cumulative sum of generating unit test for vast swaths of Maintenance screens simply doesn't justify the time spent on this screens. In fact, most people in world #1 & #2 seek out tools (like CSLA) quite specifically to generate Maintenance screens b/c they just don't justify any custom functionality.

Let's flip this into a practical application. (with no offense to Jeff Atwood here)

I can tell you right now that StackOverflow will not scale to 100x as it is currently architected. If the number of StackOverflow users grows by 100x the entire architecture will need to change. No, I've never seen the codebase, I've just listened to the podcasts (Hanselminutes and their own) but just listening to the problems it's obvious that things would need to change.

For example, to achieve 100x:

• They would need multiple databases.
  
• They would need specialized services to push data back and forth.
  
• Your rep score would be updated every 60 minutes by a service (not in real time).
• There would be a server responsible for updating RSS feeds. Data about updates would be pushed to the DB and replicated to bulk insert files on the RSS server to manage the reads against the primary.

As I said above the SOLID principles would be a guiding light in solving these problems.

• They would need to break out interfaces for "high-communication" features and build standard interfaces for communication (Web Services, Bulk Insert files, etc.).
• They would need to reduce connectivity dependence. Web servers would read from local DB copies and update themselves periodically.
  
• LINQ-to-SQL would suffer as tables became stored on different DBs. They would need an IOC model for managing connections at a table level so that they could point connections at the right spots.

Of course, in the process of growing by 100x, they would move from world #1 into #3. And the nature of these SOLID principles is that they become more valuable as the scope of the project grows and the available resources to build that project also grow.

I work for an ad network, so I live in world #2 & #4.

In one world we serve 100s of millions of impressions in a day. In another world, we operate a user interface with 100s of users. The definition of "Quality" varies dramatically between these worlds. In one world, we can build drag & drop UIs with MS AJAX and leverage the servers for good response times. In another world we write these massively scalable and distributed apps with lots of interfaces and IOC and robust, error-resistant code.

When Joel talks about Bob having "never written much code", he's talking about Bob not having written very much of the type of code that Fog Creek writes.

When Bob lays out the framework for a new project he simply has too many tools that do not provide value in Joel's world.

When Joel lays out the framework for a new project he ignores many of the tools in Bob's framework because he's never really needed them before.

And that's OK... I mean look at StackOverflow, it already has every dev on its mailing list :). It doesn't actually have to grow 100x to cover its potential market.

IT misunderstood again

So found this off the wire:

Study finds IT pay up in six areas, down elsewhere

Research firm's CEO says 'we've never seen anything like this before'

Here are the six areas:

1. Management / Methodology / Process
2. Database
3. Messaging and Communication
4. Architecture
5. Security
6. Networking

Now for skills that are down:

1. Application Development
2. SAP & Enterprise Development
3. Operating Systems
4. Web / e-commerce
5. Systems Networking

So in classic fashion, those trying to research the industry have fundamentally misunderstood the industry.  And I think that the problem is obvious, how do you actually "track skills"?

The report claims to track over 354 IT skills.  That's a lot of skills, of course, it could also just be a lot of fluff.  Anyone who's looked for a job in IT knows that the market is acronym-crazy.  It's also short on any form of acronym meaning. 

I mean, what the heck is the difference between "Networking" & "Systems Networking"? According to the stats, that difference is worth 3%+ in pay. 

What counts as "Web / e-commerce"?  It's obvious that general e-commerce is becoming commodity, that's to be expected.  But is "web" really worth less?  Facebook app development, Google App Engine, Microsoft's Azure, Amazon's AWS... these are all "Web" technologies.  Where do they fall?  Heck, Azure and AWS are also Operating Systems technologies.

So get that, skills in AWS have actually dropped in value. Right.... Huh?!?

I think the reason the CEO has "never seen anything like this before" is really that he's asking the wrong questions.  Trying to track IT pay by grouping across these massive skill sets is doomed to fail.  In fact, trying to track IT pay by any grouping of skill set is seriously flawed.

And the reasons are simple:

• There are too many IT skills and new skills are constantly being created
• Each skill has a lot of gradients
• Skills with different names can be very closely related
• Skills are constantly being picked up by experienced people

So how anyone plans to track that is beyond me.  Last year's "Database skills" are not this year's and they're not next year's.  Why you would group them under the same category each year and then pretend that they correlate demonstrates a misunderstanding about the way this industry works.

Enough to Retire?

Just came across the 10/10/4 model as presented at Mint.com. It demonstrates a pretty tenuous grasp of reality.

Comments below.

Jess:the life expectancy in the US is only 77 years according the the CDC

Is the current life expectancy or the life expectancy for people who are currently 20 and won't hit 77 for another 50+ years? Do we have a link for this?

Even if we change the model to 80 years (let's give medical science a little credit), that still leaves us with 45 years of working to 35 years of not working. That leaves us with 45 people working for 35 people not working.

Imagine that you're living in a (global) village with 80 people. 45 of those people do all of the hunting / gathering / farming / house-building. The other 35 people are either non-productive children or old people who sit around smoking pipes and eating food brought in by the other 45 people.

Either way.

If you save 10% / year for 45 years, and receive no effective pay raise between 20 & 65 (i.e.: pay raise = inflation). Then you would need ~3.25% real returns (that's returns above inflation) for basically the entire 45 years to meet the goal of having 10x your annual income. That doesn't sound like much, but here's some perspective.

Right now TIPS bonds are offering 0.7% real returns (they were offering 0.0% returns just a few months ago). Real Stock market returns over the last decade are into the negative. If you earn 0% real returns one year you have to make 7.5%+ real returns the next, just to make it up, that's not easy.

What's more, you're subject to a very critical period. At year 30 you have about 5x of your 10x. From year 30 to 45, you're only going to save 1.5x (putting you at 6.5x), which means that you're relying on 15 years of solid returns to make up that other 3.5x. If you have a 5 or 10-year drought, you could end up way short. And a 5 or 10-year drought is going to happen somewhere in those 45 years.

Finally, we made the very unsafe assumption that your "real income" doesn't change. Realistically, your income increases over time.

According to the 10/10/4 model: "...by the time you are 65, you will need 10x your income immediately prior to retirement to retire at the level you want.."

So you need 10x your final income, not your starting or even your average income.

Let's say you're 20 and making 30k today. Your "10x" number is 300k. You save 10% for 10 years and save just over 1 year's worth of income (say 31k). At the end of 10 years you make the big switch and find a new job earning 45k (again, no inflation). Awesome for you!

However, now your "10x" number is 450k, but you only have 31k in the bank. You're behind, right? You're at year 10, you should have at least 10% of your target number, but you only have 6.8%. So what if you continue to plow along for another 10 years and then get another pay raise to 60k? Now your 10x number is at 600k, you're 20 years in to the plan but you're way behind the curve. You should have saved 120k (+ interest), but you're nowhere close to that number.

And then you have to account for medical. If you're earning 60k but receiving 10k in medical benefits (may be low-balling in the US), you now need 700k in savings (not 600k).

You can see where I'm going with this. If you follow the 10% savings route and you also follow a normal pattern of increasing income throughout your career, the 10x goal is very difficult.
- Your increasing income makes previous savings insufficient.
- High medical expenses inflate your "10x" number.
- You need consistent returns well above inflation and you need them at the right times.

Don't get me wrong, I'm a savings advocate. I save 10% and then some in tax-advantaged accounts.

But I make no pretenses of making it to 10x without saving more, getting lucky or making some savvy investments.

Again, the model presented above is very broken. Readers can follow this at their own risk.

Extra Code Worst Software Waste?

Please Note: this is an older post I found in my archives. Originally dated August 13.

Another blog reply, this time to a devlicious blog post by Derek Whitaker.

re-creating massive amounts of code is expensive and time consuming

Umm... no. If codegen is expensive or time-consuming, you're doing it wrong. If members of the team have a difficult time using code, you're doing something wrong.

You're simply attributing the problems to the wrong source. Godegen is clearly not the issue here. The issue is implementation and training.

So you come up with this as a substitute:

Code you create now, but do not have a direct, immediate need for NOW is waste.

But the NOW timeline is really short-sighted.

Here's a classic. We've rigged our generated entity code to have special handling on any columns named "CreatedDate" and "ModifiedDate". Of course, lots of the primary entity tables we created simply didn't have these columns.

Now, to start with, we don't have a reporting interface or any business logic associated with these columns, so according to your logic adding these columns (i.e.: adding code) is a waste. And of course, that continues to be true until a problem arises that is solved by having these columns (except that they don't exist). I can't now go back and add these columns and suddenly get the data, but hey at least I didn't waste any time, right?

The problem with only handling immediate needs is that it opens the door to ignoring common failures.

Take my example. I know from experience that having these columns on specific tables will be useful. Maybe not now, but definitely soon. And we're not talking about full table-auditing, just a couple columns. We can get that information for a time usage so small that it's basically free and you're telling me that it's a waste because we're not going to use those columns now?

As always, good judgment and experience must be applied when making decisions about what code and features to build. I've spent a lot of time "paying for" mistakes that were wrought from the "we don't need it now" mentality.

In the case of our two dates (Created and Modified), we had to fight through support issues that relied on the non-existent data. A year ago, our product didn't have a support team, so nobody put them in, we didn't need them NOW. That cost us time and money for what should've been a "free" feature.

Turns out we'll likely need a full change history on a couple of the tables.. That's fine, we can't get that "for free", so it's OK that we didn't "waste time".

And what about features that don't get used? If I build 5 features and only 1 gets used, is that waste? Technically it is, but realistically, not every feature is going to be a hit. In fact, it's generally acknowledged that failures are a very important component of success. Maybe it just takes the implementation of 5 features to find one that "sticks", all of that "unused code" may be a "success" in some people's book.

I think you can get my point here, waste is not as simple as defining code that does or does not get used. You also have to factor opportunity costs for the code that is written and the "timing of the time". Hours are not equal, the hour when your datacenter is down has a very different value than the hour I spent writing this post.

Calling unused code the "Worst of the wastes" is difficult to swallow without a little deeper comparative analysis. But given the difficulty of even analyzing the value of unused code, I think you'd be in for a challenge there.