The evils of over-engineering

My reply to a great Slashdot comment: http://it.slashdot.org/comments.pl?sid=264749&threshold=3&commentsort=0&mode=nested&cid=20164481

Ugh, Java frameworks...Somebody needs to drag the people who make these things in a room, erase their memories, and make them use what they have created...

The end result is you end up with what should be a fairly simple task (like OO-relational mapping) have 400 page manuals because it ends up doing every little thing that people want to do. In the time it takes you to choose the right framework, download and install the binaries, wade through the required config files, sift through the quickstart, and actually get familiar with how it is done, you could have just written and tested the tedious JDBC code to load and unload an object from the database.

You know, this seems to work both ways. Ever worked with a homebrew framework that didn't have good exception-handling or some type of passable tracing? The other end of the spectrum from these IBM Java guys are the people who still hardcode strings everywhere in the system and hand-code ALL of their stored procs and data access layer.

There are actually two problems here:

1. The Principle of Least Threshold


2. Enterprise Hubris

From the top, Javaheads like your above description are trapped in the Enterprise way of thinking. And hey they're building apps that will process more transactions in one day than many other small business apps will ever process. They have the right to be a little proud, the problem is that they're not very good at "scoping downwards". These frameworks all scaled up and came together, but now they have 400 pages of documentation and 12 different config files and everything is interdependent b/c they didn't want to lose functionality.

Even written a good business logic layer (BLL)? Nor have I, nor have like 90%+ of programmers, but it's ok, most of them never need it. The BLL is just useless abstraction for the typical database front-end that we develop. BLL just gets smushed in with Entities and UI b/c it's pointlessly complicated to develop on all but the biggest systems.

On the flip side, you get the Principle of Least Threshold, which is my term for the phenomenon where the perception of quality is limited by the highest level quality one has experienced. There are millions of small developers out there (VB 6 was the second most-used langauge at one point), who have never worked with any thing close to these monstruous Struts frameworks. So they don't understand them and they don't need half of the functionality and they can't spend 4 months figuring out how the system works, so they just build their own system one piece at a time and keep re-creating the parts of the wheel that they actually need.

I'm living this, our Data Objects framework doesn't have tracing. Nobody cares but me, b/c I'm the only one who knows what a godsend it is to have instrumentation code. Just yesterday, wasted 2-3 hours diagnosing a problem that would've taken 10 minutes if we'd had minimal tracing (queries?).

Of course, the real solution lies between the two camps. The Javaheads with the arcane infrastructure and architecture knowledge need to enable mechanisms to scale the complexity of their frameworks. The other guys need to start pushing from the other direction. But most of all the leaders in the field need to learn to embrace differences and change (man that sounds corny).

When anyone starts spouting off about good software or bad software, they must at least identify the scale of the software with which they are working (and when was the last time we saw that). Unacceptable mistakes in an Enterprise Framework are just ignored in a 12-user system. In fact nearly every piece of code posted to the web is unacceptable in an Enterprise environment, b/c it fails to catch errors or instance tracing or define attributes or use fully qualified types or validate inputs or even comment in a way that is compatible with JavaDoc/NDoc/Sandcastle.

A twenty line class in an Enterprise app could have 80 lines of associated "overhead code" that just plugs the class into the framework. Is this elegant and beautiful or just clunky and cumbersome? That answer could vary by system size, by programmer skill or even by personal taste. But no one seems to want to acknowledge this reality. The small guys call the Enterprise guys "clunky" and they in turn call the small guys "clumsy". So in the end they just ignore each other and throw rocks in each other's general direction and nobody stands and up and says "Hey guys, it's a continuum, it's not black and white, our solutions can only be inspirations for yours."

We need new leaders. The guys in my CoDe & MSDN magazines aren't sitting around handing out three different solutions of different complexity for the same problem. They're writing one-solution articles to solve problems that happen at all kinds of levels. How do you store user-specific data in a web app? Well in this article we'll talk about how you can solve this problem using my framework that doesn't actually work with your chosen method of storing data... if stuff isn't in an SQL Server-persisted Session then you're SOL... blah... blah... blah..." thanks guys. These are the same guys that write articles showing drag 'n drop data connections and then wave their hands about storing encrypted connection strings in the config files or using parameters to protect from injections.

Point is, we as programmers need to get off our high horses and actually learn about other people's problems so that we can learn to develop beautiful solutions to the problem rather than just solutions.

Loving what you do

I just got off the slashdot blog with a pretty powerful post.

Here's a link to the original post.
http://it.slashdot.org/comments.pl?sid=238919&cid=19559553

And here's a copy of my reply:

So why the hell are you not doing what you love everyday? Why do you spend your days doing stuff that doesn't fill the soul? If you really love your horse, then why don't you become a professional horse trainer? Then you can spend time with her everyday.

Sure your current job allows you to "help her have a better life.", but that doesn't mean that you couldn't give her a better life while doing something that "fills the soul". I mean, it's great to hear nuggets of wisdom like this one: "Company X, the best place in the world to work, will can your ass the moment the numbers do not add up for keeping you.", until you realize that that's pretty much how everything works out in life. (didn't take me 28 years in the field BTW) If my SO doesn't get her cut in the relationship deal she leaves, if my boss doesn't pay me enough or treat me well, I leave, if I can't carry my own weight at the company, then I get dumped, if I can't pick up the rebounds then coach benches me. It's pretty darn simple.

So I go to a job I love doing every day. And as a direct result of the fact that I love my job I'm not worried about getting canned b/c I'll find another. Just b/c some company dumps me doesn't mean that I can't find deep spiritual enjoyment in my work. Being fired doesn't mean I'm inadequate at what I do either, it's just a wrong time/place. Heck your beloved horse is going to die someday soon, probably before you will, how's that any different than getting "downsized" from a job that you love?

You may be a horse whisperer masquerading as an IT guy and for that you have my deepest sympathies. But I'm an IT guy and that's tattoed on my breast. It sounds really corny, but that's what I do, I'm not here to "die a slow death for some company", I'm here to throw in my 2 cents the best way I can find. The company is my means to that end, they make it easier to do what I'm good at. I have loves outside of work, but I surprise the most people and do the most good in this world by solving and automating complex processes and problems. If I'm "dying" at a company, then I'm not really contributing as best as I can and it's time for me to go.

So if you're pissed off at companies and the job in general after 28 years, then you're just in the wrong field or suffering some major emotional breakdown. Cash out now, take the retirement money and go train horses. I mean, isn't that what you're saving for anyways? So that you can "retire" and spend the days with your girl? So if that's where you want to be, if that's what you're good at, if that's what you're passionate about, then you'll find a way to make the money from working with horses. You said it yourself, this money is just a "means to an end", so why not use your means and find an end you enjoy? Why not pick a life and then make it happen instead of picking a job and then hoping that you can find a life?