Sunday, March 13, 2016

Pragmatic Thinking & Learning: Serendipity, Revelation, and Validation

I've been thinking about context a lot lately. In my previous blog post, I wrote an entire section about context and the lack of it in traditional introductory computer programming instruction.

Yesterday being a Saturday, I was trying to score some much-needed points with my wife by cleaning up the room above our garage that also serves as my home office. While doing that, I noticed Andy Hunt's book, Pragmatic Thinking & Learning: Refactor Your Wetware on one of my bookshelves. It had been sitting there just collecting dust since I bought it from a discount book store a few years ago. Somehow, I had never gotten around to reading it.

Since I had recently posted an article about learning, I thought now might be a good time to see what the book had to say about the learning process. As I started flipping through it, a realization soon dawned on me. Before I tell you more about that, I need to take you back a few years to set the context for what it was that I realized from skimming over the first two chapters of Pragmatic Thinking & Learning.

It was right around the end of 1999…

A series of fortunate events and the unfortunate one that started it all


I remember reading Kent Beck's Extreme Programming Explained: Embrace Change just a couple of months after its first printing in September 1999. I quickly became enamored with the ideas Kent presented in his book. It all made sense to me when he described the problems he had seen on software development projects and explained how the practices of Extreme Programming (XP) addressed them.

Kent's book resonated with me because I had also experienced many of the problems he described. At the time, I was well into my second year on a multi-year, multi-million dollar project that would eventually shut down in the middle of 2000. I would move on a few months before that but there were others who were not so lucky.

That wasn't my first project ever—I had about eight years of development work experience when I joined the project—but it was the first one of that size and the first one that I was assigned to after arriving in the United States with my young family in late 1997. The people I worked with on the project were great but the project was not. It was, in fact, the worst project I've ever been on in terms of overall experience and outcome, bar none.

The experience of having been on a failure of that magnitude had lasting effects on me. Fortunately, so did having a lifeline of hope for better things as described in Kent's book. It was during that difficult time that I resolved to find a job where I could do Extreme Programming or work in a very similar way. That resolution has made all the difference for me.

An Awakening to Better Things and Better Ways


It was also there in the pages of XPE, as Kent's book would soon be known, that I first read about automated unit testing and "test-first programming," a term that would be replaced in a few years by what we commonly know it as today: test-driven development or TDD. Although the concept was new to me back then, it made a lot of sense and I wanted to try it as soon as I could.

As is often the case on projects that are in trouble, informal discussions about software development practices had, ironically (or maybe naturally), become quite a regular occurrence. During one such discussion, one of my colleagues, Mark Halloran, suggested that I read XPE. He also suggested that I read Martin Fowler's Refactoring: Improving the Design of Existing Code since it was kind of related. So, I bought both books and read them.

Mark and I have continued to stay in touch over the years but he probably doesn't realize how much he changed my life, both on a personal and professional level, with those passing suggestions for further reading. If you're reading this, Mark, thank you. I owe a lot to you.

As I read the first chapter of Fowler's Refactoring and worked my way through the long (50 pages!) first example that introduces the basic ideas of refactoring, something happened to me. It was nothing short of an epiphany. I'm not just saying that for effect either because "something clicked in my head" just doesn't come close to describing it. It was more like the recent jet.com "mind blown" commercials where I just sat there, amazed and awe-stricken.

I can't remember exactly what I said back then but I'm sure it was out loud and something along the lines of "Oh my God! I've already been doing a lot of this stuff but I've never put them all together like that!" By "like that!" I would have been referring to the purposeful and methodical way in which Fowler transformed the code and improved its design with a series of small, incremental changes.

To see the code and design literally morph in front of me was, at that moment in my career as a programmer, a revelation of a new and endless stream of possibilities. It was like a veil that I never realized was there before had suddenly been lifted and I could see everything much more clearly. At that moment, I think that I felt that I had really grokked what Fowler was doing and it was mind-blowing.

Things that I know, they know, and then some


That first refactoring example also made me feel validated as a programmer. As I said, I had already been doing a lot of the little things that Fowler showed. Many of the refactoring techniques looked familiar to me. At least the scenarios to which they applied were. And yet, when seen in their entirety, the techniques made me look at the things that I did in a totally different light. Refactoring clarified many things for me and opened my mind to many more possibilities.

It felt good though, to know that I wasn't too far behind on the curve when it came to good programming habits and that I was on the right track with the kind of things that I was already doing. I had just fallen short on scope, method, and purpose.

Whereas I usually made the same kind of little changes in isolation, Fowler's examples showed me how those changes contributed to a bigger picture. Whereas I was making changes by referring to individual notes on best practice floating around in my head, Fowler clearly explained his motivation for making each change, the ramifications on the overall design, and the systematic manner in which he fluidly and seemingly effortlessly moved from one change to the next.

Previously, I made code changes that were based on vague notions of purpose drawn from experience and guided by intuition. I usually went by feeling and my methods were haphazard and relatively clumsy at best. Fowler showed me how I could do much better and apply the changes he described with a clarity of purpose, depth in understanding. and a systematic method of execution.

Where previously all I had were a bunch of random dots of knowledge and experience in my head, Fowler showed me how to connect all those dots so I could see the big picture.

I was thoroughly impressed. The man was obviously a minor programming deity at the very least and I could only aspire to approach that level of skill when I grew up as a programmer.

Common Experiences and Amazing Serendipity


Others who read these two books around the same time must have felt much the same way. In fact, the very first Java Users Group meeting that I attended—and I have to credit Mark Halloran again for taking me to it—was also around that time, in Cincinnati, Ohio.

The featured presentation in that CinJUG meeting was about refactoring and automated testing. It was given by a pair of presenters, one of whom was the late Jim Weirich, who would eventually go on to author the Rake build tool and help create the current RubyGems package manager.

At that time, Jim was working as a consultant for the Compuware Corporation, the same company that I would join just a few months later. Jim's presentation and his enthusiasm about refactoring and automated unit testing was infectious and it strengthened my resolve to learn and get good at these things.

I still think back and wonder at the sheer serendipity of everything that happened to me because, what are the odds, right? Were these all just strange coincidences? Fate? Destiny? Divine intervention? I don't know.

Maybe it was all just pure dumb luck but it seems like too many things fell into place for me just at the right times and right places. But then again, maybe that was my one big jackpot of a lifetime. Maybe that's why I haven't been able to win big in the lottery: I already got my big win at Lady Luck's wheel.

Déjà vu all over and over again


So now we're back to this past Saturday morning with me dusting off Andy Hunt's Pragmatic Thinking & Learning: Refactor Your Wetware and getting distracted from my cleaning duties. The point-making with the wife would just have to wait.

Again, I don't remember ever reading this book since I bought it a couple of years ago. I will often just buy books with the intent of reading them later when I have some free time. I have literally over two hundred books on three bookshelves in my home office and another hundred or so more in the basement. It's normal to see a few of these books strewn around in my home office. I go through a lot of wife points because of this, hence the need to gain some back by cleaning up a bit once in a while.

I won't even tell you about the stacks of magazines that I have around the house and the hundreds more electronic books that litter my hard drives. Let's just say that my intended reading list is quite long and it would be pretty optimistic to think that I'll ever get around to checking off everything on it.

I often tell my wife that I'm just getting all these books for future reference when I need them, not really to read them from cover to cover. She just shakes her head and gives me a knowing look. If you're married, yeah, it's that look.

Anyway, I was trying to see if there was anything in Pragmatic Thinking & Learning that I might be able to reference in my presentations about test-driven development. I have been giving presentations about TDD pretty regularly over the last few years and I'm working on doing more in the near future. It was on page 6 that I stopped and started getting a feeling of deja vu.

Here's what I saw:

Tip #1: Always Consider Context.

Immediately below that, Andy writes, "Put a copy of that up on your wall or your desktop, in your conference room, on your whiteboard, or anywhere you think alone or with others."

Ironically, if you follow Andy's advice, you'll probably have people looking at it and asking you, "What's this thing with 'context' all about? Where are you coming from and where are you going with this?" More on that later.

Anyway, when I read Andy's Tip #1, I thought, "Wow! I just wrote about context a week ago. That's nice." I thought it was a nice coincidence that I had been thinking about context and now find that Andy Hunt had been thinking along the same lines when he wrote the first chapter of his book.

Skimming through some more, I saw these section titles in Chapter 1:

  • Where We're Going
  • Journey from Novice to Expert
  • This is Your Brain
  • Get in Your Right Mind
  • Debug Your Mind
  • Learn Deliberately
  • Gain Experience
  • Manage Focus

Suddenly, I thought I heard the universe faintly calling out to me. "Wow," I thought again, "some of these are the same things I have in my TDD presentation!" The brain, how it works, how it's often wired differently from what it needs to be for TDD, retraining your brain, learning deliberately (I'm borrowing J.B. Rainsberger's term, "intentional practice"). These are all topics in the latest rendition of my presentation on TDD.

Flipping back to page 6, I see a highlight quote above Tip #1 that says, "Everything is interconnected."

You don't say, Andy. I think you and I might have interconnected somehow with these ideas.

Flipping back to page 1 of the chapter, I read "Welcome! Thank you for picking up this book."

WTH is going on here?! Did I actually read this book before and somehow forget that I did?

Quickly, I go over to my computer and bring up tmux and the session where I have the beginnings of my book on TDD. I finally signed up on LeanPub on Friday to start writing this book. I had worked out a draft of the introduction chapter and I had settled on a lead off line that says, "Thank you for your interest in this book. It wasn't easy, but I finally managed to gather years of…" Looking through my earlier commits in git, however, I see that the first few revisions started with "Thanks for picking up this book."

Now I'm looking around to make sure I'm not getting punked somehow. Is there some subliminal message coming out of the TV? Maybe I'm just losing it and having a very senior moment here.

I can't recall ever reading this book before but it seems that a lot of the things that have been on my mind lately are also in there. None of the illustrations look familiar but the topic headings are spot on with the things I've been writing down recently. I really doubt that it's a lapse in memory on my part but then again, I can't be 100% sure.

So I go on to Chapter 2 - Journey from Novice to Expert.

Mind Blown, Again.


Chapter 2 appears to be all about the Dreyfus Model of Skill Acquisition. The similarities continue because I mentioned Shu Ha Ri in my last blog post. I may have subsequently edited it out but it was there when I first posted the article. The Dreyfus Model and Shu Ha Ri are pretty aligned with each other so again, my thoughts seemed to be eerily in synch with Andy's.

Then, on page 30, I found this:

Tip #3: Know what you don't know

Oh. My. God.

That is, word for exactly freaking word, what I have on one of the slides in the presentation on TDD that I'm preparing for the Agile 2016 conference! I don't yet know if my submissions will be accepted—the program selection committee won't inform successful submitters until later this month—but I'm still polishing up my material regardless. What I do know is that I was referring to the cluelessness that many developers have about basic coding and design principles.

In my presentation, I talk about "Awareness" as being a key to success in TDD. Awareness leads to recognition and sensitivity which leads to understanding. Having awareness means that you need to know what you don't know.  I then go on to list some of the important things developers should at least be aware of when they are doing test-driven development: DRY, SOLID, SLAP, GRASP, Law of Demeter, Principle of Least Astonishment/Surprise, code smells, design smells, test smells, simplicity, etc.

If you don't believe that I had this before I saw Andy's Tip #3, I have recorded a number of practice runs of the presentation with Keynote and can show you the files. I swear, I did not have prior knowledge that this was in Andy's book! You know what would be really freaky though? It's if it turns out that Andy was thinking about that infamous Donald Rumsfeld news briefing when he wrote that tip. Because that's what I was thinking about when I added that slide to my presentation.

In his book, Andy's Tip #3 about knowing what you don't know is given in the context of skill distribution and the tendency of practitioners at the lower skill levels to overestimate their own abilities. Andy writes that "metacognitive abilities, or the ability of being self-aware, tends to be possible only at the higher skill levels."

Wow, novices seem to have everything stacked up against them. Not only are they oblivious to what they don't know, they also need to get better to become self-aware. But to get better, they need to be more aware. Now they're in a Catch-22! What's a clueless novice programmer to do then?

Yes, It's All Interconnected…


I think one of the reasons that Pragmatic Learning & Thinking is resonating so much with me is that it reaffirms the point I made in my last blog post regarding context and its importance in teaching programmers how to write good software. 

Without context, it's highly unlikely that novice programmers will recognize the connections between their choices in writing software and the related principles and values of good software development practice. As long as there's a disconnect there, they will continue to make poor decisions and stumble around in the dark. More mistakes will be made as a result. 

In his book, Andy explains how the Dreyfus Model of Skills Acquisition says that novices need context-free rules so that they can still effectively complete tasks. Unlike more experienced developers, novices need recipes to follow since they don't have much experience and intuition on which to base their decisions. 

Ok, fine. The scientists who said that were a lot smarter and probably knew what they were talking about, so I guess I'll have to take their word for it. But then again, maybe I'll do that with a few grains of salt.  

I still think that novices need to have an awareness of context, even if they don't fully comprehend what they see or are able to discern the extent of it and how it relates to the things that they are doing. 

Awareness will eventually lead to recognition and sensitivity. This in turn leads to understanding. Understanding is the basis for learning. 

I submit, therefore, that learning can only start when novices have some measure of awareness of context. Even just a little bit. Having recipes to follow may be fine for them in the short term but this can also make novices too dependent on having context-free rules. You've probably seen them before, the ones who seem to just blindly follow "best practice" and conventions without exception or questioning whether it's really the best thing to do in all cases.

If they're not careful to apply critical thinking to their recipes, novices who are too dependent on context-free rules can become like people hooked on drugs. They are no longer able to function without them. They can exhibit withdrawal symptoms and become confused and disoriented. We need to help wean novice programmers off of their context-free rules and challenge them to start recognizing the interconnection of parts in the larger context.

The Unnoticed Scenery Metaphor: We Can't Assume or Leave it to Luck


We can't just assume that novice programmers will eventually notice how things are interconnected within a context. I think we all know where wrong assumptions can lead us. We have to explicitly point out some of these contexts and interconnections to help novice programmers build up their awareness.

It's like when I'm out on a road trip with my family. My kids are usually sitting in the back with their faces in their phones. My wife and I often complain that they're missing a lot of the nice scenery that we drive past. 

Occasionally, my wife will give them fair warning of something interesting that's near the road up ahead. This gives my kids some time to stop what they're doing and look up to see what it is their mom thinks they'll find so interesting. They may still shrug and go back to their phones again, but at least they saw what it was their mom wanted them to see. At least they are now aware that there was something there, even if it was of somewhat dubious interest to them.

Novice programmers are like that, too. They're often too focused on the immediate details and seldom feel they have time to look up to see what's going on around them and elsewhere in the system. They fail to see the big picture while they're eyeballs deep in their work and just trying to get it done. This is how many opportunities are missed and mistakes are made.

Again, as I said in my last post, we need to help them out more or else they'll just keep flailing about aimlessly, with no definite movement towards growth and learning. We need to break their focus on the details that are often overwhelming them and ask them to look up at the big picture once in a while, so that they can start getting their bearings.

Not everyone will be as lucky as I seem to have been in the past. We can't just leave it to luck or assume that novices will see things when they need to see them. Luck shouldn't have to play a big part in learning and growing. I think Uncle Bob said something to that effect before. We need to make the path to learning more intentional and less serendipitous for novices and everyone else alike. We really need to do this. For everyone. I don't care what the Dreyfus brothers said.

Great(?) Minds Think Alike


I'm averse to using the word "great" to describe myself. It seems too presumptuous and a bit pretentious actually. I don't consider myself a great programmer. I guess I'm pretty good but I know there are many elite programmers out there who could run circles around me. While typing with their keyboards behind their backs. And playing an RPG in the background. While drinking Mountain Dew and… well, you get the point.

However, it is nice to know that once again, after all these years since I read XPE and Refactoring, I can still feel validated in the things that I've been doing to stay as good as I can be and keep improving my skills. 

I do know a lot about basic practices, principles, mindsets, perspectives, and patterns that I have used to keep my programs clean and myself out of trouble throughout the years. I've gotten to a point where many people are actually encouraging me to write a book on the stuff that I know, particularly about TDD, and share the joy when I'm doing TDD that I'm constantly talking about.

So, yeah, I've started on the book. I won't share what I have just yet, but I will eventually, when I'm good and ready. I'll probably have a few sample chapters up on LeanPub for public review and feedback in about a month or so. Hopefully, my submissions to the Agile 2016 conference will get accepted and I'll get to present some of this stuff in Atlanta come July. That should be a good learning experience, too.

Until then, I'll try to post more regularly on this blog to see what some of these ideas of mine look like on virtual paper.

Please stay tuned for more.

No comments: