Lower Your Quality Debt Ceiling

Recently Chris Sterling from Rally delivered a great talk "Managing Quality Debt in Practice" to the Seattle Software Test Group Meetup hosted by Dynacron Group in Kirkland. Though he made many points that resonated with me, one that I felt the strongest affinity with was the "No Defect" Mindset which basically means that you don't kick the "quality" can down the road and expect to fix it in the future.

All of us at one time (at least) in our careers have experienced the phenomena I call "creative defect management". You know, where you are reviewing the defects with the Business Owner, Project Manager and the developers and yesterday's Severity 1 defect magically becomes a Severity 3 because "... we are three days to the release date and we cannot fix this in time, we will patch it after the release..."

“For every [dollar] of competitive advantage gained by cuttng quality, it costs $4 to restore it; and software is an organizational asset and decisions to cut quality must be made by  executive management and reflected in the financial statements.” http://www.infoq.com/presenta2ons/agile_quality_canary_coalmine 

Ken Schwaber

How many of us have worked on applications that have tens or even hundreds of unresolved known issues that were logged over several previous releases. What I thought was a radical idea a few years back, abandon bug tracking tools, has become a major theme for me. In my view, a better approach is to write a "test" story when you find a bug, review it during backlog grooming and if it doesn't reach the level of being assigned to the next sprint then archive it because it is likely never to be fixed.  Perhaps you let it hang in the backlog for a a couple of sprints but if it is not a high priority to be assigned in a short period of time, then it likely will never be. 

Do you Fund Your DoD?

Establishing and funding your "Definition of Done" is critical to lowering your Quality Debt. On a recent client engagement we used the following DoD:

• Accepted by Product Owner 
• We do not work on stories that are not validated by the client
• The code has been peer reviewed
• Test Passes on all CI targets
• OS
• Browser
• Locales & Languages
• PMD, checkstyle, & find bugs successfully executed and issues resolved 
• Project Management tool (e.g Rally) has been updated
• Both positive & negative tests pass 
• Every commit has relative and meaningful comment
• Don't overload checkins 
• All broken build issues are fixed, no new feature checkins until build is GREEN
• All new classes, methods, and attributes have at least one usage
• No "unnecessary / zombie code"
• No "known" defects that have not been "resolved" either with remediation or a story in the backlog

This is not trivial, it requires effort and tenacity but by establishing, evolving and adhering to a DoD you will lower your Quality Debt in short order.

A Rose By Any Other Name

A Rose By Any Other Name

Recently I was involved in a LinkedIn discussion on the future of the SDET role. One participant stated:

Test automation (like spec) has made it so easy to write tests, that testing is really strong now. 90% of the "real" projects on GitHub have large test suites. Maybe this is just the startup scene in Seattle, but anyone using OSS is likely going to have a robust testing suite... And no dedicated testers or SDET.

 

While I applaud developers who actively practice TDD or who retroactively develop robust Unit tests, I do not believe this portends the end of the SDETs. Quite the contrary, I believe the more application development teams embrace these new tools and methodologies, the more prominent the SDET’s role becomes.

Apparently I’m not alone, in the book “How Google Tests Software” they do not have the title of SDET instead they have Software Engineer in Test (SET).

At Google, we have created roles in which some engineers are responsible for making other engineers more productive and more quality-minded. These engineers often identify themselves as testers, but their actual mission is one of productivity.  Whittaker, James A.; Arbon, Jason; Carollo, Jeff (2012-03-21). How Google Tests Software (Kindle Locations 551-552). Pearson Education (USA). Kindle Edition.

Again from “How Google Tests Software” the role of the SET:

SETs are partners in the SWE codebase, but are more concerned with increasing quality and test coverage than adding new features or increasing performance. SETs write code that allows SWEs to test their features.  Whittaker, James A.; Arbon, Jason; Carollo, Jeff (2012-03-21). How Google Tests Software (Kindle Locations 566-567). Pearson Education (USA). Kindle Edition.

Having been a (American) football player in my youth, I had always thought of QA & Test as the defensive team; preventing our opponents (bugs) from scoring (making it to production).  My analogy has morphed as I have become a strong proponent of Agile and BDD/TDD. I believe SDETs are the offensive line on the project team. The SDET’s charter is to “enable” the application team to achieve it’s shared objectives of quality and velocity.

True SDETs are well positioned because they can directly impact IT’s ROI. As IT departments re-invest in training, infrastructure, tools and processes, they would do well to take the offensive and invest in strong, talented SDETs.

Time Marches on...

May Day, May Day... It is the first of May and I along with the rest of the nation are looking for better days to come.
The last few weeks I have been working on Java code for a good friend of mine Bob Arasmith. Bob is a great guy and one of the best engineers I have ever worked with / for. Our friendship goes back more than 25 years (yea I know, we don't look that old) when we first met at Verbatim in 1983. But back to the subject of working with Java.
The first thing I that hit me is that although I have been around Java from a QA perspective since the early 1990s I really had not developed a serious application with it. Java is pretty straight forward I guess but after working with Ruby and Rails for the past 3 years Java feels a bit heavy. My first project was to parse some very large text files and create a database from the contents. This was definitely not rocket science but it did give me a good task to familiarize myself with Java. I am now working on a second project that will incorporate JSW or perhaps YAJSW for monitoring a collection of applications and services. Wish me luck.

By the way, I am still looking for consulting, contracting or a permanent position, any leads would be most welcome.

I made the leap...

I finally made the leap and bought a new 17" MacBook Pro, let me just say that I believe this is the best personal computer I have ever had. My old PC workstation will undergo the transition to virtualized desktop using VMWare ESXi in the coming weeks. Once I have confidence that I can configure and maintain a virtualized environment I will move on to the server, should be a fun but challenging conversion as I am not an expert systems administrator.

I have several projects going right now, however I am on the lookout for a new contract or permanent job should the right opportunity present itself; any leads would be appreciated.

Checkout
Danny Faught's new blog Software Alchemy
Harsha Elchuri's new site TestForge.net


Until next time...

Cucuber and Selenium

In my latest contract I have been developing automated tests for a Rails application using Selenium and Cucumber. I was reasonably comfortable with Selenium, or so I thought but Cucumber was another story, no pun intended.

Cucumber allows you to define tests in natural language such as:
Given I am a valid user
When I log in
Then I should see my todo list


Pretty cool but the real beauty is in creating a DSL that lets you spit out tests nearly as fast as you can think of them. This is where the true benefit of Cucumber + Selenium really shines. Yes it takes significant time to build up the DSL but over time, you have a vocabulary that allows you to not only test the application but in true BDD form, specify it as well.

More to come over the next few weeks.

Is Development finally getting SaaSy

I spent a few minutes the other day speaking with Scott Price and getting to know a little about his company LoadStorm and their impending Beta. Cloud computing and SaaS seems to be "the next thing" lately, everyone (with the notable exception of Larry ) is proclaiming it to be the next big thing and for the most part I agree.

I think back two years ago when I started my journey as an independent contractor, I thought then there was a definite market for SaaS based tools. New companies were emerging that were more "hosted" or "ASP" than SaaS but Salesforce.com opened the door and with Amazon EC2 and S3 SMEs can afford to go green. IMHO, virtualization and multitenancy are going to have a significant impact not only on the way software is distributed, but how it is developed as well.

LoadStorm is not trying to compete with the big boys, at least not yet, but as more similar products mature and become availble, the big boys may find their market share eroding. LoadStorm, I think falls somewhere between the DYI with Open Source and the Enterprise that want and can afford the top of the line offerings.

I am going to kick the tires on LoadStorm's product and see if I can find a lightning strike.

Cloud Computing

Yesterday I heard about a new company called Zuora.com from a recruiter so I looked them up. Interesting company that is all about SaaS; I guess SaaS is a bit passe these days, everyone seems to prefer discussing "cloud computing" but no matter what you call it, it is on-demand, it is efficient, it is green and I believe it is the future of a large percentge of computer applications.
I think we will see in the near future, a large shift to cloud computing. An excerpt rom K. V. Rao, President and one of Zuora's Founders that I think sums it up very well.

What you build, you should sell.

Our customers are realizing they are diverting precious resources into critical, but non-core activities like subscriber management, billing, and payments instead of dedicating them to building and enhancing differentiation in core products. They fall behind in product differentiation and competitive advantage leading to loss of market share and revenue when operational laundry lists and details are usurping time that could be spent on revenue-generating activities.

What you can't sell, you should buy (or preferably subscribe).

This becomes even more important in tough economic times, when businesses get lean and mean, regain focus on building differentiation in the core business, and divest non-core activities or outsource them.

Addicted to Visine (use rcov to get the red out!)

So for years, being the QA guy that I am, I constantly was questioning if developers were bothering to unit test or complaining that their unit tests were not robust enough. What would I do if the roles were reversed? Well, they are not reversed as there is no QA team checking my application but still I was / am wearing the developer hat at times these days.
Big confession time.... it is harder than it looks to get into a test as you go rhythm. Test Driven Development is still not something I can claim. Not what you expect from a QA-Test guy huh? Well there you have it, I have taken the first steps on the road to recovery. I am a recovering non-testing developer. Actually I am well past that in the 12 steps, you see I develop in Rails and by habitually runing rcove I am developing a new addiction to green bars. I find myself getting anxious when I see any red these days; even when I can rationalize why I shouldn't worry. So now I develop a little application code, check the coverage then grap the Visine by coding the tests required to “get the red out”. Next step, rspec and TDD...

But what does that tell me about my years of QA experience? Was I just being a hipocrite? Perhaps a little but back when I was developing full time professionally, in C and assembler, the tools were not nearly as sophisticated. What I am really learning is that good practices, need to be habits and how do we get habits? By practice. So develop your addiction to visine and “get the red out”. It is a great “habit” for developers.

Bad Decisions Compounded

I was scheduled to return to Bangalore last Sunday but then came the attacks in Mumbai so I delayed it for a week. I don't think there is any real danger to me in B'lore but I suppose it is better to be safe.

But that isn't what I want to talk about. In my last posting, I expressed my dismay at the short sightedness of my client, penny-wise and pound foolish really are at play here. I have been trying to get my head around the quality aspects of the project and as one might expect, I have been reviewing countless bugs. Yesterday it struck me that I wasn't seeing the entire picture; sure enough, when I asked about it I was told that a few weeks back, the Excel spreadsheet that was being used to track issues had gotten too big so they archived all the closed issues. If you are like me that is like discounting all American history up until World War I. How can you understand a project when you ignore the first 50% of its history?

I am just dumbfounded that due to a "managerial" decision not to use a real issue tracking tool there continues to be bad project management on a project of this scale.

What is it that we don't understand

I have spent nearly two weeks in Bangalore working with a client and the outsourced engineering team they contracted for the project. Though I am contractually prevented from identifying the "players", my client is a household brand with a global footprint. Likewise the off-shore development partner is among the largest in the world. Together you would think that they would have this down to a science but you would be mistaken. Tools and processes are sophamoric and (IMHO) my client made a fundamental mistake by not having people embedded from day 1. This is a large project with 10s of thousands of staff hours and yet, untill recently, when they felt the project was in serious jeopady of missing key milestones and objectives, there was no direct representation of my client here in B'lore. The dev partner, eager for the potential long-term relationship, failed to demand reasonable terms, again IMHO. Now neither side is "happy"; people are working serious hours that may have been the norm for strart-ups 10-15 years ago, but neither of these are start-up. They are both companies billion dollar revenues but they short-change the effort regardless of the resources available to them. Some of it politics, some of it financial.

Now, as it often does, coincidence strikes. I exchanged email with another former collegue who is starting his own company. He has asked me to provide any insight I can give him on off shoring to India. My advice, particularly for a start-up with limited resources (both capital and time) is to seriously consider on-shoring. Sure from a finacial analysis on paper, India looks to be a much better deal but I am not sure they build in the true cost accurately. Travel, time, wear and tear on the staff that has to be on calls from 8:00 PM until midnight to discuss critical issues with the development team. The cost of doing business in India is rising, the cost of doing buisiness in the US is stabilizing or dropping in some instances. Plus, given the current financial situation, perhaps on-shoring serves a greater good.

Whatever my friend decides to do, my advice is to provide adequate resources, if he wants to off-shore, my council is that he have one of his own in B'lore and one of their staff at his site. Pay the price necessary to succeed, don't cut corners on tools or process, or it will come back to bite him in the a__.

A sign of the new economy?

Though I am not convinced it is a direct result, I find myself in an interesting position where I am being contracted by a "household name" company to help manage a project they have off-shored. I am not going to turn down the money but had they on-shored the project, in some other place than the high-tech hot-beds, they probably could have saved a considerable amount of capital.

As all two of the people who read this blog know, I am from Boise, Idaho. Boise is a pretty good sized town with a population > 200k (> 500k in the surrounding area). It has four distinct seasons, great lifestyle and good schools. The only negative I can say about it is that it still tends to be a bit conservative for me. All-in-all it would be a great place to live and raise a family. It has reasonable airline access to Portland, Seattle, Silicon Valley etc. and a good labor pool. Boise State and the University of Idaho bot have campuses there. The biggest names, at least in high-tech are HP and Micron. So why can't we on-shore to places like Boise, I am sure there are other cities that fit the bill as well? I don't know the answer but I think it may have as much to do with a false sense of financial economy than anything else. That 5:1 cost ratio we found in India during the latter part of the 1990's is now more like 2:1 today and shrinking. India is getting more expensive, in fact I have heard tales of India sub-contraction out to places like China and Viet Nam.

I believe in an open global economy but maybe we need to re-think our business strategies; move away from the short term view and look forward to a once again, expanding US economy.

Oh well, for me it is most of the next three months in Bangalore...

yet another post...

So here is another of my very infrequent posts. Not that anyone reads them. I sometimes need to write them anyway. Recently I have been focusing a lot of time and energy toward updating and improving my testing tools & skills. In my current contract I am attempting to automate the testing of a very sophisticated web app using Selenium RC - Ruby. This app's UI is nearly all GWT and that is proving to be the biggest challenge. From my understanding, GWT does not generate element IDs by default and thus I find myself having to leverage Selenium IDE to get the xpaths for each element, and believe me they are not pretty.

On another front I am diving into RSPEC & RCOV which I find to be a double edged sword. Perhaps as my skills improve it will have a big upside but for now it is time consuming. I cannot remember where I saw it, on someones blog but the key words were "test your own code" so by default I would probably not maintain 100% coverage which contradicts most of my 20+ years of experience. The thing I really like about RSPEC is the ability to tease requirements out. Being able to have a client give me a terse rudimentary description of what an object should or shouldn't do seems like a big win. The hard part, just as with any testing, is just the commitment of doing it. It always seems that you are spending too much time developing specs and not enough time writing code. I have not broached the stories yet, the specs themselves are keeping me busy.

On a personal note, I turned 50 this past Wednesday, do I feel older? Not really but as scary as it sounds, I see more of my parents in me than I ever thought I would, both the good and the not-so-good. Nature or nurture, whatever the reason there it is.

It has been a long, long time...

It has been several months since I last blogged, so i guess I should get off my ass and put something up. Not that anyone is waiting on pins and needles for a new post from BearBlu. As the saying goes, "the only thing constant is change". So currently I am working two contracts; the bigger of the two is with AgileMind in SF. AgileMind is doing some very interesting work in the area of web based education using the SCORM statndard for Learning Management Systems. Working with them is like a re-union of sorts. The VP is Andy Felong, with whom I worked with at Walmart.com; the Sr. Architect is Dorothy Gantenbien; we crossed paths at Profluent, a start-up that never reached a full start. By the way, Dorothy is an aspiring photographer, check out her work here. Finally, the configuration and build master is Trudy Reese; Trudy and I have worked together three times previously, Calico, Profluent and MS2.

I am also busy developing a small Rails app for a long-time friend who is starting her own accounting business with two other partners. I am not sure how much I can say at this time so I won't say anything more other than it is a great project to develop in Rails 2.0 and I am thoroughly enjoying it.

Speaking of Rails, I have to give a plug to Ryan Bates; he produces Railscasts and I think they are terrific. In a 3-5 minute podcast he shows me more than I could hope to learn in an hour of trial and error. If you are like me and trying to ramp-up on technique, I would most definitely recommend giving Ryan a listen.

Well back to getting my web site cleaned up and working on the Rails project. I'll try not to take so long before my next post.

What is the saying? "The only thing that is constant is change..." Sure enough. I had been working with a young start-up called BookRenter.com with a couple of really bright young engineers but alas the business wasn't generating sufficient revenue to keep me on so... I took a contract with a company in San Francisco. It feels kind of like a reunion of sorts, the VP, Andy Felong was at Walmart.com while I was there and the Sr. Architect is Dorothy Gantenbein who I worked with some time ago at another start-up. To top it off, Trudy Reese is joining them at the end of the month.

My new client is a provider of educational tools, curriculum development, assessment and not sure what else. They are re-engineering the application and needed someone to help bootstrap the QA effort, so here I go again. It will be fun and pay the bills but I haven’t forgotten that I have my own project to get back to. Oh and to make it a bit more chaotic, I am taking another short contract with a former client in Denver to help them evolve their QA process as they grow their team in Bangalore.

Today I am boning up on Xen virtualization. I want to run multiple servers on a Dell PowerEdge SC1430. Greg Joe and I experimented with this last year and now I want to roll it out for production in BearBlu's environment.

I will let you know how it goes...

The only constant is change...

Well, my one-time development partner started a "paying job" today so it is somewhat less crowded at BearBlu Software's HQ. I guess the biggest issue for me is that I have to pick up the things he was doing as well as continue the fit-n-finish of the application.

Today I am going to focus on UI Usability. Gone are the "good-old-days" of text-based client/server UIs. Today it is Ajax/Javascript if you want to have a serious web application. I have spent some time absorbing this technology and still I am just scratching the surface. In-line edits and drop-down selectors are not difficult to deploy; I even added a real-time digital clock and date field but I am sure there could be so much more given the time to explore.

But today.... Today I want to focus on the UI customization to allow users to use labels that work for them. I have looked for a Rails Recipe but did not find one. I guess I'll have to roll my own.

The Countdown Begins...

It all started harmlessly enough; I had been recently separated from the company that was paying my bills so I decided to build a new application that I felt had value... but what? As is usually the case you stick to what you "think" you know. I was a "QA / Process Guy" not a technology guru. So I started thinking about the tools I used daily and what I liked and disliked about them; could I make them better? Perhaps not but I might be able to make them more usable for the average end user and surely I had enough experience to help others. With the help of my partner in crime, we set off on this journey.

The world of developing web applications is very different from the client-server and embedded applications I developed more than a decade ago. New technologies have emerged, SOA, Ajax and the explosion of Open Source tools and applications presented me with a personal challenge to learn to program again.

In the next few weeks, my partner and I will be revealing the Beta version of our efforts. We hope you check it out and let us know what you think of it and if you can see value in it.