Lower Your Quality Debt Ceiling

Recently Chris Sterling from Rally delivered a great talk "Managing Quality Debt in Practice" to the Seattle Software Test Group Meetup hosted by Dynacron Group in Kirkland. Though he made many points that resonated with me, one that I felt the strongest affinity with was the "No Defect" Mindset which basically means that you don't kick the "quality" can down the road and expect to fix it in the future.

All of us at one time (at least) in our careers have experienced the phenomena I call "creative defect management". You know, where you are reviewing the defects with the Business Owner, Project Manager and the developers and yesterday's Severity 1 defect magically becomes a Severity 3 because "... we are three days to the release date and we cannot fix this in time, we will patch it after the release..."

“For every [dollar] of competitive advantage gained by cuttng quality, it costs $4 to restore it; and software is an organizational asset and decisions to cut quality must be made by  executive management and reflected in the financial statements.” http://www.infoq.com/presenta2ons/agile_quality_canary_coalmine 

Ken Schwaber

How many of us have worked on applications that have tens or even hundreds of unresolved known issues that were logged over several previous releases. What I thought was a radical idea a few years back, abandon bug tracking tools, has become a major theme for me. In my view, a better approach is to write a "test" story when you find a bug, review it during backlog grooming and if it doesn't reach the level of being assigned to the next sprint then archive it because it is likely never to be fixed.  Perhaps you let it hang in the backlog for a a couple of sprints but if it is not a high priority to be assigned in a short period of time, then it likely will never be. 

Do you Fund Your DoD?

Establishing and funding your "Definition of Done" is critical to lowering your Quality Debt. On a recent client engagement we used the following DoD:

• Accepted by Product Owner 
• We do not work on stories that are not validated by the client
• The code has been peer reviewed
• Test Passes on all CI targets
• OS
• Browser
• Locales & Languages
• PMD, checkstyle, & find bugs successfully executed and issues resolved 
• Project Management tool (e.g Rally) has been updated
• Both positive & negative tests pass 
• Every commit has relative and meaningful comment
• Don't overload checkins 
• All broken build issues are fixed, no new feature checkins until build is GREEN
• All new classes, methods, and attributes have at least one usage
• No "unnecessary / zombie code"
• No "known" defects that have not been "resolved" either with remediation or a story in the backlog

This is not trivial, it requires effort and tenacity but by establishing, evolving and adhering to a DoD you will lower your Quality Debt in short order.

A Rose By Any Other Name

A Rose By Any Other Name

Recently I was involved in a LinkedIn discussion on the future of the SDET role. One participant stated:

Test automation (like spec) has made it so easy to write tests, that testing is really strong now. 90% of the "real" projects on GitHub have large test suites. Maybe this is just the startup scene in Seattle, but anyone using OSS is likely going to have a robust testing suite... And no dedicated testers or SDET.

 

While I applaud developers who actively practice TDD or who retroactively develop robust Unit tests, I do not believe this portends the end of the SDETs. Quite the contrary, I believe the more application development teams embrace these new tools and methodologies, the more prominent the SDET’s role becomes.

Apparently I’m not alone, in the book “How Google Tests Software” they do not have the title of SDET instead they have Software Engineer in Test (SET).

At Google, we have created roles in which some engineers are responsible for making other engineers more productive and more quality-minded. These engineers often identify themselves as testers, but their actual mission is one of productivity.  Whittaker, James A.; Arbon, Jason; Carollo, Jeff (2012-03-21). How Google Tests Software (Kindle Locations 551-552). Pearson Education (USA). Kindle Edition.

Again from “How Google Tests Software” the role of the SET:

SETs are partners in the SWE codebase, but are more concerned with increasing quality and test coverage than adding new features or increasing performance. SETs write code that allows SWEs to test their features.  Whittaker, James A.; Arbon, Jason; Carollo, Jeff (2012-03-21). How Google Tests Software (Kindle Locations 566-567). Pearson Education (USA). Kindle Edition.

Having been a (American) football player in my youth, I had always thought of QA & Test as the defensive team; preventing our opponents (bugs) from scoring (making it to production).  My analogy has morphed as I have become a strong proponent of Agile and BDD/TDD. I believe SDETs are the offensive line on the project team. The SDET’s charter is to “enable” the application team to achieve it’s shared objectives of quality and velocity.

True SDETs are well positioned because they can directly impact IT’s ROI. As IT departments re-invest in training, infrastructure, tools and processes, they would do well to take the offensive and invest in strong, talented SDETs.

Time Marches on...

May Day, May Day... It is the first of May and I along with the rest of the nation are looking for better days to come.
The last few weeks I have been working on Java code for a good friend of mine Bob Arasmith. Bob is a great guy and one of the best engineers I have ever worked with / for. Our friendship goes back more than 25 years (yea I know, we don't look that old) when we first met at Verbatim in 1983. But back to the subject of working with Java.
The first thing I that hit me is that although I have been around Java from a QA perspective since the early 1990s I really had not developed a serious application with it. Java is pretty straight forward I guess but after working with Ruby and Rails for the past 3 years Java feels a bit heavy. My first project was to parse some very large text files and create a database from the contents. This was definitely not rocket science but it did give me a good task to familiarize myself with Java. I am now working on a second project that will incorporate JSW or perhaps YAJSW for monitoring a collection of applications and services. Wish me luck.

By the way, I am still looking for consulting, contracting or a permanent position, any leads would be most welcome.

I made the leap...

I finally made the leap and bought a new 17" MacBook Pro, let me just say that I believe this is the best personal computer I have ever had. My old PC workstation will undergo the transition to virtualized desktop using VMWare ESXi in the coming weeks. Once I have confidence that I can configure and maintain a virtualized environment I will move on to the server, should be a fun but challenging conversion as I am not an expert systems administrator.

I have several projects going right now, however I am on the lookout for a new contract or permanent job should the right opportunity present itself; any leads would be appreciated.

Checkout
Danny Faught's new blog Software Alchemy
Harsha Elchuri's new site TestForge.net


Until next time...

Cucuber and Selenium

In my latest contract I have been developing automated tests for a Rails application using Selenium and Cucumber. I was reasonably comfortable with Selenium, or so I thought but Cucumber was another story, no pun intended.

Cucumber allows you to define tests in natural language such as:
Given I am a valid user
When I log in
Then I should see my todo list


Pretty cool but the real beauty is in creating a DSL that lets you spit out tests nearly as fast as you can think of them. This is where the true benefit of Cucumber + Selenium really shines. Yes it takes significant time to build up the DSL but over time, you have a vocabulary that allows you to not only test the application but in true BDD form, specify it as well.

More to come over the next few weeks.

Is Development finally getting SaaSy

I spent a few minutes the other day speaking with Scott Price and getting to know a little about his company LoadStorm and their impending Beta. Cloud computing and SaaS seems to be "the next thing" lately, everyone (with the notable exception of Larry ) is proclaiming it to be the next big thing and for the most part I agree.

I think back two years ago when I started my journey as an independent contractor, I thought then there was a definite market for SaaS based tools. New companies were emerging that were more "hosted" or "ASP" than SaaS but Salesforce.com opened the door and with Amazon EC2 and S3 SMEs can afford to go green. IMHO, virtualization and multitenancy are going to have a significant impact not only on the way software is distributed, but how it is developed as well.

LoadStorm is not trying to compete with the big boys, at least not yet, but as more similar products mature and become availble, the big boys may find their market share eroding. LoadStorm, I think falls somewhere between the DYI with Open Source and the Enterprise that want and can afford the top of the line offerings.

I am going to kick the tires on LoadStorm's product and see if I can find a lightning strike.

Cloud Computing

Yesterday I heard about a new company called Zuora.com from a recruiter so I looked them up. Interesting company that is all about SaaS; I guess SaaS is a bit passe these days, everyone seems to prefer discussing "cloud computing" but no matter what you call it, it is on-demand, it is efficient, it is green and I believe it is the future of a large percentge of computer applications.
I think we will see in the near future, a large shift to cloud computing. An excerpt rom K. V. Rao, President and one of Zuora's Founders that I think sums it up very well.

What you build, you should sell.

Our customers are realizing they are diverting precious resources into critical, but non-core activities like subscriber management, billing, and payments instead of dedicating them to building and enhancing differentiation in core products. They fall behind in product differentiation and competitive advantage leading to loss of market share and revenue when operational laundry lists and details are usurping time that could be spent on revenue-generating activities.

What you can't sell, you should buy (or preferably subscribe).

This becomes even more important in tough economic times, when businesses get lean and mean, regain focus on building differentiation in the core business, and divest non-core activities or outsource them.