January 26, 2016

Take 3, Scene 1: Product’s Role in DevOps

On this episode of The Innovation Engine podcast, we give listeners a sneak peek at a new audio endeavor from 3Pillar called Take 3. Take 3 will cover the trends, technologies, and tools that are changing the way products – and entire companies – are built in the digital age. Each episode – or scene as we’re calling them – touches on a key topic that we think is influencing the way technology-powered products are conceived, designed, developed, and launched.

For Take 3, Scene 1, host Julia Slattery is joined by Dan Greene and Jessica Hall to talk about Product’s Role in DevOps. Among the topics they cover are why DevOps has become one of the topics du jour in the world of software development, why having the ability to deploy to production multiple times a day isn’t for everyone, and the motivation behind Dan’s TechCrunch article that was published last year titled ‘What is DevOps?’.

About the Guests

Jessica Hall is the Director of the Innovate practice at 3Pillar, where she specializes in bringing new ideas to market through rapid prototyping. Previously she built the UX team at CEB and led the creation of the Newseum’s interactive exhibits and websites. Her work has been recognized by the Web Marketing Association, American Association of Museums, The Webby Awards, Time, Graphic Design USA, Forbes, and The Washington Post.

Dan Greene is the Director of the Advanced Technology Group in the US at 3Pillar Global. Dan has 18 years of software design and development experience, with software and product architecture experience in areas including eCommerce, B2B integration, Geospatial Analysis, SOA architecture, Big Data, and Cloud Computing. He is an AWS Certified Solution Architect who worked at Oracle, ChoicePoint, and Booz Allen Hamilton prior to 3Pillar.

Listen to the Episode

Interested in hearing more? Tune in to the pilot episode of Take 3 via the SoundCloud embed below.

Follow Take 3

If you listen to Take 3 and like what you hear, you can keep tabs on Take 3 via the 3Pillar website or its home on SoundCloud at https://soundcloud.com/take3pillar. There you can listen to Take 3, Scene 2 – An Introduction to Blockchain Technology to get the downlow on a technology that many think will revolutionize industries from finance to health care to insurance.

Subscribe to The Innovation Engine

If you’re interested in hearing more episodes of The Innovation Engine, you can download and subscribe on iTunesStitcher Radio, or SoundCloud. You can also download the podcast’s dedicated iOS app from the iTunes App Store for all 89 episodes in one place.

Read the Transcription

A transcription of the episode is given below.

Julia Slattery: So to start things off—Jessica, DevOps has been on the tip of a lot of people’s tongues in the software development space in the last few years. What are some of the forces at play that cause that kind of attention to be made very early in the product life cycle?

Jessica Hall: So I was pretty excited when I heard about DevOps because I remember doing monthly releases and having to be up at like 3 in the morning or 4 in the morning on a weekend, and doing testing on a release because the only time we could do a release was once a month. And then it had to be smoke tested and user acceptance tested and it was really painful and annoying. So what I’m excited about with DevOps is that I think it speaks to a really important business seed, which is that organizations are realizing that they need to be quicker, they need to release more, they need to be more responsive to their customer’s needs.

The hard part about DevOps, the more I think it really starts to muck around with product, is that we can do a lot more and we can release a lot more. We can do all of these releases—and people would hear about Amazon, who does some insane number amount of releases an hour, and think “Great, we should be doing all this stuff.” Not all of that is good, not all of that is creating value for the customer or creating value for the business. So where I think product needs to be involved is in really understanding what are we doing, feeding the right things into this DevOps machine and monitoring on the back – if we did something, well did it work, did it impact someone, is it creating noise, is it being effective? And then being able to take the right action, so a lot of product people would be like “Oh that is an engineering thing, they’re just pushing stuff out,” but you need to understand what’s being done and what the impact is so that you can really guide and steer the product that you are responsible for.

Dan Greene: And to bring another angle to it, you have to consider when you can release something to your customers every week. That means you have to be ready to release something to your customers every week, and that’s our true product challenge. There are a lot of analysis exercises that happen today that you would feel that you have the time to figure out what the customer wants, so let me work on this later. When there is something new going out in front of your customers every week, you have to be ready. You have to work with your product management teams and your engineering teams to know what is in the pipeline coming up so you can get customers ready for that feedback, so then you can pivot and change what’s going up in the next week. But you have to be willing to take that active role and that is definitely a challenge for a lot of our organizations today.

Jessica Hall: Some of them are going to get thrown off by agile in a sense, just like “Okay, well, I’ve definitely been told, as someone who came up through the User Experience side of the product, but we have to keep the development team busy, so just make that design.” I’m like “Well, but I’m working on this one and this one has more importance to the business, it’s more important to the users,” but they need to keep the development team busy, so they’re going to focus all that energy here. And I think what I worry about in DevOps is the same thing happens, like we’re doing all this stuff to show the executives that we’re busy, we are deploying this many times, we are creating this many features. Well, that doesn’t matter if those features aren’t important, that doesn’t matter if it is not improving the experience or solving problems for customers and it doesn’t matter to the business if it’s not doing things like helping with acquisition, retention, revenue, and referral.

So I think it’s kind of the signal and the noise problem of like, yeah awesome, you can do all of these things, doesn’t mean you should. And also, I think product needs to stand, becoming that kind of I’m setting the vision and the direction and empower engineering to help achieve that rather than you certainly can’t micromanage anything in a continuous delivery, releasing all the time situation.

Dan Greene: Yeah, I mean just to reiterate: because you can deploy all the time, doesn’t mean you should because some product features will take longer than others to build and get preliminary user testing on. It’s perfectly acceptable to have a delivery that has two features and wait a month and then you might have three deliveries, one every week. These patterns drive them by what your customer needs, not necessarily by what you could potentially execute upon. Again, it’s keeping the product in mind over the engineering.

Julia Slattery: So Dan, you recently contributed to DevOps at 3Pillar with the piece that was published in TechCrunch titled, “What is DevOps?” The piece has been shared more than 3,000 times and it led to the creation of an e-book of the same title. So since you literally wrote the book, or at least this book DevOps, what would be a thing or two you think would surprise people to know about it?

Dan Greene: The thing that I wanted people to understand about DevOps was that you have to define what it means for your organization and your product line. We have customers of ours that are pursuing a continuous delivery model where as soon as code is tested and validated, it flows to production. We have other customers that take a more managed approach to it, where they have the ability to automatically deploy, but they wanted that user safeguard. They happen to have a bad experience in the past with difficulties in deployment and rollback. So they intentionally instituted a manual check to make sure that this is good before releasing. The key is that as much of it is automated as possible. So that way, it removes human error.

I have worked more overnights than I care to admit in troubleshooting production issues because people who just go and get it done. Your rockstar operations people are actually typically your biggest risk because they go in and just fix it because you’re in an emergency firefighting mode. You want to bring that knowledge base earlier in the process, so you have a chance to dry run these capabilities in development and virtual environments.

The development of the cloud as an entity has really made it possible to quickly and efficiently stand up a new environment, try this deployment – this didn’t work, strip it down, try it again. Ten years ago, that would have been a nonsensical effort to do because the provisioning and the acquisition of all that hardware would have been absolutely prohibited. Now, it’s something that can be done, and that’s why this is an exciting time for DevOps, that these virtual capabilities have led us to move these automations earlier in the process so they get a lot more exercise, a lot more testing to make sure they are right, and it’s not the Friday before a launch, we’re subtly, all-hands-on-deck working through the weekend on something. And so I think people would read the e-book and read the post on TechCrunch and really get an understanding as to what is needed for their organization.

Jessica Hall: The operations person, if they leave, like in a lot of organizations that aren’t mature in this area, you are screwed. Because they’re the only ones who know how all these things happen, they are the only ones who can do it and they are the ones who have to scramble when it gets screwed up. So having mature DevOps doesn’t just enable business by allowing you to react quicker, it saves you from that problem, from that one person who knows how everything works leaving and then you are completely screwed.

Dan Greene: Absolutely. Another interesting model that’s come out of this is the model of failing forward. Instead of a typical model where you try to deploy to your environment and if something fails, it’s now this emergency critical thing to roll everything back to the way it was. And that’s typically a very difficult, dangerous process. With the additional automation and capabilities, instead of undoing the co-change, you can apply a fix to the code or to the database or to wherever the problem is in rapid iteration to solve it. You still keep your features, you just solve them quickly. You have a means of deploying that fix very rapidly.

Julia Slattery: Okay great. And finally, getting customer feedback and being able to make adjustments based on that feedback is the key for any development team working in an agile environment. What are some ways you all go about getting that feedback and then feeding it back into the user experience and product management process?

Jessica Hall: That’s a great question because I think it is what separates great products and great companies from people who aren’t being successful. A lot of folks will go through this whole process and then never engage. So it starts with a couple of things. One is there are metrics: what are you trying to drive? And if you are trying to drive more than three or four things, don’t bother, because you aren’t going to drive anything. So what are the three or four metrics that you are trying to drive that you think are most important for your customers and your business, and then focus on tracking those and sharing them across the organization so that everybody knows what we are aiming for and how we are doing against that. So those are the numbers, but then let’s focus on the people. So if we’re trying to drive something, let’s say it is a new feature and we push it out there and there is a whole bunch of people who used it once and then abandoned it, we need to go talk to them. We need to understand why they used it for the first time, why did they abandon it and why haven’t they been back, or who is using it, how is it effective, how are they getting to it, what are they doing with it. So whatever the outcome is, you need to be able to identify how customers are engaging with it and go talk to them, understand how they are using it, maybe do a screen-share and have them walk you through the process. And they’ll tell you, “Well, I’m doing this and that and oh there is this other thing that could really help you” but you didn’t see it so maybe we need to make some adjustments to the design.

The beauty of DevOps is that we can fix it if we find it, but if we don’t talk to people, we don’t look at the metrics, we’re never going to know what’s wrong and right with it and then we’re just shooting blind. And very quickly—and we see it pretty often with some of our newer customers—they will come in and they have big problems because they’ve been meandering “I need to close this deal so I’m going to build that feature and I need to close this deal, so I am going to do this feature.” And all of a sudden, they have a muddled mess of a product that no one likes to use because they didn’t have a disciplined approach for deciding what to do, or a disciplined approach for saying “Are we hitting what we wanted to do or do we need to change course?” And so the beauty of some of these releases, for things to go more smoothly, is there should be more time for user experience and product management to focus on the customers and what’s happening, because deployments are running smoother and the development team is able to get their things done, so you’re not having to do as much troubleshooting. And so you would be refocusing your efforts on what’s really happening and how can I really make it better.

Dan Greene: And I would like to add to that, one of the things that’s typically missing in a lot of products that we see is an automated monitoring of what’s functionally going on. I’m not talking about monitoring the actual hardware health or anything like that, that’s pretty standard nowadays, but what features are people using, kind of getting background metrics of what’s going on in the system from a functional perspective in addition to the user interviews. I think getting both of those together will give all of the teams the proper feedback to focus on the right features and the right capabilities. It’s just something that we find that’s missing. A lot of people focus on very simple monitoring in terms of performance or in terms of is the server up or down, but the is feature X being used and is feature Y being used, helps us drive those user interview questions, “Hey, we noticed that this feature over here is barely being hit although we spent hundreds of developer hours on building it. Let’s ask why. Let’s get that feedback so we can improve the feature and get it out there quickly.”

Jessica Hall: That’s where the metrics kind of help point you in the direction that you need, and  the quantitative stuff says “Hey go look over here, there might be something happening” and shows what’s really going on here, what didn’t we learn. Then working together to figure out okay, we learned X, now here is what we’re going to do and then you can do things like prototyping, AB tests, or multivariate tests. And hopefully things are running so smoothly and you’re getting things done, that you have a lot more time and a lot more mind bandwidth to make those things happen.