Courtney Hemphill on Mixing Lean & Agile Development at GOTO Conf Chicago 2014 | GOTO 2014

Sure, and I agree with you, it does sound redundant. Everyone sort of thinks that they have that objective, but it's trickier than you think. And what we found, you know, we were a development shop since 2000, we've done Agile XP from the get-go, and we found that that process actually was what allowed us to deliver software as quickly as possible to our clients. What we found, though, was that oftentimes what you were giving your clients might not actually be what the market was looking for, and some of our clients sort of made that realization as well. We ended up partnering with some incredible design firms. We worked a lot with IDEO, Hot Studio, and they were UGtastic in helping us craft beautiful vision for the products. However, once you start taking sort of a vision and making it reality, as you can imagine, there are changes that need to be made, and interfaces are very specific to how users will engage with them, and they have a technical component. So we have decided, and we've been doing this for about six years, to bring sort of the design closer to the development. So the talk that I gave was how our teams of developers, and now with embedded designers that are on site at Carbon5 and that are part of our team, are working together, very iteratively on a weekly fashion, to both plan and come up with hypotheses each week that they're testing against, and that they are looking to gain an actual metric against at the end of the week for putting in front of users and really refuting or valid ating the hypothesis about that product or that feature set. So you're actually really doing experimental work, I mean, to say, okay, this is what I think is going to be the result, and then measure that and see if it actually was. And when you mentioned IDEO, that's a very interesting company to work with. If you don't know IDEO, they're a product design company. So they create things in meet space that people really use. How did that working relationship affect , you know, something that was more, we usually think of as intangible, like software.

Sure , it's a good question. And two points that I'd want to pull out there is , one, I think where you're sort of angling at is this idea of, you know, what does software mean in today's world, where it's really not just these intangible kind of digital interfaces, it really has a lot more than that. I will say, you know, one of the more interesting projects that we had with IDEO was for Genente ch, and it was for a TED conference. And we were literally putting together what was termed the genetic symphony of participants at TED. And so people would actually come to the event, they would swab using a DNA stick, and we would actually get the DNA for that particular person, and we would chart that based on sort of like the heretics of their DNA against sort of the musical composure. So it was a very sort of, you know, artistic endeavor, and it very much had a software component backing it. It was kind of the algorithms against the DNA and the musical components. We were creating a symphony of these attendees, but there was obviously an experience process that was designed by IDEO. So it was a very intense project, and I will not lie, but because it was backed by an event which happens, you know, at a moment in time. But it was an incredible engagement, and what was successful about it was every day we had communications with IDEO, every day we were sort of testing, and you know, the original vision was huge. And what we helped them to do was sort of take, you know, the easiest slices of the pie that we could do in order to make this a reality, and sort of dropping the bells and whistles and, you know, leaving some time to bring back the ones that were most critical, but trying to sort of take a stab at the full experience and making sure we get it done by that event date. And that's sort of what we have seen as to be a big success. Is that you can do a lot, but what you want to do is just enough to make it so that it is a success. And whether it's a success as an event, or as a product that you're coming to market with, that's what really is a company we're trying to optimize for. Right, and I'm trying to even imagine that musical components, because it's even a third dimension, that you've got a software product that you're delivering that's going to have an interface. People actually swabbing, and there's a physical thing that has to be done that I'm not sure how that would have even gone down. There is a lot of moving parts.

We did. IDEO brought in some musicians that were there to sort of set the score, and the scores were sort of backed, you know, each individual person has sort of a DNA profile, and there can be sort of groupings of these profiles. So what we were doing was, instead of trying to literally put a note to every potential DNA string, what we were doing is taking these DNA traits, sort of these segments of strings, and putting a note to them. So that way we were sort of minimizing the amount of sort of variables in the iteration, and that's how we managed that.

I would definitely like to have that. Well, the talk that I gave isn't about the Genentech stuff, it's about two other projects that I worked on. So we've worked on a lot of projects. This is sort of one that I've been pulling out from the IDEO. The talk actually is about two companies. One is called Bookrunner, and the other one is called Sharethru, and one was the data visualization of effectively sort of sharing of videos through different channels like Google+ and Twitter and what have you, and ways in which you can pull out sort of when a video went viral and who the influencers were. Bookrunner was a project that we did where we were comparing, we were trying to enhance the experience of kids that are in college that are trying to purchase textbooks, and allowing them to have more transparency into what their options were, you know, including Amazon, including Bookrunner, including the New Pursuit bookstore. So we were trying to do it, you know, when and where it mattered to them in the bookstore, and using SMS and mobile web kit to facilitate that endeavor. Yeah, because it could be very, I mean, I've seen sisters-in-law recently graduate, and it's ludicrous, the amount of money and requirements to buy books. It's crazy, I know, which is why I love that company, because they were being really honest and authentic about it. Right, so, you know, so you're describing how you created these products, and I'm sorry, this is very late in the day, so I'm trying to keep up, but you know, it's okay. So your presentation, you described... It's the process, yeah, the process. The presentation is, well, we're, what we did with both of these companies, and we were sort of optimizing this process during the course of these two projects, and we would sit down with the team, and this is the full team, this is, you know, developers, designers, and the product owner, it's really the balanced team at Carbon 5, and having those, you know, people, and that wasn 't just three, you know, there were more developers than just one, there were potentially more and more designers, but coming up with sort of the plan for the week, you know, particularly in the book writer case, it was, will people really understand these shelf tags, and how to get access to these prices, right? What's like the fastest way on an SMS flip phone to get to prices that we want them to see? So we would come up with how to go about this hypothesis. Well, we think it'll take, you know, two entering of numerals, and we want to engage them, you know, with a small bit of a URL, the QR, but whatever, you know, so we've come up with these hypotheses, some of which sort of had high product risk, or high technical risk, and so we were, you know, the talk is sort of how we were putting that in front of users as quickly as possible every week to make sure that we were creating a product that these users would use. The second part of the talk is really the anchor of our cadence, which is what we call it, this weekly iteration cycle, and that's when we bring in users and we would test them. So whatever we thought they would do, we had to validate that on Friday with the customer, and so we would be working and developing and designing an actual production-ready code base that we would then release, we were doing continuous release, but on Friday we would bring in, we called it five on Friday at Book Writer, right? We would , you know, have beers, bring in some students, you know, these are just, you know, we would have them try, you know, we'd give them gift certificates, and they would tell us, ah, this is great, or oh, this is not so great. Now, when you would do an experiment, if you had multiple experiments that seemed to be trending in maybe a not necessarily positive way, was there a strategy, did that ever happen, where you had to, you started on a path and then it just seemed to not be a positive, how did you...

Yeah, so we, I'll use ShareFu because that's actually the best case example of that, we went down a path for the first maybe three weeks that was sort of, we used D3, a job framework for visualization of data and interaction, and we , initially we did a spike that was pretty quick because we were, we used D3 and really harnessed sort of fake data and put it in front of users and sort of see if they could intuit what we were trying to get them to see. We failed miserably the first week. The good thing about sort of the way that we're coding is that we're putting into our code a very flexible sort of domain structure, and so at the bare minimum, you know, we know the users that we're aiming for, we know sort of their behavioral characteristics, and we know some of the domain language, and we're coding that sort of into the production code, and then we will do spikes. So the spikes are sort of opportunities for us to maybe not do as test-driven development, and they're in a branch, they're completely separate from the master, and that's the ones where we can really do some wild testing. If it works, then we fill out the test and we pull it back into the master. If it doesn't work, we can kind of let that spike go. The great thing about that is we have two opportunities. One is we can kind of go away from the norm and not be scared about, you know, messing up the core code base, and secondarily, when you have sort of an easy prototype to mess around with, you can actually real- time mess around with it, so we would, you know, mess with things in Chrome and then try it again with them. We also did this thing called hybrid prototyping , where we would take sort of the visual language in the interface, and we would put it on a whiteboard, and we would draw sort of, you know, a graph around it, and say, "Okay, use your fingers and mouse, talk us through what you would expect to see or what you would want to see. " And that saved us an entire week of coding, because they got sort of the visual language of what we had presented to them, before, and they could use that to then sort of say, "I would have preferred to have seen this," or, "This would have been more helpful. " Yeah, so, like, I click this, and now I'm expecting to see the shoes that I selected in my online store, and the ability to say this was large, small, you know, things. Whatever, yeah. I mean, for the marketers in this case, what they were doing was, it was hard for them to see immediately where they needed to click to find what they call the "influencers" in the channels. The people that will tweet about a video and make it go viral, and so they were saying, "I wish this node stood out more, I wish it was bigger, I wish it was more indicative of the amount of shares that, you know, launched from them. " So, that sort of influenced us to say, "Oh, we're just going to make that big. " And when you were describing experiments and writing these spikes, and then bringing that back in, and then backfilling tests, you know , there's a lot of people that think, "Oh, test driven development means I just sit down and write a test, and then check to see whether it did what I expected it to do. " And that's a really low-level thing, but it sounds like what you're doing was really test-driven, like literally the scientific word for test, where you just create something, and then say, "How did that?