Applying Agile to Data Warehouse Design

Applying Agile to Data Warehouse Design

I’m in the process of reading a book on Agile database warehouse design titled, appropriately enough, Agile Data Warehouse Design and by Lawrence Corr.

While Agile methodologies have been around for some time – going on two decades – they haven’t permeated all aspects of software design and development at the same pace. It’s only in recent years that Agile has been applied to data warehouse design in any significant way.

I’m sure many Agile consultants have worked on projects in the past where they were asked to come up with a complete design up-front. That’s true with data warehouse projects too where a client’s  database team wanted the entire schema designed up-front – even before the requirements for the reports the data warehouse would be supporting were identified. What would appear to be driving the design was not the business and their report priorities, but the database team and their desire to have a complete data model.

While Agile Data Warehouse Design introduces some new methods, it emphasizes a common-sense approach that is present in all Agile methodologies. In this case, build the data warehouse or data mart one piece at a time. Instead of thinking of the data warehouse as one big star schema, think of it as a collection of smaller star schemas – each one consisting of a fact table and its supporting dimension tables.

The book covers the basics of data warehouse design including an overview of fact tables, dimension tables, how to model each and as mentioned, star schemas. The book stresses the 7-Ws when designing a data warehouse – who, what, where, when, why, how and how many. These are the questions to ask when talking to business to come up with an appropriate design. “How many” is applicable for the fact tables, while the other questions apply to dimension table design.

Agile Data Warehouse Design stresses collaboration with the business stakeholders, keeping them fully engaged so that they feel like they are not just users, but owners of the data. Agile Data Warehouse Design focuses on modeling the business processes that the business owners want to measure, not the reports to be produced or the data to be collected.

I still have a way to go before I’ve finished the book and then applied what I’ve learned, but so far, it’s been a worthwhile learning experience.

Uber. Eight years ago, the company did not exist and the word was simply a rarely used adjective of German origin meaning “ultra”, like an uber intellectual. Today, Uber has become one of the most successful startups in history and the word has become a commonplace verb in English parlance. Transcending to “verb” status puts Uber in the highly exclusive class of innovative business disrupters like Google and FedEx whose business names and processes have become synonymous with an action that didn’t previously exist but is now done on a regular basis.  Who today wouldn’t understand what actions you had taken if you said, “I quickly googled the address for the nearest drop-off spot and uber-ed over there so I could fed-ex my package out on time”?

Uber owns no cars, has no drivers, and has minimal fixed assets. Instead, they created an incredibly user-friendly software that improves aspects of the taxi ride industry we didn’t know needed improvement. Not surprisingly, the full legal name is Uber Technologies, Inc.  While the only technologies typically found in traditional taxi cabs are the decades old meter clicking away the increments of the cost of your ride, the Uber software provides new value to both the driver and the customer with useful information such as the location of both the driver and the customer, time estimate for pick-up, exact pricing, car options, driving directions, and much more.

By creating this simple way to get a ride, Uber has reached another pinnacle accomplishment whereby the creativity of its business model has become a noun: uber-fication.  According to Dr. Paul Marsden in his Reading Room article, The Uberfication of Everything, “…the real genius of Uber lies in a deep understanding of convenience – what it is and why it matters.  That’s what Uberfication is all about; pivoting your business to deliver on a core under-exploited consumer need – convenience”.

One thing that every startup has is a dream and a vision. But, let’s be honest, that simply isn’t enough to successfully build a booming new business like Uber. You need the right partners, you need money, and you need passion for the project at hand. We believe that we can help in all these areas, which lead us to formalize an offering exclusively for startups.

When I formed CC Pace nearly 36 years ago, I was driven by a vision of a new model for a consulting company – one where integrity and the client’s best interest were ingrained in the firm’s culture and successful delivery could almost be guaranteed by the quality, drive and teamliness of the employees who worked there. While my dream may not have been as wide-reaching as Uber, when I think back to that time, I just remember energy, excitement and that ‘anything is possible’ feeling. Over the years, we’ve been very fortunate to work with clients in all phases, from startups to Fortune 500 organizations—all of which we value a great deal. I get excited to work with clients of all sizes, but there is something about working with startups that brings about an energy that you can’t replicate in other environments. Being a part of someone else’s vision coming to life brings me right back to where I stood over 35 years ago and is an environment in which I’ve seen our project teams thrive in.

Our experience working with startups combined with our project teams’ passion has lead us to formalize an offering to help startups get off the ground with the right technology. To enable us to work on more of these type of efforts, we are officially launching a new risk/reward program for startups. Here, we are able to combine our technical prowess with our business acumen that result in a software component that fully and effectively supports the start-up’s vision.  The premise of our offering is to build the technological platform for your business with less cash required. In exchange for this discount, we agree upon a fair share of some downstream benefits of your startup reflective of the risk we take.

If you like the idea of maintaining control of your vision while paying less up-front to get the results you need, then I would love to hear from you. Interesting companies with challenging technology needs has been a driver for us for over 35 years. For this reason, we are confident that we have the ability to help better enable your dream. After all, it’s only a matter of time before the next “Uber” shocks the world.

For more information on the risk/reward program, check out our offering here.

In my personal experience working on various software development projects, the concept of team energy often appears to be either undervalued or benignly ignored by management teams. The reasons are many. First of all, the term may be confused with “team velocity” which is a relative measurement of a team’s average output or productivity. If the velocity appears to be at either predictable or positive levels, the management team may choose to believe that team energy is also at satisfactory levels. Organizations may attempt to boost employee morale by putting together team-building exercises and outing events. This macro approach may result in generating a perceived positive effect on team energy, thus obscuring the need for focus on individual teams.  So, what is team energy and why should managers consider devoting some attention to it?

When thinking in terms of team energy, one can look at it as building credit with each individual team. One can also look at it from an analogy of having a rainy day fund. From a management’s standpoint it is important to keep team energy in the positive territory. This helps ensure that the team will likely empower themselves to exceed expectations, as well as step up during times of crisis or high pressure situations. I have worked in high energy teams, in which members voluntarily pushed themselves past regular working hours to produce deliverables. These cases did not involve any direct increase in compensation or promotion. People naturally wanted to succeed because they possessed enough energy to do so. I have also witnessed the opposite, where a team’s energy was low, deliverables were in a perpetual state of tardiness and the backlog was steadily accruing bugs. Developers and testers did not feel empowered to succeed and entered a cycle of doing the absolute bare minimum to “get the management off their back”.

Science behind building teams 
Agile methodologies, whether Scrum or Kanban, prescribe various techniques that are focused on continuous improvement that may positively affect team energy. Regardless of whether an organization has truly embraced Agile, it is difficult to find managers that would oppose efforts in improving a team’s processes of delivering faster and at a higher value. After all, who is against a boost in productivity? There is a hidden psychological component to continuous improvement that has a causal effect on team energy. This component is more associated with the experience of individual team members.

Studies of team dynamics such as ones conducted by MIT’s Human Dynamics Laboratory, and documented in Harvard Business Review suggest that there is a science to building high performing and high energy teams. One of the keys is to focus on the human brain and social dynamics of a group. Your teams may be composed of introverts as well as extroverts and a wide range of personalities, but there is a common factor that seems to persist. The studies show that humans feel good when they achieve their goals and overcome obstacles. A human brain actually rewards its owner with extra levels of dopamine when a goal is achieved. When the team feels good more often than not, the team energy goes up. When the opposite occurs, team energy goes down. Therefore, focusing on small achievable goals not only helps the organization to shift focus of deliverables, but it also fosters this psychological benefit of achievement for each individual team member.

Measurements 
An organization may choose to periodically measure team energy. One way to achieve such measurement is through anonymous surveys. Usually this is done at a more enterprise level to gauge the overall organization energy. There is certainly value in doing that, but the effort is not focused and may not necessarily apply to teams. Small teams may not produce very accurate results. There may be disincentives to be frank when answering a survey because team members may feel singled out and fear reprisals from management. In addition, more introverted team members may choose not to “rock the boat”.  A more effective and team-focused approach is to have an Agile coach periodically take team energy measurements. An opportune time may be during team retrospectives, when a team is usually more receptive to be candid.  Most importantly, these measurements do not need to be secretly stored in a manager’s vault but should be shared with the team. Adding transparency to the team building and management process will not only increase team energy, but also foster leadership skills among the more proactive and extraverted team members.

Building a new software product is a risky venture – some might even say adventure. The product ideas may not succeed in the marketplace. The technologies chosen may get in the way of success. There’s often a lot of money at stake, and corporate and personal reputations may be on the line.

I occasionally see a particular kind of team dysfunction on software development teams: the unwillingness to share risk among all the different parts of the team.

The business or product team may sit down at the beginning of a project, and with minimal input from any technical team members, draw up an exhaustive set of requirements. Binders are filled with requirements. At some point, the technical team receives all the binders, along with a mandate: Come up with an estimate. Eventually, when the estimate looks good, the business team says something along the lines of: OK, you have the requirements, build the system and don’t bother us until it’s done.

(OK, I’m exaggerating a bit for effect – no team is that dysfunctional. Right? I hope not.)

What’s wrong with this scenario? The business team expects the technical team to accept a disproportionate share of the product risk. The requirements supposedly define a successful product as envisioned by the business team. The business team assumes their job is done, and leaves implementation to the technical team. That’s unrealistic: the technical team may run into problems. Requirements may conflict. Some requirements may be much harder to achieve than originally estimated. The technical team can’t accept all the risk that the requirements will make it into code.

But the dysfunction often runs the other way too. The technical team wants “sign off” on requirements. Requirements must be fully defined, and shouldn’t change very much or “product delivery is at risk”. This is the opposite problem: now the technical team wants the business team to accept all the risk that the requirements are perfect and won’t change. That’s also unrealistic. Market dynamics may change. Budgets may change. Product development may need to start before all requirements are fully developed. The business team can’t accept all the risk that their upfront vision is perfect.

One of the reasons Agile methodologies have been successful is that they distribute risk through the team, and provide a structured framework for doing so. A smoothly functioning product development team shares risk: the business team accepts that technical circumstances may need adjustment of some requirements, and the technical team accepts that requirements may need to change and adapt to the business environment. Don’t fall into the trap of dividing the team into factions and thinking that your faction is carrying all the weight. That thinking leads to confrontation and dysfunction.

As leaders in Agile software development, we at CC Pace often encourage our clients to accept this risk sharing approach on product teams. But what about us as a company? If you founded a startup and you’ve raised some money through venture capital – very often putting your control of your company on the line for the money – what risk do we take if you hire us to build your product? Isn’t it glib of us to talk about risk sharing when it’s your company, your money, and your reputation at stake and not ours?

We’ve been giving a lot of thought to this. In the very near future, we’ll launch an exciting new offering that takes these risk sharing ideas and applies them to our client relationships as a software development consultancy. We will have more to say soon, so keep tuning in.

I used to attend Agile conferences pretty frequently, but at some point I got burned out on them and the last one I attended was a 2007 conference in Washington, DC.  This year, when the Agile Alliance conference returned to the DC region, and I decided it was time to give them a try again.

It’s interesting to see how things changed since I last attended an Agile conference.  Agile 2015 felt much more stage managed than in previous years, with its superhero party, the keynotes making at least glancing reference to it (the opening keynote, Awesome Superproblems, appears to have been retitled for the theme, since all the references in the presentation were to “wicked” problems instead of “super” problems), and making one go through the vendor to get to lunch.  It also seemed like there were mostly “experts” making presentations, whereas previously I felt like there were more presentations by community members.  I have mixed feelings about all of this, but on the whole I felt that my time was well spent.  Although I didn’t really plan it, I seem to have had three themes in mind when I picked my sessions, team building, DevOps and craftsmanship.  Today I’ll tell you about my experiences with the team building sessions.

Two of the keynotes supported this theme: Jessie Shternshus’ Individuals, Interactions and Improvisation and James Tamm’s Want Better Collaboration? Don’t be so Defensive.  I’d heard of using the skills associated with improvisation to improve collaborative skills, but the Agile analogy seemed labored.  Tamm’s presentation was much more interesting to me.  I’m not sure he’s aware of the use of the pigs and chickens story in Scrum, but he started out with a story about chickens.  Red zone and green zone chickens, to be precise.  Apparently there are those chickens (we’re outside of the scrum metaphor here, incidentally) that become star egg layers by physically abusing other chickens to suppress their egg production.  These were termed red zone chickens, while the friendly, cooperative chickens were termed green zone chickens.  Tamm described a few unpleasant solutions (such as trimming the chickens’ beaks) that people had tried to deal with the problem, and ended up by describing an experiment whereby the the red zone chickens were segregated from the green zone chickens, with the result that the green zone chickens’ egg production went up 260% while only the mortality rate went up for the red zone chickens (http://blog.pgi.com/2015/05/what-can-chickens-teach-us-about-collaboration/).  Tamm then went on to compare this to human endeavors, pointing out the signs that an organization might be in the red zone (low trust/high blame, threats and fear, and risk avoidance, for example) or in the green zone (high trust/low blame, mutual support and a sense of contribution, for example), while explaining that no organization is going to be wholly in either zone.  He wound up with showing us ways to identify when we, as individuals, are moving into the red zone and how to try to avoid it.  This was easily the most thought provoking of the three keynotes, and I picked up a copy of Tamm’s book, Radical Collaboration, to further explore these ideas.  The full presentation and slides are available at the Agile Alliance website (www.agilealliance.org).

In the normal sessions, I also attended Lyssa Adkins Coaching v. Mentoring, Jake Calabrese’s Benefiting from Conflict – Building Antifragile Relationships and Teams, and two presentations by Judith Mills: Can You Hear Me Now?  Start Listening Instead and Emotional Intelligence in Leadership.  Alas, It was only in hindsight that I realized that I’d read Adkins’ book.  In this presentation, she engaged in actual coaching and mentoring sessions with two people she’d brought along specifically for the purpose.  Unfortunately the sound in the room was poor and I feel like I lost a fair amount of the nuance of the sessions; the one thing I came away with was that mentoring seems like coaching while also being able to provide more detailed information to them.

Jake Calabrese turned out to be a dynamic and engaging speaker and I enjoyed his presentation and felt like it was useful, but that was before I went to Tamm’s keynote on collaboration.  I did enjoy one of the exercises that Calabrese did, though. After describing the four major “team toxins”: Stonewalling, Blaming, Defensiveness and Contempt, he had us take off our name badges, write down which toxin we were most prone to on a separate name badge, and go and introduce ourselves to other people in the room using that toxin as our name.  Obviously this is not something you want to do in a room full of people that work together all the time, but it was useful to talk to other people about how they used these “toxins” to react to conflict.  In the end, though, I felt that Calabrese’s toxins boil down to the signs of defensiveness that Tamm described and that Tamm’s proposals for identifying signs of defensiveness in ourselves and trying to correct them are more likely to be useful than Calabrese’s idea of a “Team Alliance.”

The two presentations by Judith Mills that I attended were a mixed bag.  I thought the presentation on listening was excellent, although there’s a certain irony in watching many of the other attendees checking their e-mail, being on Facebook, shopping, etc., while sitting in a presentation about listening (to be fair, there was probably less of that here than in other presentations).  Mills started by describing the costs of not listening well and then went into an exercise designed to show how hard listening really is: one person would make three statements and their partner would then repeat the sentences with embellishments (unfortunately, the number of people trying this at once made it difficult to hear, never mind listen.  The point was made, though).  We then discussed active listening and the habits and filters we can have that might prevent us from listening well and how communication involved more than just the words we use.  This was a worthwhile session and my only disappointment was that we didn’t get to the different types of question that one might use to promote communication and how they can be used.

Mills’ presentation on Emotional Intelligence in Leadership, on the other hand, was not what I anticipated.  I went in expecting a discussion on EI, but the presentation was more about leadership styles and came across as another description “new” leadership.  It would probably be useful for the people that haven’t experienced or heard about anything other than Taylorist scientific management, but I didn’t find anything particularly new or useful to my role in this presentation.

In previous installments in this series, I’ve talked about what Product Owners and development team members can do to ensure iteration closure. By iteration closure, I mean that the system is functioning at the end of each iteration, available for the Product Owner to test and offer feedback. It may not have complete feature sets, but what feature sets are present do function and can be tested on the actual system: no “prototypes”, no “mock-ups”, just actual functioning albeit perhaps limited code. I call this approach fully functional but not necessarily fully featured.

In this installment, I’ll take a look at the Scrum Master or Project Manager and see what they can do to ensure full functionality if not full feature sets at the end of each iteration. I’ll start out by repeating the same caveat I gave at the start of the Product Owner installment: I’m a developer, so this is going to be a developer-focused look at how the Scrum Master can assist. There’s a lot more to being a Scrum Master, and a class goes a long way to giving you full insight into the responsibilities of the role.

My personal experience is that the most important thing you as a Scrum Master can do is to watch and listen. You need to see and experience the dynamics of the team.

At Iteration Planning Meetings (IPMs), are Product Owners being intransigent about priorities or functional decomposition? Are developers resisting incremental functional delivery, wanting to complete technical infrastructure tasks first? These are the two most serious obstacles to iteration closure. Be prepared to intervene and discuss why it’s in everyone’s interest to achieve this iteration closure.

At the daily stand-up meetings, ensure that every team member speaks (that includes you!), and that they only answer the three canonical questions:

  1. What did I do since the last stand-up?
  2. What will I do today?
  3. What is in my way?

Don’t allow long-winded discussions, especially technical “solution” discussions. People will tune out.

You’re listening for:

  • Someone who always answers (1) and (2) with the same tasks every day and yet says they have no obstacles
  • Whatever people say in response to (3)

Your task immediately after the stand-up is to speak with team members who have obstacles and find out what you can do to clear the obstacles. Then address any team members who’re always doing the same task every day and find out why they’re stuck. Are they inexperienced and unwilling to ask for help? Are they not committed to the project mission and need to be redeployed?

Guard against an us-versus-them mentality on teams, where the developers see Product Owners or infrastructure teams as “the enemy” or at least an obstacle, and vice versa. These antagonistic relationships come from lack of trust, and lack of trust comes from lack of results. Again, actual working deliverables at the close of each iteration go a long way to building trust. Look for intransigence on either the developer team or with the Product Owner: both should be willing to speak freely and frankly with each other about how much work can be done in an iteration and what constitutes Minimal Value Product for this iteration. It has to be a negotiation; try to facilitate that negotiation.

Know your team as human beings – after all, that is what they are. Learn to empathize with them. How do individuals behave when they’re happy or when they’re frustrated? What does it take to keep Jim motivated? It’s probably not the same things as Bill or Sally. I’ve heard people advocate the use of Meyers-Briggs Personality Tests or similar to gain this understanding. I disagree. People are more complex than 4 or 5 letters or numbers at one moment in time. I may be an introvert today and an extrovert tomorrow, depending on how my job is going. Spend time with people to really know them, and don’t approach people as test subjects or lab rats. Approach them as human beings, the complex, satisfying, irritating, and ultimately rewarding organisms that we actually are.

Occasionally, when I speak at technical or project management meet-ups, an audience member will ask, “I’m a Scrum Manager and I can’t get the Product Owner to attend the IPM; what should I do?” or, “My CIO comes in and tasks my developer team directly without going through the IPM; how do I handle this?” I try to give them hints, but the answer I always give is, “Agile will only expose your problems; it won’t solve them.” In the end, you have to fall back on your leadership and management skills to effect the kind of change that’s necessary. There’s nothing in Scrum or XP or whatever to help you here. Like any other process or tool, just implementing something won’t make the sun come out. You still have to be a leader and a manager – that’s not going away anytime soon.

Before I close, let me point out one thing I haven’t listed as something a Scrum Master ought to be adept at: administration. I see projects where the Scrum Master thinks their primary role is to maintain the backlog, measure velocity, track completion, make sure people are updating their Jira entries, and so on. I’m not saying this isn’t important – it is. It’s very important. But if you’re doing this stuff to the exclusion of the other stuff I talked about up there, you’re kind of missing the point. Those administrative tasks give you data. You need to act on the data, or what’s the point? Velocity is decreasing. OK…what are you and the team going to do about it? That’s the important part of your role.

When we at CC Pace first started doing Agile XP projects back in 2000-2001, we had a role on each project called a Tracker. This person would be part time on the project and would do all the data collection and presentation tasks. I’d like to see this role return on more Agile projects today, because it makes it clear that that’s not the function of the Scrum Master. Your job is to lead the team to a successful delivery, whatever that takes.

So here we are at the end of my series. If there’s one mantra I want you to take away from this entire series, it’s Keep the system fully functional even if not fully featured. Full functionality – the ability of the system to offer its implemented feature set to the Product Owner for feedback – should always come before full features – the completeness of the features and the infrastructure. Of course, you must implement the complete feature set and the full infrastructure – but evolve towards it. Don’t take an approach that requires that the system be complete to be even minimally useful.

If you’re a Product Owner:

  • Understand the value proposition not just of the entire system, but of each of its components and subsets.
  • Be prepared to see, use, and test subsets, or subsets of subsets of subsets, of the total feature set. Never say, “Call me only when the system is complete.” I guarantee this: your phone will never ring.

If you’re a developer:

  • Adopt Agile Engineering techniques such as TDD, CI, CD, and so on. Don’t just go through the motions. Become really proficient in them, and understand how they enable everything else in Agile methodologies.
  • Use these techniques to embrace change, and understand that good design and good architecture demand encapsulation and abstraction. Keeping the subsystems isolated so that the system is functional even if not complete is not just good for business. It’s good engineering. A car’s engine can (and does) run even before it’s installed into the car. Just don’t expect it to take you to the grocery store.
  • Be an active team member. Contribute to the success of the mission. Don’t just take orders.

If you’re a Scrum Master:

  • Watch and listen. Develop your sense of empathy so you “plug in” to the team’s dynamics and understand your team.
  • Keep the team focused on the mission.
  • If you want to sweat the details of metrics and data, fine – but your real job is to act on the data, not to collect it. If you aren’t good at those collection details, delegate them to a tracking role.

I hope you’ve enjoyed this series. Feel free to comment and to connect with me and with CC Pace through LinkedIn. Please let me hear how you’ve managed when you were on a supposedly Agile project and realized that the sound of rushing water you heard was the project turning into a waterfall.

In Part 1 of this blog series, I presented a high-level summary of the many different opportunities that Business Analysts (BA) can pursue in an Agile BA role, often resulting in new and exciting experiences. I also highlighted some of the differences between today’s “Agile BA” and the traditional “Waterfall BA”. Finally, I presented an interesting metaphor for today’s Agile BA, one of a Major League Baseball “utility player” (in this case, Jose Oquendo – a player who accomplished the rare feat of playing at every position during his 12-year MLB career.)

Part 2 of this blog entry focuses on the five key functional areas (aka “opportunities”) where I feel Agile BA’s can contribute or take outright ownership of a certain project task or responsibility. A key point to remember is that in order for these opportunities to present themselves, circumstances need to exist which ultimately depend on the dynamics of a project and the makeup of the particular team. Surely we don’t want to step on any toes or introduce team conflict. But if the opportunity is presented and a need has clearly been established, take the bull by the horns and run with these five opportunities:

  • Project Management
  • Product Management (aka the Product Backlog)
  • Testing
  • Documentation
  • Collaboration (with Project Stakeholders and Team Members)

PROJECT MANAGEMENT – WORK WITH (OR AS) THE SCRUMMASTER

On today’s project teams, Agile BA’s are usually best-suited to provide project management/ScrumMaster support whenever the need arises. And let’s be real – with PM’s and ScrumMasters constantly being pulled in several different directions, the Agile BA can tackle a numerous amount of responsibilities associated with this role.

In all likelihood, Agile BA’s have the experience necessary to handle many of the day-to-day responsibilities of a PM or ScrumMaster. The Agile BA can facilitate any of the recurring “events” as needed – the daily scrum (or “standup”), sprint planning, sprint review and sprint retrospective.

In many cases, the Agile BA is as close to (or even sometimes more engaged) with the project’s product backlog than the actual PM. This knowledge of the past, current and future state of the product backlog enables the Agile BA to assist with several project-related artifacts – for example, sprint and release burn-up/burn-down charts.

Successfully leading and delivering on many of these crucial project events and tasks not only contributes to the success of the team, but it also provides valuable on-the-job training. For Agile BA’s who want to eventually move into a PM or ScrumMaster role, this experience is invaluable.

THE “PROXY PRODUCT OWNER” – TODAY’S “PRODUCT OWNER” REALITY

Lately, it seems that a fully-engaged Product Owner is more of a luxury than a norm on today’s Agile projects. Agile BA’s can benefit from the potential subject-matter knowledge gained and added exposure by bridging this gap and acting as a “Proxy Product Owner”. In cases where a truly-dedicated Product Owner is not a reality, no one is better suited to step into this role than the Agile BA.

BA’s usually develop a solid rapport with the customer and can act as a liaison between the customer and the project team whenever needed. And as stated earlier, the Agile BA probably has the most experience working with the project’s user stories and backlog. On fast-moving development projects, many decisions are needed real-time, and waiting for answers from an absent Product Owner usually hinders the team’s progress.

TESTING – AFTERALL, WHO KNOWS THE STORY BETTER THAN THE BA?

Many Agile teams have already moved to this model, but for teams which have not, here is another opportunity. As previously mentioned, BA’s handing their work over to testers ‘waterfall-style’ is an outdated and inefficient practice. I have seen that 2-3 fully engaged Agile BA’s can efficiently handle the workload of 2-3 full-time BA’s and 2-3 full-time testers. Instead of separating requirements and functional testing tasks for a particular piece of functionality (e.g. user story), Agile BA’s focus on the user story as a whole – from origination (story creation) through implementation (fully-tested, potentially shippable product.) This is not to say that other methods of specialized testing aren’t needed, but in many cases, the best person to drive a user story to “done” is the Agile BA.

Automated testing has also become an invaluable practice on software development projects and provides yet another opportunity for Agile BA’s to contribute in the testing arena. With working knowledge of the current state of the application and product backlog, Agile BA’s have the capability to define and develop a project’s ongoing automated testing suite. Depending on the testing tools employed by the team, BA’s can “pair” with a technical resource (e.g. java developer). In this scenario, the developer handles the technical components of the automated testing suite while the BA designs, builds and manages the suite from a functional perspective.

COLLABORATION – BEFORE YOU KNOW IT, YOU’RE THE PROJECT’S “GO-TO” PERSON

I have included “collaboration” as an opportunity for Agile BA’s because collaboration, indeed, leads to opportunities. In my previous Waterfall experiences, BA’s didn’t collaborate much. In today’s Agile world, the Agile BA can really become a project’s “go-to” person. The collaboration piece also closely ties in with the previously mentioned PM/ScrumMaster and Product Owner opportunities.

For example, facilitating a sprint review or presenting a product demo provides invaluable experience and exposure. Sprint review meetings often include executives and/or stakeholders whom otherwise do not participate at all towards the project (basically, you see them once every two weeks). Leading these sessions provides direct communication with the “customer”, providing valuable feedback which can be relayed back to the team. Since entire project teams rarely attend these informational sessions, the team will start looking to you to provide the important feedback that we all value working on Agile projects. Personally, I have always looked forward to returning to the team room and have always appreciated team members asking, “so, how did it go!?”, after each sprint demo. At the same time, I am always glad to be able to provide that feedback to the team which we can use for future success.

DOCUMENTATION – CHANGE DOCUMENTATION FROM A TEDIUS TASK TO A VALUABLE COMMODITY

Even in today’s Agile world, most software development projects require some essential “dirty work”. The BA role has certainly evolved, but we should not completely abandon our roots. While we’ve all heard repeatedly (sometimes to our detriment) that the Agile Manifesto preaches valuing working software over comprehensive documentation, certain documentation can be critical to the success of projects.

I have seen that, if applied effectively, this basic Agile tenet not only reduces redundant documentation, but it helps teams focus on where documentation actually adds value to a project. Instead of documenting a requirement or process as part of an extensive list of deliverables promised six months ago (which will never be read or will become irrelevant), we document exactly what is needed, today. Most likely, information and processes which need to be documented aren’t even known at the outset of the project. Due to the fact that writing is a core skillset possessed by most BA’s, Agile BA’s are well-positioned to accomplish many of the documentation deliverables needed over the duration of a project.

NOW, GET TO WORK!

You’ve just finished reading this blog entry and your new sprint starts next week. It’s not entirely unrealistic that you can begin working in each of the areas mentioned in this blog over the next two weeks (if you haven’t been already). Offer to facilitate your upcoming sprint planning or review session. Ask the PM if you can contribute to the upcoming metrics reports (e.g. sprint/release burndown). If you aren’t already, start testing user stories – start at a high-level, ensuring that all acceptance criteria is met. Take a few hours, dive into and familiarize yourself with the product backlog. Offer to facilitate the sprint review and invite stakeholders who have become disengaged. And finally, document that process flow which has been taking up valuable whiteboard real-estate for the last several weeks (and really needs to be erased)!

Before you know it, you’re pursuing five completely new opportunities in a matter of two weeks. It might be similar to trying out three completely new positions on a baseball diamond. More importantly, you may even finally be able to explain exactly why Jose Oquendo was such a valuable player to have on a Major League Baseball team.

For the past 3 months we’ve had the pleasure of working with a charitable organization called the Ceca Foundation.

Ceca, which is derived from “celebrating caregivers”, was established in 2013 to celebrate caregiver excellence and “to promote high patient satisfaction by recognizing and rewarding outstanding caregivers”. They do this by providing employees of caregiving facilities with a platform for recognizing and nominating their peers for the Ceca Award – a cash reward given throughout the year. These facilities include rehabilitation centers, hospitals, assisted living centers and similar organizations.

CC Pace partnered with Ceca to build their next generation, customized nomination platform.

This was one of those projects that fills you with pride. First, for the obvious reason – Ceca’s worthwhile mission. Second, the not-so-obvious reason, which was the development process. It was a great example of why I enjoy helping customers build products.

The process

For various reasons, Ceca was under a tight deadline to get the new platform up-and-running for several facilities. The Agile process turned out to be a great fit, as it allowed for frequent customer feedback and weekly deployments to a testable environment. We developed the platform using high-level feature stories, rather than detailed specifications. This allowed the team to concentrate on the desired outcome, rather than getting caught up in the technical details. At times, we had to forgo a software-based solution in favor of a manual process. When you have limited resources and time, you have to make these types of decisions.

In February, after about 3 weeks, the Ceca Foundation launched the new web platform for one facility and then quickly brought on several more. There was immediate gratification for the team as we watched the nominations flood in.

The “feel good” story

What made this project successful and enjoyable at the same time? I’m reminded of the first value in the Agile manifesto – individuals and interactions over processes and tools. Some factors were technical but most were not:

  • a motivated and enthusiastic customer (Ceca)
  • a set of agreed upon features to provide the Minimum Viable Product
  • frequent collaboration with the customer
  • a cloud-hosted environment to provide infrastructure on-demand for testing and live versions
  • a software-as-a-service model that allowed us to quickly bring on new facilities

For me, it was Agile at its most fundamental: discuss the desired features; provide a cost estimate for those features; negotiate priority with the customer; provide frequent releases of working software.

Check out the Ceca Foundation for more information. You can see a demo of the software under “Technology​“.

As I mentioned in the introductory post in this series, an issue I frequently see with underperforming Agile teams is that work always spills over from one iteration into the next. Nothing ever seems to finish, and the project feels like a waterfall project that’s adopted a few Agile ceremonies. Without completed tasks at iteration planning meetings (IPMs), there’s nothing to demo, and the feedback loop that’s absolutely fundamental to any Agile methodology is interrupted. Rather than actual product feedback, IPMs become status meetings.

In this post, let’s look at how a Product Owner can help ensure that tasks can be completed within an iteration. As a developer myself, I’m focusing on the things that make life easier for the development team. Of course, there’s a lot more to being a good Product Owner, and I encourage you to consider taking a Scrum Product Owner training class (CC Pace offers an excellent one).

First and foremost, be willing to compromise and prioritize. An anti-pattern I observe on some underperforming teams is a Product Owner who’s asked to prioritize stories and replies, “Everything is important. I need everything.” I advise such teams that when you ask for all or nothing, you will never get all; you will always get nothing. So don’t ask for “all or nothing”, which is what a Product Owner is saying when they say everything has a high priority.

As a Product Owner, understand two serious implications of saying that every task has a high priority.

  1. You are saying that everything has an equal priority. In other words, to the developer team, saying that everything has a high priority is indistinguishable from saying that everything has medium or indeed low priority. The point of priority isn’t to motivate or scare the developers, it’s to allow them to choose between tasks when time is pressing. You’re basically leaving the choice up to the developer team, which is probably not what you had in mind.
  2. You’re really delegating your job to the developer team, and that isn’t fair. You’re the Product Owner for a reason: the ultimate success of the product depends on you, and you need to make some hard choices to ensure success. You know which bits of the system have the most business value. The developers signed on to deliver functionality, not to make decisions about business value. That’s your responsibility, and it’s one of the most important responsibilities on the entire team.

The consequence of “everything has a high priority” is that the developers have no way to break epic stories down into smaller stories that fit within an iteration. Everything ends up as an epic, and the developer team tends to focus on one epic after another, attempting to deliver complete epics before moving on to the next. It’s almost certain that no epic can be completed in one 2-week iteration, and so work keeps on spilling over to the next iteration and the next and the next.

Second, work with the developer team to find out how much of each story can be delivered in an iteration. Keep in mind that “delivered” means that you should be able to observe and participate in a demo of the story at the end of the iteration. Not a “prototype”, but working software that you can observe. Encourage the developers to suggest reduced functionality that would allow the story to fit in an iteration. For example, how about dealing only with “happy path” scenarios – no errors, no exceptions? Deal with those edge cases in a later iteration. At all costs, move towards a scenario where fully functional (that is, actually working) software is favored over fully featured software. Show the team that you’re willing to work with the evolutionary and incremental approach that Agile demands.

Third, be wary of stories that are all about technical infrastructure rather than business value. Sure, the development team very often need to attend to purely technical issues, but ask how each such story adds business value. You are entitled to a response that convinces you of the business need to spend time on the infrastructure stories.

At the end of a successful IPM, you as the Product Owner should have:

  • Seen some working software – remember, fully functional but perhaps not fully featured
  • Offered the developer team your feedback on what you saw
  • Worked with the developer team to have another set of stories, each of which is deliverable within one iteration
  • Prioritized those stories into High, Medium, and Low buckets, with the mutual understanding  that nobody will work on a medium story if there are high stories remaining, and nobody will work on low stories if there are medium stories remaining
  • A clear understanding of why any technical infrastructure stories are required, and what business problem will be addressed by such stories

Finally, make yourself available for quick decisions during the iteration. No plan survives its first encounter with reality. There will always be questions and problems the developers need to talk to you about. Be available to talk to them, face to face, or with an interactive medium like instant messaging or video chat. Be prepared to make decisions…

Developer: “Sorry, I know we said we could get this story done this iteration, but blahblah happened and…”

You: “OK, how much can you get done?”

Developer: “We would have to leave out the blahblah.”

You: “Fine, go ahead.” (Or, “No, I need that, what else can we defer?”)

Be decisive.

Next post, I’ll talk about what developers should concentrate on to make sure some functionality is delivered in each iteration.