VSTS DevOps 101: Continuous Integration (CI) with Automated Builds
This is the second in a series of posts about our experience using Visual Studio Team Services (VSTS) to build a deployment pipeline for an ASP.NET Core Web Application. Future posts will cover release artifacts and deployment to Azure cloud services.
It’s assumed that you have an ASP.NET Core project set up in VSTS and connected to a Git repository.
See the previous blog post for details.
The goal of this post is to set up your ASP.NET Core project to automatically build and run Unit Tests after every commit to source code repository (I.e. Continuous Integration).
Here is a video summarizing the steps described below:
Select the project from our previous post.
- Select “ + New Definition”
- Select ASP.NET Core Template
- Enter any Name that can help you to identify this build.
- Select an appropriate Agent queue. In this example, we will use Hosted VS2017. Use this agent if you’re using Visual Studio 2017 and you want the VSTS service to maintain your queue.
- To simplify the process, use the default value for other fields.
- Go to “Trigger” and Enable continuous integrations. This will cause a build to automatically kick off after every code commit.
- Save the definition.
Adding a Test Project to the Solution
- Open the solution (from the previous post) in Visual Studio.
- Add a new Test Project.
- Select .NET Core > Unit Test Project. We will name this project MyFirstApp.Tests. Note: the default build definition will look for test projects under folders that end with the word, “Tests“. So, make sure that a new folder that contains this word is created when you add your Unit Test Project.
- For a proof of concept, we are going to write a dummy Test Method. Enter the following code in UnitTest1.cs
- Rebuild the project.
- Commit the changes locally and push it to the remote source repo.
- Back in VSTS, you can see that a Build has been triggered.
- Click on the build number #2018XXXX.X to view the details of the build. Normally this will take a few minutes to complete.
- Ensure all of the steps passed. You can click on each step to view log details.
We’ll demonstrate how to deploy builds to different environments, either via push-button deployment or triggered automatically after each build (I.e. continuous deployment).
This is the first in a series of posts about our experience using Visual Studio Team Services (VSTS) to build a deployment pipeline for an ASP.NET Core Web application. Future posts will cover automated builds, release artifacts and deployment to Azure cloud services.
The purpose of this post is to show how to create a new Visual Studio Team Services, or VSTS, project, set up a Git code repository and integrate everything through your Visual Studio IDE.
- Microsoft Account (to be used to create VSTS Account) https://account.microsoft.com
- Visual Studio Team Services (VSTS) Account (Free version – 5 users)
- Visual Studio IDE, Recommended 2015+ (Free version – Community Edition)
Here is a video summarizing the steps described below:
Creating a New Project
- If you haven’t already done so, create a VSTS account using your Microsoft Account. Your Microsoft Account is the “owner” of this VSTS account.
- Log into VSTS. By default, the URL should look like (VSTS account name).visualstudio.com. For this demo, The VSTS account that we will be using is https://ccpacetest.visualstudio.com, and the Microsoft user is CCPaceTest@outlook.com
- Create a new project. This Microsoft Account is the “owner” of this project.
Connecting VSTS Project to Visual Studio IDE
- There are many ways you can work on your project. A list of supported IDE’s is listed in the dropdown below. For this demo, we will use Visual Studio 2017 Professional Edition.
- Click “Clone in Visual Studio”. This will prompt you to launch your Visual Studio.
- If this is your first time running Visual Studio, you will be prompted to log in to your Visual Studio account. To simplify the process, log in to the Microsoft Account that you used to create the demo project previously. This should give you the admin access to the project.
- Go to View > Team Explorer > Manage Connection. Visual Studio will automatically show you a list of the hosted repositories for the account you used to log in. If you are following the previous steps, you should be able to see HelloWorld.Demo project. If you are not seeing a project that you are expecting to have access to, verify the account you log in to or check with the project owner to make sure you are given the right permission.
5. Connect to the project.
6. If this is the first time you are accessing this project in your local environment, you will be prompted to clone the repository to your local git repository.
Initial Check In
- Within the Team Explorer, click the Home button. Under the “Solutions”, select “new…”. Using this method will ensure the solution is added to the right folder.
- For this Demo, we will create a new project using the ASP.NET Core Web Application template. The solution name doesn’t have to be the same as the project name in VSTS, but to avoid confusion, it is recommended to use the same name.
- Once the solution is created, go back to Team Explorer and select “Changes”, you should be able to view all the items you have just added.
- Enter a comment and click “Commit All”. This will update your local repository.
- To “Push” these changes to the remote Repository, click “sync”, and finally “push”.
- You can verify this by logging into your VSTS, go to “Code” and you should be able to see all of the codes you have just checked in.
Collaborating with other Team Members
- You can add additional members to your project by going to the Settings > Security
- Please note that:
a: If your VSTS account is Azure Active Directory backed, then you can only add email addresses that are internal to the tenant.
b: You must add email addresses for users who have “personal” Microsoft accounts unless your VSTS account uses your organization’s directory to authenticate users and control account access through Azure Active Directory (Azure AD). If new users don’t have Microsoft accounts, have them sign up.
‘Agile’… ‘Lean’… ‘Fitnesse’… ‘Fit’… ‘(Win)Runner’… ‘Cucumber’… ‘Eggplant’… ‘Lime’… As 2018 draws near, one might hear a few of these words bantered around the water cooler at this time of year as part of the trendy discussion topic: our personal New Year’s resolutions to get back into shape and eat healthy. While many of my well-intentioned colleagues are indeed well on their way to a healthier 2018, many of these words were actually discussed during a strategy session I attended recently – which, surprisingly, based on the fact that many of these words are truly foods – did not cover new diet and exercise trends for 2018. Instead, this planning session agenda focused on another trendy discussion topic in our office as we close out 2017 and flip the calendar over to 2018: software test automation.
“SOFTWARE TEST AUTOMATION?!?” you ask?
“SERIOUSLY – cucumbers and limes and fitness(e)?!?”
This thought came to mind after the planning session and gave me a chuckle. I thought, “If a complete stranger walked by our meeting room and heard these words thrown around, what would they think we were talking about?”
This humorous thought resonated further when recently – and rather coincidentally – a client asked me for a high-level, summary explanation as to how I would implement automated testing on a software development project. It was a broad and rather open-ended question – not meant to be technical in nature or to solicit a solution. Rather, how would I, with a background in Agile Business Analysis and Testing (i.e. I am not a developer) go about kickstarting and implementing a test automation framework for a particular software development project?
This all got me thinking. I’ve never seen an official survey, but I assume many people employed or with an interest in software development could provide a reasonable and well-informed response if ever asked to define or discuss software test automation, the many benefits of automated testing and how the practice delivers requisite business value. I believe, however, that there is a substantial dividing line between understanding the general concepts of test automation and successfully implementing a high-quality and sustainable automated testing suite. In other words, those who are considered experts in this domain are truly experts – they possess a unique and sought-after skill set and are very good at what they do. There really isn’t any middle ground, in my opinion.
My reasoning here is that getting from ‘Point A’ (simply understanding the concepts) to ‘Point B’ (implementing and maintaining an effective and sustainable test automation platform) is often an arduous and laborious effort, which unfortunately, in many cases, does not always result in success. At a fundamental level, the journey to a successful test automation practice involves the following:
- Financial investment: Like with any software development quality assurance initiative, test automation requires a significant financial investment (in both tools and personnel). The notion here, however – like any other reasonable investment – is that an upfront financial investment should provide a solid return down the line if the venture is successful. This is not simply a two-point ‘spike’ user story assigned to someone to research the latest test automation tools. To use the poker metaphor – if you are ready to commit, then you should go all-in.
- Time investment: How many software development project teams have you heard stating that they have extra time on their hands? Surely, not many, if any at all. Kicking off an automated testing initiative also requires a significant upfront time investment. Resources otherwise assigned to standard analysis, development or testing tasks will need to shift roles and contribute to the automated testing effort. Researching and learning the technical aspects of automated testing tools, along with the actual effort to design, build out and execute a suite of automated tests requires an exceptional team effort. Reassigning team tasks initially will reduce a team’s velocity, although similar to the financial investment concept, the hope is significant time savings and improved quality down the line in later sprints as larger deployments and major releases draw near.
- Dedicated resources with unique, sought after skill sets: In my experience, I’ve seen that usually the highest rated employees with the most institutional/system knowledge and experience are called on to manage and drive automated testing efforts. These highly rated employees are also more than likely the most expensive, as the roles require a unique technical and analytical skill set, along with a significant knowledge of corresponding business processes. Because these organizational ‘all-stars’ initially will be focused solely on the test automation effort, other quality assurance tasks will inherently assume added risk. This risk needs to be mitigated in order to prevent a reduction in quality in other organizational efforts.
It turns out that the coincidental internal automated testing discussion and timely client question – along with the ongoing challenge in the QA domain associated with the aforementioned ‘Point A to Point B’ metaphor – led to a documented, bulleted list response to the client’s question. Let’s call it an Agile test automation best-practices checklist. This list can be found below and provides several concepts and ideas an organization could utilize in order to incorporate test automation into their current software testing/QA practice. Since I was familiar with the client’s organization, personnel and product offerings, I could provide a bit more detail than necessary. The idea here is not the ‘what’, as you will not find any specific automation tools mentioned. Instead, this list covers the ‘how’: the process-oriented concepts of test automation along with the associated benefits of each concept.
This list should provide your team with a handy starting point, or a ‘bridge’ between Point A and Point B. If your team can identify with many of the concepts in this list and relate them to your current testing process and procedures, then further pursuing an automated testing initiative should be a reasonable option for your team or project.
More importantly, this list can be used as a tool and foundation for non-technical members of a software development team (e.g. BA, Tester, ScrumMaster, etc.) in order to start the conversation – essentially, to decide if automated testing fits in with your established process and procedures, whether or not it will provide a return on investment and to ensure if you do indeed embark down the test automation path, that you continue to progress forward as applications, personnel and teams mature, grow and inevitably, change. Understand these concepts and when to apply them, and you can learn more about cucumbers, limes and eggplants as you progress further down the test automation path:
To successfully implement and advance an effective and sustainable automated testing initiative, I make every effort to follow the following strategy which combines proven Agile test automation best-practices with personal, hands-on project and testing experience. As such, this is not an all-inclusive list, rather just one IT consultant’s answer to a client’s question:
For folks new to the world of test automation and for those who had absolutely no idea that ‘Cucumber’ is not only a healthy vegetable but is also the name of an automated testing tool, I hope this blog entry is a good start for your journey into the world of test automation. For the ‘experts’ out there, please respond and let me know if I missed any important steps or tasks, or, how you might do things differently. After all, we’re all in this together, and as more knowledge is spread throughout the IT world, the more we can further enhance our processes.
So, if you’ll excuse me now, I’m going to go ahead and plan out my 2018 New Year’s resolution exercise regimen and diet. Any additional thoughts on test automation will have to wait until next year.
Boy this summer flew by quickly! CC Pace’s summer intern, Niels, enjoyed his last day here in the CC Pace office on Friday, August 18th. Niels made the rounds, said his final farewells, and then he was off, all set to return to The University of Maryland, Baltimore County, for his last hurrah. Niels is entering his senior year at UMBC, and we here at CC Pace wish him all the best. We will miss him.
Niels left a solid impression in a short amount of time here at CC Pace. In a matter of 10 weeks, Niels interacted with and was able to enhance several internal processes for virtually all of CC Pace’s internal departments including Staffing, Recruiting, IT, Accounting and Financial Services (AFA), Sales and Marketing. On his last day, I walked Niels around the office and as he was thanked by many of the individuals he worked with, there were even a few hugs thrown around. Many folks also expressed wishes that Niels’ and our paths will hopefully soon cross again. In short, Niels made a very solid impression on a large group of my colleagues in a relatively short amount of time.
Back in June I gladly accepted the challenge of filling Niels’ ‘mentor’ role as he embarked on his internship. I’d like to think I did an admirable job, which I hope Niels will prove many times over in the years to come as he advances his way up the corporate ladder. As our summer internship program came to a close, I couldn’t help reminiscing back to my days as a corporate intern more than 20 years ago. Our situations were similar; I also interned during the spring/summer semesters of my junior year at Penn State University, with the assurance of knowing I had one more year of college remaining before I entered the ‘real world’. My internship was only a taste of the ‘corporate world’ and what was in store for me, and I still had one more year to learn and figure things out (and of course, one more year of fun in the Penn State football student section – priorities, priorities…)
Penn State’s Business School has a fantastic internship program, and I was very fortunate to obtain an internship at General Electric’s (GE) Corporate Telecommunications office in Princeton, NJ. My role as an intern at GE was providing support to the senior staff in the design and implementation of voice, data and video-conferencing services for GE businesses worldwide. Needless to say, this was both a challenging and rewarding experience for a 21-year-old college student, participating in the implementation of GE’s groundbreaking Global Telecommunications Network during the early years of the internet, among other things.
As I reminisced back to my eight months at GE, I couldn’t help but notice the similarities between my internship and a few of the ‘lessons learned’ I took away from my experience 20+ years ago, and how they compared or contrasted to my recent observations and feedback I provided to Niels as his mentor. Of course, there are pronounced differences – after all, many things have changed in the last 20 years – the technology we use every day is clearly the biggest distinction. I would be remiss not to also mention the obvious generation gap – I am a proud ‘Gen X’er’, raised on Atari and MTV, while Niels is a proud Millennial, raised on the Internet and smartphones. We actually had a lot of fun joking about the whole ‘generation gap thing’ and I’m sure we both learned a lot about each other’s demographic group. Niels wasn’t the only person who learned something new over the summer – I learned quite a bit myself.
In summary, my reminiscing back to the late 90’s
certainly helped make my daily music choices easier for a few weeks this summer led to the vision for this blog post. I thought it would be interesting to list a few notable experiences and lessons I learned as an intern at GE, 20 odd years ago, along with how my experiences compared or contrasted with what I observed in the last 10 weeks working side-by-side with our intern, Niels. These observations are based on my role as his mentor, and were provided as feedback to Niels in his summary review, and they are in no particular order.
Have you similarly had the opportunity to engage in both roles within the intern/mentor relationship as I have? Maybe your example isn’t separated by 20 years, but several years? Perhaps you’ve only had the chance to fulfill one of these roles in your career and would love the opportunity to experience the other? In any case, see if you recognize some of the lessons you may have learned in the past and how they present themselves today. I think you’ll be amazed at how even though ‘the more things change, the more they stay the same’.
PowerApps is one of the most recent additions to the Microsoft Office suite of products. PowerApps has been marketed as “programming for non-programmers”, but make no mistake; the seamless interconnectivity PowerApps has with other software products allows it to be leveraged in highly complex enterprise applications. PowerApps basic is included in an Office 365 License, but for additional features and advanced data connections, a plan must be purchased. When I was brought on to CC Pace as an intern to assist with organizational change regarding SharePoint, I assumed that the old way of using SharePoint Designer and InfoPath would be the framework I would be working with. However, as I began to learn more about PowerApps and CC Pace’s specific organizational structure and needs, I realized that it was essential to work with a framework geared towards the future.
Solutions with PowerApps
While Big Data and data warehousing become common practices, data analytics and organized data representation become more and more valuable. On the small to medium organizational scale, bringing together scattered data which is stored in a wealth of different applied business practices and software options has been extremely difficult. This one of the areas where PowerApps can create immense value for your organization. Rather than forcing an extreme and expensive organizational change where everyone submits expense reports, recruitment forms, and software documentation to a brand new custom database management system, PowerApps can be used to pull data from its varying locations and organize it.
PowerApps is an excellent solution for data entry applications, and this is the primary domain I’ve been working in. A properly designed PowerApp allows the end user to easily manipulate entries from all sorts of different database management systems. Create, Read, Update, Delete (CRUD) applications have been around and necessary for a long time, and PowerApps makes it easy to create these types of applications. Input validation and automated checks can even help to prevent mistakes and improve productivity. If your organization is constantly filling out their purchase orders with incorrectly calculated sales tax, a non-existent department code, or forgetting to add any number of fields, PowerApps allows some of those mistakes to be caught extremely early.
Integration with Flow (the upgraded version of SharePoint designer WorkFlows), allows for even greater flexibility using PowerApps. An approval email can be created to ensure to prevent mistakes from being entered into a database management system, push notifications can be created when PowerApps actions are taken, the possibilities are (almost) endless.
Pros and Cons
There are both advantages and disadvantages to leveraging software in an enterprise solution that is still under active development. One of the disadvantages is that many of the features that a user might expect to be included aren’t possible yet. While Flow integration with PowerApps is quite powerful, it is missing several key features, such as an ability to attach a document directly from PowerApps, or to write data over multiple records at a time (i.e. add multiple rows to a SQL database). Additionally, I would not assume that PowerApps is an extremely simple, programming free solution to business apps. Knowledge of the different data types as well as the use of functions gives PowerApps a steep learning curve. While you may not be writing any plaintext code, other than HTML, PowerApps still requires a good amount of knowledge of technology and programming concepts.
The main advantage to PowerApps being new software is just that; it’s brand new software. You may have heard that PowerApps is currently on track to replace, at least partially, the now shelved InfoPath project. InfoPath may continue to work until 2026, but without any new updates to the program, it may become obsolete on newer environments well before that. Here at CC Pace, we focus on innovation and investing in the solutions of tomorrow, and using PowerApps internally rather than creating a soon non-supported InfoPath framework was the right choice.
As a programmer and cybersecurity enthusiast, creating pieces of enterprise systems is something I never knew I would be so interested in. I’m Niels Verhoeven, a summer IT Intern at CC Pace Systems. I study Information Systems with a focus on Cybersecurity Informatics at University of Maryland, Baltimore County. My experiences at CC Pace and my programming background have given me quite a bit of insight into how users, systems, and business can fit together, improving productivity and quality of work.
Is your business undergoing an Agile Transformation? Are you wondering how DevOps fits into that transformation and what a DevOps roadmap looks like?
Check out a webinar we offered recently, and send us any questions you might have!
Recently, I was part of a successful implementation of a project at a big financial institution. The project was the center of attention within the organization mainly because of its value addition to the line of business and their operations.
The project was essentially a migration project and the team partnered with the product vendor to implement it. At the very core of this project was a batch process that integrated with several other external systems. These multiple integration points with the external systems and the timely coordination with all the other implementation partners made this project even more challenging.
I joined the project as a Technical Consultant at a rather critical juncture where there were only a few batch cycles that we could run in the test regions before deploying it into production. Having worked on Agile/Scrum/XP projects in the past and with experience working on DevOps projects, I identified a few areas where we could improve to either enhance the existing development environment or to streamline the builds and releases. Like with most projects, as the release deadline approaches, the team’s focus almost always revolves around ‘implementing functionality’ while everything else gets pushed to the backburner. This project was no different in that sense.
When the time had finally come to deploy the application into production, it was quite challenging in itself because it was a four-day continuous effort with the team working multiple shifts to support the deployment. At the end of it, needless to say, the whole team breathed a huge sigh of relief when we deployed the application rather uneventfully, even a few hours earlier than what we had originally anticipated.
Once the application was deployed to production, ensuring the stability of the batch process became the team’s highest priority. It was during this time, I felt the resistance to any new change or enhancement. Even fixes to non-critical production issues were delayed because of the fear that they could potentially jeopardize the stability of the batch.
The team dreaded deployments.
I felt it was time for me to build my case to have the team reassess the development, build and deployment processes in a way that would improve the confidence level of any new change that is being introduced. During one of my meetings with my client manager, I discussed a few areas where we could improve in this regard. My client manager was quickly onboard with some of the ideas and he suggested I summarize my observations and recommendations. Here are a few at a high level:
It’s common for these suggestions to fall through the cracks while building application functionality. In my experience, I have noticed they don’t get as much attention because they are not considered ‘project work’. What project teams, especially the stakeholders, fail to realize is the value in implementing some of the above suggestions. Project teams should not consider this as additional work but rather treat it as part of the project and include the tasks in their estimations for a better, cleaner end product.
“Once I started looking around behind the port frames, I figured I could just….”
And so began a summer of endless sailboat projects and no sailing. One project lead to the start of another without resolving the first. What does this possibly have to do with software development and Agile techniques?
My old man and I own and are restoring an older sailboat. He is also in the IT profession, and is steeped in classic waterfall development methodology. After another frustrating day of talking past each other, he asked how I felt things could be handled differently in our boat projects.
“Stop starting and start finishing!”
It is the key mindset for Agile. Take a small task that provides value, focus on it, and get it done. It eliminates distraction and gives the user something usable quickly.
Applying this mindset outside of software may not be intuitive, but can pay dividends quickly. On the boat, we cleared space on the bulkhead, grabbed a stack of post-its and planned through the next project, rewiring the boat. The discussion started with the goal of the project. “We’re just to tear everything out and rewire everything.” Talk about ignoring non-breaking changes! I suggested that we focus on always having a working product – a sail-able boat – and break the project into smaller tasks that can be worked from start to finish in short, manageable pieces of time.
Approaching the project from that angle, we quickly developed a list of sub tasks, prioritized them, and put them up on our make-shift Kanban board. This was planning was so intuitive and rewarding on its own that we did the same for other projects we want to tackle before April.
So stop starting, start finishing, and start providing value quicker for your stakeholders.
Building a new software product is a risky venture – some might even say adventure. The product ideas may not succeed in the marketplace. The technologies chosen may get in the way of success. There’s often a lot of money at stake, and corporate and personal reputations may be on the line.
I occasionally see a particular kind of team dysfunction on software development teams: the unwillingness to share risk among all the different parts of the team.
The business or product team may sit down at the beginning of a project, and with minimal input from any technical team members, draw up an exhaustive set of requirements. Binders are filled with requirements. At some point, the technical team receives all the binders, along with a mandate: Come up with an estimate. Eventually, when the estimate looks good, the business team says something along the lines of: OK, you have the requirements, build the system and don’t bother us until it’s done.
(OK, I’m exaggerating a bit for effect – no team is that dysfunctional. Right? I hope not.)
What’s wrong with this scenario? The business team expects the technical team to accept a disproportionate share of the product risk. The requirements supposedly define a successful product as envisioned by the business team. The business team assumes their job is done, and leaves implementation to the technical team. That’s unrealistic: the technical team may run into problems. Requirements may conflict. Some requirements may be much harder to achieve than originally estimated. The technical team can’t accept all the risk that the requirements will make it into code.
But the dysfunction often runs the other way too. The technical team wants “sign off” on requirements. Requirements must be fully defined, and shouldn’t change very much or “product delivery is at risk”. This is the opposite problem: now the technical team wants the business team to accept all the risk that the requirements are perfect and won’t change. That’s also unrealistic. Market dynamics may change. Budgets may change. Product development may need to start before all requirements are fully developed. The business team can’t accept all the risk that their upfront vision is perfect.
One of the reasons Agile methodologies have been successful is that they distribute risk through the team, and provide a structured framework for doing so. A smoothly functioning product development team shares risk: the business team accepts that technical circumstances may need adjustment of some requirements, and the technical team accepts that requirements may need to change and adapt to the business environment. Don’t fall into the trap of dividing the team into factions and thinking that your faction is carrying all the weight. That thinking leads to confrontation and dysfunction.
As leaders in Agile software development, we at CC Pace often encourage our clients to accept this risk sharing approach on product teams. But what about us as a company? If you founded a startup and you’ve raised some money through venture capital – very often putting your control of your company on the line for the money – what risk do we take if you hire us to build your product? Isn’t it glib of us to talk about risk sharing when it’s your company, your money, and your reputation at stake and not ours?
We’ve been giving a lot of thought to this. In the very near future, we’ll launch an exciting new offering that takes these risk sharing ideas and applies them to our client relationships as a software development consultancy. We will have more to say soon, so keep tuning in.
Recently, I attended a meetup for Loudoun’s Tech Startups in Ashburn, VA. It was a great opportunity to discuss ideas in various stages of development, as well as the resources available to bring these ideas to market. It was encouraging to see so many motivated entrepreneurs share their experiences in a local setting, with Loudoun showing its promise as a business incubator.
Michelle Chance from the Innovative Solutions Consortium gave a great speech about the services provided by her organization, such as “hard challenge” events, think-tank style meetings and student/recent graduate mentoring.
The hard challenge events seemed particularly interesting to me since they provide a collaborative environment for solving difficult technology problems. Organizations compete for the best solution and receive awards for most disruptive and innovative technologies.
Another sponsor of the event was the Mason Enterprise Center, which provides consultation and training to small business owners and entrepreneurs. They have regional offices in Fairfax, Fauquier, Loudoun and Prince William counties.
We attended this event because of our past experience developing collaborative solutions for startups. We’ve found that the Agile development philosophy fits nicely with the entrepreneurial spirit of organizations that want to quickly build a product that provides value. In fact, principles such as Minimum viable product and Continuous deployment are core to the Lean startup philosophy championed by Eric Ries. This method encourages startups to build a minimum amount of high-value features that can be released quickly. With frequent releases, a company can immediately begin collecting and responding to customer feedback.
If you’re at any stage in the startup process and have a technical idea that you want to explore or expand, there are plenty of resources available in the NoVa area. Also, I encourage you to attend one of the various meetups for startups, such as the Loudoun Tech Startups group.
Senior IT managers starting a new project often have to answer the question: build or buy? Meaning, should we look for a packaged solution that does mostly what we need, or should we embark on a custom software development project?
Coders and application-level programmers also face a similar problem when building a software product. To get some part of the functionality completed, should we use that framework we read about, or should we roll our own code? If we write our own code, we know we can get everything we need and nothing we don’t – but it could take a lot of time that we may not have. So, how do we decide?
Your project may (and probably does) vary, but I typically base my decision by distinguishing between infrastructure and business logic.
I consider code to be infrastructure-related if it’s related to the technology required to implement the product. On the other hand, business logic is core to the business problem being solved. It is the reason the product is being built.
Think of it this way: a completely non-technical Product Owner wouldn’t care how you solve an infrastructure issue, but would deeply care about how you implement business logic. It’s the easiest way to distinguish between the two types of problems.
Examples of infrastructure issues: do I use a relational or non-relational database? How important are ACID transactions? Which database will I use? Which transactional framework will I use?
Examples of business logic problems: how do I handle an order file sent by an external vendor if there’s an XML syntax error? How important is it to find a partial match for a record if an exact match cannot be found? How do you define partial?
Note that a business logic question could be technical in nature (XML syntax error) but how you choose to solve it is critical to the Product Owner. And a seemingly infrastructure-related question might constitute business logic – for example, if you are a database company building a new product.
After this long preamble, finally my advice: Strongly favor using existing frameworks to solve infrastructure problems, but prefer rolling your own code for business logic problems.
My rationale is simple: you are (or should be) expert in solving the business logic problems, but probably not the infrastructure problems.
If you’re working on a system to match names against a data warehouse of records, your team knows or can figure out all the details of what that involves, because that’s what the system is fundamentally all about. Your Product Owner has a product idea that includes market differentiators and intellectual property, making it very unlikely that an existing matching framework will fulfill all requirements. (If an existing framework does meet all the requirements, why is the product being developed at all?)
Secondly, the worst thing you want to do as a developer is to use an existing business logic framework “to make things simple”, find that it doesn’t handle your Product Owner’s requirements, and then start pushing back on requirements because “our technology platform doesn’t allow X or Y”. For any software developer with professional pride: I’m sorry, but that’s just weak sauce. Again, the whole point of the project is to build a unique product. If you can’t deliver that to the Product Owner, you’re not holding up your end of the bargain.
On the other hand, you are very likely not experts on transactional frameworks, message buses, XML parsing technology, or elastic cloud clusters. Oracle, Microsoft, Amazon, etc., have large expert teams and have put their own intellectual property into their products, making it highly unlikely you’ll be able to build infrastructure that works as reliably and is as bug free.
Sometimes the choice is harder. You need to validate a custom file format. Should you use an existing framework to handle validations or roll your own code? It depends. It may not even be possible to tell when the need arises. You may need to use an existing framework and see how easy it is to extend and adapt. Later, if you find you’re spending more time extending and adapting than rolling your own optimized code, you can change the implementation of your validation subsystem. Such big changes are much easier if you’ve consistently followed Agile engineering practices such as Test Driven Design.
As always, apply a fundamental Agile principle to any such decision: how can I spend my programming time generating the most business value?
As I write this blog entry, I’m hoping that the curiosity (or confusion) of the title captures an audience. Readers will ask themselves, “Who in the heck is Jose Oquendo? I’ve never seen his name among the likes of the Agile pioneers. Has he written a book on Agile or Scrum? Maybe I saw his name on one of the Agile blogs or discussion threads that I frequent?”
In fact, you won’t find Oquendo’s name in any of those places. In the spirit of baseball season (and warmer days ahead!), Jose Oquendo was actually a Major League Baseball player in the 1980’s, playing most of his career with the St. Louis Cardinals.
Perhaps curiosity has gotten the better of you yet again, and you look up Oquendo’s statistics. You’ll discover that Oquendo wasn’t a great hitter, statistically-speaking. His career .256 batting average and 14 homeruns over a 12 year MLB career is hardly astonishing.
People who followed Major League Baseball in the 1980’s, however, would most likely recognize Oquendo’s name, and more specifically, the feat which made him unique as a player. Oquendo has done something that only a handful of players have ever done in the long history of Major League Baseball – he’s played EVERY POSITION on the baseball diamond (all nine positions in total).
Oquendo was an average defensive player and his value obviously wasn’t driven from his aforementioned offensive statistics. He was, however, one of the most valuable players on those successful Cardinal teams of the 80’s, as the unique quality he brought to his team was derived from a term referred to in baseball lingo as “The Utility Player”. (Interestingly enough, Oquendo’s nickname during his career was “Secret Weapon”.)
Over the course of a 162-game baseball season, players get tired, injured and need days off. Trades are executed, changing the dynamic of a team with one phone call. Further complicating matters, baseball teams are limited to a set number of roster spots. Due to these realities and constraints of a grueling baseball season, every team needs a player like Oquendo who can step up and fill in when opportunities and challenges present themselves. And that is precisely why Oquendo was able to remain in the big leagues for an amazing 12 years, despite the glaring deficiency in his previously noted statistics.
Oquendo’s unique accomplishment leads us directly into the topic of the Agile Business Analyst (BA), as today’s Agile BA is your team’s “Utility Player”. Today’s Agile BA is your team’s Jose Oquendo.
A LITTLE HISTORY – THE “WATERFALL BUSINESS ANALYST”
Before we get into the opportunities afforded to BA’s in today’s Agile world, first, a little walk down memory lane. Historically (and generally) speaking – as these are only my personal observations and experiences – a Business Analyst on a Waterfall project wrote requirements. Maybe they also wrote test cases to be “handed off” and used later. In many cases, requirements were written and reviewed anywhere from six to nine months before documented functionality was even implemented. As we know, especially in today’s world, a lot can change in six months.
I can remember personally writing requirements for a project in this “Waterfall BA” role. After moving onto another project entirely, I was told several months down the road, “’Project ABC’ was implemented this past weekend – nice work.” Even then, it amazed me that many times I never even had the opportunity to see the results of my work. Usually, I was already working on an entirely new project, or more specifically, another completely new set of requirements.
From a communications perspective, BA’s collaborated up-front mostly with potential users or sellers of the software in order to define requirements. Collaboration with developers was less common and usually limited to a specific timeframe. I actually worked on a project where a Development Manager once informed our team during a stressful phase of a project, “please do not disturb the developers over the next several weeks unless absolutely necessary.” (So much for collaboration…) In retrospective, it’s amazing to me that this directive seemed entirely normal to me at the time.
Communication with testers seemed even rarer – by the very definition, on a Waterfall project, I’ve already passed my knowledge on to the testers – it’s now their responsibility. I’m more or less out of the loop. By the time the specific requirements are being tested, I’m already off onto an entirely new project.
In my personal opinion the monotony of the BA role on a Waterfall project was sometimes unbearable. Month-long requirements cycles, workdays with little or no variation, and some days with little or no collaboration with other team members outside of standard team meetings became a day to day, week to week, month to month grind, with no end in sight.
AND NOW INTRODUCING… THE “AGILE BUSINESS ANALYST”
Fast-forward several years (and several Agile project experiences) and I have found that the role of today’s Agile Business Analyst has been significantly enhanced on teams practicing Agile methodologies and more specifically, Scrum. Simply as a result of team set-up, structure, responsibilities – and most importantly, opportunities – I feel that Agile teams have enhanced the role of the Business Analyst by providing opportunities which were never seemingly available on teams using the traditional Waterfall approach. There are new opportunities for me to bring value to my team and my project as a true “Utility Player”, my team’s Jose Oquendo.
The role of the Agile BA is really what one makes of it. I can remain content with the day to day “traditional” responsibilities and barriers associated with the BA role if I so choose; back to the baseball analogy – I can remain content playing one position. Or, I can pursue all of the opportunities provided to me in this newly-defined role, benefitting from new and exciting experiences as a result; I can play many different positions, each one further contributing to the short and long-term success of the team.
Today, as an Agile BA, I have opportunities – in the form of different roles and responsibilities – which not only enhance my role within the team but also allow me to add significant value to the project. These roles and responsibilities span not only across functional areas of expertise (e.g. Project Management, Testing, etc.) but they also span over the entire lifetime of a software development project (i.e. Project Kickoff to Implementation). In this sense, Agile BA’s are not only more valuable to their respective teams, they are more valuable AND for a longer period of time – basically, the entire lifespan of a project. I have seen specifically that Agile BA’s can greatly enhance their impact on project teams and the quality of their projects in the following five areas:
- Project Management
- Product Management (aka the Product Backlog)
- Collaboration (with Project Stakeholders and Team Members)
We’ll elaborate – in a follow-up blog entry – specifically how Agile BA’s can enhance their role and add value to a project by directly contributing to the five functional areas listed above.
For the past 3 months we’ve had the pleasure of working with a charitable organization called the Ceca Foundation.
Ceca, which is derived from “celebrating caregivers”, was established in 2013 to celebrate caregiver excellence and “to promote high patient satisfaction by recognizing and rewarding outstanding caregivers”. They do this by providing employees of caregiving facilities with a platform for recognizing and nominating their peers for the Ceca Award – a cash reward given throughout the year. These facilities include rehabilitation centers, hospitals, assisted living centers and similar organizations.
CC Pace partnered with Ceca to build their next generation, customized nomination platform.
This was one of those projects that fills you with pride. First, for the obvious reason – Ceca’s worthwhile mission. Second, the not-so-obvious reason, which was the development process. It was a great example of why I enjoy helping customers build products.
For various reasons, Ceca was under a tight deadline to get the new platform up-and-running for several facilities. The Agile process turned out to be a great fit, as it allowed for frequent customer feedback and weekly deployments to a testable environment. We developed the platform using high-level feature stories, rather than detailed specifications. This allowed the team to concentrate on the desired outcome, rather than getting caught up in the technical details. At times, we had to forgo a software-based solution in favor of a manual process. When you have limited resources and time, you have to make these types of decisions.
In February, after about 3 weeks, the Ceca Foundation launched the new web platform for one facility and then quickly brought on several more. There was immediate gratification for the team as we watched the nominations flood in.
The “feel good” story
What made this project successful and enjoyable at the same time? I’m reminded of the first value in the Agile manifesto – individuals and interactions over processes and tools. Some factors were technical but most were not:
- a motivated and enthusiastic customer (Ceca)
- a set of agreed upon features to provide the Minimum Viable Product
- frequent collaboration with the customer
- a cloud-hosted environment to provide infrastructure on-demand for testing and live versions
- a software-as-a-service model that allowed us to quickly bring on new facilities
For me, it was Agile at its most fundamental: discuss the desired features; provide a cost estimate for those features; negotiate priority with the customer; provide frequent releases of working software.