Optimizing the Organization: Where Does Testing Fit into your Org Structure?

Where then does testing fit into your organization? Is it just something people do? Is it a central component to be shared, or is it just something that each group does on their own. All groups face this struggle when they discover that you can’t just make it an additional duty for someone and get good value. When you have decided that you want to build a real optimization team within your organization, you are challenged with those very questions. What I want to propose are a couple of frameworks that I have seen work to great success, and what fundamentally makes them successful. It may not be possible to move mountains quickly to get to these structures immediately, but it is important to understand why they work and to think of ways to move towards those directions.

One of the core challenges is that testing really isn’t a full on marketing discipline, nor an analytics discipline, IT discipline or really anything else that is in most groups normal organization structure. To most groups starting out, testing is thought of as a feature done in order to prove that their current efforts are better then their prior efforts. Testing instead is a unique discipline that takes parts of all of those, but is also at its best when it is showing the inefficiencies in your internal processes and mindset. for any group to succeed, you need to have people, alignment, and the correct mindset, otherwise it all goes to waste. When testing is not allowed to be a new discipline, it suffers exponentially the inefficiencies of the discipline it is placed under.

The first thing to understand is that for larger organizations, testing works best in a hub and spoke model; meaning that you have a central team that then works with members of the various business units to improve their actions. While this might be the best model, it only works if you have established clear rules and the correct mindset in those other groups as well as your own.

Education will always be the primary role of the testing team.

Each of the frameworks below shows the central team, that would then work with an individual or team in each structure that follows a similar format. Central alignment allows you to separate the resources and to insure that you are not just adding on testing to existing duties. This format allows you the benefit of creating a central knowledge base, while leveraging local knowledge, resources, and structure to work. The accumulation and sharing of knowledge, and the design of your efforts to accomplish this task is the primary goal of your larger structure. For this to work, testing can not be dictated only by the business unit. It must instead be a collaborative effort where both sides work together to create a continuous culture of optimization, one that is focused on being “wrong” over being “right”. There can never be a time where just someone coming up with an “idea” is allowed to be the end all of what is tested. No idea by itself is sacred, and no one, the CEO down to your janitor, should be allowed to just throw something up because they think “it will work”.

Framework #1:

In this first model, you see that we have a manager of testing who works through or acts as a project manager for external resources (IT and creative). This person may have analysts working under them, but fundamentally they work with other groups in a dotted line creating a cross functional team that tackles testing. While you may not have direct full time people on the team, in this structure the same people work together regularly to advance the organizations optimization efforts. You will also notice that while they may work under analytics, they are not analytics, with a separate team handling those responsibilities. The disciplines are dramatically different, and there is a lot of value for bringing different data types together, but if you have just your standard analytics team also doing testing, you will never achieve anything close to the value that you can and should receive.

The limitations of this type of structure is a heavier need for sponsorship to allow the freedom and align the teams on central goals. You will now have personal or team goals for the various business units that may be opposite of the central or optimization team goals. It is extremely easy when resources are not “owned” for those to go to projects based on political or popular reasons and not the value they may bring to the organization. It is easy to talk about working together and having access to resources, but that tends to last only as long as there is not a fire that someone feels needs to be put out. Other limitations are the constants pull to do other types of work, especially for report pulling, as well as the need to make it clear what people are measured on despite being on very different teams. Despite those limitations, a lot of good can come if you have clear and strong leadership, accountability, and the right people in place. This is one of the most common structures for mature groups, and is one that will allow some level of success and expansion throughout the organization.

Framework #2:

In this second example, you see that you have a full optimization team, one without dotted lines and one who is independently part of the entire data team. Here you have technical and creative resources, but who only tangentially part of the larger marketing and IT teams. These resources are not directly part of their team, but the same group continuously work together to grow and expand the organizations testing efforts. The benefits of this structure is the ability to really develop the skills of the members, the central role within the larger organization, and the ability to have a separate charter and to really focus on improving the site as a whole, not just the smaller components within.

This is the preferred structures that I have seen for larger organizations. This allows for consistent resources, the ability to do the right thing for the site, not just the business unit, and an independent role for analytics and just marketing. It doesn’t create confusion for the team to do things that help only one group or person, and it allows for a clear line between testing and any other group. The goal is to work together to improve, not for teams to fight over who gets credit. The limitations are the need to constantly be working with and educating the various business units, and the complications of owning the impact for the team’s actions.

In order for either framework to function properly, the executive sponsor must take an active role in keeping people aligned and accountable for site goals, not just personal goals. You will never be able to function if you cant first deal with petty infighting and a lack of accountability towards a common central goal. If you do not have that, then don’t wait for it to happen and instead seek out sponsorship to make it happen. You don’t need or even want higher level executives to be dealing with the day to day operations, but you need an umbrella by which to separate the disciplines, align the resources, and make sure that the system itself is being used to its highest function. As much as we may want to pretend we live in a world where everyone works together towards a common goal, it is rare that this is the case. The executive sponsor’s primary responsibility is to raise the level of discourse away from petty individual goals and to hold people accountable for actions that make everyone better.

These are but two of the more common examples I have seen for successful programs. In all cases, the real success is far more about developing skills, challenging ideas, getting buy-in and accountability, and more than anything treating optimization as an independent function that never starts and never ends. The team is meant to be the best friend of all business units, not their worst enemy. It is meant to show them the efficiency of their efforts and to make sure that what they think matters really does. If you are just going to leverage testing to just push the same ideas, to prove value for other tools, or to make someone look good then you have no chance of building a world class organization. This means that you have to fight the battles to get past the initial resistance and educate people as to why their ideas won’t work and why you must challenge all ideas in order to understand the value, not just directly but relative to other courses of action.

There are no magic bullets to make your organization a great optimization organization. No matter what structure and path you choose to go down, it takes time and a lot of hard work to get people to understand how to make it work. The hardest task is always going to be getting past poor misconceptions and petty internal battles for control and who gets to claim success. Once you have gotten past those points, aligning in a way that insures success is the next step on the path to a great program. Programs fail when they stop fighting necessary fights and just go with the flow. If you want to make your organization the best it can be however, you can never stop improving and you can never stop trying to get people to align in the best ways possible.