Reduced Operational Complexities of Data Collection in Audience Measurement for Operations and Data Science Teams in 30+ Countries with an enterprise data platform

• Introduction
• 30+ Countries, 6 Continents, 1 Goal: Streamline Data Collection
• Instrumenting Technical Support and Data Science Personnel
• Designing Internal Transformation
• The Core Challenge: Reimagining Problem-Solving for Expert Users
  • Understanding End Users Through Real-World Observations
  • Incorporating Quantitative Data
  • Reframing Problems with Workshops
• Testing Prototypes: Real-Life Scenarios in a New Environment
• Making Strategic Decisions
  • Technology Decisions
  • Product Architecture Decisions
  • Information Architecture Decisions
  • Design Decisions
• Unified Platform: Global Brain for Global Operations
• Bittersweet Success Amidst Unprecedented Challenges
• The Three Most Important Things I Learned

After decades of growth through acquisitions, the global setup became inevitably fragmented. This fragmentation led to increased maintenance overhead and limited opportunities for automation.

Years of internal process optimisations had reached their limits, resulting in inefficient collaboration due to role-specific toolchains and disconnected work environments.

Our team was assembled to:

1. Consolidate and harmonise international data into a common platform to enable automation, integration, and multi-country products and analytics.
2. Streamline data collection.
3. Reduce maintenance overhead through effective process automation.
4. Provide secure and reliable access to task-related data across devices and roles.
5. Improve hardware stock management.
6. Simplify knowledge management to reduce training costs.

The end users of the new Panel Tools were a diverse group of internal professionals, primarily from operations and data science teams.

Our main challenge in considering and prioritising user needs was the global differences in their work environments and practices.

These global differences included:

1. Work Styles: Ranging from customer-facing roles focused on interviewing, to quality control specialists working with data, and technical support staff focused on hardware installation.
2. Work Locations: From office-based positions to roles operating in the field.
3. Team Structures: Spanning small teams to large teams with multiple management layers.
4. Work Modes: Including fixed area responsibilities and teams operating on a first-come, first-served basis.

Mapping and understanding these challenges was our first step in designing practical solutions through cross-functional collaboration.

I led product design and UX research, establishing a team of designers and assisting in scaling the project's development team.

Collaborating with the product owner and tech lead, I built strong relationships with stakeholders and subject matter experts (SMEs) responsible for 30+ markets on six continents.

Collaborating across domains on product strategy and architecture, I partnered with leaders from product, data science, hardware engineering, software development, and operations while reporting to the VP of Technology and later to the Global SVP of UX.

Initially, my focus was on:

1. Contextualising complexity to enable problem reframing.
2. Facilitating workshops with SMEs to resurface and unlock expertise trapped in siloed work environments and create a shared working context.
3. Developing tools and frameworks to streamline research and collaboration at scale among domain experts and stakeholders.
4. Recruiting and onboarding a global group of test users.
5. Establishing an unmoderated, remote usability testing framework.

As the team gained momentum and we began resolving problems, my focus shifted to:

1. Iterating on design, prototypes, and experiments.
2. Experimenting with dual-track agile, integrating research into development cycles for course correction on product architecture and strategy to support evolving stakeholder goals.
3. Promoting a culture of evidence-based decision-making within the S&P 500 company.

With a strong foundation rooted in cross-organisational collaboration, we were ready to start untangling the project’s complexities.

Initially, redesigning and re-platforming multiple software products with individual UIs and complex data models into a unified solution felt unworkable.

The diverse operating models, data governance policies, and privacy regulations across more than 30 markets further complicated the task. The project's scope became even more daunting, with stakeholders distributed across six continents.

Multiple local considerations complicated the scope even further:

• Different roles in different markets deal with different numbers of objects and different types of workflows.
• Power outages in South Africa
• Severe weather events and long distances in Australia
• Inaccessible home areas due to religion-driven social norms in Saudi Arabia
• Markets with ongoing client and audit commitments constrained flexibility and prevented rapid changes.

To start unpacking this challenge, we needed to understand the backstage of day-to-day audience measurement studies in more detail.

We began by shadowing personnel in offices and on location to understand the day-to-day experiences of end users in real-world contexts of proprietary hardware and regulatory constraints.

Following the shadowing sessions, we conducted group expert usability walkthroughs. These sessions helped us quickly understand non-linear and multi-intent workflows across different roles beyond the dynamics of organisational units.

All qualitative insights were invaluable, but the final step in preparing the team for workshops was incorporating quantitative data for a comprehensive data-driven context.

Deep dives with subject matter experts helped get the right data and the data right.

Insights gathered from observations served as context for historical data analysis.

By reviewing production data, like meter logs, support tickets, visit summaries, and panel performance, we quantified the problems and identified opportunities.

After gathering both qualitative and quantitative data, we assembled experts to reframe problems at their core.

In-person and online workshops, adapting popular formats for the enterprise, brought stakeholders and subject matter experts together and made everyone feel included and invested in the process.

Workshops worked wonders in resurfacing requirements and expertise locked in files and specialised work environments into a shared working context.

In parallel with facilitating workshops, together with the product owner, we started a global group of volunteer testers. Country managers selected individuals representing local diversity in age, tenure, tech savviness, and more.

Once we identified and refined the hypotheses worth testing, we invited our global group of future users to experiment with our prototypes.

With a little help from my tech lead (who wrote a custom API for the Webflow CMS), we utilised low-code development tools to build working prototypes, enabling users to work on familiar problems through search, filters, and forms.

By integrating Hotjar and Google Analytics into prototypes, we transformed them into the backbone of our unmoderated, remote usability testing framework.

Unmoderated, remote usability testing allowed us to collect user feedback across different time zones without the constraints of scheduling live sessions.

We agreed with subject matter experts and stakeholders to:

1. Rely on task-based scenarios with expected results.
2. Utilise session recordings and task completion rates to validate successful design elements or identify areas requiring further iteration.
3. Leverage surveys to collect ideas, motivations, and pain points in a safe, anonymous environment outside of titles and reporting chains.

Armed with insights from these tests, we were prepared to make critical decisions that would shape the product's technology and architecture.

1. Translate procedural, ticket-based workflows into a flexible object-attribute system using object-oriented design concepts.
2. Unify data models to simplify integration, enable multi-market products, and consolidate workflows scattered across role-specific and market-specific toolchains into a single environment.

1. Advocate for data democratisation and simplified language to establish a single source of truth, replacing:
   • Role and/or market-specific definitions
   • Inconsistent naming conventions
   • Diverse taxonomies
2. Avoid linear workflows by offering multiple entry and exit points, acknowledging that solutions can vary across products and markets.
3. Provide multiple data presentation modes for the same data to support different workflows such as:
   • Cards for case-by-case investigation.
   • Tables for batch operations.
   • Maps for process inspection.
   • Charts for decision-making.

1. Ensure key components are scalable by 10, 100, or 1,000 factors to accommodate market differences.
2. Design with datasets from multiple locales to support different date formats, item numbers, label lengths, etc.
3. Plan variants for feature toggling on component, pattern and feature levels to support a variety of use cases, devices, and locales.
4. Utilise card metaphor extensively to surface contextual information while providing larger interactive areas.

These decisions, focused on process automation, scalability, and data democratization across the organization, resulted from close collaboration among cross-functional teams. Technology, product, design, and stakeholders from various markets worked together, basing their choices on meticulously gathered and analyzed evidence.

By addressing immediate challenges while ensuring our platform's ability to evolve, these strategic choices provided scalable and lasting value to the organization.

Through iterative design cycles informed by research and refined by subject matter expert reviews, we developed a proof of concept demonstrating flexibility and readiness to support a variety of setups.

We simplified the multirole setup by using object-oriented programming concepts as interface metaphors.

This approach allowed us to apply the same pattern of operations on objects and attributes across different roles, all integrated with the same data sources.

With minor adjustments, the same interface displaying a list of cards and following similar interaction logic could present:

1. Recruiters with individuals to interview.
2. Technicians with meters to install.
3. Managers with team members and targets.
4. Leaders with market performance data.

The introduction of GraphQL facilitated smooth performance in creating relationships between objects and querying these relationships, both manually and automatically, to support expanding and evolving schemas.

By combining object-oriented information architecture with object-oriented programming concepts used as interface metaphors, such as polymorphism, abstraction, and encapsulation, we provided a single front-end platform.

The unified platform successfully addressed all the challenges related to:

1. Data governance policies
2. Privacy regulations
3. Client commitments
4. Regulatory requirements
5. Team setups and workflows across 30+ countries

The unified platform was ready to democratise data, empowering our global teams to work more efficiently and support the organisation's strategic goals across all markets.

The project results made a company-wide impact and attracted top leadership attention.

I was awarded the CEO's Annual Award for Global Top Performers for two consecutive years. My work visibility on the C-level also contributed to a global investment decision to set up a UX org with an SVP of UX and 30+ FTEs.

In the third year, I was promoted to a director position, reflecting my work's significant contributions and impact.

However, despite the remarkable progress and traction gained in the first 18 months, the onset of the COVID-19 pandemic forced stakeholders to gradually shift priorities and address the immediate challenges of the “new normal”.

As a result:

1. Parts of the scope related to the self-service model were prioritised and evolved into features within participant-facing products. (LINK →)
2. In various markets, other processes were provisionally covered using APIs, microservices, or custom combinations of third-party tools.
3. Global supply chain disruptions towards the end of the pandemic impacted our ability to proceed with the in-house approach as planned.

Despite shifting priorities, our team's dedication ensured that the foundational work we accomplished would benefit the organization for years to come.

This foundational work significantly reduced the timelines and budgets of multiple projects, saving substantial time and resources.

I'm grateful for the recognition and opportunities during this time and proud of our work's lasting impact on the organisation.

Despite its bittersweet conclusion, this transformative journey reinforced the importance of adaptability, collaboration, and innovation in overcoming complex challenges.

During this transformative experience in UX leadership and design strategy on a global scale, I learned three invaluable lessons:

Focus on core value creation and invest in skills that multiply what you can already do. Read more → “Five career-changing skills if you are working on software.”

Domain expertise turbocharges any project. Read more → “Get the most of your subject matter experts.”

Data is the most underutilised asset in most companies. Read more → “What, when and why makes data so valuable?”