Validity Engage
If Validity provides controls for users to see how competitors and the industry is performing in their email marketing campaigns, then we will see an increase in Engage sign ups, because we assume users struggle to understand the market.
- Lead product designer for the market intelligence agent
- Collaborate with SDRs, AMs, and key stakeholders to validate product-market fit
- Collaborate with design team to build, scale, and standardize design system for multi-agent use.
Developed a core UX loop
The featured insight work informed future implementations and thinking for not only the market intelligence tool but our other two agents and how users interact with AskEngage across the app.
Informed visual design and UI direction for the app
Having no visual design or UI direction in place, my exploration of the tool helped inform team discussions and decisions for the design system and visual language of Engage.
Improved my personal process
Utilizing Claude + VS Code sped up my iteration process and speed to value for users, for faster learning and improvements.
Why Launch a New Product?
Validity has a suite of tools and services addressing the full email lifecycle. In its current state, Validity lacked an AI angle for its existing platform and felt it was lagging behind the competition and how email teams were working in the new AI era. Enter the Validity Engage Platform. The Engage platform has three core agents addressing email marketers' needs in both the pre-send and post-send workflows.
The Market Intelligence agent allows users to always have a pulse on what is happening from a strategic perspective in the email landscape, all while being trained on the Validity Intelligence Network.
The Validity Intelligence Network is a network of global email signals from every inbox across the world. With 25+ years' worth of data, over 2.5 billion data points, and 30,000+ brands, the Validity Intelligence Network is the foundation used by the Market Intelligence agent.
A Proof of Concept
Behind the scenes, our data science team built and trained an LLM to create an internal tool to explore insights we could provide.
The output categorized signals into what was and wasn't working and included a verbose summary of the selected competitor with email examples based on the LLM's findings.
This tool was the foundation to shape our MVP experience and allowed me to learn more about our data and possibilities.
Getting Started for MVP – Team Alignment
After a walkthrough of the Validity Intelligence Network, and the proof-of-concept tool, I held a two-day assumption mapping session with developers, data scientists, and a PM to ideate and align on core assumptions we would start designing and building our agent experience around.
From the Workshop
After taking a look at the competitive landscape and ideating, we had our "leap of faith" assumptions that we wanted to focus on and later validate with users as an MVP for the market intelligence agent:
Users will want to see relevant insight data for specific industries vs. an individual company
Marketers already follow their competition in some manner, but struggle to see the larger picture in their industry and how it is moving.
Users care more about overall email strategy from competitors than individual journeys
Journeys refer to an email journey, i.e checkout experience, and the team believed how competitors were positioning themselves in the landscape would be more valuable.
Users want "low effort" email strategy recommendations
Bite-sized, easily digestible findings would be more impactful, implementable, and measurable for marketers as they work on their email strategy.
These assumptions and a core hypothesis would be the anchor for the team moving forward as we developed a proof of concept and scaled the agent.
Anchoring the MVP Features
For MVP, the team was focused on enriching the data we have from the internal tool and making it valuable and digestible for users. We hypothesized the tool would only be as powerful if the information we were providing was accurate, relevant, and actionable.
The team anchored around three core features for launch that address our assumptions:
Featured Insights
A weekly roundup of 3 to 6 insights the LLM ranks based on a user's tracked brands.
Custom Insights
Controls users can specify what the LLM should look at in their industry and be notified the moment it happens.
Signal Feed
Real-time signals for what is happening in a user's email marketing landscape.
Below, I'll take a closer look at designing and implementing featured insights for the market intelligence agent.
Featured Insights
Featured insights are the first thing users see when they land in the tool. We needed to design a feature that users saw value in right away.
Key considerations:
1. What type of information does an email marketer care about in their strategy?
2. What information can the LLM accurately pull for the insights?
3. How can a user dive deeper and interact with an insight after generation?
4. From a business perspective, how do insights consume credits? How can we ensure users want to continue consuming credits?
The Core UX Loop
The team brainstormed what information would be relevant for users based on our existing industry knowledge and our internal tool, and devised a core loop that would serve as the basis for featured insights
The loop included generating insights based on a user's company and tracked brands, then allowing the user to click on the card to learn more, or clicking on a prompt pill to kick off an AskEngage response.
The insight itself would include the following:
- An LLM-generated header element
- A core metric (expressed as a whole number or percentage)
- An LLM-generated "prompt-pill"
For speed to value, a user's company and tracked brands that were required to run a featured insight report were set up internally by our team on the back end and then later addressed as a front-end feature for users. We were prioritizing whether the information present in the insights was valuable and accurate for users.
AskEngage
AskEngage is a feature we paired with all agents, allowing users the freedom to ask questions based on our Validity Intelligence Network.
It acted as a core mechanic for credit consumption and a foundation for diving deeper with featured insights.
My Design Process. Building with Claude + VS Code
The product team was equipped with Claude licenses for their design needs, and we were able to develop live-coded prototypes in a separate repo, hooked up to our design system. This was a fantastic way to move fast and iterate quickly to get a proof of concept in front of users.
This was my first time implementing an AI-driven workflow into my personal process, and I wanted to highlight some points from my experience:
Pros:
- Ability to iterate extremely fast and efficiently.
- Real interaction states and responsiveness for features gave me better control and understanding of expectations in a live production environment.
- Team buy-in. Because WIP designs were a build, I felt the team and outside stakeholders had a better understanding of the product vision and direction compared to viewing a static Figma file.
Cons:
- Convoluted feedback loop and visibility for larger R&D team. We found it difficult for the team at large to see what designers were working on week over week. We solved this by creating a living webpage with our branch history and implementing a branch naming convention.
- Claude artifacts. When working on a new branch, I found Claude would not transfer everything I needed from a previous branch. I hypothesize this can be fixed/improved by creating a better claude.design skill.
- Handoffs. Because the designer's repo was separate from production, I experimented by handing off features by exporting branching into an HTML file. My squad found this was as time-consuming as using the Claude MCP in Figma.
While working on the Market Intelligence features, I would wireframe concepts in Figma, translate the wireframes into a buildable prototype, and iterate in VS Code based on team feedback.
Concepting Card Designs
Now that I had the kit of parts for a feature insight, I started concepting designs for the look and feel of the card. At this time, we did not have a finalized direction for a brand or UI, so I took it as an opportunity to push the boundaries and have an open discussion with the product team about the look and feel of Engage. While designing, I considered the visual hierarchy, the interaction of clicking a prompt pill, and the layout structure of the dashboard view, as work for custom insights and the signal feed was starting soon.
Claude
Claude helped me rapidly explore visual direction and iterate on placement of the core components of a featured insight card. I was easily able to pull these cards back into Figma via their MCP to utilize for future exploration.
Exploring Featured Insight Page Layout
Users could click a featured insight card and explore in-depth relevant email examples that the LLM pulled, and chat with AskEngage to dive deeper. I explored the layout, how users could ask additional questions, and the content of the page.
From prompt exploration of our internal tool, I knew our LLM could do a lot more than just pull email creatives. I also used this opportunity to push the types of content we could pull from featured insights, such as data visualization charts, rankings, and more.
Insight Page Decision
Due to the large lift in data enrichment and development work, the team decided to focus on organizing and displaying email creatives paired with the ability to dive deeper into an insight with AskEngage for MVP. Pulling in charts, rankings, and other information would come later. We wanted to stay diligent and focus on our core vision.
Addressing Feedback
Our feedback loop included key stakeholders from the leadership team, SDR's, AM's, and beta users of Engage. Feedback was aggregated via a Slack channel, and I created a Claude project to capture recorded calls by our sales development and account manager teams.
In the next iteration, I was focused on solving the following:
Insight quality skews quantitative. There's no coverage of qualitative strategy shifts like new product launches, positioning changes, or shifts in who a brand is targeting.
The featured insights were too structured for the content required in the card UI. I wanted to take a look at what it looked like to strip back the card UI to its bare bones.
Insights aren't organized by how actionable they are. Positive findings (keep doing this) and negative findings (a competitor is outperforming you) get no visual distinction, and easy wins get buried next to harder, more strategic findings.
How can I think about adding a visual distinction to the card UI that lets users easily understand the content of the insight?
Users are forced into a fixed count of six cards even when only two or three findings are genuinely fine.
For the initial report, we required the LLM to always generate 6 cards for users. With this change, I wanted to take a look at different layout structures and hierarchy for the section.
Lean into email!
Users struggled to connect featured insights were tied to email creatives and performance.
Additional Card Exploration
This round, I used a mix of Claude and Figma wireframing to explore a variety of additional layouts and card designs for featured insights.
Where I Landed
After an additional round of exploration and continued feedback, the insight cards stripped back any "AI-gradient" coloring to reduce the amount of clutter for users. Cards included logo previews, email creative thumbnails, and a more dynamic label + heading, and subheader for better content clarity. Prompt pills were removed from the card UI and, replaced by an action footer with an explore link and bookmarks icons. When clicking the link, users would be navigated to the insights page, and the AskEngage panel would start an LLM response so users could go more in-depth with the insight.
Where the Product Stands
Using the framework above, I balanced incremental improvements to featured insights with rapid experimentation across custom insights and signal feed work, incorporating feedback to continuously refine the product and accelerate toward product-market fit.
What I Learned
Ship fast to learn
Building a 0-to-1 product and finding product-market fit means you might miss the mark a couple of times, and the only way you're going to figure it out is by shipping and learning. Then repeating the process 20x.
Tight-knit collaboration
Two other product designers were building different agent experiences on the Engage platform. It was imperative, to be in constant communication, discussing the look and feel of Engage, collaborate on product direction, and standardize design system patterns.
If I Had More Time
Make insights more actionable for users
Users noted that insights weren’t as actionable as they would like. Users could digest the information, save it for later, and even download the findings for their own needs. But the agent wasn’t telling users how the insights were fitting into existing marketing strategies.
Meet the user where they are working
Internal workflows are evolving at companies, and I believe it's important to meet the user where they're at. What would it look like to receive featured insights directly in Slack? Or be able to chat with our agent in a Claude project?
Continue to ideate on how to increase credit consumption for users
As the core business model, how can the market intelligent agent continue to introduce value for users while addressing credit usage from an account's credit package? Can they upload their own email to run a competitive analysis and see what improvements to make based on our industry knowledge?
Track core metrics to inform product direction
- Credit consumption by feature
- Time-to-value from sign-up to first featured insight report generated
- % of users who generated an insight and then interacted with AskEngage
More Projects
