At Microsoft Ignite, Microsoft Dynamics 365 Marketing announced a range of new AI features. We strongly believe in the power of AI to help businesses and their customers grow. We also recognize that these new technologies have the potential for misuse and harm.
That’s why in Dynamics 365 Marketing, we are taking an intentional and rigorous approach to upholding Microsoft’s responsible AI principles. AI requires scrutiny, thoughtfulness, and research to first understand potential impacts on people and society, and then seek solutions that mitigate harm.
As Satya Nadella says, Microsoft runs on trust. And trust must be earned in the short term as well as the long term. In Dynamics 365 Marketing, we see responsible AI as an opportunity to demonstrate trustworthiness as well as a path for innovation—a way to minimize harm and expand our capacity to provide useful and delightful experiences for our customers and their customers.
Let’s take a closer look at some of the work we’re doing on a new AI feature called Content ideas.
What is Content ideas and how does it work?
The Content ideas feature in Dynamics 365 Marketing helps marketers get inspiration for emails and create their best content faster. Marketers can type in a few key points, and Content ideas will generate original content based on those key points. Under the hood, Content ideas also references the customer’s past marketing emails so it can generate ideas that are similar in tone, structure, and style. This is a powerful advancement I’m very excited about, where marketers never have to start from a blank slate when writing content.
The AI technology behind this feature is a large language model called GPT-3, developed by OpenAI and currently available in an invite-only preview as part of Microsoft’s Azure OpenAI Service as well as through OpenAI’s API. GPT-3 can perform a wide range of natural language tasks, including summarizing text, analyzing text for sentiment, and—as it’s applied in Content ideas—generating original text that looks like a human wrote it. GPT-3 is one of the leading examples of the type of AI model the industry is moving toward, rapidly accelerating AI capabilities that bring value to customers.
But large language models such as GPT-3 come with risks, including generating content that isn’t factual or content that reflects the biases of the dataset used for training—which, in the case of GPT-3, was approximately 45TB of text from the internet. To mitigate such risks, OpenAI and Microsoft are committed to helping customers identify potential safety issues that could arise from using GPT-3 and providing best practices for safety. And at Microsoft, as we incorporate these kinds of technologies into our products, we’re also investing deep thought into how risks might show up for our customers in our specific scenarios, and how we can address those challenges.
Here, we’ll zoom in on one area we’re looking at closely for Content ideas: the user experience (UX).
UX questions for a responsible AI approach
Dynamics 365 Marketing has focused on human-centered research to deeply understand the needs and aspirations of marketing content creators, as well as explorations in UI design and data science, to translate responsible AI principles into a powerful UX that elevates and empowers human expertise.
As Charles Lamanna, Corporate Vice President of our Business Applications and Platform says, “An emerging technology like GPT-3 is such an exciting breakthrough in innovation. I’m proud of products like Dynamics 365 Marketing, where teams are working across engineering and design to intentionally think about how we responsibly bring AI to our customers.”
Early research leads us to these key questions:
1. How might we build transparency around how Content ideas works, so people can use the feature to meet their specific needs?
Setting clear expectations about Content ideas’ capabilities and limitations is essential, both to help people achieve their goals and to prevent people from using it in a way that isn’t intended. The more people understand how GPT-3 uses their key points to generate original content, the easier it is for them to craft key points that will get them helpful suggestions. In the current UX for Content ideas, we offer a “Learn more” panel from multiple points during onboarding. This panel is structured similar to a FAQ, addressing top questions about what the feature does and how the technology works. We’re also using design principles such as progressive disclosure to give people relevant information at just the moment they need it. For example, after marketers have submitted their key points and are waiting for Content ideas to generate suggestions, the loading screen sets expectations around potentially seeing unexpected results and offers tips for what to do next if none of the suggestions are a good fit. We’re continuing to explore ways to help people better understand how their choices affect the system outputs.
2. Once people understand how the technology works, how might we give people more control over the system?
A foundational pillar in human-AI collaboration is making sure people have meaningful oversight and control. The right amount of control helps people make the system work for their goals and context, and helps them build confidence in the system. With Content ideas, we want to empower content creators’ expertise and give them the right levers and buttons so they can use the system in ways that work for them, while automating parts of the process that don’t require human judgment. For example, we frame the feature as a brainstorming and writing partner, rather than a magical tool that does all the writing for you. In the end, the author is in charge—Content ideas makes suggestions that they can choose to use, edit, or ignore. Our research has also shown that content creators want more granular control over generated suggestions, such as being able to copy and paste smaller sections from different suggestions, and the ability to instruct the system on additional attributes such as audience and tone. We’re exploring how to integrate these potential interactions and others along these lines.
3. Once people understand how the technology works and how they can influence it, how might we help them understand their accountability and feel confident about their responsibility for the final content?
Large pre-trained language models like GPT-3 are general purpose and don’t always produce perfectly accurate results, particularly for tasks that require specific knowledge like the latest pricing data for a product. This means that even with detailed key points to start with, Content ideas might include color variations, prices, or sale dates that could look realistic but might not be correct. We want to make sure content creators feel confident in their responsibility as final owners of the content, making sure they have robust opportunities throughout the experience to check for accuracy and edit as appropriate. Additionally, in our “Learn more” panel, we directly answer the question, “Can I use the suggestions word for word?” (The short answer: Yes, as long as you review carefully for accuracy and appropriateness.) As we move forward, we’re exploring ideas such as a reminder to check for accuracy before someone adds a suggestion to their draft, or a feature to flag details that might benefit from a close read.
4. How might we measure the success of our UX to capture how well we are building trust, supporting creativity, and empowering user confidence in using Content ideas to meet their goals?
Success in UX is often measured by things like: Were we able to help someone accomplish a task more quickly? Was the task done at a higher quality? And are people satisfied with the result? Content ideas invites us to consider additional ways in which people might have a successful experience. For example, since the feature can offer a range of possible ideas for a content creator to consider, if someone is looking for multiple avenues of inspiration, creativity might look like generating many ideas and then building new ideas from there—rather than copying and pasting a single idea. In our research for Content ideas, we’re considering how to qualitatively assess people’s experiences—such as how much they felt that the feature helped them become more creative, and how confident they were over having control over the final text—so that we have a more holistic understanding of where we can improve the experience to support a range of user goals. We’re also exploring ways of gathering feedback in the UI to help us understand the usefulness of generated ideas.
These are hard questions, and we don’t have all the answers yet. But we are committed to developing solutions that minimize harm and empower human expertise, while always providing our customers and our users an amazing experience. Ultimately the goal is to build high-quality experiences that establish appropriate trust, bringing sustainable value to people and businesses. We’re educating ourselves and trying to learn quickly so that we can achieve this vision for our future and yours. I’m proud that Content ideas is one of many areas Microsoft is looking at when it comes to responsibly implementing AI technologies like GPT-3, such as the recently launched Ten Guidelines for Product Leaders to Implement AI Responsibly and the new responsible AI dashboard.
Learn more
Content ideas is available in preview as of October 2021. Learn about this and other new AI-powered capabilities and more in the 2021 release wave 2 for Dynamics 365 Marketing.
And to learn more about how your organization can elevate your customer experiences, visit the Dynamics 365 Marketing webpage and sign up for a free Dynamics 365 Marketing trial to explore real-time customer journey orchestration and the other rich capabilities offered in Dynamics 365 Marketing.
