Intelligent Technology Depends on Machine Learning Access to User Data

Some years ago, I wrote about how Outlook uses machine learning to predict words to insert in messages. This was an early example of machine learning in Outlook. Text prediction is common practice today and we almost expect applications to include machine learning to help us compose notes, documents, and responses. Given the introduction of ChatGPT and Bing’s AI Bot, some worry about the prospect of increasing amounts of machine-generated text and its effect on human creativeness. It’s definitely a story to follow.

Over the last few years, Microsoft has steadily increased the use of “intelligent technology” in Outlook. Currently, the range of features covers features like birthday detection to text predictions to suggested replies, controlled through OWA settings (Figure 1). Regretfully, the Set-MailboxMessageConfiguration cmdlet doesn’t currently support updating these settings for a mailbox.

The combination of Microsoft Research and product engineering groups has driven the introduction of intelligent technology in OWA. For example, Outlook’s suggested replies feature is underpinned by the Azure Machine Learning Service.

Outlook Desktop Lags in Intelligence

Outlook desktop clients receive the intelligent technology features after OWA. This lag has always existed, but at least we can respond to email with an emoji. Oddly, there’s been a few recent reports of Outlook for Windows failing to display the “show text predictions while typing” setting in its options (here’s an example). I don’t see the setting on one PC and do on another, both of which run the same build of Outlook click to run. I even updated the system registry at HKCU\SOFTWARE\Microsoft\Office\16.0\Common\MailSettings to set the InlineTextPrediction DWORD value to 1 to enable text predictions with no effect.

Microsoft Processing of User Data

One thing that people get worried about is the notion that Microsoft “reads” their email to create suggested replies and to build models for text predictions. It’s true that Microsoft processes email to create the suggestions and predictions used by Outlook, but the important thing is that the data used by the learning models constructed to help machine learning understand how individual users work with text remain in user mailboxes. Microsoft doesn’t gather information from the 380-odd million active Office 365 users to improve its detection algorithms. The general foundation for the models come from public data (and I imagine, messages circulating within Microsoft), but the tweaks to make those models personal remain private to the user.

In its user documentation for suggested replies, Microsoft says that “Suggested replies are generated by a computer algorithm and use natural language processing and machine learning technologies to provide response options.” It also says that “Outlook uses a machine learning model to continually improve the accuracy of the suggestions. This model runs on the same servers as your mailbox within your organization. No message content is transmitted or stored outside of your organization.”

These statements don’t mean that the machine learning code runs on 300K Exchange Online mailbox servers. Instead, Microsoft uses a concept called Privacy Preserving Machine Learning (PPML) to transfer data to specialized AI computers in the Microsoft cloud. After processing, Microsoft erases the source information from the AI computers and background agents update mailboxes with user-specific results. It is this information that Outlook consumes locally when dealing with messages.

Email is worldwide, but the structures and syntax used by different languages means that Microsoft’s machine learning processes is limited to certain languages. For instance, at the time of writing, suggested replies are available in only 22 languages.

I’ve heard (but can cite no public evidence) that AI processing occurs on a tenant basis to allow some consolidation of generic results at the tenant level. For instance, if many users in a tenant use “OK” as a standard response, it’s likely that machine learning will consider “OK” as a prime candidate to be a suggested response for everyone in that tenant. The consolidated generic data remains in the tenant.

Viva Insights Processes User Email Too

In addition to the way Microsoft processes user email to understand text patterns, Viva Insights looks through email to detect commitments made by users. Its MyAnalytics predecessor started to scan emails for commitments in 2018. When users open the Viva Insights add-in or use the Viva Insights app in Teams, they see recommendations and insights derived from the contents of the calendar and inbox folders from their mailbox.

Among the information Viva Insights highlights are messages that might contain commitments that the user needs to follow up. Viva Insights displays details of the messages it has found and prompts the users to either note the potential task as complete or add it as a personal To Do task (Figure 2).

Viva Insights also finds messages where the user asks recipients to do something and prompts them to either follow up or mark the task as done.

There’s lots of deep research into finding commitments in email and highlighting those commitments to users. But again, the important thing is that the data used by Viva Insights remains in user mailboxes and is under the control of users.

Worrying About the Data Used by Machine Learning in Outlook

Those with responsibility for compliance and privacy in an organization are usually the people most worried about the processing of user data. With the growth of machine learning and AI-powered “experiences” and the resultant need for access to user data to learn from, this is a good concern to have. In the case of Microsoft 365, many “connected experiences” exist where people consume a cloud service without realizing where data comes from or is consumed.

Personally, I’m not concerned about how machine learning processes my email as the outcome is useful (when it works), but I realize that others have different feelings. It’s a topic for every organization to work through and figure out how happy they are to have Microsoft process their data to create new features.

To finish off, Figure 3 shows how Bing chat answered my question about how Outlook uses machine learning…

---

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Viva Briefing Highlights Data from Viva Insights

Microsoft’s history with the generation of personal insights for users based on their work patterns and activities goes back to the purchase of Volometrix in 2015. Volometrix helped organizations to figure out how to be more efficient based on information stored in user mailboxes and calendars, which later became Delve Analytics, MyAnalytics, and finally Viva Insights.

Viva Insights still aims to help people understand how they work so that they can make better use of their time. The Viva Insights suite includes the Viva Insights add-in for Outlook, the Viva Insights app for Teams, the twice-monthly digest email, and the daily briefing email. All surface information gleamed from user interaction with Microsoft 365 captured in the Graph.

Pausing Viva Briefings

Message center notification MC486289 (December 15) says that Microsoft plans to pause sending the Viva Briefing daily email to users who signed up to receive these messages. From an email perspective, Viva Briefing (Figure 1) and digest messages are not real email because Viva injects them directly into user mailboxes. Although the messages are mail items, they do not pass through the Exchange Online transport system and therefore are immune to processing by components like mail flow rules. Microsoft stamps the messages as coming from a trusted sender, so that makes the direct injection acceptable!

Microsoft plans to pause sending Viva Briefing messages after 15 January 2023. Following the normal time required to deploy changes within Microsoft 365, no users should receive these messages after 1 February 2023. Resumption will follow sometime later in 2023. I haven’t received a Viva Briefing message since last Monday. Perhaps my work life isn’t interesting enough to warrant a briefing, or maybe the pause kicked in early for the holiday period.

More Personalized Information

The pause is to allow Microsoft to make changes to the content of the Viva Briefing messages “to be more personalized for each recipient.” I don’t know what this means because the whole point of Viva Briefing is to deliver personalized content to the recipient. For example, Figure 2 shows items found by Cortana (lurking under the covers of Viva Insights) to remind me about things I might like to follow-up. This information comes from email in my mailbox, so it’s highly personalized.

Cortana finds follow-up items by scanning messages for key words and phrases that indicate when the recipient or sender might be committing to an action. The first item in Figure 1 is an example where Cortana highlights that fact that the mailbox owner made a commitment to take an action. The second item is a variation where the mailbox owner asked a recipient to do something.

I don’t depend on the Viva Briefing to find follow-up actions for me, but I do find the prompts to be moderately useful. Sometimes, Cortana highlights something that I have forgotten to do and proves its worth. I suspect that people who have busier calendars and take on more commitments than I do find the briefing email more valuable.

Finding Who’s Using Viva Briefing

Exchange Online automatically enables new mailboxes to receive the Viva Briefing email. However, users won’t receive briefing messages unless they are active. For instance, if you create a test mailbox and only use it from time to time, there’s no email activity for Cortana to analyze and highlight, so there’s no reason to send a briefing. Perhaps the reduced level of email traffic over the last few days is the reason why I haven’t received a briefing message since Monday.

To discover what mailboxes are enabled for Viva Briefing, run PowerShell to find the set of user mailboxes and check each mailbox with the Get-UserBriefingConfig cmdlet. Here’s an example:

$EnabledMbx = 0; $NonEnabledMbx = 0; [array]$EnabledUsers = $Null; [array]$NonEnabledUsers = $Null
[array]$Mbx = Get-ExoMailbox -RecipientTypeDetails UserMailbox -ResultSize Unlimited
Write-Host ("Checking {0} mailboxes for Viva Briefing status" -f $Mbx.count)
ForEach ($M in $Mbx) {
   $Status = Get-UserBriefingConfig -Identity $M.UserPrincipalName
   If ($Status.IsEnabled -eq $True) {
      $EnabledMbx++
      $EnabledUsers += $M.DisplayName
   } Else {
      $NonEnabledMbx ++
      $NonEnabledUsers += $M.DisplayName }
}
[string]$EnabledUsers = $EnabledUsers -Join ", " 
Write-Host ("Viva Briefing is enabled for {0} mailboxes and disabled for {1} mailboxes. The following mailboxes are enabled: {2}" -f $EnabledMbx, $NonEnabledMbx, $EnabledUsers)

Waiting for Briefings

Microsoft will likely describe the improvements they make to increase the personalized content in Viva Briefing messages when they relaunch the service. Until then, we’ll just have to track commitments and action items using Outlook tasks, To Do, Planner, Project, or any of the other methods available in Microsoft 365.

---

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.