Project Cortex deep dive | Intelligent knowledge management built on SharePoint (Microsoft Ignite)


(Upbeat music) – Hello everyone, and welcome
to Microsoft Mechanics Live. Coming up, we’re joined
by Naomi Moneypenny, for a deep dive on Project Cortex, a brand new service that
builds a rich knowledge network of the people, knowledge, and
skills for your organization. Now we’re gonna show you how it works under the covers with Microsoft
Graphs, Search, and AI and a new approach with machine teaching to help extract important data. But please join me in
welcoming Naomi Moneypenny’s gonna guide us through Project Cortex. Welcome to the show. (Applause)
– Thank you so much Jeremy. It’s awesome to be here and awesome to see so many people. – Alright, so we’re here to do a deep dive on Project Cortex. So at first glance we think about this, it looks a lot like kind
of knowledge management, ECM, Enterprise Contact
Management, Enterprise search. How should we think about Project Cortex? – Well, it’s really, it is a combination of all of those things. So, it’s really why we think
this is so revolutionary. It’s the culmination of a bunch of work that we have with Microsoft
Research, Microsoft Search, Microsoft Graph, Azure, Microsoft 365, and our Power Platform teams. – Alright, but we wanna dig into how it actually works. We got a room full of
admins, computer scientists, how does it work? – Alright, I’d love to show you. So, a little bit of important currencies that we think about inside
of our organizations. They’re really the
data, and then of course all of you people who
bring their knowledge and expertise and all of the skills. So here, I can see I’m in an email and this happens quite
often to us at Microsoft, here’s an acronym that
I don’t know, right? – [Jeremy] It happens…
too much at Microsoft. – [Naomi] It does.
(laughing) It really does. So I see this acronym, I can immediately get
that highlight on the page and I am seeing a topic
card as a part of that. And you can see this topic card has understandably what that acronym means, and then I can see there’s
a quick description here and there’s some people and the resources that are connected here. And so it gives me a quick
overview of what’s happening. Now if I go to a SharePoint page, maybe I’m catching up on
some news about my team or another team that
I’m not familiar with. You’ll see right here in the context of that beautiful communications page that I’m getting these highlights here. So these highlight’ll let me
explore a little bit more. Notice that we only have
the highlighted term once on the page, right. We’re not trying to have
a whole bunch of those every time the word
Project Core is mentioned. We’re actually surfacing the
right kind of information. You can see I’ve got a
couple of actions on there like following as well
as suggesting an edit to this topic card, so that’ll help us with the curation that
we talk about later. So when I go to this page
I actually get a topic page that’s been generated by AI,
and that’s a wonderful thing here is that AI is doing that heavy lifting behind the scenes. It’s actually mining through
all of the content that we have in the Microsoft Graph and then generating this page as part of it. So you can see I’ve
got a description here. It’s extracted that automatically. I’ve got people who are referenced here. I’ve got some resources
that have been added as well as ones that have been curated and then ones that have
been discovered by AI too. And then this is quite possibly
my favorite feature here which really helps you
to understand the context of topics across your organization and so looking at how
things might be connected, maybe projects that you didn’t know about inside of your organization,
really being able to bring all of that together right here. – It’s really a great
visual view of kind of where emanates from and all
the different connected parts. – Mm-hmm, exactly, and
then of course we have here as part of our year of Yammer too. Looking at the way that we
can learn from tacit knowledge inside of an organization, so often times people will
have communities of practice. They wanna be able to get
the whole of the answers within those, and here I can just bring all of that information directly right here on top of a topic page. So that’s kind of a great way to think about that topic page. Now let me show you a little bit about how those things roll together so for example we’d have lots
of projects and campaigns and initiatives inside of an organization, and so we’re building a knowledge center as kind of a new destination there. And then this knowledge
center is a rich place to be able to keep up to
date with what you need. And you’ll see all these areas here, but of course because it’s powered by the wisdom SharePoint here, when I got to edit this page you’ll see that I can easily add
in different web parts. So I click on add web part, pretty normal, but you’ll see as part of this we have an exciting new range of knowledge powered web parts. And those web parts enable you to take this great functionality and put it into the all
the other destinations that you have inside of SharePoint. So you already have your
internet portals and other sites, you can use all of this great capability to make all of those experiences richer. So this is really then of
course surfaced as it needed to go through your every day work, and it’s connecting that
people and knowledge and knowledge and skills all around the information
in your organization. – Awesome, so all these things
that are really accurate out of the box, you don’t have to worry about kind of building
these sites from scratch. You can see the integrated
experiences in context there. The nice thing is it
actually helps then unlock and democratize all the
different tribal knowledge that might exist within your organization. So the things that you’d
normally have to do like get coffee meetings
with a dozen different people as you come in, or also
as somebody leaves, not enough coffees in the
world will actually bring that knowledge back sometimes, if they’ve left the organization, maybe gone to a competitor,
all that stuff is able to be figured out by Project
Cortex kind of in the span of how it’s actually detecting and extracting the different
information signals that it’s getting through the Graph. And so you have all the
knowledge and the skills to be able to collaborate with
the folks that you need to. But how did it end up doing all of that work kind of behind the scenes? – Well it’s really
bringing together a number of those technologies that we talked about under the covers, so we’ve
got Microsoft Search. We’ve got of course the
power of the Microsoft Graph, which helps to understand
all of the relationships and the activities around
the people, the entities, and the information that you’re probably already familiar with, capabilities like Most
Recently Used Documents or recommending content when
you’re inside of SharePoint as well as thinking about
AI and machine learning. And so we really have this
foundation for trusted knowledge. And so when you look at how
this works, it’s working by systematic inferencing. You think about the
unsupervised machine learning that it’s using to infer
things, and it’s able to build up a huge knowledge base starting with very little information. So in this example that we’re showing, the Project Cortex is able to infer that CORE in this case is a noun, and it’s able to distill that through all of the source materials, those files, those documents that we showed earlier. And it infers other properties to pull out canonical concepts based on the frequency and
reference and associations. Then it discovers more
correlated signals across those files, the
services, and the content, and it’s able to find all
of that related material. So as that’s building up, it’s really using a graphing structure to understand the relationships. It’s weighting the
importance of the people who are in the organization
and the contributions and the projects as well
as even your relationships inside of the organization’s org chart. So of course the more that it infers, the more confidence it builds in the reliability of the system. So gradually it’s getting
a rich schema of data and associated people on all
of the kinds of properties and it’s able to present what
we think is a really great way of a consumable way of how
you’re working every single day. – Awesome, so it’s all machine generated, but what is the role then
of kind of the project owner or the project manager and
the experts then in this case, given that the machine learning’s
building a great baseline? – So it’s a great question,
really thinking about how we’re generating all
these pages automatically, and they really are wiki-like
so that experts are able to contribute to these and
to refine the information and the wording and how it’s presented. And then of course in
the future you’ll be able to apply this to how you get work done. So for example, if we’re thinking about our construction company example, you heard earlier this
week from Simon Denton that Mark McDonald has
a construction company, and we’re thinking about how we generate a pitch
for a new campaign. And even though we might just start off with just a client name and a project name that are known at this
point, you’d be able to understand what needs to
be done in terms of steps to create the pitch and see
who else might have worked with that client to help
create a virtual team. And then of course as
you work on this pitch, Project Cortex will have surfaced
that relevant information to you to kick start your
knowledge and understanding. – And we saw this is
a really rich baseline of organizational intelligence
that you get out of the box, but there are things that we can do to make it even smarter, right? – That is really true. So the system will really
make itself smarter with all of that built in
AI, but we can also help you to get the right information
from your organization, from different formats
such as the forms piece that we have with AI Builder. So I’d love to be able to
show that to you right now. When I think about a
purchase order, for example, that’s coming into your organization, so I’m just putting up an example one here to show you what this looks like. You can see I’ve got a bunch
of standard fields here, and I’ve got a status message
here on the bottom too. And then I can actually pop over to my Accounting Content Center here, and so I check out this Content Center. It’s kind of a new destination if you will for having a business
process, and it’s a great way to be able to consolidate
all of that information. So I wanted to show you an example of one that we’ve already set up earlier. So here’s a document
library, and you can see that it’s actually mined
already a bunch of content from that purchase order. Now if I wanted to upload one
of these new purchase orders, I’m gonna add this in here
and just grab one of my files from my desktop here. I’m gonna grab one of
these, and I’ll call it, let’s grab one of these
Fabrikam ones here for example. It should be great. We’re gonna upload that,
and you’ll see this would be the sort of typical end
user experience, right. There’s a model. It’s already been set up
here for it to process, and you can see it’s
uploaded the file right now. And then we’ll give it a second, and it’ll actually kick
off the flow for it and be able to extract that information. – So you can upload a PDF,
or in this case a docx files, those kind of things,
it’s gonna be able to take that semi-structured, I
would say, kind of content and be able to schematize it effectively so it can actually use AI and
intelligence over it right? – That’s exactly right, so
there’s a wonderful thing here is that we’re able to use
this right in the context of the work that you’re doing and be able to understand all
of this information coming in. And it is extracting that
information automatically so you can see it’s populating
content here, directly here. So it’s kind of a great
experience from that perspective, and I can show you a couple
of different other experiences that we have too. So let me show you inside
of the Content Center. If I was to create one of these models, which you might be asking,
“How can I do that?” So in this case the document library. I’m gonna go ahead and be able to create my document library,
so we’ll say ChapmanPO, something like that. I’m just gonna create a
purchase order documents because you know you have a lot of purchase orders Jeremy all the time. – [Jeremy] Yeah, a lot, yeah,
I’m purchasing a lot of stuff. – [Naomi] You see we’ve
natively integrated right into AI Builder right
here, so I can create a model. And that’ll send me over to
the AI Builder interface, and you’ll see we go directly into the Forms Processing area. So I’m just gonna call this
Chapman, let’s do that, create this library.
– This is all Power Platform AI Builder?
– Exactly. Exactly. And it’s the wonderful
thing here is we think this is a great opportunity
to be able to put all of that machine teaching
information directly, machine learning information excuse me, directly into the Power Platform as well as using structured
and unstructured documents too. So I’m gonna upload a bunch
of those purchase orders, and we’re gonna ask the
Power Platform here, in this case AI Builder, to be able to analyze those documents. And it’s gonna be looking for those fields that we saw earlier, so you
remember in Word here we had, in this case, there was a benefits notice. We also have an idea
around how we would put all of that information together
from your purchase order. So there’s kind of two
different steps here if you think about it. So the AI Builder really is
really focused on this area where the formatting of the
document looks very similar to each other, and so it’s
a great way to be able to put all of that information together. And then when it has that
information we’re able to easily show that inside
of the Power Platform, and so you can see it’s
done some analysis here. – [Jeremy] There it is. – I’m gonna grab some of
the fields that I want, and you’ll see that it’s
actually loading that up. And here we have some analysis, so for example, it’s put bounding boxes automatically here.
– It’s actually found some things I think that it’s,
– Exactly. – that are unique, yeah. – [Naomi] That’s right. So you can see I can pick out
the date field, the PO field, etc, the Ship To, you
know whatever I wanted to extract from here. I’m just gonna extract a
couple of those fields here. I’m gonna click done, and
then we’re gonna go next. And then we’ll go ahead
and train this model and use this model the
way that I want to do. So it usually trains pretty quickly, obviously we’ve given some very
standardized examples here, and then we’ll can check out my model. So I’m gonna say go, and in this case here it’s saying my model’s ready. So we’re ready to
publish it to SharePoint. And so if you think about
it we’re now creating that connection effectively
between SharePoint and AI Builder so that I can apply it to the document library. So this is a great opportunity
and one of the reasons that we really wanted
to make this connection to the Power Platform because
we really think it’s the best around understanding both the construction of the structured documents that you have inside of the Power Platform as well as the unstructured data that
you have inside of SharePoint. So you can see here it’s gonna
process this form information from AI Builders and SharePoint. I’m gonna click Continue, and
it will use that flow inside of my ChapmanPO library, so
you can see behind the scenes what it’s done is actually created all of this information here, and obviously we wanna not
have the end users have to deal with that, but
basically it’s extracting that information and
making it all possible. And so when I go back to
your document library here that we created you’ll notice
that it’s actually created some columns here automatically for us. So you see how it’s actually created, so there’s the Modified By, the DATE, and the PO ones automatically. So those have been created
by AI Builder directly in here which means that once
I now have that information I can go ahead and upload a bunch of files and we can start processing
some of that information. So I’m just gonna grab different
ones than we had before. So I’m gonna grab those. I’ll upload these documents
– And these are image files now right?
– Yep. – [Jeremy] So it’s even using less structured data than before? – [Naomi] Exactly, so it’s jpegs or PDFs, and so as we grab that information and you’ll see it process
it in just a second here. So we’ll have flow kick
off in the background and have all of that information too. So here’s a great piece of it, It’s all good. It’s connecting, and
then it’ll mine through all of that data that we had to. So I just wanted to
showcase how that all works. – So is this kind of teaching then, teaching it how to work
using machine teaching? Or is that something that’s… – So that’s really different. – [Jeremy] Okay. – So it’s an interesting thing
here to be able to notice around these different kinds of documents. So if you saw before we had
this HR benefits example as well and that’s a very unstructured document, and so you might think
about how you use that inside of your organization. So one of the areas that
I wanted to show you is how can you actually use
some of that information too. So this is the machine teaching example. I’m showing here that machine
teaching, in this case, we can see we’ve extracted a
whole bunch of that information from those HR benefits,
and we’ve even got things like Retention labels
that are part of it too. So all of those areas are here. And then if I wanted to teach the machine how to actually understand
some of this I could select these training files. I’m gonna start training, and in this case I’m gonna click on HRBenefits. I’m gonna grab one of our extractors that we’ve already built in advance so I’m gonna grab HR Benefits Provider, and then I can actually help to teach the machine the same
way as I would teach a person. – [Jeremy] Wow. – So for example if we were looking at these documents
together, I wouldn’t say, “Oh Jeremy, you need to
go match every single one “of these patient names
and patient providers.” I’d probably tell you to
look for some key words that are coming before it or after it, and that would actually
help you to understand this. So in this case you can
see that there’s some of these status column here. It’s all gray right now. If I go ahead and test my model you’ll see that it’s gonna analyze this information, and it’ll actually show me where it actually has a great result. So if I’ve taught it already that the word insurance
provider, for example, comes before the name of the provider, you can see all of that coming through. And you see that a lot of
these have gone green already so that means that my model’s working. Now if I click on one where
I haven’t gotten a green dot, you’ll see that it’s
actually trying to identify that automatically so that
green text is actually surfacing up that machine teaching
information right there. So these are kind of the
lots of different ways that you can bring this
information together and actually be able to use that inside of your organization. Now while we were doing that you noticed that the AI Builder
model processed earlier. – Yep, looks like it’s
– So here’s, for example, – fully filled in now.
– Exactly. – [Jeremy] All the columns are populated. – [Naomi] Got all that great
metadata as part of this. And then of course you can
use this as part of this too to be able to actually kick off a flow. So the wonderful thing here is I could actually get a
status message in this case from understanding what
this is happening here, and then I can actually post
this directly into Teams too. So it’s a great way to have all of those work flow coming together. – It’s amazing using the Power Platform showing how something that’s very low code can actually add all this intelligence, and you’re even able to integrate things like approval steps and kind
of the knowledge mining process that was in there. You could use Power Platform
even further as part of how you integrate with
Teams, but what happens then, so you worked a bit with unstructured data in terms of the jpegs, are
there other types of data that you can work with? – Yeah, so those machine teaching
piece that we were looking at earlier, so this
information is obviously that very unstructured
metadata, and so we’re able to extract that automatically
in the same way. So it really is using the same
benefits that we have here. So we’ve really used
machine teaching to improve all of this accuracy further, and then we’re able to do that. – Awesome, so I know that a
lot of admins are watching and probably wondering about the best way to approach security compliance cause there’s a lot of information here. You’re talking about purchase
orders and all the things that are kind of very
important cause there’s a lot of financial transactions and
stuff that we’re even showing here in this demonstration today. But are there ways to be
able to lock that down and make sure that we’ve got
our information protected? – Yeah, so it really just works like any of the other productivity
solutions that we have out there. So whether that’s Outlook,
Teams, whether it’s file sharing OneDrive or SharePoint, and any of your existing
information policies, so information classification
and labeling becomes more important and even where the labeling of your information is
tied to specific policies such as Microsoft information protection. – And we should show kind
of what the differentiated permissioning then would
– Oh, that’s great. – look like because like the
rest of the Microsoft Graph, it’s kind of looking at
what you have access to, what I have access to, and based on that it’s actually tailoring the way that it’s surfacing up information. So looks like you’ve got
yours already ready to go, and I’ll just open up on mine. On my machine, I’ve actually
got lesser permissions, unfortunately, than what you have. So I’m gonna go ahead and go
into the same project atlas and the same topic card. I’ll click show more, and
if I scroll down you’ll see in this case if we look at
our two people card areas, I’ve got a few less documents let’s say, for example, than you do. We’ve both got six people
that are connected to it, but you’ve got five different documents. I’ve only got permissions to view four, so it means that we’re able
to use the permissioning and the Search components
there to just expose what I have access to. So all of this is a very
early look at Project Cortex, but still you’ve done a great
job in terms of bringing all these things together in
such a short amount of time. But when can people expect
to start seeing this and start trying it out? – So we really have a
number of organizations that are already kicking the tires here. So it’s in private preview right now, and we will be available
in the first half of 2020. – Okay, very cool stuff,
so everybody excited about what they’re seeing with Project Cortex? (audience applauds and cheers) Okay, so we’ve already
shown a little bit in terms of what’s in Word, why
don’t you show us again some of the in-context experiences that we can expect moving forward cause it’s gonna get even better
in terms of our day to day. – Exactly, so we really
think about this being shown in all of those Office applications that you use every single day, so in this case I’m in Microsoft Word. And I can go ahead and use
the Smart Lookup functionality in this case and actually
do that Smart Lookup. And I will get that same
topic card being shown here, and it’s a great experience too. And then I can even use this
inside of Microsoft Teams. So not only can I just
get those notifications of work flow coming through Teams, I could also ask a Knowledge bot to help curate a one on one
discussion that we’ve had and add that to the topic page as well. And then we’re really continuing
to build out more and more of those in context experiences across all of the Office applications. – This is really amazing stuff. It brings together kind
of all the AI pieces, the Graph pieces, all the
knowledge that people need to really get up and running
quickly in their work. Amazing stuff with Project Cortex. I think everyone here is loving this, but where can folks go to learn more? – So they can really sign up for our Project Cortex
preview aka.ms/ProjectCortex. – All right, and of course
these are all really great tips and of course keep watching
Microsoft Mechanics. We’ll continue covering
Project Cortex and much more. So if you’re a SharePoint
admin, an architect, somebody’s just interested
in great technology, keep checking back. That’s all the time we have for this show. Thank for watching, and
we’ll see you next time. (audience applauds) (upbeat music)

, , , , , , , , , , , , , , , , , , , , , ,

Post navigation

Leave a Reply

Your email address will not be published. Required fields are marked *