Session	Vinay K. Chaudhri
Duration	1 hour
Date/Time	20 May 2020 16:00 GMT
	9:00am PDT/12:00pm EDT
	5:00pm BST/6:00pm CEST
Convener	KenBaclawski
Track	Whither

Ontology Summit 2020 Vinay K. Chaudhri

Knowledge graphs, closely related to ontologies and semantic networks, have emerged in the last few years to be an important semantic technology and research area. As structured representations of semantic knowledge that are stored in a graph, KGs are lightweight versions of semantic networks that scale to massive datasets such as the entire World Wide Web. Industry has devoted a great deal of effort to the development of knowledge graphs, and they are now critical to the functions of intelligent virtual assistants such as Siri and Alexa. Some of the research communities where KGs are relevant are Ontologies, Big Data, Linked Data, Open Knowledge Network, Artificial Intelligence, Deep Learning, and many others.

Agenda

Vinay K. Chaudhri Textbook Open Knowledge Network
- 10 minute presentation giving an overview of our progress and proposed research for the Phase II of the Textbook Open Knowledge Network (TOKN) project.
- We ask that participants read the TOKN Executive Summary
Feedback and discussion of the overview
Video Recording
YouTube Video

Abstract: College students today face the challenge of mastering concepts in the new subject areas, relating those concepts across multiple disciplines, and one size fits all nature of textbooks. Intelligent Textbooks (ITB) using Artificial Intelligence (AI) and knowledge graphs (KG) solve these problems by allowing students to dynamically interact with the textbook content, increasing their ability to understand concepts, increasing engagement, and thereby, improving academic performance. ITBs offer students easy access to definitions and descriptions of concepts, make connections across different sections of the syllabus, and allow students to pose their own questions. Initial trials of ITBs that utilize KGs have been found to improve student grade outcomes by a full letter grade over the control group that was using a conventional textbook. ITBs have been found especially helpful for underperforming students, thus, broadening participation.

The potential of ITBs to facilitate better learning has been extremely difficult to realize without major investments of time, money, and expertise. The reason is that KGs are currently constructed using human subject-matter experts in a process that is extremely expensive and time consuming. Due to the large investment required, publishers and ed tech providers keep their KGs proprietary, eliminating their utility outside of the scope of the project for which they were created. I will discuss our efforts to create an open source Textbook Open Knowledge Network (TOKN) that can be freely used for creating ITBs and a variety of education technology applications. We are also working towards a novel process and tools for creating the KGs that combine automatic construction of a KG with validation by humans to ensure high accuracy. We envision a community of educators who would co-create TOKN, and eventually take the ownership for its future development and evolution.

Conference Call Information

Date: Wednesday, 20 May 2020
Start Time: 9:00am PDT / 12:00pm EDT / 6:00pm CEST / 5:00pm BST / 1600 UTC
- ref: World Clock
Expected Call Duration: 1 hour
The Video Conference URL is https://zoom.us/j/689971575
- iPhone one-tap :
  - US: +16699006833,,689971575# or +16465588665,,689971575#
- Telephone:
  - Dial(for higher quality, dial a number based on your current location): US: +1 669 900 6833 or +1 646 558 8665
  - Meeting ID: 689 971 575
  - International numbers available: https://zoom.us/u/Iuuiouo
Chat Room: http://bit.ly/2LkAbKj
- If the chat room is not available, then use the Zoom chat room.

Attendees

Discussion

The Role of Ontologies

[11:53] Ravi Sharma: Vinay - Protege is a platform language for ontology yet in your 2 page summary you do not directly address or mention Ontology.

[11:55] Ravi Sharma: You are in relation to MAG through KGs, and you mention Taxonomy only in relation to MAG?

[12:20] Mike Bennett: The O word gets a lot of different reactions and can also end up being reduced to DL based reasoning application ontologies. These would not be a good fit to this business requirement

[12:21] Mike Bennett: Presumably the book also has to deal with operations like negation that aren't covered in DL or FOL ontologies?

[12:23] David Eddy: @MikeB... story is told that in early days, the term for "automobile" or "car" was "horseless carriage" since that could invoke a known mental image. Cars were for railroads.

[12:23] Mike Bennett: Sales pitch: Imagine you could look at the books on your shelf and have the knowledge in them march into your brain.

[12:27] Janet Singer: The discussions in the CS520 have raised serious questions about what O terminology adds to the work they are doing (it is rarely mentioned). This is a great opportunity for clarification if the O community is up to the challenge

[12:52] Janet Singer: Questions for the O community: If all the material framed in terms of Os disappeared from the face of the earth, what difference would it make to the practical, value-producing work being done with KGs? Is ontology just an archaic name that can more clearly be addressed (and *is being addressed* in the massive KG work) in terms of taxonomies, rules, domain theories, world models, schemas, etc.? Does adding ontological language just complicate everything or are there specific benefits? How can those be better isolated and communicated?

[12:54] Mike Bennett: @Janet an ontology is a (kind of) schema for a KG. Without it all you have is 'here is a thing' and 'here is a relationship'.

[12:54] John Sowa: That is a major reason why I have ZERO faith in the ISO standard effort for ontologies.

[12:54] David Eddy: missing piece is the KG or the CONTENT in the graph?

[12:54] John Sowa: We need standards for narrow subjects.

[12:55] John Sowa: But the upper level must be flexible.

[12:56] John Sowa: For the upper level, a typical dictionary written for humans is more useful than *any* current candidates for a T:\LO.

[12:57] Mike Bennett: The differences in the ways that different fields conceptualize things is why you can't use a 'realist' TLO like BFO.

[12:58] Janet Singer: @Mike: The people working with KGs who have spoken at the Stanford seminar are accomplishing what is attributed by people here to Os without using the specific language.

[12:59] Mike Bennett: @Janet Right. But they are using ontologies. You don't have to use the O word and for most audiences you would not, in order to have an ontology

[12:59] Bobbin Teegarden: @Janet ... and that could be a good thing -- they pick up the idea and make it their own...?

[13:01] John Sowa: Mike, I agree with you. But the technology I was discussing can *read* multiple textbooks and use them to interpret documents on those subjects.

[13:01] Janet Singer: @Bobbin: Yes!

[13:05] Ram D. Sriram: @John Sowa: I agree with your comment "We need standards for narrow subjects"

Usability Issues

[12:09] Mike Bennett: I like how the button is labeled 'Enquire' rather than 'Search'. Thinking in the language of people not IT implementers.

[12:11] Mike Bennett: Also being able to click through from one piece of knowledge to the next gives the user a feeling of control. It's a lot like how we experience our own minds, where you can be aware of something generally or drill down into detailed meaning. Knowing you drill through means you don't always need to.

[12:11] Bobbin Teegarden: Vinay: The visualizations include all those very creative right brainers who are usually ignored in educational approaches. Great inclusion!

[12:12] Leia Dickerson: What planned usability testing is in this work? I find the question of seeing if the audience thinks it will be useful would be best for the target population.

[12:16] Janet Singer: Less important to emphasize students today facing complex material (as if that is new). More interesting to focus on how can today's semantic technology, automation and crowdsourcing techniques address the perennial problems in learning new subjects, especially for the groups you mentioned (e.g shy students, underserved communities)

[12:19] Leia Dickerson: Also, will students be the only ones targeted in user testing? I'm assuming professors and other professionals (librarians, TAs, etc.) may need to interact with the applications and be able to direct student use of them when there are questions.

[12:24] Bobbin Teegarden: Ralph Abraham wrote an entire 4 series textbook on the behavior of dynamical systems, ALL in pictures with annotations instead of text: incredibly effective on teaching very complex concepts. A request: less words, more diagrams ;0)...

[12:25] Mike Bennett: @Bobbin diagrams with ontological links...

[12:26] Ravi Sharma: @Mike I though he is using domain Knowledge to begin with such as biology!

[12:28] Bobbin Teegarden: @Mike yes, more knowledge graphs. And are there diagrams beyond KGs, or KGs with pictures of a topic (e.g. a picture of a particular virus) rather than a circle with a name in it...

[12:28] Ravi Sharma: @Mike - yes on the pitch but intermediate step of query inquisitive mind, logic and building on previous level of domain knowledge and general knowledge!

[12:30] Ravi Sharma: @Bobbin - great example and yes on visuals and process steps.

[12:31] Ravi Sharma: @Leia- I thought he would use experts and crowdsourcing during testing .

[12:51] Leia Dickerson: @Vinay-- There is literature on accessibility and ESL issues. My best suggestion is to work with any accessibility and/or ESL concerned librarians for initial conversations. They will be able to guide you on how to start.

[12:52] Leia Dickerson: Licensing allows for a lot more money than print textbook sales.

[12:57] Leia Dickerson: @Vinay-- Perhaps contact this group at Stanford Libraries? https://library.stanford.edu/department/cidr/about

Learner Modeling

[12:26] Mark Underwood @knowlengr: I wonder how successful this can be without learner modeling (and resultant ontologies) ... This interesting approach seems purely textbook-based, and textbooks can stray quite far from some ideal knowledge representation

[12:27] Mark Underwood @knowlengr: Stated differently: start with what the learner knows, not what a book knows

[12:29] Mark Underwood @knowlengr: The question-building apparatus appears to embed armchair psychometrics

Industrial Training

[12:06] David Eddy: @Vinay... what about "knowledge" that NEVER appears in textbooks?

[12:07] David Eddy: @Vinay... particularly the messy terminology inside organizations?

[12:17] Mike Bennett: Industrial training is a good area and quite lucrative, e.g. on complex products in SCADA and process control.

Cognitive Memory

[12:27] John Sowa: The Cognitive Memory system created its own knowledge graph (in conceptual graphs) by automatically reading documents and discussing them with the human reader.

[12:28] John Sowa: The system automatically became more knowledgeable as humans used it.

[12:29] John Sowa: See http://jfsowa.com/talks/cogmem.pdf (the last example at the end).

[12:31] John Sowa: Janet, Cogmem requires a preliminary ontology for NLP. But it creates its ontology from the documents it reads -- with the help and guidance of the people who use it.

[12:40] Bobbin Teegarden: @JohnS please hurry the tool!

[12:41] David Eddy: 80%+ "politics" and <10% tech

[12:46] John Sowa: Bobbin. The tool itself (Cognitive Memory) is useless without a great deal of supporting software (mostly NLP).

[12:46] Ravi Sharma: @John - Thanks for explaining that cogmem creates not only ontology but more/

[12:47] John Sowa: There were several painful issues, including one employee who ran away with the funding. That is the reason why VivoMind went bankrupt.

[12:49] John Sowa: Another problem was Newt Gingrich, who shut down the gov't and caused the funding to be dropped from a major award.

[12:50] John Sowa: There were other managerial problems, funding issues, etc.

[12:50] Ravi Sharma: @John - Wish you a silver lining and new possibilities to use these concepts which are unique and are your team's IP?

[12:51] John Sowa: We did speak with Google, IBM, and others. But they had their own internal competitors who did not want to be replaced.

[12:51] John Sowa: But there are some ongoing projects.

[12:53] John Sowa: @Ravi, please read the examples in cogmem.pdf. Multidomain issues are not a problem if you have the option of growing your ontology automatically.

[13:02] John Sowa: I understand Vinay's skepticism. There is work going on to commercialize these things.

[13:02] David Eddy: @JFS... did the VivoMind work go down to level of reading load modules as "document?"

[13:02] Mike Bennett: @John I remain curious about how this kind of system makes the distinction between e.g. contextual versus universal concepts and so on. Even humans need help with it as Vinay also confirms. Unless the resultant ontology is intended to be used only in the scope in which it was populated?

[13:04] John Sowa: The first major application read the comments and the text in COBOL programs to provide the basis for interpreting the manuals and emails about the programs.

[13:06] John Sowa: In fact, Vinay's skepticism is a major reason why IBM and Google were skeptical.

[13:06] John Sowa: General principle: no expert wants to admit that other experts have something better.

[13:07] David Eddy: @JFS... so what about when the experts are long gone and cannot defend their turf?

Miscellaneous Feedback

[11:56] Ravi Sharma: @Vinay - What is reason for including Psychology when there is sufficient data info books and tools using Protege and bio ontologies?

[12:13] Ravi Sharma: use hasty cost reduction is counter to that on propagation or multiple subscriptions which will effectively reduce cost while keeping quality very objective. Instead we should concentrate on propagation of books to reduce costs

[12:15] Ravi Sharma: @Vinay - for crowdsourcing how do you filter the maturity and knowledge of the crowd to qualify

[12:16] Mike Bennett: Will you need some foundational and cross-domain ontologies to seed the process?

[12:18] Terry Longstreth: @Vinay: do future plans include prototyping the system with course creation ab initio (without an existing full/formal textbook)?

[12:25] David Eddy: @Vinay... what is the percentage of cost to provide useful content for the KGs?

[12:27] David Eddy: @Vinay... building PDFs from MSWord (or fancy text processors?)

[12:47] Ravi Sharma: @Vinay - your examples of e-contract execution and Tax are already solved. What is not solved are new problems related to remote and integrated manufacturing, multidomain aspects of solving a problem etc.

[12:58] Mark Underwood @knowlengr: IMHO It won't take much to improve on the textbook, a form frozen in amber for a few hundred years of unproven pedagogy

Endorsements

[12:01] Ravi Sharma: @Vinay - what is so wonderful about ITB and TOKN is that memory and memory recall based learning and testing will be challenged to find analytical and logical aspects of what the student or learner understands rather than what they have memorized. I see great learning disciplines and technologies leading towards Knowledge bases and Knowledge Management systems and frankly life situations demand more than ordinary textbook knowledge that many times in void of integration, interconnection and interdoamin klearning? Next gen for sure

[12:54] Ravi Sharma: @Vinay - deep and interdoamin learning for Projects and products is a strong point.

[12:59] Mark Underwood @knowlengr: Thanks for another interesting session - be well all

[13:04] Mike Bennett: @Vinay also thanks for these Stanford sessions, much appreciated.

[13:04] Bobbin Teegarden: @Vinay yes, great Stanford sessions!

[13:05] Ravi Sharma: Vinayat talk and thanks great Questions.

[13:05] David Eddy: @JFS... thx

[13:06] Leia Dickerson: Thank you!!

Possible Followup Session

[13:05] Mike Bennett: Was there a part 2 set of slides we were going to see?

[added later] Ken Baclawski: Not yet. Perhaps at a second session in the summer.

[13:05] Bobbin Teegarden: Can we have him back for the second session?

Resources

Video Recording
YouTube Video
Vinay Chaudhri is running a Knowledge Graphs Seminar (CS 520) at Stanford
- Sessions are on Tuesdays from 4:30PM to 6:20PM PDT (For the time in your timezone see World Clock)
- Connection URL: https://stanford.zoom.us/j/620240910

Previous Meetings

... further results

Next Meetings