From OntologPSMW

Jump to: navigation, search
[ ]


Ontology Summit 2013: Panel Session-05 - Thu 2013-02-14     (1)

Summit Theme: "Ontology Evaluation Across the Ontology Lifecycle"     (1A)

Summit Track Title: Track-D: Software Environments for Evaluating Ontologies     (1B)

Session Topic: Software Environments for Evaluating Ontologies - I     (1C)

Session Co-chairs: Mr. PeterYim (Ontolog; CIM3) and Dr. MichaelDenny (MITRE) - intro slides     (1D)

Panelists / Briefings:     (1E)

  • Dr. MichaelDenny (MITRE) - "Ontology Quality and Fitness: A Survey of Software Support" slides     (1F)
  • Professor MichaelGruninger (U of Toronto) - "Ontology Evaluation Workflow in COLORE" slides     (1G)
  • Ms. JeanneHolm (; NASA/JPL) - "Evaluating US Open Data for Discovery, Interoperability, and Innovation" slides     (1H)
  • Mr. GavinMatthews (Vertical Search Works) - "Assuring broad quality in large-scale ontologies" slides     (1I)

Abstract     (1K)

OntologySummit2013 Session-05: "Software Environments for Evaluating Ontologies-I" - intro slides     (1K1)

This is our 8th Ontology Summit, a joint initiative by NIST, Ontolog, NCOR, NCBO, IAOA & NCO_NITRD with the support of our co-sponsors. The theme adopted for this Ontology Summit is: "Ontology Evaluation Across the Ontology Lifecycle."     (1K2)

Currently, there is no agreed methodology for development of ontologies, and there are no universally agreed metrics for ontology evaluation. At the same time, everybody agrees that there are a lot of badly engineered ontologies out there, thus people use -- at least implicitly -- some criteria for the evaluation of ontologies.     (1K3)

During this Ontology Summit, we seek to identify best practices for ontology development and evaluation. We will consider the entire lifecycle of an ontology -- from requirements gathering and analysis, through to design and implementation. In this endeavor, the Summit will seek collaboration with the software engineering and knowledge acquisition communities. Research in these fields has led to several mature models for the software lifecycle and the design of knowledge-based systems, and we expect that fruitful interaction among all participants will lead to a consensus for a methodology within ontological engineering. Following earlier Ontology Summit practice, the synthesized results of this season's discourse will be published as a Communiqu��.     (1K4)

At the Launch Event on 17 Jan 2013, the organizing team provided an overview of the program, and how we will be framing the discourse around the theme of of this OntologySummit. Today's session is one of the events planned.     (1K5)

Further to our Track-D Mission, we will, in this 5th virtual panel session of the Summit, start by laying out a plan to survey the landscape of software environments, systems and tools that feature "ontology evaluation." This will be followed by briefings from our panelists who will share with us their experience and insights on how a few (exemplary) software environments/systems/tools support ontology evaluation, and help assure quality in the development of ontology or ontology-driven application. Among them, we will include software environments that have "real needs" to evaluate whether and which ontologies would best meet their needs, and how practitioners are assuring the quality of those ontologies in "real applications'. A Q&A and open discussion among all panelists and participants will follow the briefings.     (1K6)

More details about this Ontology Summit is available at: OntologySummit2013 (homepage for this summit)     (1K7)

Briefings     (1K8)

  • Dr. MichaelDenny (MITRE) - "Ontology Quality and Fitness: A Survey of Software Support" slides     (1K8A)
    • Abstract: ... A survey will be conducted across the Ontology Summit community and perhaps more broadly to identify software that evaluates and/or promotes ontology quality and fitness. Software providers will be asked to describe those capabilities of their software that measure or assure ontology quality and fitness. The survey will be structured around a checklist of those ontology factors recognized in the other Ontology Summit tracks that could potentially be addressed in a software environment. The overall survey method and an initial list of ontology evaluation factors will be discussed in this session.     (1K8A1)
COLORE is an open repository of first-order ontologies that serves as a testbed for ontology evaluation and integration techniques, and which supports the design, evaluation, and application of ontologies in first-order logic. This talk will explore the different techniques that are used when an ontology is uploaded into COLORE.     (1K8C1)
In particular, we consider different workflows (sequences of evaluation steps) that are used to verify and     (1K8C2)

modularize an ontology. Although not all of these ontology evaluation workflows are completely automated, we will discuss current directions for implementation.     (1K8D)

  • Ms. JeanneHolm (; NASA/JPL) - "Evaluating US Open Data for Discovery, Interoperability, and Innovation" slides     (1K8E)
    • Abstract: ... With the release of the US Digital Strategy and discussions around new open data policies, the US project is working at creating the next generation of open data capabilities. As " 2.0" evolves, there are changes ranging from interfaces (like to open source (the Open Government Platform). In the meantime, the project continues to open up hundreds of thousands of government datasets from hundreds of government agencies. Finding, connecting, and navigating through this growing, connected collection of data and linked resources becomes a challenge. This talk will discuss progress to date and invite ideas on the work underway around the world on government ontologies, taxonomies, and vocabularies.     (1K8E1)
  • Mr. GavinMatthews (Vertical Search Works) - "Assuring broad quality in large-scale ontologies" slides     (1K8F)
    • Abstract: ... In developing and extending ontologies that support a range of semantic processing applications, it is easy to introduce unintended consequences, unnecessarily limited solutions, or hidden errors. From my experience at Cycorp and Vertical Search Works, I will try to provide some answers to the following questions: How can you measure the efficacy of the ontology at solving real-world tasks? How can you use introspection to reason about the validity of assertions, and identify missing information? How can you facilitate effective human review of ontology?     (1K8F1)

Agenda     (1L)

OntologySummit2013 - Panel Session-05     (1L1)

  • Session Format: this is a virtual session conducted over an augmented conference call     (1L2)

Proceedings     (1M)

Please refer to the above     (1M1)

IM Chat Transcript captured during the session    (1M2)

see raw transcript here.     (1M2A)

(for better clarity, the version below is a re-organized and lightly edited chat-transcript.)     (1M2B)

Participants are welcome to make light edits to their own contributions as they see fit.     (1M2C)

-- begin in-session chat-transcript --     (1M2D)

[08:59] Peter P. Yim: Welcome to the     (1M2E)

Ontology Summit 2013: Virtual Panel Session-05 - Thu 2013-02-14     (1M2F)

Summit Theme: Ontology Evaluation Across the Ontology Lifecycle     (1M2G)

  • Summit Track Title: Track-D: Software Environments for Evaluating Ontologies     (1M2H)

Session Topic: Software Environments for Evaluating Ontologies - I     (1M2I)

Panelists / Briefings:     (1M2K)

and Innovation"     (1M2O)

Logistics:     (1M2Q)

  • (if you haven't already done so) please click on "settings" (top center) and morph from "anonymous" to your RealName (in WikiWord format)     (1M2T)
    • for Linux Skype users: please note that the dial-pad is only available on v4.1 (or later or the earlier Skype versions 2.x,)     (1M2X1)

if the dialpad button is not shown in the call window you need to press the "d" hotkey to enable it.     (1M2Y)

Proceedings:     (1M2AG)

[08:51] marcelaVegetti morphed into Marcela Vegetti     (1M2AH)

[09:19] MatthewWest1 morphed into Matthew West     (1M2AI)

[09:19] anonymous1 morphed into Max Petrenko     (1M2AJ)

[09:21] anonymous morphed into Kevin Simkins     (1M2AK)

[09:23] anonymous morphed into Gavin Matthews     (1M2AL)

[09:24] anonymous morphed into Michael Denny     (1M2AM)

[09:27] anonymous morphed into Carmen Chui     (1M2AN)

[09:27] anonymous morphed into MarcosMartinezRomero     (1M2AO)

[09:29] anonymous morphed into Jeanne Holm     (1M2AP)

[09:32] anonymous morphed into Lamar Henderson     (1M2AQ)

[09:33] anonymous morphed into Doug Foxvog     (1M2AR)

[09:39] anonymous morphed into Paul Pope     (1M2AS)

[09:40] SimonSpero1 morphed into Simon Spero     (1M2AT)

[09:42] anonymous morphed into Lamar Henderson     (1M2AU)

[09:42] Clare Paul morphed into Clare Paul     (1M2AV)

[09:43] Joanne Luciano: Call for help with software development- is there anyone in the community who     (1M2AW)

knows SADI services? (if not, ok check out I have a generalized framework     (1M2AX)

approach that can be used for development of ontologies and evaluation, providing context for     (1M2AY)

intrinsic and extrinsic evaluation. If you can help, or know anyone who can, please get in touch     (1M2AZ)

[09:36] Peter P. Yim: == Peter P. Yim opens the session on behalf of the co-chairs ... see: the [0-Chair] slides     (1M2AAB)

[09:43] Peter P. Yim: == Michael Denny presenting ... see: the [1-Denny] slides     (1M2AAJ)

[09:44] anonymous morphed into Dan Cerys     (1M2AAK)

[09:44] anonymous morphed into Akram     (1M2AAL)

[09:50] Amanda Vizedom: @MichaelDenny - cutting out all human-required evaluation factors might miss     (1M2AAM)

much opportunity. Why not include all factors, noting that there may be much that the evaluation     (1M2AAN)

environment can do to *assist* with non-automated evaluation and with the tracking and management of     (1M2AAO)

evaluation of all factors?     (1M2AAP)

[09:54] Joanne Luciano: +1 to Amanda's comment -- use computers to assist humans     (1M2AAQ)

[09:51] anonymous morphed into Will Burns     (1M2AAR)

[09:57] Simon Spero: Not perfect, but not bad:     (1M2AAS)

[10:01] Simon Spero: More academic: Tourangeau, R., Rips, L.J., and Rasinski, K. (2000),     (1M2AAU)

The Psychology of Survey Response, Cambridge: Cambridge University Press.     (1M2AAV)

[11:25] Michael Denny: @SimonSpero: Thanks for the pointers to survey methodology. I am hoping you     (1M2AAY)

will be able to help field this survey. I would like to come up with the simplest of implementations     (1M2AAZ)

that will allow continuing contributions and visibility of results.     (1M2AAAA)

[09:57] Doug Foxvog: @MichaelDenny: you suggest characterizing breadth, depth, use considerations,     (1M2AAAB)

etc. of ontologies. Do you have an ontology which can be used to express these features of ontologies?     (1M2AAAC)

[09:58] Terry Longstreth: Perhaps an element of the Maintenance phase, but I prefer adding     (1M2AAAD)

Retirement' to include archiving, sequestering, or erasing any component     (1M2AAAE)

[09:58] Bob Schloss: [Apologies - last minute unavoidable conflict came up. I will look over the 3     (1M2AAAF)

sets of slides from the rest of today tomorrow or next week]     (1M2AAAG)

[09:58] Peter P. Yim: @BobSchloss: thanks for coming (albeit briefly)     (1M2AAAH)

[10:01] Amanda Vizedom: Suggestion for additional evaluation factors (very important for some common     (1M2AAAI)

applications): Interpreting slide 8 #10 to mean core logical inference: inference type, along with     (1M2AAAJ)

power (e.g., integrated support for probabilistic reasoning/ uncertainty); integrated lexical     (1M2AAAK)

coverage (languages, support for NLP); integrated topic-relation coverage (facets, subtopic, support     (1M2AAAL)

for document classification).     (1M2AAAM)

[10:01] Joanne Luciano: What are the research questions that the survey seeks to obtain (I didn't see     (1M2AAAN)

that -maybe I missed it).     (1M2AAAO)

[10:02] Doug Foxvog: Michael Denny brings up "assess[ing] query performance" for an ontology. Isn't     (1M2AAAP)

this to a large extent a property of the inference engine used, not necessarily of the ontology?     (1M2AAAQ)

[10:05] Terry Longstreth: @doug: For the extrinsics - Track B - Todd and I have discussed (argued)     (1M2AAAR)

this. He won. Basically, if we can't see the ontology except as it presents behaviors, the     (1M2AAAS)

interpreting mechanism (whether a person, inference engine or generated software) is part of the     (1M2AAAT)

[10:04] Amanda Vizedom: @doug I'd say both (that is, it is a characteristic of a specific ontology in     (1M2AAAV)

a specific reasoning environment).     (1M2AAAW)

[10:09] Doug Foxvog: re: Amanda's comment regarding bullet point "Assess the inferencing power of an     (1M2AAAX)

ontology". The inferencing power of an *ontology* would be a feature of the kinds of expressions     (1M2AAAY)

used: inheritance, argument constraints, properties of relations, existence and types of rules,     (1M2AAAZ)

expression of uncertainty, etc. Support for actually inferencing using such statements would depend     (1M2AAAAA)

upon the inference engine.     (1M2AAAAB)

[10:03] Joanne Luciano: suggest adding an "open question" -- to ask what survey respondents think is     (1M2AAAAC)

important but was not asked on survey.     (1M2AAAAD)

[10:04] Marcela Vegetti: +1 to Joanne's comment     (1M2AAAAE)

[10:03] Peter P. Yim: Michael Denny: ideas and feedback solicited on the survey design between now and Feb-22     (1M2AAAAF)

[10:20] Michael Denny: re: all commenters, thank you -- the current set of software capabilities     (1M2AAAAG)

under each ontology development phase are concluded by a solicitation to the software provider to     (1M2AAAAH)

add their software capabilities that are not already covered. The objective of this survey is simply     (1M2AAAAI)

to compile an inventory of software resources that help users evaluate or promote ontology quality     (1M2AAAAJ)

and fitness. There is no present intent to develop these findings into an ontology of     (1M2AAAAK)

evaluation/fitness factors or capabilities.     (1M2AAAAL)

[10:23] Amanda Vizedom: @MichaelDenny: Is there a notion of modularity in the survey, as     (1M2AAAAM)

@MichaelGruninger just discussed I didn't see it, but could have missed)? If not, it seems well     (1M2AAAAN)

worth adding, since it can be important to suitability and there is so much recent and ongoing work     (1M2AAAAO)

on it. Yes?     (1M2AAAAP)

[10:36] Michael Denny: @AmandaVizedom: Modules are covered under Management phase (as #4), but should     (1M2AAAAQ)

probably appear in Build phase as well.     (1M2AAAAR)

[10:05] Peter P. Yim: == Michael Grüninger presenting ... see: the [2-Gruninger] slides     (1M2AAAAS)

[10:05] anonymous morphed into Mike Bennett     (1M2AAAAT)

[10:06] Simon Spero: Can you count how many different ontologies you have?     (1M2AAAAU)

[10:16] Joanne Luciano: @SimonSpero Can you define "different"?     (1M2AAAAV)

[10:22] Simon Spero: Joanne: no.     (1M2AAAAW)

[10:08] anonymous morphed into Paul Pope     (1M2AAAAX)

[10:18] Alan Rector: There is work in Manchester on various similar metrics - contact Bijan Parsia -     (1M2AAAAY)

bparsia [at]     (1M2AAAAZ)

[10:21] Peter P. Yim: @AlanRector - thank you, Alan, I believe Bijan Parsia is being contacted and     (1M2AAAAAA)

invited to present at another track session ... right, Leo Obrst / Steve Ray (Track-A)?     (1M2AAAAAB)

[10:18] Joanne Luciano: @PeterYim (and other organizers) - do we have any slots in the summit for     (1M2AAAAAC)

discussion or any time in the face-to-face set aside for "discussion"     (1M2AAAAAD)

[10:23] Peter P. Yim: @JoanneLuciano - we will have "discussion" time at the end of each and every     (1M2AAAAAE)

virtual session. I am sure there will be plenty of discussion time during the face-to-face Symposium     (1M2AAAAAF)

too (although Mike Dean & Ram D. Sriram, our Symposium co-chairs will be in the best position to provide     (1M2AAAAAG)

specifics ... probably closer to the time.)     (1M2AAAAAH)

[10:27] Amanda Vizedom: @MichaelGruninger: Can you say more about "Intended Models" as used in the     (1M2AAAAAI)

COLORE sense, and how they might relate to things found "in the wild" - that is, in non-research     (1M2AAAAAJ)

[10:38] Michael Grüninger: @AmandaVizedom: Suppose you are looking for an organization ontology with     (1M2AAAAAL)

relations such as supervises, authorizes, manages. You could use an org chart from your company as     (1M2AAAAAM)

an example of your intended semantics of the relations. We would translate such a chart into a set     (1M2AAAAAN)

of relations with their extensions (i.e. a model) and then search the repository.     (1M2AAAAAO)

[10:31] Simon Spero: @MichaelGruninger: is there anyway to deal with conservative/definitional     (1M2AAAAAP)

extensions without making CLIF's quantification of predicates unhelpful in the cases where they be     (1M2AAAAAQ)

most useful? Would making CL sorted be enough?     (1M2AAAAAR)

[10:35] Michael Grüninger: @SimonSpero: all of the work we have done so far has used essentially     (1M2AAAAAS)

good-old fashioned FOL, and really hasn't exercised the full quantification of predicates in CL. We     (1M2AAAAAT)

translated CLIF to Prover9 and do the theorem proving there. Once there is a full CL theorem prover,     (1M2AAAAAU)

we can use it.     (1M2AAAAAV)

[10:31] Alan Rector: On modules - there is an OWL module extractor as a web app on the     (1M2AAAAAW) site. It is also available by request to download but new features     (1M2AAAAAX)

are being added regularly. [above link updated from original post, thanks to prompt from Simon Spero     (1M2AAAAAY)

[10:32] Fabian Neuhaus: @Till. Michael mentioned "Mossakowski, T., Codescu, M., Kutz, O., Lange, C.,     (1M2AAAAAAA)

and Gruninger, M. (2013) Proof Support for Common Logic." in his presentation. Is the paper available?     (1M2AAAAAAB)

[10:43] Fabian Neuhaus: @Michael: I think we need to scope the notion of "evaluation". Much of your     (1M2AAAAAAC)

presentation covered an analysis of the relationships between ontology (e.g., conservative     (1M2AAAAAAD)

extension, non-conservative extension) etc. This is interesting, but this does not tell you anything     (1M2AAAAAAE)

about the quality of an ontology or a set of ontologies. .     (1M2AAAAAAF)

[10:47] Michael Grüninger: @Fabian: First of all, one way in which we evaluate the quality of an     (1M2AAAAAAG)

ontology is wrt intended models, and we use the relationships between the ontologies in the     (1M2AAAAAAH)

repository to do this verification. In other words, consistency alone is not enough. Second, in past     (1M2AAAAAAI)

few weeks, people have raised the idea of ontology comparison as being part of evaluation; it does     (1M2AAAAAAJ)

not address quality per se, but it does evaluate two ontologies against each other e.g. what are     (1M2AAAAAAK)

their similarities and differences?     (1M2AAAAAAL)

[10:52] Fabian Neuhaus: @Michael, second point. Yes, this is why I think we need to scope it, in     (1M2AAAAAAM)

particular for the sake of the communique. If we use the word "evaluation" to mean "analyze" ontologies     (1M2AAAAAAN)

or their use in systems under any conceivable aspect, including their structure, community use,     (1M2AAAAAAO)

and intellectual property rights " then we will have trouble to get to a reasonable communique.     (1M2AAAAAAP)

[10:55] Fabian Neuhaus: @Michael, first point. I understood the point, and agree that one very good     (1M2AAAAAAQ)

way to evaluate ontologies is to use intended models.     (1M2AAAAAAR)

[10:55] Will Burns: Intellectual property rights as an issue makes the assumption that the data     (1M2AAAAAAS)

itself has a context which uniquely identifies it and grants it the ability to be copyrighted. In     (1M2AAAAAAT)

this manner, I suppose we're still addressing the ontology aspect as a 1:1 storage and retrieval     (1M2AAAAAAU)

methodology whereby the files themselves are married to context. Unfortunately that doesn't *quite*     (1M2AAAAAAV)

work out in a situation where the volume of data is no longer inherent with context.     (1M2AAAAAAW)

[10:57] Simon Spero: @WillBurns: In US, Australia, and possibly Canada, "mere data" cannot be     (1M2AAAAAAX)

copyrighted (beyond selection, coordination, and arrangement).     (1M2AAAAAAY)

[10:58] Will Burns: @SimonSpero Correct. Specific and unique arrangements can be copyrighted, but the     (1M2AAAAAAZ)

question becomes what happens when we remove all three up front and still manage to store it all?     (1M2AAAAAAAA)

[11:02] Michael Denny: @FabianNeuhaus: By and large I was being driven by the full range of     (1M2AAAAAAAB)

evaluation factors mentioned in the three tracks which includes, for example, "intellectual property     (1M2AAAAAAAC)

rights". Such usage requirements that go into a user's choice to adopt particular ontologies as     (1M2AAAAAAAD)

components may determine the overall quality or fitness of their ontology product. That said, I also     (1M2AAAAAAAE)

hesitated to include evaluation of this factor as a candidate software capability. I chose to     (1M2AAAAAAAF)

include it to be faithful to the tracks' identifications of factors.     (1M2AAAAAAAG)

[10:27] Peter P. Yim: == Jeanne Holm presenting ... see: the [3-Holm] slides     (1M2AAAAAAAH)

[10:43] Steve Ray: Looks like the open government github platform shouldn't have a "t". Instead, it     (1M2AAAAAAAI)

[this is in reference to JeanneHolm's slide#15, which is now corrected. Thanks, Steve.]     (1M2AAAAAAAK)

[10:39] Joanne Luciano: Would like to bring everyone's attention to some tools from our lab at RPI     (1M2AAAAAAAL)

(Tetherless World Constellation) that to take gov't data and turn it to RDF - there's also data     (1M2AAAAAAAM)

quality tools available. This is the RPI site: Linked Data     (1M2AAAAAAAN)

[10:48] Joanne Luciano: @JeanneHolm Semantic Ecology and Environmental Portal uses gov't data:     (1M2AAAAAAAP)

[10:49] Jeanne Holm: Thanks Joanne! Great pointer.     (1M2AAAAAAAR)

[11:13] Scott Hills: @JeanneHolm, on your slide 16, can you offer a link to more info about     (1M2AAAAAAAS)

ADMS-based metadata?     (1M2AAAAAAAT)

[11:15] Jeanne Holm: @ScottHills You can find out more about ADMS (a European Union initiative) go to     (1M2AAAAAAAU)

[11:16] Scott Hills: @JeanneHolm, thanks.     (1M2AAAAAAAW)

[10:51] Peter P. Yim: == Gavin Matthews presenting ... see: the [4-Matthews] slides     (1M2AAAAAAAX)

[10:58] Doug Foxvog: @Gavin: you show a "RT:" (related term?) link. Does the ontology specify the     (1M2AAAAAAAY)

type of relation? Or is the number of relations very limited?     (1M2AAAAAAAZ)

[10:59] Simon Spero: @doug RT is usually defined as a residual category of all relations that aren't     (1M2AAAAAAAAA)

equivalence or hierarchical.     (1M2AAAAAAAAB)

[11:00] Till Mossakowski: slide 6: what does "BT" mean? subconcept? why is "woman musician" a     (1M2AAAAAAAAC)

subconcept of "famous musician"?     (1M2AAAAAAAAD)

[11:01] Simon Spero: @Till: That looks like an incorrect BT     (1M2AAAAAAAAE)

[11:02] Simon Spero: @Till: ?N BT ?B means that everything about ?N is also about ?B     (1M2AAAAAAAAF)

[11:03] Doug Foxvog: The ontology on slide 6 shows both "woman musician" and "musical performer" as     (1M2AAAAAAAAG)

subtypes of "famous musician". Yet "famous musical performer" is a subtype of "musical performer".     (1M2AAAAAAAAH)

The class "famous musician" probably is not mis-named since it is a subtype of "celebrity". There     (1M2AAAAAAAAI)

seems to be a problem here.     (1M2AAAAAAAAJ)

[11:04] Simon Spero: @Till: Diagnostic sentence frame - " It's about a woman musician but it's not     (1M2AAAAAAAAK)

about a famous musician" should be unacceptable     (1M2AAAAAAAAL)

[11:15] Gavin Matthews: @TillMossakowski: BT (Broader Term) means sub-collection/sub-type. I'm not     (1M2AAAAAAAAM)

claiming our ontology is perfect. I wanted to show how we detected errors. :)     (1M2AAAAAAAAN)

[11:16] Simon Spero: @Gavin: Is it restricted to ~genls?     (1M2AAAAAAAAO)

[11:18] Simon Spero: @Gavin: In controlled vocabularies, BT can apply to genls, isa, or intentionally     (1M2AAAAAAAAP)

necessary parts (where the absence of a part requires explanation)     (1M2AAAAAAAAQ)

[11:19] Simon Spero: @Gavin: there are BTG, BTP, and BTI which are spec preds of BT     (1M2AAAAAAAAR)

[11:20] Simon Spero: @Gavin: but undifferentiated broader does not allow you to infer a genls     (1M2AAAAAAAAS)

[11:21] Gavin Matthews: @SimonSpero: We use BT to mean roughly "genls". We distinguish ISA meaning     (1M2AAAAAAAAT)

isa. For practical reasons, we permit a certain underspecification.     (1M2AAAAAAAAU)

[11:24] Amanda Vizedom: During last week's discussion of development methodology, we really talked     (1M2AAAAAAAAV)

very little about how evaluation can guide development. The evaluation Gavin discusses is used by     (1M2AAAAAAAAW)

ontology developers strongly; we might want to talk about that.     (1M2AAAAAAAAX)

[11:05] Matthew West: Sorry need to drop off now. Considerable food for thought here.     (1M2AAAAAAAAY)

[11:06] Simon Spero: What's needed is D. Allan Cruse control..     (1M2AAAAAAAAAA)

[11:08] Doug Foxvog: @Simon: yes, "related term" might be all that is needed for generic web     (1M2AAAAAAAAAB)

searching. But it would be useful if a search for owners of a sports team would yield different     (1M2AAAAAAAAAC)

links than for members of the team, or for supporters of the team.     (1M2AAAAAAAAAD)

[11:09] Simon Spero: @doug: Yes - RT can be specialized (he described a few, I think).     (1M2AAAAAAAAAE)

[11:24] Gavin Matthews: @DougFoxvog: Yes, "RT" means related term. It is our vaguest relation, and we     (1M2AAAAAAAAAF)

constantly work to specialise it. Relations like "plays role in", "located in", and "has theme" are     (1M2AAAAAAAAAG)

considered to be sub-relations of RT, but in practice all of those horizontal links have     (1M2AAAAAAAAAH)

approximately the same effect on disambiguation.     (1M2AAAAAAAAAI)

[11:30] Simon Spero: @Gavin: Underspecification is necessary - there have been attempts to create     (1M2AAAAAAAAAJ)

more specific coverings of associative relationships ( RT), but they all ran in to the problem that     (1M2AAAAAAAAAK)

they were incomplete, and not useful for IR purposes     (1M2AAAAAAAAAL)

[11:09] Amanda Vizedom: I would call everyone's attention to Gavin's slide 9. This level of testing     (1M2AAAAAAAAAM)

is something almost no one imagines applying to ontologies, even in contexts where the rest of the     (1M2AAAAAAAAAN)

software environment is so tested. Yet it's entirely possible, and once established, extremely valuable.     (1M2AAAAAAAAAO)

[11:11] Peter P. Yim: == Q&A and Open Discussion ...     (1M2AAAAAAAAAQ)

[11:13] Michael Grüninger: @Fabian: A similar comment can be made about ontology metrics i.e. what do     (1M2AAAAAAAAAR)

they have to do with ontology quality? I get more insight from knowing that an ontology I have is     (1M2AAAAAAAAAS)

inconsistent with another ontology than I get from knowing that there are 40 axioms with 5 relations,     (1M2AAAAAAAAAT)

12 classes with 3 siblings.     (1M2AAAAAAAAAU)

[11:14] Till Mossakowski: @Michael: what does "inconsistent with" mean? The union being inconsistent?     (1M2AAAAAAAAAV)

[11:14] Fabian Neuhaus: @Michael. I would (and have) made the same comment about these metrics.     (1M2AAAAAAAAAW)

[11:20] Michael Grüninger: @Till: Yes, I meant that two ontologies are mutually inconsistent. For     (1M2AAAAAAAAAX)

example, one ontology entails that time is discrete and the other ontology entails that time is dense.     (1M2AAAAAAAAAY)

[11:26] Joanne Luciano: One simple way evaluation can be used to guide development is with the use of     (1M2AAAAAAAAAZ)

reasoners. Whenever we add a class or an assertion, we run the reasoner and check if we get what we     (1M2AAAAAAAAAAA)

expect. Of course this is by hand, and scaling this is another story.     (1M2AAAAAAAAAAB)

[11:32] Steve Ray: Must run. Thanks.     (1M2AAAAAAAAAAD)

[11:33] Alan Rector: On tracking environments: We have used Redmine successfully and found it easier     (1M2AAAAAAAAAAE)

than Bugzilla because of the good Wiki and other facilities. So many of the issues that come up are     (1M2AAAAAAAAAAF)

more complex issues than simple "bugs".     (1M2AAAAAAAAAAG)

[11:35] Simon Spero: @AlanRector: bug report - running the owltoskos web tool on the skos ontology crashed.     (1M2AAAAAAAAAAH)

[11:38] Alan Rector: @SimonSpero - please email Bijan Parsia - bparsia [at] - and Sean     (1M2AAAAAAAAAAI)

Becchofer - sean.bechhofer [at] - they will redirect it to whomever is dealing with     (1M2AAAAAAAAAAJ)

[11:35] Joanne Luciano: Please email me if you'd like to collaborate on a generalized ontology     (1M2AAAAAAAAAAL)

evaluation and development framework. jluciano [at] Thanks everyone     (1M2AAAAAAAAAAM)

[11:37] Peter P. Yim: join us again, same time next week, for Ontology Summit 2013 session-06: "Ontology     (1M2AAAAAAAAAAN)

Summit 2013: Synthesis-I" - Co-chairs: Michael Grüninger & Matthew West -     (1M2AAAAAAAAAAO)

[11:38] Gavin Matthews: Thanks very much everyone. I've found it very interesting.     (1M2AAAAAAAAAAQ)

[11:40] Peter P. Yim: -- session ended: 11:39 am PST --     (1M2AAAAAAAAAAR)

-- end of in-session chat-transcript --     (1M2AAAAAAAAAAS)

Additional Resources     (1N)

For the record ...     (1N6)

How To Join (while the session is in progress)     (1O)

  • Dial-in:     (1O4D)
    • Phone (US): +1 (206) 402-0100 ... (long distance cost may apply)     (1O4D1)
    • Skype: joinconference (i.e. make a skype call to the contact with skypeID="joinconference") ... (generally free-of-charge, when connecting from your computer)     (1O4D2)
      • when prompted enter Conference ID: 141184#     (1O4D2A)
      • Unfamiliar with how to do this on Skype? ...     (1O4D2B)
        • Add the contact "joinconference" to your skype contact list first. To participate in the teleconference, make a skype call to "joinconference", then open the dial pad (see platform-specific instructions below) and enter the Conference ID: 141184# when prompted.     (1O4D2B1)
      • Can't find Skype Dial pad? ...     (1O4D2C)
        • for Windows Skype users: Can't find Skype Dial pad? ... it's under the "Call" dropdown menu as "Show Dial pad"     (1O4D2C1)
        • for Linux Skype users: please note that the dial-pad is only available on v4.1 (or later; or on the earlier Skype versions 2.x,) if the dialpad button is not shown in the call window you need to press the "d" hotkey to enable it. ... (ref.)     (1O4D2C2)
  • Shared-screen support (VNC session), if applicable, will be started 5 minutes before the call at:     (1O4E)
    • view-only password: "ontolog"     (1O4E1)
    • if you plan to be logging into this shared-screen option (which the speaker may be navigating), and you are not familiar with the process, please try to call in 5 minutes before the start of the session so that we can work out the connection logistics. Help on this will generally not be available once the presentation starts.     (1O4E2)
    • people behind corporate firewalls may have difficulty accessing this. If that is the case, please download the slides above (where applicable) and running them locally. The speaker(s) will prompt you to advance the slides during the talk.     (1O4E3)
    • instructions: once you got access to the page, click on the "settings" button, and identify yourself (by modifying the Name field from "anonymous" to your real name, like "JaneDoe").     (1O4F1)
    • You can indicate that you want to ask a question verbally by clicking on the "hand" button, and wait for the moderator to call on you; or, type and send your question into the chat window at the bottom of the screen.     (1O4F2)
    • thanks to the folks, one can now use a jabber/xmpp client (e.g. gtalk) to join this chatroom. Just add the room as a buddy - (in our case here) ... Handy for mobile devices!     (1O4F3)
  • Discussions and Q & A:     (1O4G)
    • Nominally, when a presentation is in progress, the moderator will mute everyone, except for the speaker.     (1O4G1)
    • To un-mute, press "*7" ... To mute, press "*6" (please mute your phone, especially if you are in a noisy surrounding, or if you are introducing noise, echoes, etc. into the conference line.)     (1O4G2)
    • we will usually save all questions and discussions till after all presentations are through. You are encouraged to jot down questions onto the chat-area in the mean time (that way, they get documented; and you might even get some answers in the interim, through the chat.)     (1O4G3)
    • During the Q&A / discussion segment (when everyone is muted), If you want to speak or have questions or remarks to make, please raise your hand (virtually) by clicking on the "hand button" (lower right) on the chat session page. You may speak when acknowledged by the session moderator (again, press "*7" on your phone to un-mute). Test your voice and introduce yourself first before proceeding with your remarks, please. (Please remember to click on the "hand button" again (to lower your hand) and press "*6" on your phone to mute yourself after you are done speaking.)     (1O4G4)
  • RSVP to with your affiliation appreciated, ... or simply just by adding yourself to the "Expected Attendee" list below (if you are a member of the community already.)     (1O4I)
  • Please note that this session may be recorded, and if so, the audio archive is expected to be made available as open content, along with the proceedings of the call to our community membership and the public at-large under our prevailing open IPR policy.     (1O4K)
    • Caveat: to allow us to share some of the latest in commercial deployment of ontology-based technologies, this session will be featured with a special waiver to commercial organizations on the Ontolog IPR Policy. Panelists from such organizations are welcome to talk about their proprietary (non-open) technologies if they so desire (on the condition that proprietary portions of their presentation are to be specifically stated as such). However, (despite the waiver) it should be noted that we will (as usual) be making available the entire proceedings, including all slides, recorded audio, etc., of the session to the community and the public at large from this Ontolog site (with those organizations' consent.)     (1O4K1)

Attendees     (1P)

This page has been migrated from the OntologWiki - Click here for original page     (1P4)