From OntologPSMW

(Difference between revisions)
Jump to: navigation, search
(Proceedings)
 
Line 46: Line 46:
  
 
== Proceedings ==
 
== Proceedings ==
 +
MikeBennett: Challenge activity for Summit
 +
 +
MikeBennett: The Sumit starts on Wed 16th
 +
 +
MikeBennett: Would help to be able to announce this Challenge if we have something.
 +
 +
MikeBennett: See 14:34 in last month's notes
 +
 +
MikeBennett: Challenge idea is based on the ESWC Requests for Challenges. KB sent them a request for advice (as they said you could do)  but got no response.
 +
 +
MikeBennett: THis was not specifically a proposal for a challenge to do at ESWC.
 +
 +
MikeBennett: Challenge (sic) i to come up with such a proposal. Evaluation methodologies are not available.
 +
 +
MikeBennett: KB was proposing to submit a Challenge proposal to ESWC, rather than suggesting that the Summit itself runs a Challenge. Would write up something for the k/o meeting for the group to come up with some methodology for evaluating the quality of explanations for explainable AI.
 +
 +
MikeBennett: There was a talk at one of the summit sessions on 3 levels of explanation:
 +
 +
MikeBennett: 1. Technical explanation - describing the mathematical specification
 +
 +
MikeBennett: 2. gets progressively more into communicating with people (who are not familiar with the mathematics behind the ML algorithms).
 +
 +
MikeBennett: Good tech explanation is dependent on ontologies. Enables a narratiev of dialog with the individual in explaining what is happening.
 +
 +
MikeBennett: Presenter works in an XAI program. People do not have a clear what an explanation is, or don't have the same idea of what is required for a solution.
 +
 +
MikeBennett: Hence needed to reiterate the notion of evaluation criteria. Other than simply asking if a person is happy with the explanation. Else the state of the art for explanation for AI systems is not well developed.
 +
 +
MikeBennett: Given ISWC deadline of 15 Jan, we would not have the chance for someone in the Summit to work together on submitting a proposal.
 +
 +
MikeBennett: The issue is that there are no evaluation criteria.
 +
 +
MikeBennett: ESWC was Dec 15 deasline for proposals.
 +
 +
MikeBennett: ISWC is 15 Jan
 +
 +
MikeBennett: If ISWC accepted our proposal then once we know f it accepted we could encourage folks at the Summit to submit a response.
 +
 +
MikeBennett: We can mention that it is proposed and what the submission date for entries is.
 +
 +
MikeBennett: If not accepted for ESWC or ISWC it can be done at the Summit anyway.
 +
 +
MikeBennett: Ken will take the proposal and add it to the agenda for the 16 Jan k/o meeting.
 +
 +
MikeBennett: KB submitted something to ESWC via EasyChair rather than as an email. Needs some feedback.
 +
 +
MikeBennett: KB We can describe the Organizing Committee of the Summit to be the Org Ctee for the Challenge. This is already in the draft proposal.
 +
 +
MikeBennett: So there may be enough already to meet the requirements of a Challenge proposal to one of the other. If we get any feedback from ESWC we can include it in the submission for ISWC.
 +
 +
MikeBennett: A weakness may be that the submissions require some evaluation criteria but this is the very thing no-one has.
 +
 +
MikeBennett: If we had the meta-criteria that could be framed as a challenge in its own right. But we  do not.
 +
 +
MikeBennett: Or we can run the meta-challenge within the Summit itself as a challenge.
 +
 +
MikeBennett: KB can mention this on the 16 Jan call.
 +
 +
MikeBennett: IF we had some clue as to how we would evaluate an explanation this would help indicate how an explanatiojn is constructed.
 +
 +
MikeBennett: For example there is a knowledge aspect, a social aspect, linguistic aspects. Mathematical aspect (see above)
 +
 +
MikeBennett: This can be done at the Summit as a workshop, or as one of the workshop problems for the face to face.
 +
 +
KenBaclawski: The Doron session of the summit is relevant: http://ontologforum.org/index.php/ConferenceCall_2018_11_28
 +
 +
ToddSchneider: https://www.dnvgl.com/events/international-industrial-ontologies-workshops-134292
 +
 +
MikeBennett: This is the IOF event in Oslo in 1st week of Feb.
 +
 +
MikeBennett: Oslo, Norway on 4 - 7 February 2019
 +
 +
MikeBennett: Barry Smith pushing IOF to adopt BFO as their universal ontology for this. Some push-back. BS expected a decision on this in November. Some discussion will follow. MG pushing to get some requirements for the proposed industrial ontology.
 +
 +
MikeBennett: BFO has no Change Request mechanism. Not clear how that aligns with the notion of an ISO standard. Proposal seems to be for extensions and as when required.
 +
 +
MikeBennett: Need to drop the Realism requirement and allow for abstract notion.
 +
 +
MikeBennett: Should we as a group be looking to do something on the ISO TLO standard liaison, feedback etc.?
 +
 +
MikeBennett: Given we are also interfacing with Semantic Web, can we do something on TLO for that community?
 +
 +
MikeBennett: MB Yes; some SemWeb folks run scared of TLO perhaps without good reason e.g. misaapplication of these concepts at client sites.
 +
 +
MikeBennett: TS: WE can come up with examples of good usage and describe the representation challenges. e.g. what a function is, when realized. how to treat in BFO (concretized) v others.
 +
 +
MikeBennett: This is explaining an  UO which is a kind of explaining something. How we can you explain an AO and does it contribute to the explanation. MAy be a contributiojn to understanding what UO items to use.
 +
 +
MikeBennett: What are the criteria for having a good explanation? Just giving an abstract account of the UO elements may not be practical.
 +
 +
MikeBennett: So there is explainable UO versus Explainable AI.
 +
 +
MikeBennett: BFO 2.0 - one important aspect is documentation. Many TLOs fall short on that.
 +
 +
MikeBennett: Can we create a resource on the use, common mistakes etc. for TLO elements.
 +
 +
MikeBennett: All agree this is a good idea.
 +
 +
MikeBennett: NExt steps:
 +
 +
MikeBennett: Collect some, review them, think about how to present these.
 +
 +
MikeBennett: Put a bunch of things into a review of a paticular example/
 +
 +
MikeBennett: MB: Identify the common problems, e.g. behind Continuant v Occurrent; relatiev things et. and describe how the leading ontologies address these common issues in different ways.
 +
 +
MikeBennett: What venue?
 +
 +
MikeBennett: Something on the IAOA website.
 +
 +
MikeBennett: A blog?
 +
 +
MikeBennett: Something where people can put comments and responses. A blog (as  configured in our wiki structure) would cover this.
 +
 +
MikeBennett: WE can do this within our SWAO Wiki structure and promote it to the IAOA once it is mature enough.
 +
 +
MikeBennett: Review when or whether to allow for a response capability - given it can get heated and the heatdedness can turn other participant off. Hence not a mailer.
 +
 +
MikeBennett: First collect some material.
 +
 +
MikeBennett: Topics:
 +
 +
MikeBennett: Things that happen (Continuant v Occurrent);
 +
 +
MikeBennett: Thins in contexts (Relative Things / qua entities)
 +
 +
MikeBennett: Units and measures (Sweet ontology etc.)
 +
 +
MikeBennett: Make clear we are not taking a 4D approach, not treating everything as a process - i.e. not being locked in to one or another practitioner's set of assumptionsm OR when we are, make a given ssuption or decision explicit. Help people understand the decisions and modeling choices.
 +
 +
MikeBennett: e.g. in IOF, the different people have different notions even of wht is ontological analysis versus modeling design decisions.
 +
 +
MikeBennett: We need to present the options, assumptions etc. rather than define a right and wrong approach to these things.
 +
 +
MikeBennett: List what the TLO considerations are as above;
 +
 +
MikeBennett: List what are the paradigms within which one has to think these things: 4D v 3D; Realist or not; treatments fo time and temporality and so on. Whether you can have multiple inheritance
 +
 +
MikeBennett: Action Circulate the list of problem areas and the list of pardigmatic considerations, as outlined above, via email, then start to put a wiki page together.
 +
 +
MikeBennett: AoB?
 +
 +
MikeBennett: Next Meeting?
 +
 +
MikeBennett: 4 Feb (during the IOF thing in OSLO)
 +
 +
MikeBennett: Defer to 11 Feb?
 +
 +
MikeBennett: Separately, Andrea can't make this time slot generally.
 +
 +
MikeBennett: We will move these calls to 2 - 3 going forward. Create a new / update existing Outlook entry.
 +
 +
MikeBennett: Add this to the Ontology Events Google Calendar.
 +
 +
MikeBennett: This should go in the IAOA Google Calendar. Can also go in the Shed and SWAO spreadsheet-generated one.
 +
 +
MikeBennett: MB will update the 2019 Events spreadsheet and share with IAOA. They may choose to include or ignore some of the more specialist stuff.
 +
 +
MikeBennett: Can include AI, Blockchain, other things of relevance to ontologists but not directly about ontology.
 +
 +
MikeBennett: Confirmed we will meet on 11th.
  
 
== Attendees ==
 
== Attendees ==

Latest revision as of 14:12, 11 February 2019

[ ]
    (1)
Number 62
Duration 1 hour
Date/Time January 7 2019 18:30 GMT
10:30 PST/1:30pm EST
6:30pm GMT/7:30pm CET
Convener Mike Bennett

IAOA Semantic Web Applied Ontology (SWAO) SIG     (2)

Meetings are normally on the first Monday of the month at these times.     (3)

Contents

[edit] Agenda     (5)

1. Housekeeping     (5A)

2. Events     (5B)

3. Discussion of new initiatives     (5F)

5. Next Meeting     (5M)


[edit] Proceedings     (6)

MikeBennett: Challenge activity for Summit     (6A)

MikeBennett: The Sumit starts on Wed 16th     (6B)

MikeBennett: Would help to be able to announce this Challenge if we have something.     (6C)

MikeBennett: See 14:34 in last month's notes     (6D)

MikeBennett: Challenge idea is based on the ESWC Requests for Challenges. KB sent them a request for advice (as they said you could do) but got no response.     (6E)

MikeBennett: THis was not specifically a proposal for a challenge to do at ESWC.     (6F)

MikeBennett: Challenge (sic) i to come up with such a proposal. Evaluation methodologies are not available.     (6G)

MikeBennett: KB was proposing to submit a Challenge proposal to ESWC, rather than suggesting that the Summit itself runs a Challenge. Would write up something for the k/o meeting for the group to come up with some methodology for evaluating the quality of explanations for explainable AI.     (6H)

MikeBennett: There was a talk at one of the summit sessions on 3 levels of explanation:     (6I)

MikeBennett: 1. Technical explanation - describing the mathematical specification     (6J)

MikeBennett: 2. gets progressively more into communicating with people (who are not familiar with the mathematics behind the ML algorithms).     (6K)

MikeBennett: Good tech explanation is dependent on ontologies. Enables a narratiev of dialog with the individual in explaining what is happening.     (6L)

MikeBennett: Presenter works in an XAI program. People do not have a clear what an explanation is, or don't have the same idea of what is required for a solution.     (6M)

MikeBennett: Hence needed to reiterate the notion of evaluation criteria. Other than simply asking if a person is happy with the explanation. Else the state of the art for explanation for AI systems is not well developed.     (6N)

MikeBennett: Given ISWC deadline of 15 Jan, we would not have the chance for someone in the Summit to work together on submitting a proposal.     (6O)

MikeBennett: The issue is that there are no evaluation criteria.     (6P)

MikeBennett: ESWC was Dec 15 deasline for proposals.     (6Q)

MikeBennett: ISWC is 15 Jan     (6R)

MikeBennett: If ISWC accepted our proposal then once we know f it accepted we could encourage folks at the Summit to submit a response.     (6S)

MikeBennett: We can mention that it is proposed and what the submission date for entries is.     (6T)

MikeBennett: If not accepted for ESWC or ISWC it can be done at the Summit anyway.     (6U)

MikeBennett: Ken will take the proposal and add it to the agenda for the 16 Jan k/o meeting.     (6V)

MikeBennett: KB submitted something to ESWC via EasyChair rather than as an email. Needs some feedback.     (6W)

MikeBennett: KB We can describe the Organizing Committee of the Summit to be the Org Ctee for the Challenge. This is already in the draft proposal.     (6X)

MikeBennett: So there may be enough already to meet the requirements of a Challenge proposal to one of the other. If we get any feedback from ESWC we can include it in the submission for ISWC.     (6Y)

MikeBennett: A weakness may be that the submissions require some evaluation criteria but this is the very thing no-one has.     (6Z)

MikeBennett: If we had the meta-criteria that could be framed as a challenge in its own right. But we do not.     (6AA)

MikeBennett: Or we can run the meta-challenge within the Summit itself as a challenge.     (6AB)

MikeBennett: KB can mention this on the 16 Jan call.     (6AC)

MikeBennett: IF we had some clue as to how we would evaluate an explanation this would help indicate how an explanatiojn is constructed.     (6AD)

MikeBennett: For example there is a knowledge aspect, a social aspect, linguistic aspects. Mathematical aspect (see above)     (6AE)

MikeBennett: This can be done at the Summit as a workshop, or as one of the workshop problems for the face to face.     (6AF)

KenBaclawski: The Doron session of the summit is relevant: http://ontologforum.org/index.php/ConferenceCall_2018_11_28     (6AG)

MikeBennett: This is the IOF event in Oslo in 1st week of Feb.     (6AI)

MikeBennett: Oslo, Norway on 4 - 7 February 2019     (6AJ)

MikeBennett: Barry Smith pushing IOF to adopt BFO as their universal ontology for this. Some push-back. BS expected a decision on this in November. Some discussion will follow. MG pushing to get some requirements for the proposed industrial ontology.     (6AK)

MikeBennett: BFO has no Change Request mechanism. Not clear how that aligns with the notion of an ISO standard. Proposal seems to be for extensions and as when required.     (6AL)

MikeBennett: Need to drop the Realism requirement and allow for abstract notion.     (6AM)

MikeBennett: Should we as a group be looking to do something on the ISO TLO standard liaison, feedback etc.?     (6AN)

MikeBennett: Given we are also interfacing with Semantic Web, can we do something on TLO for that community?     (6AO)

MikeBennett: MB Yes; some SemWeb folks run scared of TLO perhaps without good reason e.g. misaapplication of these concepts at client sites.     (6AP)

MikeBennett: TS: WE can come up with examples of good usage and describe the representation challenges. e.g. what a function is, when realized. how to treat in BFO (concretized) v others.     (6AQ)

MikeBennett: This is explaining an UO which is a kind of explaining something. How we can you explain an AO and does it contribute to the explanation. MAy be a contributiojn to understanding what UO items to use.     (6AR)

MikeBennett: What are the criteria for having a good explanation? Just giving an abstract account of the UO elements may not be practical.     (6AS)

MikeBennett: So there is explainable UO versus Explainable AI.     (6AT)

MikeBennett: BFO 2.0 - one important aspect is documentation. Many TLOs fall short on that.     (6AU)

MikeBennett: Can we create a resource on the use, common mistakes etc. for TLO elements.     (6AV)

MikeBennett: All agree this is a good idea.     (6AW)

MikeBennett: NExt steps:     (6AX)

MikeBennett: Collect some, review them, think about how to present these.     (6AY)

MikeBennett: Put a bunch of things into a review of a paticular example/     (6AZ)

MikeBennett: MB: Identify the common problems, e.g. behind Continuant v Occurrent; relatiev things et. and describe how the leading ontologies address these common issues in different ways.     (6AAA)

MikeBennett: What venue?     (6AAB)

MikeBennett: Something on the IAOA website.     (6AAC)

MikeBennett: A blog?     (6AAD)

MikeBennett: Something where people can put comments and responses. A blog (as configured in our wiki structure) would cover this.     (6AAE)

MikeBennett: WE can do this within our SWAO Wiki structure and promote it to the IAOA once it is mature enough.     (6AAF)

MikeBennett: Review when or whether to allow for a response capability - given it can get heated and the heatdedness can turn other participant off. Hence not a mailer.     (6AAG)

MikeBennett: First collect some material.     (6AAH)

MikeBennett: Topics:     (6AAI)

MikeBennett: Things that happen (Continuant v Occurrent);     (6AAJ)

MikeBennett: Thins in contexts (Relative Things / qua entities)     (6AAK)

MikeBennett: Units and measures (Sweet ontology etc.)     (6AAL)

MikeBennett: Make clear we are not taking a 4D approach, not treating everything as a process - i.e. not being locked in to one or another practitioner's set of assumptionsm OR when we are, make a given ssuption or decision explicit. Help people understand the decisions and modeling choices.     (6AAM)

MikeBennett: e.g. in IOF, the different people have different notions even of wht is ontological analysis versus modeling design decisions.     (6AAN)

MikeBennett: We need to present the options, assumptions etc. rather than define a right and wrong approach to these things.     (6AAO)

MikeBennett: List what the TLO considerations are as above;     (6AAP)

MikeBennett: List what are the paradigms within which one has to think these things: 4D v 3D; Realist or not; treatments fo time and temporality and so on. Whether you can have multiple inheritance     (6AAQ)

MikeBennett: Action Circulate the list of problem areas and the list of pardigmatic considerations, as outlined above, via email, then start to put a wiki page together.     (6AAR)

MikeBennett: AoB?     (6AAS)

MikeBennett: Next Meeting?     (6AAT)

MikeBennett: 4 Feb (during the IOF thing in OSLO)     (6AAU)

MikeBennett: Defer to 11 Feb?     (6AAV)

MikeBennett: Separately, Andrea can't make this time slot generally.     (6AAW)

MikeBennett: We will move these calls to 2 - 3 going forward. Create a new / update existing Outlook entry.     (6AAX)

MikeBennett: Add this to the Ontology Events Google Calendar.     (6AAY)

MikeBennett: This should go in the IAOA Google Calendar. Can also go in the Shed and SWAO spreadsheet-generated one.     (6AAZ)

MikeBennett: MB will update the 2019 Events spreadsheet and share with IAOA. They may choose to include or ignore some of the more specialist stuff.     (6AAAA)

MikeBennett: Can include AI, Blockchain, other things of relevance to ontologists but not directly about ontology.     (6AAAB)

MikeBennett: Confirmed we will meet on 11th.     (6AAAC)

[edit] Attendees     (7)

[edit] Next Meetings     (8)

[edit] Previous Meetings     (9)