Actions

Ontolog Forum

This is one of the "OntologySummit2013_Hackathon_Clinics" Projects ...

Hackathon Project (HC-05): Ontology of Ontology Evaluation

Project Co-champions: Amanda Vizedom, Ali Hashemi,

Event Date / Time: (Day-1) Sat 2013.03.30; optionally extending to Sun 2013.03.31 ... see: [ details]

Archived work output can be found here

Participant Volunteers

Add your name below, along with any time constraints or other participation-related comments. Thank you!

Updated Logistical Details

This wiki page will be the go-to place for the most current and accurate information; any additional channels used or sub-events scheduled will be listed here.

Update after close of session, Sunday, 2013-Mar-31: We have a variety of action items to follow up on, as captured in the chat transcript. For the moment, I (Amanda) am changing the Google docs to read only, mostly to avoid confusing folks who may follow these links and expect new edits to the docs to be read and/or incorporated. The end-of-day files have also been saved, by Peter, to the shared web-dav workspace (and that is where new output should go).

Update for Sunday, 2013-Mar-31: We will meet at 10 am, probably for 3 hrs or so. Audio via usual teleconference bridge, supplied by Peter (confirmed). Continuing to review, consolidate, and finalize the conceptual model spread across the comments in https://docs.google.com/drawings/d/1l3nc_lJgNlD1lGDY1kN7DbvzbYBZMfkn12l1_r_RIU4/edit?usp=sharing (currently view only) and the various subdiagrams in the https://drive.google.com/folderview?id=0B_5lZBgIG6LRQnZSdUFWX2FnYTg&usp=sharing shared Google drive (currenly editable by anyone with the URL).

  • Activities will be in session:
    • Sat 2013.03.30 (Primary Hack Day)
      • starting at 7:00am PDT / 10:00am EDT / 3:00pm CET / 14:00 UTC with kick-off open webcast segment

(though chat room will be open and some folks starting to work a bit earlier, from: 6:00am PDT / 09:00am EDT / 3:00pm CET / 13:00 UTC)

      • to: 18:00pm EDT
    • Sun 2013.03.31 (Optional Extension)
      • from 7:00am PDT / 10:00am EDT / 3:00pm CET / 14:00 UTC
      • to: 13:00 EDT
  • Open webcast segment
    • up to 1 Hr. 45 min. (expecting to be on the shorter side)
    • starting: Sat 2013.03.30 - 7:00am PDT / 10:00am EDT / 3:00pm CET / 14:00 UTC
    • Dial-in for the "Open Webcast" segment:
      • Phone (US): +1 (206) 402-0100 ... Conference ID: 141184# ; or
      • Skype: joinconference ... Conference ID: 141184#
  • Project chat-room (up when in session): http://webconf.soaphub.org/conf/room/hc-05
    • Currently there are no plans to use subgroup chat rooms, but if that changes in-session, such subgroup chat-room will be designated (using ../room/hc-05-1 ; ../room/hc-05-2 etc.), announced in the main chat, and listed here on the project wiki page.
  • Live Collaboration and Shared files:
  • Google Drive: https://drive.google.com/folderview?id=0B_5lZBgIG6LRQnZSdUFWX2FnYTg&usp=sharing
    • This is a shared Google Drive, set to be viewable and editable by anyone with the URL.
    • It is recommended, though not required, that participants explicitly add themselves as editors, so that they can easily find files and, if they are Google drive users, have synced copies of the drive and all files ready-to-hand.
    • All editors and viewers of any file in the drive can see who else is working in it and what they are doing, in real time. This supports our real-time collaborative editing goals.
    • Finished files (and periodic checkpoints, if desired) can be upload to shared static file share work space supported by cim.
  • Update: we exceeded Google Hangout limit (10 people) on Saturday, but were able to use the teleconference bridge for the remainder of our time. We will also be able to use the bridge on Sunday.
  • Tools for development activities
    • Requirements gathering: harvesting content and/or excerpts from summit materials into one collection
      • Shared document in Google drive, shared in hangout.
    • Conceptual Model development
      • Google's simple text and Drawing tools autosaved to shared drive.
    • Formal representation (day 2):
      • This will depend in part on the characteristics of the resulting conceptual model. A representation language that is appropriate to that model will be chosen by participants. Tools for that language, available to all participants, will then be selected for day 2 work.

Abstract

The 2013 Ontology Summit is bringing together a rich range of perspectives on ontology evaluation. Some of these perspectives are fairly abstract, some are encoded in methods and practices, and some are encoded in tools. Critical interaction via summit sessions and discussion has resulted in greater sharing of knowledge and in richer understandings of ontology evaluation at multiple elements. This enrichment is likely to be apparent in the activities and future products of summit participants. Another manifestation of this enrichment will be the summit Communique. The goal of this "hackathons & clinics" activity is to add another manifestation: a formal ontology representing ontology evaluation elements, factors, relationships, processes, etc., as they have emerged from summit discussion.

The scope of this project is staged to increase with available resources or as time permits, beginning with representation of the most important general concepts and their relationships to one another, through representation of specific metrics, tools, ontology characteristics, evaluation processes, and multi-faceted organizations thereof. Scope and prioritization will be requirements-driven, based on use cases. Given the very short duration of the Hackathon, the ambitions of this immediate project have been reigned in a bit. Our deliverable for the primary Hackathon day is now focused on producing a rigorous, readily formalizable conceptual model that covers as much of the target content as is feasible, considered with an eye toward many possible uses and informed by the summit discussions. Our formalized deliverable is moved to the second, optional extension day, and prioritizes support for the Ontology Summit PSMW (see HC-04).

see: [ Project document ]

Collaborators Sought

  • Formal and informal modelers to participate in the requirements and content gathering and conceptual modeling to take place on the primary hackathon day
  • Formal ontologists willing and able to participate in the extension activity of formalizing the resulting conceptual model.

Activities Scheduled

Primary Hackathon Day: Saturday, 2013-03-30

Activities

  • Gathering Requirements (whole team, extracting this content into single, shared document)
    • Technical requirements for use in PSMW, to support organization, presentation, and reuse of Ontology Summit 2013 materials and content (i.e., for use by HC-04 team).
    • Coverage requirements from various materials presented during the summit, especially those that identify:
      • evaluatable characteristics of ontologies,
      • metrics and methods for such evaluation,
      • lifecycle stages, maturity models of ontologies and/or evaluation
      • useful faceting characteristics of ontology characteristics
        • characterizing ontology internally, characterizing ontology externally, characterizing relation of ontology specifics to particular use or application context
        • context dependence-independence of relevance of characteristic to quality/suitability
        • automatability ...
        • ...
  • Developing Conceptual Model (whole team, collaborative)
    • Organizing, clarifying and documenting specifications of material above into a conceptual model
    • Critically examining draft models against requirements, refining and revising

deliverable

  • A rigorous but informal conceptual model of significant aspects of ontology evaluation, as discussed and developed during the summit. Though informal, this model shall be of sufficient clarity, quality, and explicitness to support effective and accurate formalization by all or some of the team.

Optional Extension day: Sunday, 2013-03-31

Activities

  • Formalizing the conceptual model developed on Saturday
  • At minimum, formalizing this model in such a way to meet requirements for PSMW use.
  • Possibly also formalizing in an ontology language suitable for use by other tools (ontology development environments, evaluation tools, and/or repositories).

deliverables

  • A formal ontology of ontology evaluation (at minimum: sufficient for PSMW use; preferably also sufficient to be used in other environments and to itself be evaluated by some ontology evaluation tool(s)).



Retrieved from "https://ontologforum.com/index.php?title=OntologySummit2013_Hackathon_Clinics_OntologyOfOntologyEvaluation&oldid=2221"