BITPlan ceur-ws Wiki - User contributions [en]

Workdocumentation 2023-04-06

2023-04-06T09:33:29Z

Tim Holzheim: /* Participants */

{{PageSequence|prev=Workdocumentation 2023-03-30|next=|category=Workdocumentation}}
= Prototype Feedback =
== Date & Time ==
2023-04-06 11:00 - 12:00
== Participants ==
* [[User:Wf|Wf]] ([[User talk:Wf|talk]]) 10:55, 6 April 2023 (CEST)
* Daniel Mietchen
* [[User:Tim Holzheim|Tim Holzheim]] ([[User talk:Tim Holzheim|talk]])

== Agenda ==
* Prototypes
* Possible Queries

== Prototypes ==
The three main prototypes are:

* Single Point of truth server:
http://ceurspt.wikidata.dbis.rwth-aachen.de/index.html

* Volume browser: http://cvb.bitplan.com/

* Semantic Media Wiki:
https://ceur-ws.bitplan.com/index.php/Main_Page
== Possible Queries ==
* https://cr.bitplan.com/index.php/List_of_Queries

Workdocumentation 2023-03-27

2023-03-27T07:29:46Z

Tim Holzheim: /* Individual problems */

= Participants =
* Tim
* Wolfgang
= Agenda =
* CEUR-SPT
* Projectmeeting preperation

= CEUR-SPT =
= Projectmeeting preparation =
== Feedback on UI / HTML ==
* [https://ceur-ws.org/ Original CEUR-WS static page]
* [http://cvb.bitplan.com Volume Browser]]
* [http://ceurspt.wikidata.dbis.rwth-aachen.de/Vol-3362.html CEUR-SPT ]
* [http://ceur-ws.bitplan.com Semantic Mediawiki]]
=== Ideas ===
* Help Button
* Link to Open Source Projects / Issues
* Link to Ticket System

== Feedback on Structure / Ontology /Link ML ==
* [https://docs.google.com/spreadsheets/d/1rDyzmiphqrnwugid9-y4B6nRUUBjxusHXvmxzkEeGKg/edit#gid=0 Progress]
* Series handling e.g. [https://ceur-ws.org/iaoa.html iaoa]

== Feedback on Software ==
* Tickets - Severity / Milestones
* Documentation
* Code Review
* Tests / CI

== Fixing errors in the original HTML files ==

=== Individual problems ===
* [https://ceur-ws.org/Vol-2836/ Vol-2836: Proceedings of the Conference on Digital Curation Technologies (Qurator 2021) ]
** publication date: 2019-03-24
** submitted: 2021-01-29

=== Systematic problems ===
* newlines in acronyms ..
* failed parsing for papers
* duplicate paper ids
* paper-1 pointing to complete proceedings
* trailing blanks in url https://stackoverflow.com/questions/75816859/why-is-a-valid-uri-not-valid-in-linkml/75819565#75819565
* ftp link instead of http link
* bibo / dcterms annotations
* logos
* emphasized text
* other HTML pecularities / exotic cases

Workdocumentation 2023-03-27

2023-03-27T07:15:43Z

Tim Holzheim: /* Individual problems */

= Participants =
* Tim
* Wolfgang
= Agenda =
* CEUR-SPT
* Projectmeeting preperation

= CEUR-SPT =
= Projectmeeting preparation =
== Feedback on UI / HTML ==
* Volume Browser
* CEUR-SPT
* Semantic Mediawiki
== Feedback on Structure / Ontology /Link ML ==
* [https://docs.google.com/spreadsheets/d/1rDyzmiphqrnwugid9-y4B6nRUUBjxusHXvmxzkEeGKg/edit#gid=0 Progress]

== Feedback on Software ==
* Tickets - Severity / Milestones
* Documentation
* Code Review
* Tests / CI

== Fixing errors in the original HTML files ==

=== Individual problems ===
* [https://ceur-ws.org/Vol-2836/| Vol-2836: Proceedings of the Conference on Digital Curation Technologies (Qurator 2021) ]
** publication date: 2019-03-24
** submitted: 2021-01-29

=== Systematic problems ===
* newlines in acronyms ..
* failed parsing for papers
* duplicate paper ids
* paper-1 pointing to complete proceedings
* trailing blanks in url https://stackoverflow.com/questions/75816859/why-is-a-valid-uri-not-valid-in-linkml/75819565#75819565
* ftp link instead of http link
* bibo / dcterms annotations
* logos
* emphasized text
* other HTML pecularities / exotic cases

Editor Extraction and Reconciliation

2023-03-09T09:47:53Z

Tim Holzheim: /* Wikidata Reconciliation */

= Editor Extraction =

* covered volumes 1-3354
** optimized for volumes 600+
* 11764 Editor records
* for 228 volumes no editors could be extracted
[[File:volume_editor_distribution.png|400px]]

= Reconciliation =
== dblp reconciliation ==

=== Volume Editors of CEUR-WS in dblp ===
<source lang="sparql">
PREFIX dblp: <https://dblp.org/rdf/schema#>
SELECT DISTINCT ?vol_number
(GROUP_CONCAT(DISTINCT ?name; separator="|") as ?names)
(GROUP_CONCAT(DISTINCT ?dblp_id; separator="|") as ?concat_dblp_id)
WHERE {
?volume dblp:publishedIn "CEUR Workshop Proceedings" ;
dblp:publishedInSeries "CEUR Workshop Proceedings" ;
dblp:publishedInSeriesVolume ?vol_number;
dblp:hasSignature ?editors.
?editors dblp:signatureDblpName ?name ;
dblp:signatureCreator ?dblp_id ;
dblp:signatureOrdinal ?editor_ordinal ;
dblp:signaturePublication ?dblp_publication_id ;
a dblp:EditorSignature.
}
GROUP BY ?vol_number
</source>

=== Volume Editors of CEUR-WS in dblp with identifiers ===
<source lang="sparql">
PREFIX datacite: <http://purl.org/spar/datacite/>
PREFIX dblp: <https://dblp.org/rdf/schema#>
PREFIX litre: <http://purl.org/spar/literal/>
SELECT DISTINCT
(group_concat(DISTINCT ?nameVar;separator='|') as ?name)
(group_concat(DISTINCT ?homepageVar;separator='|') as ?homepage)
(group_concat(DISTINCT ?affiliationVar;separator='|') as ?affiliation)
(group_concat(DISTINCT ?dblpVar;separator='|') as ?dblp)
(group_concat(DISTINCT ?wikidataVar;separator='|') as ?wikidata)
(group_concat(DISTINCT ?orcidVar;separator='|') as ?orcid)
(group_concat(DISTINCT ?googleScholarVar;separator='|') as ?googleScholar)
(group_concat(DISTINCT ?acmVar;separator='|') as ?acm)
(group_concat(DISTINCT ?twitterVar;separator='|') as ?twitter)
(group_concat(DISTINCT ?githubVar;separator='|') as ?github)
(group_concat(DISTINCT ?viafVar;separator='|') as ?viaf)
(group_concat(DISTINCT ?scigraphVar;separator='|') as ?scigraph)
(group_concat(DISTINCT ?zbmathVar;separator='|') as ?zbmath)
(group_concat(DISTINCT ?researchGateVar;separator='|') as ?researchGate)
(group_concat(DISTINCT ?mathGenealogyVar;separator='|') as ?mathGenealogy)
(group_concat(DISTINCT ?locVar;separator='|') as ?loc)
(group_concat(DISTINCT ?linkedinVar;separator='|') as ?linkedin)
(group_concat(DISTINCT ?lattesVar;separator='|') as ?lattes)
(group_concat(DISTINCT ?isniVar;separator='|') as ?isni)
(group_concat(DISTINCT ?ieeeVar;separator='|') as ?ieee)
(group_concat(DISTINCT ?geprisVar;separator='|') as ?gepris)
(group_concat(DISTINCT ?gndVar;separator='|') as ?gnd)
WHERE{
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volume;
dblp:editedBy ?editor.
?editor dblp:primaryCreatorName ?nameVar.
OPTIONAL{?editor dblp:primaryHomepage ?homepageVar.}
OPTIONAL{?editor dblp:primaryAffiliation ?affiliationVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?dblp_blank.
?dblp_blank datacite:usesIdentifierScheme datacite:dblp;
litre:hasLiteralValue ?dblpVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?wikidata_blank.
?wikidata_blank datacite:usesIdentifierScheme datacite:wikidata;
litre:hasLiteralValue ?wikidataVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?orcid_blank.
?orcid_blank datacite:usesIdentifierScheme datacite:orcid;
litre:hasLiteralValue ?orcidVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?googleScholar_blank.
?googleScholar_blank datacite:usesIdentifierScheme datacite:google-scholar;
litre:hasLiteralValue ?googleScholarVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?acm_blank.
?acm_blank datacite:usesIdentifierScheme datacite:acm;
litre:hasLiteralValue ?acmVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?twitter_blank.
?twitter_blank datacite:usesIdentifierScheme datacite:twitter;
litre:hasLiteralValue ?twitterVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?github_blank.
?github_blank datacite:usesIdentifierScheme datacite:github;
litre:hasLiteralValue ?githubVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?viaf_blank.
?viaf_blank datacite:usesIdentifierScheme datacite:viaf;
litre:hasLiteralValue ?viafVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?scigraph_blank.
?scigraph_blank datacite:usesIdentifierScheme datacite:scigraph;
litre:hasLiteralValue ?scigraphVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?zbmath_blank.
?zbmath_blank datacite:usesIdentifierScheme datacite:zbmath;
litre:hasLiteralValue ?zbmathVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?researchGate_blank.
?researchGate_blank datacite:usesIdentifierScheme datacite:research-gate;
litre:hasLiteralValue ?researchGateVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?mathGenealogy_blank.
?mathGenealogy_blank datacite:usesIdentifierScheme datacite:math-genealogy;
litre:hasLiteralValue ?mathGenealogyVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?loc_blank.
?loc_blank datacite:usesIdentifierScheme datacite:loc;
litre:hasLiteralValue ?locVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?linkedin_blank.
?linkedin_blank datacite:usesIdentifierScheme datacite:linkedin;
litre:hasLiteralValue ?linkedinVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?lattes_blank.
?lattes_blank datacite:usesIdentifierScheme datacite:lattes;
litre:hasLiteralValue ?lattesVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?isni_blank.
?isni_blank datacite:usesIdentifierScheme datacite:isni;
litre:hasLiteralValue ?isniVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?ieee_blank.
?ieee_blank datacite:usesIdentifierScheme datacite:ieee;
litre:hasLiteralValue ?ieeeVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?gepris_blank.
?gepris_blank datacite:usesIdentifierScheme datacite:gepris;
litre:hasLiteralValue ?geprisVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?gnd_blank.
?gnd_blank datacite:usesIdentifierScheme datacite:gnd;
litre:hasLiteralValue ?gndVar.}
}
GROUP BY ?editor

</source>
=== Comparing Extracted and dblp Editors ===
* editor by volume comparison
** 2233 volume the extracted editors match the dblp editors
** 807 volumes are missing in dblp (editors extracted)
** 27 volumes more editors were extracted than in dblp
** 387 volumes dblp has more editors than we could extract
* 9321 out of 11764 editor records can be reconciled
** 79.23%

== Wikidata Reconciliation ==
Using the ids queried from dblp to find the corresponding wikidata entry.

Current strategy:
;Input: List of different identifiers that are known about a editor
;Output: SPARQL query

Example:
*Input:
** homepage: http://www.stefandecker.org
** gnd id: 173443443
** dblp author id: d/StefanDecker
* Output:
<source lang="sparql">
PREFIX wdt: <http://www.wikidata.org/prop/direct/>
PREFIX wikibase: <http://wikiba.se/ontology#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?person ?personLabel
WHERE
{
{OPTIONAL{ ?person wdt:P856 <http://www.stefandecker.org>.} }
UNION
{OPTIONAL{ ?person wdt:P227 "173443443".} } # gnd
UNION
{OPTIONAL{ ?person wdt:P2456 "d/StefanDecker".} } # dblp
?person rdfs:label ?personLabel. FILTER(lang(?personLabel)="en")
}
</source>
Depending on the available identifiers the query is adjusted accordingly by adding the corresponding OPTIONAL clauses.

Running these queries for all 4942 editors known by dblp we get:
* 1467 editors were found in wikidata
* 62 editor records in dblp have a conflict with wikidata
* 3413 dblp editor records were not found in wikidata

The figure below shows the distribution of the available identifiers depending of the three categories identified, conflict, unkown

[[File:editors_wikidata_reconciliation.png|800px]]

File:Editors wikidata reconciliation.png

2023-03-09T09:45:18Z

Tim Holzheim: Tim Holzheim uploaded a new version of File:Editors wikidata reconciliation.png

File uploaded with MsUpload

Editor Extraction and Reconciliation

2023-03-09T09:44:30Z

Tim Holzheim: /* Wikidata Reconciliation */

= Editor Extraction =

* covered volumes 1-3354
** optimized for volumes 600+
* 11764 Editor records
* for 228 volumes no editors could be extracted
[[File:volume_editor_distribution.png|400px]]

= Reconciliation =
== dblp reconciliation ==

=== Volume Editors of CEUR-WS in dblp ===
<source lang="sparql">
PREFIX dblp: <https://dblp.org/rdf/schema#>
SELECT DISTINCT ?vol_number
(GROUP_CONCAT(DISTINCT ?name; separator="|") as ?names)
(GROUP_CONCAT(DISTINCT ?dblp_id; separator="|") as ?concat_dblp_id)
WHERE {
?volume dblp:publishedIn "CEUR Workshop Proceedings" ;
dblp:publishedInSeries "CEUR Workshop Proceedings" ;
dblp:publishedInSeriesVolume ?vol_number;
dblp:hasSignature ?editors.
?editors dblp:signatureDblpName ?name ;
dblp:signatureCreator ?dblp_id ;
dblp:signatureOrdinal ?editor_ordinal ;
dblp:signaturePublication ?dblp_publication_id ;
a dblp:EditorSignature.
}
GROUP BY ?vol_number
</source>

=== Volume Editors of CEUR-WS in dblp with identifiers ===
<source lang="sparql">
PREFIX datacite: <http://purl.org/spar/datacite/>
PREFIX dblp: <https://dblp.org/rdf/schema#>
PREFIX litre: <http://purl.org/spar/literal/>
SELECT DISTINCT
(group_concat(DISTINCT ?nameVar;separator='|') as ?name)
(group_concat(DISTINCT ?homepageVar;separator='|') as ?homepage)
(group_concat(DISTINCT ?affiliationVar;separator='|') as ?affiliation)
(group_concat(DISTINCT ?dblpVar;separator='|') as ?dblp)
(group_concat(DISTINCT ?wikidataVar;separator='|') as ?wikidata)
(group_concat(DISTINCT ?orcidVar;separator='|') as ?orcid)
(group_concat(DISTINCT ?googleScholarVar;separator='|') as ?googleScholar)
(group_concat(DISTINCT ?acmVar;separator='|') as ?acm)
(group_concat(DISTINCT ?twitterVar;separator='|') as ?twitter)
(group_concat(DISTINCT ?githubVar;separator='|') as ?github)
(group_concat(DISTINCT ?viafVar;separator='|') as ?viaf)
(group_concat(DISTINCT ?scigraphVar;separator='|') as ?scigraph)
(group_concat(DISTINCT ?zbmathVar;separator='|') as ?zbmath)
(group_concat(DISTINCT ?researchGateVar;separator='|') as ?researchGate)
(group_concat(DISTINCT ?mathGenealogyVar;separator='|') as ?mathGenealogy)
(group_concat(DISTINCT ?locVar;separator='|') as ?loc)
(group_concat(DISTINCT ?linkedinVar;separator='|') as ?linkedin)
(group_concat(DISTINCT ?lattesVar;separator='|') as ?lattes)
(group_concat(DISTINCT ?isniVar;separator='|') as ?isni)
(group_concat(DISTINCT ?ieeeVar;separator='|') as ?ieee)
(group_concat(DISTINCT ?geprisVar;separator='|') as ?gepris)
(group_concat(DISTINCT ?gndVar;separator='|') as ?gnd)
WHERE{
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volume;
dblp:editedBy ?editor.
?editor dblp:primaryCreatorName ?nameVar.
OPTIONAL{?editor dblp:primaryHomepage ?homepageVar.}
OPTIONAL{?editor dblp:primaryAffiliation ?affiliationVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?dblp_blank.
?dblp_blank datacite:usesIdentifierScheme datacite:dblp;
litre:hasLiteralValue ?dblpVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?wikidata_blank.
?wikidata_blank datacite:usesIdentifierScheme datacite:wikidata;
litre:hasLiteralValue ?wikidataVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?orcid_blank.
?orcid_blank datacite:usesIdentifierScheme datacite:orcid;
litre:hasLiteralValue ?orcidVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?googleScholar_blank.
?googleScholar_blank datacite:usesIdentifierScheme datacite:google-scholar;
litre:hasLiteralValue ?googleScholarVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?acm_blank.
?acm_blank datacite:usesIdentifierScheme datacite:acm;
litre:hasLiteralValue ?acmVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?twitter_blank.
?twitter_blank datacite:usesIdentifierScheme datacite:twitter;
litre:hasLiteralValue ?twitterVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?github_blank.
?github_blank datacite:usesIdentifierScheme datacite:github;
litre:hasLiteralValue ?githubVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?viaf_blank.
?viaf_blank datacite:usesIdentifierScheme datacite:viaf;
litre:hasLiteralValue ?viafVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?scigraph_blank.
?scigraph_blank datacite:usesIdentifierScheme datacite:scigraph;
litre:hasLiteralValue ?scigraphVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?zbmath_blank.
?zbmath_blank datacite:usesIdentifierScheme datacite:zbmath;
litre:hasLiteralValue ?zbmathVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?researchGate_blank.
?researchGate_blank datacite:usesIdentifierScheme datacite:research-gate;
litre:hasLiteralValue ?researchGateVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?mathGenealogy_blank.
?mathGenealogy_blank datacite:usesIdentifierScheme datacite:math-genealogy;
litre:hasLiteralValue ?mathGenealogyVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?loc_blank.
?loc_blank datacite:usesIdentifierScheme datacite:loc;
litre:hasLiteralValue ?locVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?linkedin_blank.
?linkedin_blank datacite:usesIdentifierScheme datacite:linkedin;
litre:hasLiteralValue ?linkedinVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?lattes_blank.
?lattes_blank datacite:usesIdentifierScheme datacite:lattes;
litre:hasLiteralValue ?lattesVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?isni_blank.
?isni_blank datacite:usesIdentifierScheme datacite:isni;
litre:hasLiteralValue ?isniVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?ieee_blank.
?ieee_blank datacite:usesIdentifierScheme datacite:ieee;
litre:hasLiteralValue ?ieeeVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?gepris_blank.
?gepris_blank datacite:usesIdentifierScheme datacite:gepris;
litre:hasLiteralValue ?geprisVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?gnd_blank.
?gnd_blank datacite:usesIdentifierScheme datacite:gnd;
litre:hasLiteralValue ?gndVar.}
}
GROUP BY ?editor

</source>
=== Comparing Extracted and dblp Editors ===
* editor by volume comparison
** 2233 volume the extracted editors match the dblp editors
** 807 volumes are missing in dblp (editors extracted)
** 27 volumes more editors were extracted than in dblp
** 387 volumes dblp has more editors than we could extract
* 9321 out of 11764 editor records can be reconciled
** 79.23%

== Wikidata Reconciliation ==
Using the ids queried from dblp to find the corresponding wikidata entry.

Current strategy:
;Input: List of different identifiers that are known about a editor
;Output: SPARQL query

Example:
*Input:
** homepage: http://www.stefandecker.org
** gnd id: 173443443
** dblp author id: d/StefanDecker
* Output:
<source lang="sparql">
PREFIX wdt: <http://www.wikidata.org/prop/direct/>
PREFIX wikibase: <http://wikiba.se/ontology#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?person ?personLabel
WHERE
{
{OPTIONAL{ ?person wdt:P856 <http://www.stefandecker.org>.} }
UNION
{OPTIONAL{ ?person wdt:P227 "173443443".} } # gnd
UNION
{OPTIONAL{ ?person wdt:P2456 "d/StefanDecker".} } # dblp
?person rdfs:label ?personLabel. FILTER(lang(?personLabel)="en")
}
</source>

Running these queries for all 4942 editors known by dblp we get:
* 1467 editors were found in wikidata
* 62 editor records in dblp have a conflict with wikidata
* 3413 dblp editor records were not found in wikidata

The figure below shows the distribution of the available identifiers depending of the three categories identified, conflict, unkown
[[File:editors_wikidata_reconciliation.png|800px]]

File:Editors wikidata reconciliation.png

2023-03-09T09:33:39Z

Tim Holzheim: File uploaded with MsUpload

File uploaded with MsUpload

Workdocumentation 2023-03-09

2023-03-09T09:17:35Z

Tim Holzheim: /* Paper Template Example */

== Add properties to CeurwsSchema ==
Add properties url,urn,k10plus,dblp to CeurwsSchema
* https://ceur-ws.bitplan.com/index.php?title=CeurwsSchema&type=revision&diff=1126&oldid=968
<source lang='bash'>
py-yprinciple-gen % ypgen --wikiId ceur-ws --context CeurwsSchema --topics Volume --genViaMwApi --noDry

generating Category for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume&type=revision&diff=1133&oldid=931(498)
generating Concept for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume&type=revision&diff=1134&oldid=932(223)
generating Form for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume&type=revision&diff=1135&oldid=933(487)
generating Help for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume&type=revision&diff=1136&oldid=934(223)
generating List of for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=List of Volumes&type=revision&diff=1137&oldid=1008(350)
generating Template for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume&type=revision&diff=1138&oldid=936(1582)
generating Property for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume number&type=revision&diff=924&oldid=924(80)
generating Property for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume acronym&type=revision&diff=938&oldid=938(80)
generating Property for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume wikidataid&type=revision&diff=939&oldid=939(80)
generating Property for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume title&type=revision&diff=940&oldid=940(80)
generating Property for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume description&type=revision&diff=937&oldid=937(80)
generating Property for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume url&type=revision&diff=1139&oldid=0(0)
generating Property for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume date&type=revision&diff=1140&oldid=0(0)
generating Property for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume dblp&type=revision&diff=1141&oldid=0(0)
generating Property for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume k10plus&type=revision&diff=1142&oldid=0(0)
generating Property for Volume via Mediawiki Api...
diff: https://ceur-ws.bitplan.com/index.php?title=Volume urn&type=revision&diff=1143&oldid=0(0)
generating Python for Volume via Mediawiki Api...
diff: (1764)
</source>
== Sync new fields with wikidata ==
<source lang='bash'>
smwsync -u -t ceur-ws --context CeurwsSchema --topic Volume -p url urn k10plus dblp --progress
updating cache for CeurwsSchema:Volume from wiki ceur-ws ...
stored 8 Volume items to /Users/wf/.smwsync/ceur-ws/CeurwsSchema/Volume.json
8 Volume items to sync ...
Vol-3346→dlbp: 100%|████████████████████████████| 32/32 [00:40<00:00, 1.26s/it]
</source>
<source lang='bash'>
smwsync -u -t ceur-ws --context CeurwsSchema --topic Volume -pkv Q113542713 -p desc title
</source>

= Paper Template Example=
<pre>
{|class="wikitable" style=""
! colspan="2"| Paper
|-
![[Title::@@@]]
| {{#show: {{PAGENAME}}| ?title}}
|-
![[Author::@@@]]
|{{#show: {{PAGENAME}}| ?author}}
|-
![[DOI::@@@]]
|{{#show: {{PAGENAME}}| ?doi}}
|-
![[Publication date::@@@]]
|{{#show: {{PAGENAME}}| ?publication_date}}
|-
![[Wikidata id::@@@]]
|{{#show: {{PAGENAME}}| ?wikidata_id}}
|-
![[Event::@@@]]
|{{#show: {{PAGENAME}}| ?event}}
|}

</pre>
* https://www.semantic-mediawiki.org/wiki/Help:Property_links

Workdocumentation 2023-03-09

2023-03-09T09:15:58Z

Tim Holzheim:

Editor Extraction and Reconciliation

2023-03-09T09:00:42Z

Tim Holzheim: /* Volume Editors of CEUR-WS in dblp */

= Editor Extraction =

* covered volumes 1-3354
** optimized for volumes 600+
* 11764 Editor records
* for 228 volumes no editors could be extracted
[[File:volume_editor_distribution.png|400px]]

= Reconciliation =
== dblp reconciliation ==

=== Volume Editors of CEUR-WS in dblp ===
<source lang="sparql">
PREFIX dblp: <https://dblp.org/rdf/schema#>
SELECT DISTINCT ?vol_number
(GROUP_CONCAT(DISTINCT ?name; separator="|") as ?names)
(GROUP_CONCAT(DISTINCT ?dblp_id; separator="|") as ?concat_dblp_id)
WHERE {
?volume dblp:publishedIn "CEUR Workshop Proceedings" ;
dblp:publishedInSeries "CEUR Workshop Proceedings" ;
dblp:publishedInSeriesVolume ?vol_number;
dblp:hasSignature ?editors.
?editors dblp:signatureDblpName ?name ;
dblp:signatureCreator ?dblp_id ;
dblp:signatureOrdinal ?editor_ordinal ;
dblp:signaturePublication ?dblp_publication_id ;
a dblp:EditorSignature.
}
GROUP BY ?vol_number
</source>

=== Volume Editors of CEUR-WS in dblp with identifiers ===
<source lang="sparql">
PREFIX datacite: <http://purl.org/spar/datacite/>
PREFIX dblp: <https://dblp.org/rdf/schema#>
PREFIX litre: <http://purl.org/spar/literal/>
SELECT DISTINCT
(group_concat(DISTINCT ?nameVar;separator='|') as ?name)
(group_concat(DISTINCT ?homepageVar;separator='|') as ?homepage)
(group_concat(DISTINCT ?affiliationVar;separator='|') as ?affiliation)
(group_concat(DISTINCT ?dblpVar;separator='|') as ?dblp)
(group_concat(DISTINCT ?wikidataVar;separator='|') as ?wikidata)
(group_concat(DISTINCT ?orcidVar;separator='|') as ?orcid)
(group_concat(DISTINCT ?googleScholarVar;separator='|') as ?googleScholar)
(group_concat(DISTINCT ?acmVar;separator='|') as ?acm)
(group_concat(DISTINCT ?twitterVar;separator='|') as ?twitter)
(group_concat(DISTINCT ?githubVar;separator='|') as ?github)
(group_concat(DISTINCT ?viafVar;separator='|') as ?viaf)
(group_concat(DISTINCT ?scigraphVar;separator='|') as ?scigraph)
(group_concat(DISTINCT ?zbmathVar;separator='|') as ?zbmath)
(group_concat(DISTINCT ?researchGateVar;separator='|') as ?researchGate)
(group_concat(DISTINCT ?mathGenealogyVar;separator='|') as ?mathGenealogy)
(group_concat(DISTINCT ?locVar;separator='|') as ?loc)
(group_concat(DISTINCT ?linkedinVar;separator='|') as ?linkedin)
(group_concat(DISTINCT ?lattesVar;separator='|') as ?lattes)
(group_concat(DISTINCT ?isniVar;separator='|') as ?isni)
(group_concat(DISTINCT ?ieeeVar;separator='|') as ?ieee)
(group_concat(DISTINCT ?geprisVar;separator='|') as ?gepris)
(group_concat(DISTINCT ?gndVar;separator='|') as ?gnd)
WHERE{
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volume;
dblp:editedBy ?editor.
?editor dblp:primaryCreatorName ?nameVar.
OPTIONAL{?editor dblp:primaryHomepage ?homepageVar.}
OPTIONAL{?editor dblp:primaryAffiliation ?affiliationVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?dblp_blank.
?dblp_blank datacite:usesIdentifierScheme datacite:dblp;
litre:hasLiteralValue ?dblpVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?wikidata_blank.
?wikidata_blank datacite:usesIdentifierScheme datacite:wikidata;
litre:hasLiteralValue ?wikidataVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?orcid_blank.
?orcid_blank datacite:usesIdentifierScheme datacite:orcid;
litre:hasLiteralValue ?orcidVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?googleScholar_blank.
?googleScholar_blank datacite:usesIdentifierScheme datacite:google-scholar;
litre:hasLiteralValue ?googleScholarVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?acm_blank.
?acm_blank datacite:usesIdentifierScheme datacite:acm;
litre:hasLiteralValue ?acmVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?twitter_blank.
?twitter_blank datacite:usesIdentifierScheme datacite:twitter;
litre:hasLiteralValue ?twitterVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?github_blank.
?github_blank datacite:usesIdentifierScheme datacite:github;
litre:hasLiteralValue ?githubVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?viaf_blank.
?viaf_blank datacite:usesIdentifierScheme datacite:viaf;
litre:hasLiteralValue ?viafVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?scigraph_blank.
?scigraph_blank datacite:usesIdentifierScheme datacite:scigraph;
litre:hasLiteralValue ?scigraphVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?zbmath_blank.
?zbmath_blank datacite:usesIdentifierScheme datacite:zbmath;
litre:hasLiteralValue ?zbmathVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?researchGate_blank.
?researchGate_blank datacite:usesIdentifierScheme datacite:research-gate;
litre:hasLiteralValue ?researchGateVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?mathGenealogy_blank.
?mathGenealogy_blank datacite:usesIdentifierScheme datacite:math-genealogy;
litre:hasLiteralValue ?mathGenealogyVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?loc_blank.
?loc_blank datacite:usesIdentifierScheme datacite:loc;
litre:hasLiteralValue ?locVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?linkedin_blank.
?linkedin_blank datacite:usesIdentifierScheme datacite:linkedin;
litre:hasLiteralValue ?linkedinVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?lattes_blank.
?lattes_blank datacite:usesIdentifierScheme datacite:lattes;
litre:hasLiteralValue ?lattesVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?isni_blank.
?isni_blank datacite:usesIdentifierScheme datacite:isni;
litre:hasLiteralValue ?isniVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?ieee_blank.
?ieee_blank datacite:usesIdentifierScheme datacite:ieee;
litre:hasLiteralValue ?ieeeVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?gepris_blank.
?gepris_blank datacite:usesIdentifierScheme datacite:gepris;
litre:hasLiteralValue ?geprisVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?gnd_blank.
?gnd_blank datacite:usesIdentifierScheme datacite:gnd;
litre:hasLiteralValue ?gndVar.}
}
GROUP BY ?editor

</source>
=== Comparing Extracted and dblp Editors ===
* editor by volume comparison
** 2233 volume the extracted editors match the dblp editors
** 807 volumes are missing in dblp (editors extracted)
** 27 volumes more editors were extracted than in dblp
** 387 volumes dblp has more editors than we could extract
* 9321 out of 11764 editor records can be reconciled
** 79.23%

== Wikidata Reconciliation ==
Using the ids queried from dblp to find the corresponding wikidata entry.

Current strategy:
;Input: List of different identifiers that are known about a editor
;Output: SPARQL query

Example:
*Input:
** homepage: http://www.stefandecker.org
** gnd id: 173443443
** dblp author id: d/StefanDecker
* Output:
<source lang="sparql">
PREFIX wdt: <http://www.wikidata.org/prop/direct/>
PREFIX wikibase: <http://wikiba.se/ontology#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?person ?personLabel
WHERE
{
{OPTIONAL{ ?person wdt:P856 <http://www.stefandecker.org>.} }
UNION
{OPTIONAL{ ?person wdt:P227 "173443443".} } # gnd
UNION
{OPTIONAL{ ?person wdt:P2456 "d/StefanDecker".} } # dblp
?person rdfs:label ?personLabel. FILTER(lang(?personLabel)="en")
}
</source>

Running these queries for all 4942 editors known by dblp we get:
* Queries currently running...[[User:Tim Holzheim|Tim Holzheim]] ([[User talk:Tim Holzheim|talk]]) 09:49, 9 March 2023 (CET)

Editor Extraction and Reconciliation

2023-03-09T08:57:35Z

Tim Holzheim: /* Comparing Extracted and dblp Editors */

= Editor Extraction =

* covered volumes 1-3354
** optimized for volumes 600+
* 11764 Editor records
* for 228 volumes no editors could be extracted
[[File:volume_editor_distribution.png|400px]]

= Reconciliation =
== dblp reconciliation ==

=== Volume Editors of CEUR-WS in dblp ===
<source lang="sparql">
PREFIX dblp: <https://dblp.org/rdf/schema#>
SELECT DISTINCT ?vol_number (GROUP_CONCAT(?name; separator="|") as ?names) (GROUP_CONCAT(?dblp_id; separator="|") as ?concat_dblp_id)
WHERE {
?volume dblp:publishedIn "CEUR Workshop Proceedings" ;
dblp:publishedInSeries "CEUR Workshop Proceedings" ;
dblp:publishedInSeriesVolume ?vol_number;
dblp:hasSignature ?editors.
?editors dblp:signatureDblpName ?name ;
dblp:signatureCreator ?dblp_id ;
dblp:signatureOrdinal ?editor_ordinal ;
dblp:signaturePublication ?dblp_publication_id ;
a dblp:EditorSignature.
}
GROUP BY ?vol_number
</source>

=== Volume Editors of CEUR-WS in dblp with identifiers ===
<source lang="sparql">
PREFIX datacite: <http://purl.org/spar/datacite/>
PREFIX dblp: <https://dblp.org/rdf/schema#>
PREFIX litre: <http://purl.org/spar/literal/>
SELECT DISTINCT
(group_concat(DISTINCT ?nameVar;separator='|') as ?name)
(group_concat(DISTINCT ?homepageVar;separator='|') as ?homepage)
(group_concat(DISTINCT ?affiliationVar;separator='|') as ?affiliation)
(group_concat(DISTINCT ?dblpVar;separator='|') as ?dblp)
(group_concat(DISTINCT ?wikidataVar;separator='|') as ?wikidata)
(group_concat(DISTINCT ?orcidVar;separator='|') as ?orcid)
(group_concat(DISTINCT ?googleScholarVar;separator='|') as ?googleScholar)
(group_concat(DISTINCT ?acmVar;separator='|') as ?acm)
(group_concat(DISTINCT ?twitterVar;separator='|') as ?twitter)
(group_concat(DISTINCT ?githubVar;separator='|') as ?github)
(group_concat(DISTINCT ?viafVar;separator='|') as ?viaf)
(group_concat(DISTINCT ?scigraphVar;separator='|') as ?scigraph)
(group_concat(DISTINCT ?zbmathVar;separator='|') as ?zbmath)
(group_concat(DISTINCT ?researchGateVar;separator='|') as ?researchGate)
(group_concat(DISTINCT ?mathGenealogyVar;separator='|') as ?mathGenealogy)
(group_concat(DISTINCT ?locVar;separator='|') as ?loc)
(group_concat(DISTINCT ?linkedinVar;separator='|') as ?linkedin)
(group_concat(DISTINCT ?lattesVar;separator='|') as ?lattes)
(group_concat(DISTINCT ?isniVar;separator='|') as ?isni)
(group_concat(DISTINCT ?ieeeVar;separator='|') as ?ieee)
(group_concat(DISTINCT ?geprisVar;separator='|') as ?gepris)
(group_concat(DISTINCT ?gndVar;separator='|') as ?gnd)
WHERE{
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volume;
dblp:editedBy ?editor.
?editor dblp:primaryCreatorName ?nameVar.
OPTIONAL{?editor dblp:primaryHomepage ?homepageVar.}
OPTIONAL{?editor dblp:primaryAffiliation ?affiliationVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?dblp_blank.
?dblp_blank datacite:usesIdentifierScheme datacite:dblp;
litre:hasLiteralValue ?dblpVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?wikidata_blank.
?wikidata_blank datacite:usesIdentifierScheme datacite:wikidata;
litre:hasLiteralValue ?wikidataVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?orcid_blank.
?orcid_blank datacite:usesIdentifierScheme datacite:orcid;
litre:hasLiteralValue ?orcidVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?googleScholar_blank.
?googleScholar_blank datacite:usesIdentifierScheme datacite:google-scholar;
litre:hasLiteralValue ?googleScholarVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?acm_blank.
?acm_blank datacite:usesIdentifierScheme datacite:acm;
litre:hasLiteralValue ?acmVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?twitter_blank.
?twitter_blank datacite:usesIdentifierScheme datacite:twitter;
litre:hasLiteralValue ?twitterVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?github_blank.
?github_blank datacite:usesIdentifierScheme datacite:github;
litre:hasLiteralValue ?githubVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?viaf_blank.
?viaf_blank datacite:usesIdentifierScheme datacite:viaf;
litre:hasLiteralValue ?viafVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?scigraph_blank.
?scigraph_blank datacite:usesIdentifierScheme datacite:scigraph;
litre:hasLiteralValue ?scigraphVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?zbmath_blank.
?zbmath_blank datacite:usesIdentifierScheme datacite:zbmath;
litre:hasLiteralValue ?zbmathVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?researchGate_blank.
?researchGate_blank datacite:usesIdentifierScheme datacite:research-gate;
litre:hasLiteralValue ?researchGateVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?mathGenealogy_blank.
?mathGenealogy_blank datacite:usesIdentifierScheme datacite:math-genealogy;
litre:hasLiteralValue ?mathGenealogyVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?loc_blank.
?loc_blank datacite:usesIdentifierScheme datacite:loc;
litre:hasLiteralValue ?locVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?linkedin_blank.
?linkedin_blank datacite:usesIdentifierScheme datacite:linkedin;
litre:hasLiteralValue ?linkedinVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?lattes_blank.
?lattes_blank datacite:usesIdentifierScheme datacite:lattes;
litre:hasLiteralValue ?lattesVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?isni_blank.
?isni_blank datacite:usesIdentifierScheme datacite:isni;
litre:hasLiteralValue ?isniVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?ieee_blank.
?ieee_blank datacite:usesIdentifierScheme datacite:ieee;
litre:hasLiteralValue ?ieeeVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?gepris_blank.
?gepris_blank datacite:usesIdentifierScheme datacite:gepris;
litre:hasLiteralValue ?geprisVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?gnd_blank.
?gnd_blank datacite:usesIdentifierScheme datacite:gnd;
litre:hasLiteralValue ?gndVar.}
}
GROUP BY ?editor

</source>
=== Comparing Extracted and dblp Editors ===
* editor by volume comparison
** 2233 volume the extracted editors match the dblp editors
** 807 volumes are missing in dblp (editors extracted)
** 27 volumes more editors were extracted than in dblp
** 387 volumes dblp has more editors than we could extract
* 9321 out of 11764 editor records can be reconciled
** 79.23%

== Wikidata Reconciliation ==
Using the ids queried from dblp to find the corresponding wikidata entry.

Current strategy:
;Input: List of different identifiers that are known about a editor
;Output: SPARQL query

Example:
*Input:
** homepage: http://www.stefandecker.org
** gnd id: 173443443
** dblp author id: d/StefanDecker
* Output:
<source lang="sparql">
PREFIX wdt: <http://www.wikidata.org/prop/direct/>
PREFIX wikibase: <http://wikiba.se/ontology#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?person ?personLabel
WHERE
{
{OPTIONAL{ ?person wdt:P856 <http://www.stefandecker.org>.} }
UNION
{OPTIONAL{ ?person wdt:P227 "173443443".} } # gnd
UNION
{OPTIONAL{ ?person wdt:P2456 "d/StefanDecker".} } # dblp
?person rdfs:label ?personLabel. FILTER(lang(?personLabel)="en")
}
</source>

Running these queries for all 4942 editors known by dblp we get:
* Queries currently running...[[User:Tim Holzheim|Tim Holzheim]] ([[User talk:Tim Holzheim|talk]]) 09:49, 9 March 2023 (CET)

Editor Extraction and Reconciliation

2023-03-09T08:49:22Z

Tim Holzheim: Created page with "= Editor Extraction = * covered volumes 1-3354 ** optimized for volumes 600+ * 11764 Editor records * for 228 volumes no editors could be extracted File:volume_editor_distr..."

= Editor Extraction =

* covered volumes 1-3354
** optimized for volumes 600+
* 11764 Editor records
* for 228 volumes no editors could be extracted
[[File:volume_editor_distribution.png|400px]]

= Reconciliation =
== dblp reconciliation ==

=== Volume Editors of CEUR-WS in dblp ===
<source lang="sparql">
PREFIX dblp: <https://dblp.org/rdf/schema#>
SELECT DISTINCT ?vol_number (GROUP_CONCAT(?name; separator="|") as ?names) (GROUP_CONCAT(?dblp_id; separator="|") as ?concat_dblp_id)
WHERE {
?volume dblp:publishedIn "CEUR Workshop Proceedings" ;
dblp:publishedInSeries "CEUR Workshop Proceedings" ;
dblp:publishedInSeriesVolume ?vol_number;
dblp:hasSignature ?editors.
?editors dblp:signatureDblpName ?name ;
dblp:signatureCreator ?dblp_id ;
dblp:signatureOrdinal ?editor_ordinal ;
dblp:signaturePublication ?dblp_publication_id ;
a dblp:EditorSignature.
}
GROUP BY ?vol_number
</source>

=== Volume Editors of CEUR-WS in dblp with identifiers ===
<source lang="sparql">
PREFIX datacite: <http://purl.org/spar/datacite/>
PREFIX dblp: <https://dblp.org/rdf/schema#>
PREFIX litre: <http://purl.org/spar/literal/>
SELECT DISTINCT
(group_concat(DISTINCT ?nameVar;separator='|') as ?name)
(group_concat(DISTINCT ?homepageVar;separator='|') as ?homepage)
(group_concat(DISTINCT ?affiliationVar;separator='|') as ?affiliation)
(group_concat(DISTINCT ?dblpVar;separator='|') as ?dblp)
(group_concat(DISTINCT ?wikidataVar;separator='|') as ?wikidata)
(group_concat(DISTINCT ?orcidVar;separator='|') as ?orcid)
(group_concat(DISTINCT ?googleScholarVar;separator='|') as ?googleScholar)
(group_concat(DISTINCT ?acmVar;separator='|') as ?acm)
(group_concat(DISTINCT ?twitterVar;separator='|') as ?twitter)
(group_concat(DISTINCT ?githubVar;separator='|') as ?github)
(group_concat(DISTINCT ?viafVar;separator='|') as ?viaf)
(group_concat(DISTINCT ?scigraphVar;separator='|') as ?scigraph)
(group_concat(DISTINCT ?zbmathVar;separator='|') as ?zbmath)
(group_concat(DISTINCT ?researchGateVar;separator='|') as ?researchGate)
(group_concat(DISTINCT ?mathGenealogyVar;separator='|') as ?mathGenealogy)
(group_concat(DISTINCT ?locVar;separator='|') as ?loc)
(group_concat(DISTINCT ?linkedinVar;separator='|') as ?linkedin)
(group_concat(DISTINCT ?lattesVar;separator='|') as ?lattes)
(group_concat(DISTINCT ?isniVar;separator='|') as ?isni)
(group_concat(DISTINCT ?ieeeVar;separator='|') as ?ieee)
(group_concat(DISTINCT ?geprisVar;separator='|') as ?gepris)
(group_concat(DISTINCT ?gndVar;separator='|') as ?gnd)
WHERE{
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volume;
dblp:editedBy ?editor.
?editor dblp:primaryCreatorName ?nameVar.
OPTIONAL{?editor dblp:primaryHomepage ?homepageVar.}
OPTIONAL{?editor dblp:primaryAffiliation ?affiliationVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?dblp_blank.
?dblp_blank datacite:usesIdentifierScheme datacite:dblp;
litre:hasLiteralValue ?dblpVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?wikidata_blank.
?wikidata_blank datacite:usesIdentifierScheme datacite:wikidata;
litre:hasLiteralValue ?wikidataVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?orcid_blank.
?orcid_blank datacite:usesIdentifierScheme datacite:orcid;
litre:hasLiteralValue ?orcidVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?googleScholar_blank.
?googleScholar_blank datacite:usesIdentifierScheme datacite:google-scholar;
litre:hasLiteralValue ?googleScholarVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?acm_blank.
?acm_blank datacite:usesIdentifierScheme datacite:acm;
litre:hasLiteralValue ?acmVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?twitter_blank.
?twitter_blank datacite:usesIdentifierScheme datacite:twitter;
litre:hasLiteralValue ?twitterVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?github_blank.
?github_blank datacite:usesIdentifierScheme datacite:github;
litre:hasLiteralValue ?githubVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?viaf_blank.
?viaf_blank datacite:usesIdentifierScheme datacite:viaf;
litre:hasLiteralValue ?viafVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?scigraph_blank.
?scigraph_blank datacite:usesIdentifierScheme datacite:scigraph;
litre:hasLiteralValue ?scigraphVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?zbmath_blank.
?zbmath_blank datacite:usesIdentifierScheme datacite:zbmath;
litre:hasLiteralValue ?zbmathVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?researchGate_blank.
?researchGate_blank datacite:usesIdentifierScheme datacite:research-gate;
litre:hasLiteralValue ?researchGateVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?mathGenealogy_blank.
?mathGenealogy_blank datacite:usesIdentifierScheme datacite:math-genealogy;
litre:hasLiteralValue ?mathGenealogyVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?loc_blank.
?loc_blank datacite:usesIdentifierScheme datacite:loc;
litre:hasLiteralValue ?locVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?linkedin_blank.
?linkedin_blank datacite:usesIdentifierScheme datacite:linkedin;
litre:hasLiteralValue ?linkedinVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?lattes_blank.
?lattes_blank datacite:usesIdentifierScheme datacite:lattes;
litre:hasLiteralValue ?lattesVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?isni_blank.
?isni_blank datacite:usesIdentifierScheme datacite:isni;
litre:hasLiteralValue ?isniVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?ieee_blank.
?ieee_blank datacite:usesIdentifierScheme datacite:ieee;
litre:hasLiteralValue ?ieeeVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?gepris_blank.
?gepris_blank datacite:usesIdentifierScheme datacite:gepris;
litre:hasLiteralValue ?geprisVar.}
OPTIONAL{
?editor datacite:hasIdentifier ?gnd_blank.
?gnd_blank datacite:usesIdentifierScheme datacite:gnd;
litre:hasLiteralValue ?gndVar.}
}
GROUP BY ?editor

</source>
=== Comparing Extracted and dblp Editors ===
* editor by volume comparison
** 2233 volume the extracted editors match the dblp editors
** 807 volumes are missing in dblp (editors extracted)
** 27 volumes more editors were extracted than in dblp
** 387 volumes dblp has more editors than we could extract

== Wikidata Reconciliation ==
Using the ids queried from dblp to find the corresponding wikidata entry.

Current strategy:
;Input: List of different identifiers that are known about a editor
;Output: SPARQL query

Example:
*Input:
** homepage: http://www.stefandecker.org
** gnd id: 173443443
** dblp author id: d/StefanDecker
* Output:
<source lang="sparql">
PREFIX wdt: <http://www.wikidata.org/prop/direct/>
PREFIX wikibase: <http://wikiba.se/ontology#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?person ?personLabel
WHERE
{
{OPTIONAL{ ?person wdt:P856 <http://www.stefandecker.org>.} }
UNION
{OPTIONAL{ ?person wdt:P227 "173443443".} } # gnd
UNION
{OPTIONAL{ ?person wdt:P2456 "d/StefanDecker".} } # dblp
?person rdfs:label ?personLabel. FILTER(lang(?personLabel)="en")
}
</source>

Running these queries for all 4942 editors known by dblp we get:
* Queries currently running...[[User:Tim Holzheim|Tim Holzheim]] ([[User talk:Tim Holzheim|talk]]) 09:49, 9 March 2023 (CET)

File:Volume editor distribution.png

2023-03-09T08:16:24Z

Tim Holzheim: File uploaded with MsUpload

File uploaded with MsUpload

Wikidata Synchronization for the CEUR-WS publishing platform Use-Case of

2022-10-17T12:24:49Z

Tim Holzheim: Created page with "CEUR Workshop Proceedings is a free open-access publication service for academic workshops. Data about the proceedings, events, papers, editors and authors are stored in a se..."

CEUR Workshop Proceedings is a free open-access publication service for academic workshops.
Data about the proceedings, events, papers, editors and authors are stored in a semi-structured html format.
In a effort to semantify the data and move it into a semantic mediawiki the goal was also to integrate the data into wikidata.
Since wikidata is one of the largest knowledge graphs with a large portion covering scholarly articles, adding the ceur-ws data to it increases accessibility and the cross referencing of the contained enties.
For the historic records the data needs to be extracted to create the wiki pages and wikidata items.
Additionally for new proceedings and for the curration of existing once data from the smw wiki needs to be published to wikidata.
The CEUR-WS browser in our pyCEURmake project aims to introduce such a bidirectional api for edits on wikidata and a smw wiki.

Workdocumentation 2022-09-06

2022-09-06T07:40:19Z

Tim Holzheim:

{{PageSequence|prev=Workdocumentation 2022-08-29|next=Workdocumentation 2022-09-07|category=Workdocumentation}}
== Definition of Done ==

=== ToDo ===
* Extraction of Location
* Extraction of start time & end time
* Extraction of Papers
** Name of the paper
** authors (with dblp id)
* Extraction of Editors
** name of the editors
** affiliation of the editors

* connecting the events to its series

Workdocumentation 2022-08-29

2022-09-06T07:39:32Z

Tim Holzheim:

{{PageSequence|prev=Workdocumentation 2022-08-23|next=Workdocumentation 2022-09-06|category=Workdocumentation}}
= Participants =
* Wolfgang
* Beyza
= CEUR-WS Wikidata Sync =

CEUR-WS Platform siehe http://ceur-ws.org/

Etwa 3200 Proceedings Volumes

Synchronisation mit Wikidata, k10plus und dblp

Dokumentiert in https://ceur-ws.bitplan.com/index.php/Workdocumentation_2022-08-12 und folgende - siehe auch RQ Wiki

Nutzung dblp SPARQL Abfrage via https://qlever.cs.uni-freiburg.de/dblp bzw. lokale Endpoints

CEUR-WS Browser Frontend: http://ceur-ws-browser.bitplan.com/

Beispiel: http://ceur-ws-browser.bitplan.com/volume/3111

Bewusst noch keine Detaildaten wie Ort/Datum - Link-First Ansatz ...

Beispiel für komplette Serie: https://scholia.toolforge.org/event-series/Q56846035 - erzeugt von Lars Willighagen mit https://citation.js.org/

Ziel: Paper/Author Links see also https://github.com/zotero/translators/issues/2015

Wie soll die ConfIDent Anbindung erfolgen?

= Justpy =
<source lang='bash'>
scripts/findexamples
wf@fix justpy % scripts/findexamples | grep plot_test
jp.justpy(plot_test1)
jp.justpy(plot_test2)
jp.justpy(plot_test3)
jp.justpy(plot_test4)
jp.justpy(plot_test5)
jp.justpy(plot_test6)
jp.justpy(plot_test7)
jp.justpy(plot_test8)
jp.justpy(plot_test9)
</source>
* https://github.com/justpy-org/justpy/issues/468
* https://github.com/justpy-org/justpy/issues/464

Workdocumentation 2022-09-06

2022-09-06T07:39:30Z

Tim Holzheim: Created page with "{{PageSequence|prev=Workdocumentation 2022-08-29|next=Workdocumentation 2022-09-07|category=Workdocumentation}}"

{{PageSequence|prev=Workdocumentation 2022-08-29|next=Workdocumentation 2022-09-07|category=Workdocumentation}}

Workdocumentation 2022-08-16

2022-08-17T11:09:15Z

Tim Holzheim: /* Further Queries */

{{PageSequence|prev=Workdocumentation 2022-08-15|next=Workdocumentation 2022-08-17|category=Workdocumentation}}
= Participants =
* Tim
* Wolfgang
= Agenda =
dblp Qlever

= dblp Qlever =
* https://github.com/WolfgangFahl/pyCEURmake/issues/18

== on RWTH Aachen DBIS i5 server ==
<source lang='bash' highlight='1,12-13'>
wf@confident:/hd/torterra/dblp2022$ wget https://dblp.org/rdf/dblp.nt.gz
--2022-08-16 12:00:18-- https://dblp.org/rdf/dblp.nt.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2789108619 (2.6G) [application/x-gzip]
Saving to: ‘dblp.nt.gz’

dblp.nt.gz 36%[======> ] 980.73M 38.9MB/s eta 45s
dblp.nt.gz 100%[===================>] 2.60G 34.1MB/s in 76s
2022-08-16 12:01:34 (35.1 MB/s) - ‘dblp.nt.gz’ saved [2789108619/2789108619]
gunzip dblp.nt.gz
ls -l
total 34405612
-rw-rw-r-- 1 wf wf 35231339037 Aug 16 00:16 dblp.nt
</source>
<source lang='bash' highlight='1'>
wget https://dblp.org/rdf/dblp.ttl.gz
--2022-08-16 15:36:37-- https://dblp.org/rdf/dblp.ttl.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1065586620 (1016M) [application/x-gzip]
Saving to: ‘dblp.ttl.gz’
dblp.ttl.gz 47%[===========> ] 483.91M 43.1MB/s eta 16s
dblp.ttl.gz 100%[=========================>] 1016M 32.7MB/s in 29s
2022-08-16 15:37:06 (35.3 MB/s) - ‘dblp.ttl.gz’ saved [1065586620/1065586620]
gunzip dblp.ttl.gz
</source>

== QLever installation ==
https://wiki.bitplan.com/index.php/WikiData_Import_2022-01-29
=== Environment ===
<source lang='bash' highlight='1,7,9'>
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal
wf@confident:/hd/torterra/qlever$ docker --version
Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1
wf@confident:/hd/torterra/qlever$ free -h
total used free shared buff/cache available
Mem: 15Gi 1.2Gi 12Gi 45Mi 1.6Gi 13Gi
Swap: 11Gi 0B 11Gi
</source>
===== Disk space =====
<source lang='bash'>
df | grep -v loop | grep -v tmp | grep -v udev
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 114226348 55228964 53148860 51% /
/dev/sdb1 3844590624 3266511864 382711544 90% /hd/torterra
</source>

=== QLever code clone ===
<source lang='bash' highlight='1-2'>
export QLEVER_HOME=$(pwd)
date;git clone --recursive https://github.com/ad-freiburg/qlever qlever-code;date
Tue Aug 16 15:10:09 CEST 2022
Cloning into 'qlever-code'...
remote: Enumerating objects: 14625, done.
remote: Counting objects: 100% (279/279), done.
remote: Compressing objects: 100% (227/227), done.
remote: Total 14625 (delta 133), reused 131 (delta 52), pack-reused 14346
Receiving objects: 100% (14625/14625), 190.60 MiB | 6.48 MiB/s, done.
...
Submodule path 'third_party/stxxl/extlib/foxxll/extlib/tlx': checked out 'ef81a598d9880cc7d242afc47de7328634f07f1d'
Tue Aug 16 15:10:56 CEST 2022
</source>
==== Build ====
<source lang='bash' highlight='1'>
date;sudo docker build --file Dockerfiles/Dockerfile.Ubuntu20.04 -t qlever .;date
Tue Aug 16 15:13:02 CEST 2022
Sending build context to Docker daemon 453MB
Step 1/43 : FROM ubuntu:20.04 as base
20.04: Pulling from library/ubuntu
3b65ec22a9e9: Pull complete

Removing intermediate container 1ccc2a50364e
---> d0018440a4cd
Successfully built d0018440a4cd
Successfully tagged qlever:latest
Tue Aug 16 15:25:28 CEST 2022
</source>

=== qlever control ===
<source lang='bash'>
git clone https://github.com/ad-freiburg/qlever-controlCloning into 'qlever-control'...
remote: Enumerating objects: 368, done.
remote: Counting objects: 100% (208/208), done.
remote: Compressing objects: 100% (135/135), done.
remote: Total 368 (delta 75), reused 183 (delta 72), pack-reused 160
Receiving objects: 100% (368/368), 117.76 KiB | 7.36 MiB/s, done.
Resolving deltas: 100% (130/130), done.
</source>
== dblp stardog ==
<source lang='bash' highlight='1'>
docker pull stardog/stardog:latest
latest: Pulling from stardog/stardog
2d473b07cdd5: Pull complete
b0eac9aee9aa: Pull complete
8d5b89da19bc: Pull complete
91c2bc930138: Pull complete
265d7b96dd8f: Pull complete
Digest: sha256:7fc70e1bd3d17bdb1440f0cd810294b5318f1c53935425bb51526da4a949afc0
Status: Downloaded newer image for stardog/stardog:latest
docker.io/stardog/stardog:latest
</source>

= DBLP versus CEUR-WS Queries =
== All Volumes known to dblp ==
* expected 70% of 3185 volumes found 75%
* actual 75%
<source lang="sparql">
SELECT (COUNT(?proceeding) as ?count) (MIN(xsd:integer(?volNumber)) as ?min) (MAX(xsd:integer(?volNumber)) as ?max)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volNumber .
}
LIMIT 5000
</source>
{| class="wikitable"
| count || min || max
|-
| 2375 || 1 || 3157
|}

== All papers ==
* expected 70% of ~50000 papers
<source lang="sparql">
SELECT (COUNT(?paper) as ?count)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
?paper dblp:publishedAsPartOf ?proceeding.
}
</source>
{| class="wikitable"
| count
|-
| 44275
|}

== All authors and editors ==
* authors: papers expected: <1:1 and >1:3 found 1.6 distinct authors in relation to distinct papers
* editors: volumes 3:1 found 4625 editors for 2377 volumes
<source lang="sparql">
SELECT (COUNT(DISTINCT ?author) as ?numberOfAuthors)
(COUNT(DISTINCT ?paper) as ?numberOfPapers)
(COUNT(DISTINCT ?editor) as ?numberOfEditors)
(COUNT(DISTINCT ?proceeding) as ?numberOfVolumes)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
OPTIONAL{?proceeding dblp:editedBy ?editor}
OPTIONAL{
?paper dblp:publishedAsPartOf ?proceeding.
OPTIONAL{?paper dblp:authoredBy ?author}
}

}

</source>
{| class="wikitable"
| numberOfAuthors || numberOfPapers || numberOfEditors || numberOfVolumes
|-
| 69846 || 44275 || 4625 || 2377
|}
Note: There are proceedings of ceurws without an volumeId

Namely:
* https://dblp.org/rec/conf/www/2017ldow
* https://dblp.org/rec/conf/semweb/2017hybridsemstats

== Cross-check against wikidata ==
* volumes in dblp and wikidata
<source lang="sparql">
SELECT DISTINCT ?proceeding ?wdProceedings ?urn
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
datacite:hasIdentifier [
datacite:usesIdentifierScheme datacite:urn ;
litre:hasLiteralValue ?urn ;
a datacite:ResourceIdentifier
] .
service <https://query.wikidata.org/sparql> {
?wdProceedings wdt:P179 wd:Q27230297;
wdt:P4109 ?urn
}
}
</source>

* volumes in wikidata missing in dblp
<source lang="sparql">
SELECT DISTINCT ?wdProceedings ?urn
WHERE {
service <https://query.wikidata.org/sparql> {
?wdProceedings wdt:P179 wd:Q27230297;
wdt:P4109 ?urn
}
MINUS{
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
datacite:hasIdentifier [
datacite:usesIdentifierScheme datacite:urn ;
litre:hasLiteralValue ?urn ;
a datacite:ResourceIdentifier
] .
}
}
</source>
* volumes in dblp missing in wikidata
<source lang="sparql">
SELECT DISTINCT ?proceedings ?urn
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
datacite:hasIdentifier [
datacite:usesIdentifierScheme datacite:urn ;
litre:hasLiteralValue ?urn ;
a datacite:ResourceIdentifier
] .

MINUS{
service <https://query.wikidata.org/sparql> {
?wdProceedings wdt:P179 wd:Q27230297;
wdt:P4109 ?urn.
}
}
}
</source>
== Further Queries ==
* Cross-check against wikidata
** 53000 dblp authors out of 2.3m
** all editors MUST be in dblp acording to the rules of ceur-ws
** authors can be in dblp
* Disambiguation problem only on ceur-ws side
** idea: calculate distance of potential candidate authors to authors in the same volume

Workdocumentation 2022-08-16

2022-08-17T10:31:54Z

Tim Holzheim: /* All authors and editors = */

{{PageSequence|prev=Workdocumentation 2022-08-15|next=Workdocumentation 2022-08-17|category=Workdocumentation}}
= Participants =
* Tim
* Wolfgang
= Agenda =
dblp Qlever

= dblp Qlever =
* https://github.com/WolfgangFahl/pyCEURmake/issues/18

== on RWTH Aachen DBIS i5 server ==
<source lang='bash' highlight='1,12-13'>
wf@confident:/hd/torterra/dblp2022$ wget https://dblp.org/rdf/dblp.nt.gz
--2022-08-16 12:00:18-- https://dblp.org/rdf/dblp.nt.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2789108619 (2.6G) [application/x-gzip]
Saving to: ‘dblp.nt.gz’

dblp.nt.gz 36%[======> ] 980.73M 38.9MB/s eta 45s
dblp.nt.gz 100%[===================>] 2.60G 34.1MB/s in 76s
2022-08-16 12:01:34 (35.1 MB/s) - ‘dblp.nt.gz’ saved [2789108619/2789108619]
gunzip dblp.nt.gz
ls -l
total 34405612
-rw-rw-r-- 1 wf wf 35231339037 Aug 16 00:16 dblp.nt
</source>
<source lang='bash' highlight='1'>
wget https://dblp.org/rdf/dblp.ttl.gz
--2022-08-16 15:36:37-- https://dblp.org/rdf/dblp.ttl.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1065586620 (1016M) [application/x-gzip]
Saving to: ‘dblp.ttl.gz’
dblp.ttl.gz 47%[===========> ] 483.91M 43.1MB/s eta 16s
dblp.ttl.gz 100%[=========================>] 1016M 32.7MB/s in 29s
2022-08-16 15:37:06 (35.3 MB/s) - ‘dblp.ttl.gz’ saved [1065586620/1065586620]
gunzip dblp.ttl.gz
</source>

== QLever installation ==
https://wiki.bitplan.com/index.php/WikiData_Import_2022-01-29
=== Environment ===
<source lang='bash' highlight='1,7,9'>
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal
wf@confident:/hd/torterra/qlever$ docker --version
Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1
wf@confident:/hd/torterra/qlever$ free -h
total used free shared buff/cache available
Mem: 15Gi 1.2Gi 12Gi 45Mi 1.6Gi 13Gi
Swap: 11Gi 0B 11Gi
</source>
===== Disk space =====
<source lang='bash'>
df | grep -v loop | grep -v tmp | grep -v udev
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 114226348 55228964 53148860 51% /
/dev/sdb1 3844590624 3266511864 382711544 90% /hd/torterra
</source>

=== QLever code clone ===
<source lang='bash' highlight='1-2'>
export QLEVER_HOME=$(pwd)
date;git clone --recursive https://github.com/ad-freiburg/qlever qlever-code;date
Tue Aug 16 15:10:09 CEST 2022
Cloning into 'qlever-code'...
remote: Enumerating objects: 14625, done.
remote: Counting objects: 100% (279/279), done.
remote: Compressing objects: 100% (227/227), done.
remote: Total 14625 (delta 133), reused 131 (delta 52), pack-reused 14346
Receiving objects: 100% (14625/14625), 190.60 MiB | 6.48 MiB/s, done.
...
Submodule path 'third_party/stxxl/extlib/foxxll/extlib/tlx': checked out 'ef81a598d9880cc7d242afc47de7328634f07f1d'
Tue Aug 16 15:10:56 CEST 2022
</source>
==== Build ====
<source lang='bash' highlight='1'>
date;sudo docker build --file Dockerfiles/Dockerfile.Ubuntu20.04 -t qlever .;date
Tue Aug 16 15:13:02 CEST 2022
Sending build context to Docker daemon 453MB
Step 1/43 : FROM ubuntu:20.04 as base
20.04: Pulling from library/ubuntu
3b65ec22a9e9: Pull complete

Removing intermediate container 1ccc2a50364e
---> d0018440a4cd
Successfully built d0018440a4cd
Successfully tagged qlever:latest
Tue Aug 16 15:25:28 CEST 2022
</source>

=== qlever control ===
<source lang='bash'>
git clone https://github.com/ad-freiburg/qlever-controlCloning into 'qlever-control'...
remote: Enumerating objects: 368, done.
remote: Counting objects: 100% (208/208), done.
remote: Compressing objects: 100% (135/135), done.
remote: Total 368 (delta 75), reused 183 (delta 72), pack-reused 160
Receiving objects: 100% (368/368), 117.76 KiB | 7.36 MiB/s, done.
Resolving deltas: 100% (130/130), done.
</source>
== dblp stardog ==
<source lang='bash' highlight='1'>
docker pull stardog/stardog:latest
latest: Pulling from stardog/stardog
2d473b07cdd5: Pull complete
b0eac9aee9aa: Pull complete
8d5b89da19bc: Pull complete
91c2bc930138: Pull complete
265d7b96dd8f: Pull complete
Digest: sha256:7fc70e1bd3d17bdb1440f0cd810294b5318f1c53935425bb51526da4a949afc0
Status: Downloaded newer image for stardog/stardog:latest
docker.io/stardog/stardog:latest
</source>

= DBLP versus CEUR-WS Queries =
== All Volumes known to dblp ==
* expected 70% of 3185 volumes found 75%
* actual 75%
<source lang="sparql">
SELECT (COUNT(?proceeding) as ?count) (MIN(xsd:integer(?volNumber)) as ?min) (MAX(xsd:integer(?volNumber)) as ?max)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volNumber .
}
LIMIT 5000
</source>
{| class="wikitable"
| count || min || max
|-
| 2375 || 1 || 3157
|}

== All papers ==
* expected 70% of ~50000 papers
<source lang="sparql">
SELECT (COUNT(?paper) as ?count)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
?paper dblp:publishedAsPartOf ?proceeding.
}
</source>
{| class="wikitable"
| count
|-
| 44275
|}

== All authors and editors ==
* authors: papers expected: <1:1 and >1:3 found 1.6 distinct authors in relation to distinct papers
* editors: volumes 3:1 found 4625 editors for 2377 volumes
<source lang="sparql">
SELECT (COUNT(DISTINCT ?author) as ?numberOfAuthors)
(COUNT(DISTINCT ?paper) as ?numberOfPapers)
(COUNT(DISTINCT ?editor) as ?numberOfEditors)
(COUNT(DISTINCT ?proceeding) as ?numberOfVolumes)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
OPTIONAL{?proceeding dblp:editedBy ?editor}
OPTIONAL{
?paper dblp:publishedAsPartOf ?proceeding.
OPTIONAL{?paper dblp:authoredBy ?author}
}

}

</source>
{| class="wikitable"
| numberOfAuthors || numberOfPapers || numberOfEditors || numberOfVolumes
|-
| 69846 || 44275 || 4625 || 2377
|}
Note: There are proceedings of ceurws without an volumeId

Namely:
* https://dblp.org/rec/conf/www/2017ldow
* https://dblp.org/rec/conf/semweb/2017hybridsemstats

== Further Queries ==
* Cross-check against wikidata
** 53000 dblp authors out of 2.3m
** all editors MUST be in dblp acording to the rules of ceur-ws
** authors can be in dblp
* Disambiguation problem only on ceur-ws side
** idea: calculate distance of potential candidate authors to authors in the same volume

Workdocumentation 2022-08-16

2022-08-17T10:28:59Z

Tim Holzheim: /* DBLP versus CEUR-WS Queries */

{{PageSequence|prev=Workdocumentation 2022-08-15|next=Workdocumentation 2022-08-17|category=Workdocumentation}}
= Participants =
* Tim
* Wolfgang
= Agenda =
dblp Qlever

= dblp Qlever =
* https://github.com/WolfgangFahl/pyCEURmake/issues/18

== on RWTH Aachen DBIS i5 server ==
<source lang='bash' highlight='1,12-13'>
wf@confident:/hd/torterra/dblp2022$ wget https://dblp.org/rdf/dblp.nt.gz
--2022-08-16 12:00:18-- https://dblp.org/rdf/dblp.nt.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2789108619 (2.6G) [application/x-gzip]
Saving to: ‘dblp.nt.gz’

dblp.nt.gz 36%[======> ] 980.73M 38.9MB/s eta 45s
dblp.nt.gz 100%[===================>] 2.60G 34.1MB/s in 76s
2022-08-16 12:01:34 (35.1 MB/s) - ‘dblp.nt.gz’ saved [2789108619/2789108619]
gunzip dblp.nt.gz
ls -l
total 34405612
-rw-rw-r-- 1 wf wf 35231339037 Aug 16 00:16 dblp.nt
</source>
<source lang='bash' highlight='1'>
wget https://dblp.org/rdf/dblp.ttl.gz
--2022-08-16 15:36:37-- https://dblp.org/rdf/dblp.ttl.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1065586620 (1016M) [application/x-gzip]
Saving to: ‘dblp.ttl.gz’
dblp.ttl.gz 47%[===========> ] 483.91M 43.1MB/s eta 16s
dblp.ttl.gz 100%[=========================>] 1016M 32.7MB/s in 29s
2022-08-16 15:37:06 (35.3 MB/s) - ‘dblp.ttl.gz’ saved [1065586620/1065586620]
gunzip dblp.ttl.gz
</source>

== QLever installation ==
https://wiki.bitplan.com/index.php/WikiData_Import_2022-01-29
=== Environment ===
<source lang='bash' highlight='1,7,9'>
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal
wf@confident:/hd/torterra/qlever$ docker --version
Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1
wf@confident:/hd/torterra/qlever$ free -h
total used free shared buff/cache available
Mem: 15Gi 1.2Gi 12Gi 45Mi 1.6Gi 13Gi
Swap: 11Gi 0B 11Gi
</source>
===== Disk space =====
<source lang='bash'>
df | grep -v loop | grep -v tmp | grep -v udev
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 114226348 55228964 53148860 51% /
/dev/sdb1 3844590624 3266511864 382711544 90% /hd/torterra
</source>

=== QLever code clone ===
<source lang='bash' highlight='1-2'>
export QLEVER_HOME=$(pwd)
date;git clone --recursive https://github.com/ad-freiburg/qlever qlever-code;date
Tue Aug 16 15:10:09 CEST 2022
Cloning into 'qlever-code'...
remote: Enumerating objects: 14625, done.
remote: Counting objects: 100% (279/279), done.
remote: Compressing objects: 100% (227/227), done.
remote: Total 14625 (delta 133), reused 131 (delta 52), pack-reused 14346
Receiving objects: 100% (14625/14625), 190.60 MiB | 6.48 MiB/s, done.
...
Submodule path 'third_party/stxxl/extlib/foxxll/extlib/tlx': checked out 'ef81a598d9880cc7d242afc47de7328634f07f1d'
Tue Aug 16 15:10:56 CEST 2022
</source>
==== Build ====
<source lang='bash' highlight='1'>
date;sudo docker build --file Dockerfiles/Dockerfile.Ubuntu20.04 -t qlever .;date
Tue Aug 16 15:13:02 CEST 2022
Sending build context to Docker daemon 453MB
Step 1/43 : FROM ubuntu:20.04 as base
20.04: Pulling from library/ubuntu
3b65ec22a9e9: Pull complete

Removing intermediate container 1ccc2a50364e
---> d0018440a4cd
Successfully built d0018440a4cd
Successfully tagged qlever:latest
Tue Aug 16 15:25:28 CEST 2022
</source>

=== qlever control ===
<source lang='bash'>
git clone https://github.com/ad-freiburg/qlever-controlCloning into 'qlever-control'...
remote: Enumerating objects: 368, done.
remote: Counting objects: 100% (208/208), done.
remote: Compressing objects: 100% (135/135), done.
remote: Total 368 (delta 75), reused 183 (delta 72), pack-reused 160
Receiving objects: 100% (368/368), 117.76 KiB | 7.36 MiB/s, done.
Resolving deltas: 100% (130/130), done.
</source>
== dblp stardog ==
<source lang='bash' highlight='1'>
docker pull stardog/stardog:latest
latest: Pulling from stardog/stardog
2d473b07cdd5: Pull complete
b0eac9aee9aa: Pull complete
8d5b89da19bc: Pull complete
91c2bc930138: Pull complete
265d7b96dd8f: Pull complete
Digest: sha256:7fc70e1bd3d17bdb1440f0cd810294b5318f1c53935425bb51526da4a949afc0
Status: Downloaded newer image for stardog/stardog:latest
docker.io/stardog/stardog:latest
</source>

= DBLP versus CEUR-WS Queries =
== All Volumes known to dblp ==
* expected 70% of 3185 volumes found 75%
* actual 75%
<source lang="sparql">
SELECT (COUNT(?proceeding) as ?count) (MIN(xsd:integer(?volNumber)) as ?min) (MAX(xsd:integer(?volNumber)) as ?max)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volNumber .
}
LIMIT 5000
</source>
{| class="wikitable"
| count || min || max
|-
| 2375 || 1 || 3157
|}

== All papers ==
* expected 70% of ~50000 papers
<source lang="sparql">
SELECT (COUNT(?paper) as ?count)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
?paper dblp:publishedAsPartOf ?proceeding.
}
</source>
{| class="wikitable"
| count
|-
| 44275
|}

== All authors and editors ===
* authors: papers expected: <1:1 and >1:3 found 1.6 distinct authors in relation to distinct papers
* editors: volumes 3:1 found 4625 editors for 2377 volumes
<source lang="sparql">
SELECT (COUNT(DISTINCT ?author) as ?numberOfAuthors)
(COUNT(DISTINCT ?paper) as ?numberOfPapers)
(COUNT(DISTINCT ?editor) as ?numberOfEditors)
(COUNT(DISTINCT ?proceeding) as ?numberOfVolumes)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
OPTIONAL{?proceeding dblp:editedBy ?editor}
OPTIONAL{
?paper dblp:publishedAsPartOf ?proceeding.
OPTIONAL{?paper dblp:authoredBy ?author}
}

}

</source>
{| class="wikitable"
| numberOfAuthors || numberOfPapers || numberOfEditors || numberOfVolumes
|-
| 69846 || 44275 || 4625 || 2377
|}
Note: There are proceedings of ceurws without an volumeId
Namely:
* https://dblp.org/rec/conf/www/2017ldow
* https://dblp.org/rec/conf/semweb/2017hybridsemstats

== Further Queries ==
* Cross-check against wikidata
** 53000 dblp authors out of 2.3m
** all editors MUST be in dblp acording to the rules of ceur-ws
** authors can be in dblp
* Disambiguation problem only on ceur-ws side
** idea: calculate distance of potential candidate authors to authors in the same volume

Workdocumentation 2022-08-16

2022-08-17T10:26:35Z

Tim Holzheim: /* All authors and editors = */

{{PageSequence|prev=Workdocumentation 2022-08-15|next=Workdocumentation 2022-08-17|category=Workdocumentation}}
= Participants =
* Tim
* Wolfgang
= Agenda =
dblp Qlever

= dblp Qlever =
* https://github.com/WolfgangFahl/pyCEURmake/issues/18

== on RWTH Aachen DBIS i5 server ==
<source lang='bash' highlight='1,12-13'>
wf@confident:/hd/torterra/dblp2022$ wget https://dblp.org/rdf/dblp.nt.gz
--2022-08-16 12:00:18-- https://dblp.org/rdf/dblp.nt.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2789108619 (2.6G) [application/x-gzip]
Saving to: ‘dblp.nt.gz’

dblp.nt.gz 36%[======> ] 980.73M 38.9MB/s eta 45s
dblp.nt.gz 100%[===================>] 2.60G 34.1MB/s in 76s
2022-08-16 12:01:34 (35.1 MB/s) - ‘dblp.nt.gz’ saved [2789108619/2789108619]
gunzip dblp.nt.gz
ls -l
total 34405612
-rw-rw-r-- 1 wf wf 35231339037 Aug 16 00:16 dblp.nt
</source>
<source lang='bash' highlight='1'>
wget https://dblp.org/rdf/dblp.ttl.gz
--2022-08-16 15:36:37-- https://dblp.org/rdf/dblp.ttl.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1065586620 (1016M) [application/x-gzip]
Saving to: ‘dblp.ttl.gz’
dblp.ttl.gz 47%[===========> ] 483.91M 43.1MB/s eta 16s
dblp.ttl.gz 100%[=========================>] 1016M 32.7MB/s in 29s
2022-08-16 15:37:06 (35.3 MB/s) - ‘dblp.ttl.gz’ saved [1065586620/1065586620]
gunzip dblp.ttl.gz
</source>

== QLever installation ==
https://wiki.bitplan.com/index.php/WikiData_Import_2022-01-29
=== Environment ===
<source lang='bash' highlight='1,7,9'>
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal
wf@confident:/hd/torterra/qlever$ docker --version
Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1
wf@confident:/hd/torterra/qlever$ free -h
total used free shared buff/cache available
Mem: 15Gi 1.2Gi 12Gi 45Mi 1.6Gi 13Gi
Swap: 11Gi 0B 11Gi
</source>
===== Disk space =====
<source lang='bash'>
df | grep -v loop | grep -v tmp | grep -v udev
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 114226348 55228964 53148860 51% /
/dev/sdb1 3844590624 3266511864 382711544 90% /hd/torterra
</source>

=== QLever code clone ===
<source lang='bash' highlight='1-2'>
export QLEVER_HOME=$(pwd)
date;git clone --recursive https://github.com/ad-freiburg/qlever qlever-code;date
Tue Aug 16 15:10:09 CEST 2022
Cloning into 'qlever-code'...
remote: Enumerating objects: 14625, done.
remote: Counting objects: 100% (279/279), done.
remote: Compressing objects: 100% (227/227), done.
remote: Total 14625 (delta 133), reused 131 (delta 52), pack-reused 14346
Receiving objects: 100% (14625/14625), 190.60 MiB | 6.48 MiB/s, done.
...
Submodule path 'third_party/stxxl/extlib/foxxll/extlib/tlx': checked out 'ef81a598d9880cc7d242afc47de7328634f07f1d'
Tue Aug 16 15:10:56 CEST 2022
</source>
==== Build ====
<source lang='bash' highlight='1'>
date;sudo docker build --file Dockerfiles/Dockerfile.Ubuntu20.04 -t qlever .;date
Tue Aug 16 15:13:02 CEST 2022
Sending build context to Docker daemon 453MB
Step 1/43 : FROM ubuntu:20.04 as base
20.04: Pulling from library/ubuntu
3b65ec22a9e9: Pull complete

Removing intermediate container 1ccc2a50364e
---> d0018440a4cd
Successfully built d0018440a4cd
Successfully tagged qlever:latest
Tue Aug 16 15:25:28 CEST 2022
</source>

=== qlever control ===
<source lang='bash'>
git clone https://github.com/ad-freiburg/qlever-controlCloning into 'qlever-control'...
remote: Enumerating objects: 368, done.
remote: Counting objects: 100% (208/208), done.
remote: Compressing objects: 100% (135/135), done.
remote: Total 368 (delta 75), reused 183 (delta 72), pack-reused 160
Receiving objects: 100% (368/368), 117.76 KiB | 7.36 MiB/s, done.
Resolving deltas: 100% (130/130), done.
</source>
== dblp stardog ==
<source lang='bash' highlight='1'>
docker pull stardog/stardog:latest
latest: Pulling from stardog/stardog
2d473b07cdd5: Pull complete
b0eac9aee9aa: Pull complete
8d5b89da19bc: Pull complete
91c2bc930138: Pull complete
265d7b96dd8f: Pull complete
Digest: sha256:7fc70e1bd3d17bdb1440f0cd810294b5318f1c53935425bb51526da4a949afc0
Status: Downloaded newer image for stardog/stardog:latest
docker.io/stardog/stardog:latest
</source>

= DBLP versus CEUR-WS Queries =
== All Volumes known to dblp ==
** expected 70% of 3185 volumes found 75%
** actual 75%
<source lang="sparql">
SELECT (COUNT(?proceeding) as ?count) (MIN(xsd:integer(?volNumber)) as ?min) (MAX(xsd:integer(?volNumber)) as ?max)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volNumber .
}
LIMIT 5000
</source>
{| class="wikitable"
| count || min || max
|-
| 2375 || 1 || 3157
|}

== All papers ==
** expected 70% of ~50000 papers
<source lang="sparql">
SELECT (COUNT(?paper) as ?count)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
?paper dblp:publishedAsPartOf ?proceeding.
}
</source>
{| class="wikitable"
| count
|-
| 44275
|}

== All authors and editors ===
** authors: papers expected: <1:1 and >1:3 found 1.6 distinct authors in relation to distinct papers
** editors: volumes 3:1 found 4625 editors for 2377 volumes
<source lang="sparql">
SELECT (COUNT(DISTINCT ?author) as ?numberOfAuthors)
(COUNT(DISTINCT ?paper) as ?numberOfPapers)
(COUNT(DISTINCT ?editor) as ?numberOfEditors)
(COUNT(DISTINCT ?proceeding) as ?numberOfVolumes)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
OPTIONAL{?proceeding dblp:editedBy ?editor}
OPTIONAL{
?paper dblp:publishedAsPartOf ?proceeding.
OPTIONAL{?paper dblp:authoredBy ?author}
}

}

</source>
{| class="wikitable"
| numberOfAuthors || numberOfPapers || numberOfEditors || numberOfVolumes
|-
| 69846 || 44275 || 4625 || 2377
|}
Note: There are proceedings of ceurws without an volumeId
* Cross-check against wikidata
** 53000 dblp authors out of 2.3m
** all editors MUST be in dblp acording to the rules of ceur-ws
** authors can be in dblp
* Disambiguation problem only on ceur-ws side
** idea: calculate distance of potential candidate authors to authors in the same volume

Workdocumentation 2022-08-16

2022-08-17T10:18:56Z

Tim Holzheim: /* DBLP Queries */

{{PageSequence|prev=Workdocumentation 2022-08-15|next=Workdocumentation 2022-08-17|category=Workdocumentation}}
= Participants =
* Tim
* Wolfgang
= Agenda =
dblp Qlever

= dblp Qlever =
* https://github.com/WolfgangFahl/pyCEURmake/issues/18

== on RWTH Aachen DBIS i5 server ==
<source lang='bash' highlight='1,12-13'>
wf@confident:/hd/torterra/dblp2022$ wget https://dblp.org/rdf/dblp.nt.gz
--2022-08-16 12:00:18-- https://dblp.org/rdf/dblp.nt.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2789108619 (2.6G) [application/x-gzip]
Saving to: ‘dblp.nt.gz’

dblp.nt.gz 36%[======> ] 980.73M 38.9MB/s eta 45s
dblp.nt.gz 100%[===================>] 2.60G 34.1MB/s in 76s
2022-08-16 12:01:34 (35.1 MB/s) - ‘dblp.nt.gz’ saved [2789108619/2789108619]
gunzip dblp.nt.gz
ls -l
total 34405612
-rw-rw-r-- 1 wf wf 35231339037 Aug 16 00:16 dblp.nt
</source>
<source lang='bash' highlight='1'>
wget https://dblp.org/rdf/dblp.ttl.gz
--2022-08-16 15:36:37-- https://dblp.org/rdf/dblp.ttl.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1065586620 (1016M) [application/x-gzip]
Saving to: ‘dblp.ttl.gz’
dblp.ttl.gz 47%[===========> ] 483.91M 43.1MB/s eta 16s
dblp.ttl.gz 100%[=========================>] 1016M 32.7MB/s in 29s
2022-08-16 15:37:06 (35.3 MB/s) - ‘dblp.ttl.gz’ saved [1065586620/1065586620]
gunzip dblp.ttl.gz
</source>

== QLever installation ==
https://wiki.bitplan.com/index.php/WikiData_Import_2022-01-29
=== Environment ===
<source lang='bash' highlight='1,7,9'>
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal
wf@confident:/hd/torterra/qlever$ docker --version
Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1
wf@confident:/hd/torterra/qlever$ free -h
total used free shared buff/cache available
Mem: 15Gi 1.2Gi 12Gi 45Mi 1.6Gi 13Gi
Swap: 11Gi 0B 11Gi
</source>
===== Disk space =====
<source lang='bash'>
df | grep -v loop | grep -v tmp | grep -v udev
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 114226348 55228964 53148860 51% /
/dev/sdb1 3844590624 3266511864 382711544 90% /hd/torterra
</source>

=== QLever code clone ===
<source lang='bash' highlight='1-2'>
export QLEVER_HOME=$(pwd)
date;git clone --recursive https://github.com/ad-freiburg/qlever qlever-code;date
Tue Aug 16 15:10:09 CEST 2022
Cloning into 'qlever-code'...
remote: Enumerating objects: 14625, done.
remote: Counting objects: 100% (279/279), done.
remote: Compressing objects: 100% (227/227), done.
remote: Total 14625 (delta 133), reused 131 (delta 52), pack-reused 14346
Receiving objects: 100% (14625/14625), 190.60 MiB | 6.48 MiB/s, done.
...
Submodule path 'third_party/stxxl/extlib/foxxll/extlib/tlx': checked out 'ef81a598d9880cc7d242afc47de7328634f07f1d'
Tue Aug 16 15:10:56 CEST 2022
</source>
==== Build ====
<source lang='bash' highlight='1'>
date;sudo docker build --file Dockerfiles/Dockerfile.Ubuntu20.04 -t qlever .;date
Tue Aug 16 15:13:02 CEST 2022
Sending build context to Docker daemon 453MB
Step 1/43 : FROM ubuntu:20.04 as base
20.04: Pulling from library/ubuntu
3b65ec22a9e9: Pull complete

Removing intermediate container 1ccc2a50364e
---> d0018440a4cd
Successfully built d0018440a4cd
Successfully tagged qlever:latest
Tue Aug 16 15:25:28 CEST 2022
</source>

=== qlever control ===
<source lang='bash'>
git clone https://github.com/ad-freiburg/qlever-controlCloning into 'qlever-control'...
remote: Enumerating objects: 368, done.
remote: Counting objects: 100% (208/208), done.
remote: Compressing objects: 100% (135/135), done.
remote: Total 368 (delta 75), reused 183 (delta 72), pack-reused 160
Receiving objects: 100% (368/368), 117.76 KiB | 7.36 MiB/s, done.
Resolving deltas: 100% (130/130), done.
</source>
== dblp stardog ==
<source lang='bash' highlight='1'>
docker pull stardog/stardog:latest
latest: Pulling from stardog/stardog
2d473b07cdd5: Pull complete
b0eac9aee9aa: Pull complete
8d5b89da19bc: Pull complete
91c2bc930138: Pull complete
265d7b96dd8f: Pull complete
Digest: sha256:7fc70e1bd3d17bdb1440f0cd810294b5318f1c53935425bb51526da4a949afc0
Status: Downloaded newer image for stardog/stardog:latest
docker.io/stardog/stardog:latest
</source>

= DBLP Queries =
* All volumes known to dblp (from ceur-ws)
** expected 70% of 3185 volumes
** actual 75%
<source lang="sparql">
SELECT (COUNT(?proceeding) as ?count) (MIN(xsd:integer(?volNumber)) as ?min) (MAX(xsd:integer(?volNumber)) as ?max)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volNumber .
}
LIMIT 5000
</source>
{| class="wikitable"
| count || min || max
|-
| 2375 || 1 || 3157
|}

* All papers known to dblp (from ceur-ws)
** expected 70% of ~50000 papers
<source lang="sparql">
SELECT (COUNT(?paper) as ?count)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
?paper dblp:publishedAsPartOf ?proceeding.
}
</source>
{| class="wikitable"
| count
|-
| 44275
|}

* All authors and editors
** authors: papers <1:1 and >1:3
** editors: volumes 3:1
<source lang="sparql">
SELECT (COUNT(DISTINCT ?author) as ?numberOfAuthors)
(COUNT(DISTINCT ?paper) as ?numberOfPapers)
(COUNT(DISTINCT ?editor) as ?numberOfEditors)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
OPTIONAL{?proceeding dblp:editedBy ?editor}
OPTIONAL{
?paper dblp:publishedAsPartOf ?proceeding.
OPTIONAL{?paper dblp:authoredBy ?author}
}

}
</source>
{| class="wikitable"
| numberOfAuthors || numberOfPapers || numberOfEditors
|-
| 69846 || 44275 || 4625
|}

* Cross-check against wikidata
** 53000 dblp authors out of 2.3m
** all editors MUST be in dblp acording to the rules of ceur-ws
** authors can be in dblp
* Disambiguation problem only on ceur-ws side
** idea: calculate distance of potential candidate authors to authors in the same volume

Workdocumentation 2022-08-16

2022-08-17T10:16:14Z

Tim Holzheim: /* DBLP Queries */

{{PageSequence|prev=Workdocumentation 2022-08-15|next=Workdocumentation 2022-08-17|category=Workdocumentation}}
= Participants =
* Tim
* Wolfgang
= Agenda =
dblp Qlever

= dblp Qlever =
* https://github.com/WolfgangFahl/pyCEURmake/issues/18

== on RWTH Aachen DBIS i5 server ==
<source lang='bash' highlight='1,12-13'>
wf@confident:/hd/torterra/dblp2022$ wget https://dblp.org/rdf/dblp.nt.gz
--2022-08-16 12:00:18-- https://dblp.org/rdf/dblp.nt.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2789108619 (2.6G) [application/x-gzip]
Saving to: ‘dblp.nt.gz’

dblp.nt.gz 36%[======> ] 980.73M 38.9MB/s eta 45s
dblp.nt.gz 100%[===================>] 2.60G 34.1MB/s in 76s
2022-08-16 12:01:34 (35.1 MB/s) - ‘dblp.nt.gz’ saved [2789108619/2789108619]
gunzip dblp.nt.gz
ls -l
total 34405612
-rw-rw-r-- 1 wf wf 35231339037 Aug 16 00:16 dblp.nt
</source>
<source lang='bash' highlight='1'>
wget https://dblp.org/rdf/dblp.ttl.gz
--2022-08-16 15:36:37-- https://dblp.org/rdf/dblp.ttl.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1065586620 (1016M) [application/x-gzip]
Saving to: ‘dblp.ttl.gz’
dblp.ttl.gz 47%[===========> ] 483.91M 43.1MB/s eta 16s
dblp.ttl.gz 100%[=========================>] 1016M 32.7MB/s in 29s
2022-08-16 15:37:06 (35.3 MB/s) - ‘dblp.ttl.gz’ saved [1065586620/1065586620]
gunzip dblp.ttl.gz
</source>

== QLever installation ==
https://wiki.bitplan.com/index.php/WikiData_Import_2022-01-29
=== Environment ===
<source lang='bash' highlight='1,7,9'>
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal
wf@confident:/hd/torterra/qlever$ docker --version
Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1
wf@confident:/hd/torterra/qlever$ free -h
total used free shared buff/cache available
Mem: 15Gi 1.2Gi 12Gi 45Mi 1.6Gi 13Gi
Swap: 11Gi 0B 11Gi
</source>
===== Disk space =====
<source lang='bash'>
df | grep -v loop | grep -v tmp | grep -v udev
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 114226348 55228964 53148860 51% /
/dev/sdb1 3844590624 3266511864 382711544 90% /hd/torterra
</source>

=== QLever code clone ===
<source lang='bash' highlight='1-2'>
export QLEVER_HOME=$(pwd)
date;git clone --recursive https://github.com/ad-freiburg/qlever qlever-code;date
Tue Aug 16 15:10:09 CEST 2022
Cloning into 'qlever-code'...
remote: Enumerating objects: 14625, done.
remote: Counting objects: 100% (279/279), done.
remote: Compressing objects: 100% (227/227), done.
remote: Total 14625 (delta 133), reused 131 (delta 52), pack-reused 14346
Receiving objects: 100% (14625/14625), 190.60 MiB | 6.48 MiB/s, done.
...
Submodule path 'third_party/stxxl/extlib/foxxll/extlib/tlx': checked out 'ef81a598d9880cc7d242afc47de7328634f07f1d'
Tue Aug 16 15:10:56 CEST 2022
</source>
==== Build ====
<source lang='bash' highlight='1'>
date;sudo docker build --file Dockerfiles/Dockerfile.Ubuntu20.04 -t qlever .;date
Tue Aug 16 15:13:02 CEST 2022
Sending build context to Docker daemon 453MB
Step 1/43 : FROM ubuntu:20.04 as base
20.04: Pulling from library/ubuntu
3b65ec22a9e9: Pull complete

Removing intermediate container 1ccc2a50364e
---> d0018440a4cd
Successfully built d0018440a4cd
Successfully tagged qlever:latest
Tue Aug 16 15:25:28 CEST 2022
</source>

=== qlever control ===
<source lang='bash'>
git clone https://github.com/ad-freiburg/qlever-controlCloning into 'qlever-control'...
remote: Enumerating objects: 368, done.
remote: Counting objects: 100% (208/208), done.
remote: Compressing objects: 100% (135/135), done.
remote: Total 368 (delta 75), reused 183 (delta 72), pack-reused 160
Receiving objects: 100% (368/368), 117.76 KiB | 7.36 MiB/s, done.
Resolving deltas: 100% (130/130), done.
</source>
== dblp stardog ==
<source lang='bash' highlight='1'>
docker pull stardog/stardog:latest
latest: Pulling from stardog/stardog
2d473b07cdd5: Pull complete
b0eac9aee9aa: Pull complete
8d5b89da19bc: Pull complete
91c2bc930138: Pull complete
265d7b96dd8f: Pull complete
Digest: sha256:7fc70e1bd3d17bdb1440f0cd810294b5318f1c53935425bb51526da4a949afc0
Status: Downloaded newer image for stardog/stardog:latest
docker.io/stardog/stardog:latest
</source>

= DBLP Queries =
* All volumes known to dblp (from ceur-ws)
** expected 70% of 3185 volumes
<source lang="sparql">
SELECT (COUNT(?proceeding) as ?count) (MIN(xsd:integer(?volNumber)) as ?min) (MAX(xsd:integer(?volNumber)) as ?max)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings";
dblp:publishedInSeriesVolume ?volNumber .
}
LIMIT 5000
</source>
{| class="wikitable"
| count || min || max
|-
| 2375 || 1 || 3157
|}

* All papers known to dblp (from ceur-ws)
** expected 70% of ~50000 papers
<source lang="sparql">
SELECT (COUNT(?paper) as ?count)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
?paper dblp:publishedAsPartOf ?proceeding.
}
</source>
{| class="wikitable"
| count
|-
| 44275
|}

* All authors and editors
** authors: papers <1:1 and >1:3
** editors: volumes 3:1
<source lang="sparql">
SELECT (COUNT(DISTINCT ?author) as ?numberOfAuthors)
(COUNT(DISTINCT ?paper) as ?numberOfPapers)
(COUNT(DISTINCT ?editor) as ?numberOfEditors)
WHERE {
?proceeding dblp:publishedIn "CEUR Workshop Proceedings".
OPTIONAL{?proceeding dblp:editedBy ?editor}
OPTIONAL{
?paper dblp:publishedAsPartOf ?proceeding.
OPTIONAL{?paper dblp:authoredBy ?author}
}

}
</source>
{| class="wikitable"
| numberOfAuthors || numberOfPapers || numberOfEditors
|-
| 69846 || 44275 || 4625
|}

* Cross-check against wikidata
** 53000 dblp authors out of 2.3m
** all editors MUST be in dblp acording to the rules of ceur-ws
** authors can be in dblp
* Disambiguation problem only on ceur-ws side
** idea: calculate distance of potential candidate authors to authors in the same volume

Workdocumentation 2022-08-16

2022-08-16T14:40:10Z

Tim Holzheim:

Workdocumentation 2022-08-16

2022-08-16T13:22:50Z

Tim Holzheim:

Workdocumentation 2022-08-16

2022-08-16T13:22:23Z

Tim Holzheim:

Workdocumentation 2022-08-16

2022-08-16T13:22:00Z

Tim Holzheim: /* = QLever code clone */

Workdocumentation 2022-08-16

2022-08-16T13:21:29Z

Tim Holzheim: /* Environment */