Workdocumentation 2022-08-23

edit

Agenda

new server

New Server

• Mount two hard disks
• create "wikidata" group
• Install docker according to https://docs.docker.com/engine/install/ubuntu/

qlever dblp

clone qlever-control

wf@wikidata:/hd/mantax/qlever$ git clone https://github.com/ad-freiburg/qlever-control
Cloning into 'qlever-control'...
remote: Enumerating objects: 399, done.
remote: Counting objects: 100% (239/239), done.
remote: Compressing objects: 100% (150/150), done.
remote: Total 399 (delta 94), reused 211 (delta 88), pack-reused 160
Receiving objects: 100% (399/399), 125.08 KiB | 7.36 MiB/s, done.
Resolving deltas: 100% (149/149), done.

qlever dblp

wf@wikidata:/hd/mantax/qlever$ mkdir dblp
wf@wikidata:/hd/mantax/qlever$ cd dblp
wf@wikidata:/hd/mantax/qlever/dblp$ . ../qlever-control/qlever dblp

QLEVER CONFIG

Checking your PATH ...
Added the directory "/hd/mantax/qlever/qlever-control" to your PATH

Setting up bash autocompletion ...
Done, number of completions: 35

Creating new Qleverfile ...
No pre-configuration name specified (as argument of ". qlever"). Copied default
Qleverfile to current directory, please edit and check.

Setup is complete
Type qlever and use autocompletion to see which actions are available. Add a
"show" in the end to see what an action does without executing it (for example,
qlever index show). Edit your local Qleverfile to change settings. A typical
sequence of actions if you have used a preconfigured Qleverfile is:

qlever get-data
qlever index
qlever start
qlever example-query

qlever dblp get-data

qlever get-data

This is the "qlever" script, call without argument for help

Executing "get-data":

wget -nc -O dblp.ttl.gz https://dblp.org/rdf/dblp.ttl.gz

Getting data using GET_DATA_CMD from Qleverfile ...

--2022-08-24 08:42:22--  https://dblp.org/rdf/dblp.ttl.gz
Resolving dblp.org (dblp.org)... 192.76.146.204
Connecting to dblp.org (dblp.org)|192.76.146.204|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1068155173 (1019M) [application/x-gzip]
Saving to: ‘dblp.ttl.gz’

dblp.ttl.gz         100%[===================>]   1019M  39.2MB/s    in 30s     

2022-08-24 08:42:52 (33.8 MB/s) - ‘dblp.ttl.gz’ saved [1068155173/1068155173]

qlever dblp index

qlever index

This is the "qlever" script, call without argument for help

Executing "index":

docker run -it --rm -u 10000:10000 -v /hd/mantax/qlever/dblp:/index -w /index --entrypoint bash --name qlever.dblp.index-build adfreiburg/qlever -c "zcat dblp.ttl.gz | IndexBuilderMain -F ttl -f - -i dblp -s dblp.settings.json --text-words-from-literals | tee dblp.index-log.txt"

Unable to find image 'adfreiburg/qlever:latest' locally
latest: Pulling from adfreiburg/qlever
125a6e411906: Pull complete 
7c46a5754a97: Pull complete 
ac188e4fc015: Pull complete 
e574534926c8: Pull complete 
e013d756805d: Pull complete 
e839a7c7b682: Pull complete 
072cb0a0501c: Pull complete 
a4a3efb708f4: Pull complete 
Digest: sha256:23ddc354c1b85d85ce56e0659875b7c9a89bc4c6778f3d7e91140d882c681bec
Status: Downloaded newer image for adfreiburg/qlever:latest
2022-08-24 06:50:21.033	- INFO:  QLever IndexBuilder, compiled on Wed Aug 24 00:11:41 UTC 2022 using git hash 3d1a56
2022-08-24 06:50:21.034	- INFO:  You specified the input format: TTL
2022-08-24 06:50:21.034	- INFO:  Locale was not specified in settings file, default is en_US
2022-08-24 06:50:21.034	- INFO:  You specified "locale = en_US" and "ignore-punctuation = 0"
2022-08-24 06:50:21.035	- INFO:  You specified "num-triples-per-batch = 5,000,000", choose a lower value if the index builder runs out of memory
2022-08-24 06:50:21.035	- INFO:  Integers that cannot be represented by QLever will throw an exception (this is the default behavior)
2022-08-24 06:50:21.035	- INFO:  Processing input triples from /dev/stdin ...
2022-08-24 06:51:50.419	- INFO:  Input triples processed: 100,000,000
2022-08-24 06:53:13.171	- INFO:  Input triples processed: 200,000,000