Wikipedia:Bots/Requests for approval/KMLbot

KMLbot

Operator: Evad37 (talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)

Time filed: 05:33, Saturday, September 3, 2016 (UTC)

Automatic, Supervised, or Manual: Automatic

Programming language(s): SPARQL + PetScan + AWB

Source code available: Yes, see function details below

Function overview: Adds {{Attached KML}} to articles which have KML files available through Wikidata

Links to relevant discussions (where appropriate): Template talk:Attached KML § Proposal: Use Wikidata and new module

Edit period(s): Around once a week

Estimated number of pages affected: ~450 for initial run, probably much less in subsequent runs (depends on KML creation rate on other wikis)

Exclusion compliant (Yes/No): ~~Not through {{bots}}~~ Yes, per below

Already has a bot flag (Yes/No): No

Function details:

(1) Get a list of Wikidata items which have both a KML file and an article on English Wikipedia.
Done manually with the following SPARQL query to Wikidata Query Service:

SELECT ?article
WHERE
{
        ?article schema:about ?item 
        ;       schema:isPartOf <https://en.wikipedia.org/> .
        ?item wdt:P3096 ?kml .
        SERVICE wikibase:label { bd:serviceParam wikibase:language "en" }
}

Output saved as CSV, opened with MS Excel. Titles extracted from URLs using formula like =LEFT(MID(A2,31,999),LEN(A2)-30). But these are precent-encoded, so then decode them using a web-based URL decoder.

(2) Filter out articles which already have {{Attached KML}}, or are a disambiguation page, or have been excluded through {{No KML|reason=}} (template yet to be created; would add pages to a hidden category [[Category:Pages which should not use KML from Wikidata]]).
Done manually through a PetScan query like the following (pasting the decoded list of articles into the "Manual list" box):

https://petscan.wmflabs.org/?language=en&project=wikipedia&depth=1&categories=Attached%20KML%20tracking%20categories%0D%0AAll%20Disambiguation%20pages%0D%0APages%20which%20should%20not%20use%20KML%20from%20Wikidata&combination=union&ns%5B0%5D=1&manual_list_wiki=enwiki&source_combination=manual%20NOT%20categories&interface_language=en&active_tab=tab_other_sources

Output as "Wiki" and save as a UTF-8 text file (for input to AWB)
(3) Use AWB to add {{Attached KML}} to the end of each article (prior to DEFAULTSORT, interwikis, categories and stub templates).
Specifically, use the following options only:
Options tab: (none)

More tab: Append text {{Attached KML}} with "Sort meta data after" checked

Disambig tab: (none)

Skip tab: Skip if page is in use, or page is redirect, or page doesn't exist

Bots tab: Auto-save with delay 20 seconds, nudge if stuck, skip if first nudge doesn't help

Start tab: Summary Added {{Attached KML}} to make use of KML from Wikidata ([[User:KMLbot|more information]], [[User talk:KML bot|report problems]])

Notes:

Applying here per Special:Diff/737483526
The page at User:KMLbot would have info like what I started drafting at User:Evad37/KML, but adjusted to match whatever approval is given for the bot.
The bot would be not only be exclusion-compliant through {{bots}}, but would also be exclusion-compliant through the (yet to be created) {{No KML}} template populating [[Category:Pages which should not use KML from Wikidata]] (hidden tracking category also yet to be created). This allows excluded pages to be filtered out with the PetScan query, would enable tracking of such article through the category, and would encourage editors to provide a reason why the KML shouldn't be used (so that problems could possibly be fixed for all wikis, rather than just ignored at the English Wikipedia). - Evad37 [talk] 05:33, 3 September 2016 (UTC)[reply]

Discussion

Unless you turn the option off, AWB is exclusion compliant via {{bots}}. — JJMC89 (T·C) 05:55, 3 September 2016 (UTC)[reply]

Oh, okay. Didn't realise that, but I don't think it makes a big difference to the proposal. - Evad37 [talk] 06:25, 3 September 2016 (UTC) Adjusted above - Evad37 [talk] 06:27, 3 September 2016 (UTC)[reply]

Approved for trial (50 edits). Please provide a link to the relevant contributions and/or diffs when the trial is complete. — Earwig ^talk 17:16, 3 September 2016 (UTC)[reply]

Trial completed. Notes:

Edits 1-5: edit summary malformed
Edits 2-10: disambiguation pages included - because typo in petscan (capital D)
Edits 11-14: tried to fix in petscan - something went wrong, all of these had no KML in wikidata - reverted these edits
Edits 15-20: Started over (Wikidata query, decode titles, petscan query, save as UTF8 text file). Generally okay, but AWB inconsistent in adding new lines - some have a single space (as intended), some with 2 (which makes a gap in rendered page), some with none (??) - may be due to extra lines being rearranged by "Sort meta data after".
Edits 21-25: Try option "use 0 newlines" - all appears to be okay now;
Edits 25-50: Went ahead with remainder of trial edits. Spot-checked about 15 of these while the bot was editing and saw no problems - Evad37 [talk] 00:44, 6 September 2016 (UTC)[reply]