WebDAV and MediaWiki

A few weeks back at EMWCon 2016, Angelika Müller from Hallo Welt! spoke about the sales life cycle for enterprise wiki services, featuring their  Blue Spice MediaWiki Enterprise Distribution. During her talk, Angelika hinted at a project they had completed for some customers* involving WebDAV. Unfortunately, the constraints of that engagement prevented them from speaking openly about it at the conference; the one year “gag order” was set to expire a few days after the conference.

wikiDAV0.pngWell, that gag order is up and the Blue Spice team invited us to join a webinar today to show of what they’ve done. Very cool stuff. (I have some screengrabs from the webinar, but they are on a different computer. I will add them in tomorrow.) 

The webinar was a demo conducted by Markus Glaser from the Blue Spice team, showing off how the WebDAV integration with MediaWiki, through Blue Spice, works. For those not familiar, WebDAV

stands for “Web-based Distributed Authoring and Versioning”. It is a set of extensions to the HTTP protocol which allows users to collaboratively edit and manage files on remote web servers.

Basically, it lets you interact with files on a remote web server as if they were local files. In the enterprise this typically means Microsoft Office files from a Windows computer.

To start the demo, Markus showed how you can access a wiki through Windows Explorer, with the ability to open and edit the actual wiki articles. Though this is a cool feature, as Markus noted it not an especially interesting use case. Opening a page and editing it on a text editor on your computer will be slower and more cumbersome than simply editing the page on the wiki, and you will still need to use wikitext for the content.

The really interesting, and valuable, aspect of WebDAV integration with wiki goes back to those MS Office files. With the access to the wiki through Windows Explorer, you can directly add files to the wiki through your computer file system. For example by dragging and dropping from another folder, or simply saving a new file to the Media folder (aka namespace) on the wiki.

In addition to adding files through Explorer you can open, edit, and save the file as if the file were stored locally on your computer. Which is, of course, what WebDAV does. The really great part, though, was that you can also open the file from within the wiki, edit it, and save it directly back to the wiki. No downloading, editing, and then uploading again. If you’ve ever had to do that, you know how nice this feature is. In both cases, the versioning info is maintained on the file’s wiki page.

One of the challenges with the way MediaWiki handles files, of course, is that they typically all go into a single namespace, such as “File”. Not only does this make it hard to keep things organized, all of the files have the same permissions. To address this, Markus demonstrated the Extension:NSFileRepo in action.

The NSFileRepo extension restricts access to upload and read files and images to a given set of user groups associated with protected namespaces. Using this extension (within the security limitations noted above), you can protect not only pages and areas of your wiki, but also any uploaded images or files within those namespaces.

The WebDAV integration can be used on a “vanilla” MediaWiki install, but some of the features Markus demonstrated today are specific to Blue Spice. Installation is a bit more involved than simply installing an extension; the way that WebDAV works requires some web server configuration as well.

tl;dr Using WebDAV integration with MediaWiki provides a much more user friendly experience for uploading and working with files on MediaWiki.

This slideshow requires JavaScript.

* They built the WebDAV integration as a crowdsourced project, where several customers contributed to the project to ensure it was fully funded. Those customers agreed to making the project available to general customers, but only after a certain amount of time (a year) had passed. An interesting approach to getting open source work funded, one I’ll probably come back to again later. 

Videos (links to) – my talks at EMWCon 2016

I presented the following at Enterprise MediaWiki Conference (EMWCon) 2016 on 25 – 26 May. This talk, along with all of the other talks from the conference, are available on Livestream courtesy of the Internet Society.

Note: Unfortunately I am unable to embed the videos here, I can’t find a WordPress shortcode for Livestream videos and am unable to install the Embedly plugin. Will keep looking for options. 

http://livestream.com/internetsociety/emwcon/videos/124063646

http://livestream.com/internetsociety/emwcon/videos/124154391

 

 

EMWCon 2016 – some notes (create camp)

EMWCon_cc1

Spent some time this morning discussing various ideas for projects to work on. Including:

  • HTML2Wiki
  • Semantic Form Themes
  • Mermaid
  • Make site faster
  • Extension certification
    • AD / Vagrant roles
    • BPM Setup for an extension cert service
  • Extension manager
    • Extension interdependency management
  • Extension screenshots and working links to examples on mediawiki.org
  • Reification / provenance in SMW
  • Semantic forms validation

Most are somewhat technical (definitely beyond my skill level with MW), but many of those do require some non-technical participation. And some are longer term ideas (screenshots of extensions, for example) that can be continued to be worked on over time.

The one that most appealed to me was Lex’s presentation for creating an “Extension Certification” process for MediaWiki extensions. Would tie in with the potential Enterprise MediaWiki Foundation (EMF?) that we discussed on Day 1.

Basic process is straightforward (developer creates extension, runs acceptance tests, submits to EMF for review, they certify), but the implementation is a bit less so. Quite involved on the developer side, somewhat automatic at the review level. End result would be the “EMF Seal of Approval” for the extension, showing which core versions the extension has been tested against.

This type of process would would go a long way for Enterprise users, especially when trying to convince management, IA, etc that the extension can be trusted and presents (relatively) low-risk in implementing.

You can keep up with progress on the EMWCon 2016 page.

EMWCon_cc2

EMWCon 2016 – some notes (day 2)

Live-blogging (kind of) again, day 2. Also, livestream.

Notifications in MediaWiki

EMWCon_yaron

Yaron discussing the current status of notification (poor), and what would make an ideal notification system for MediaWiki. Page creation, edit, etc in different sets of pages, such as all pages, pages in namespace, etc, and certain people notified, e.g. users in a user group, specified list, users signed up to be notified, maybe external users via email.

Notification done by maybe email, Extension:Echo, or custom webhook of some kind. Echo (you can see on mediawiki.org) is planned to be added into MW Core. Several other extensions rely on Echo.

Many potential pitfalls. Too few options not helpful, too many confusing. Regular users vs. admins vs. “superadmins”.

“Looked at SharePoint for bad notification design, and SharePoint didn’t disappoint.”

Google Summer of Code, Yaron is co-mentoring a project to create a “page notifications extension”, started this past Monday. Goal is to create a framework that covers all the possible options, to make notifications in MediaWiki useful.

Jason mentioned Extension:Notificator, which allows users to set up notifications of others. NASA’s Enterprise Media Wiki team released Extension:WatchAnalytics and others that provide some insight into watching.

Inside Wiki Sales

EMWCON_am

Angelika Müller from Hallo Welt! talking about selling wiki services, especially Blue Spice, their enterprise wiki distro. Covered the basic lifecycle of sales, from contact to requirements to estimates to doing the work and shipping the final product. Challenges include technical and organizational.

All development, as much as possible, should be customer driven. After all, they are the ones who know what they want.

Discussed some of their big use cases.

  • Wiki farms
  • Document management – they first ask, why not use a document management tool, why a wiki? Things to address – setting rights for files, managing and editing
  • Quality management

In many ways, the same challenges any design / development shop has, but some uniqueness based on the product (MediaWiki based).

A big topic of interest is how they are using WebDAV with MS Office to allow their users to edit / save MS Office documents in-place inside the wiki, instead of having to download – edit – save – upload the file.  This was developed as custom work for a group of customers who came together to fund it, with a restriction that the code for doing it could not be shared for one year (which is almost up).

Anja offered that their WebDAV developer will have a webinar next week to discuss with those who are interested.

Lightning Talks

CWIX Wiki for Interoperability Testing

An overview of the CWIX 2016 wiki.

Making Cool Directories with MediaWiki

Yaron showed some examples of “cool” directory sites. Can you do this with MediaWiki? Functionally, yes. Visually / UI, no.

Question: how can we make our display as cool as theirs? Important for external facing wikis, but can be useful and important internally as well.

Check out migadv. And MITRE’s Gestalt.

Another option is to leverage the MediaWiki API to use MediaWiki as back end and build a custom front end using Javascript, etc.

Social Semantic

EMWCon_jb2

Jason Bock sharing some of the things we’ve been working on using SMW for bringing some social features to MediaWiki. Started with a brief overview and history of milWiki (see milSuite article on Wikipedia) and installing SMW.

If you are going to use SMW, you should definitely use Semantic Forms. “Has default form” and “has alternate form” are crucial to make sure people are using forms to enter semantic data.

Create Templates based on Queries, makes it so the user doesn’t need to understand how to create queries, they can just use them. Browse pages are the most popular click from homepage, a fact of life in a hierarchical-minded organization. Query forms, with filters, provides an “Amazon” like search / filter experience.

A 12 step training guide for general users to learn how to use SMW to create their own projects, covers basic overview through the more complex concepts. Crucial if you want your users to be able to create their own solutions.

Three examples of user created semantic projects: IT ticketing system, app store, unit training.

Lunch

Anatomy of a Cyber Taxonomy Development Wiki

EMWCon_cc

Cindy Cicalese from MITRE (Extensions by MITRE). Demonstrated live at the Malware ATtribute Enumeration and Characterization wiki.

Purpose of the wiki is to help generate an ontology for describing malware. Includes consistant iconography to provide better navigation and visual indicators of what you’re looking at. Built around hiearchies to work (hierarchy builder?). Uses a combination of Cargo and SMW, but the only SMW still in use will ultimately be converted to Cargo. It is an example of how the two can coexist, use each where best suited. (nts: Cargo vs. SMW?)

The “Graph this page” option uses MITRE’s VIKI extension. Very nice, reminds me of The Brain.

Cindy walked through some key aspects of the site, how it’s structured and configured. Went down the rabbit hole of walking through some of the code. (I stood at the top of the hole and watched her walk around, a bit beyond my own current knowledge 🙂

Contracting with the GPL

EMWCon_gr

Greg Rundlett talking about the challenges – and joys? – of doing MediaWiki work for large enterprise clients.

Challenge 1 – Lead time: Make sure lead time and discovery are in your pricing model

Challenge 2 – Payment: Large enterprises have set processes that may not be in your best interest. Negotiate for the best terms you can get (net 14 would be good).

Challenge 3 – Insurance: E&O (errors and omissions) and general liability

Challenge 4-6 – “All Your Base Are Belong to Us”. The challenge is how to convince the client to allow you to publish the custom work as GPL.

The “Master Services Agreement” is a standard contract, typically a take-it-or-leave-it thing. But, you should try to write details into the agreement where details are allowed. This is especially needed for the code, how much can you “keep” for reuse or publishing to open repos.

Collaboration and cooperation, more is needed among consultants in EMW space. We aren’t competing against each other, we are competing against the Microsofts. 

Comment from Mark – we need more consultants like freephile as members of the MediaWiki stakeholders.

A bit of discussion – and disagreement – around GPL and how it applies to custom/proprietary code development.

Lightning Talk

Me – MediaWiki as part of larger Enterprise Social Network

Technical Collaboration – WMF

EMWCon_ckoerner

Chris Koerner providing some insight into the technical collaboration team at WikiMedia Foundation.

The team primarily focuses on supporting the WMF and its official efforts, but they do take input from 3d party users and work on things that provide value to them as well.

Phabricator – bug tracking. Look at in more detail later.

The team hosts several events – developer summit, hackathons, Wikimania, and others. If you are doing good things with MediaWiki, let them know so they can get the word out.

WMF – the ongoing saga

General discussion of topics related to the WikiMedia Foundation. Didn’t catch it all, so no real notes here.

EMWCon 2016 – some notes (day 1)

A live blog (of sorts) for Enterprise MediaWiki Conference (EMWCon) 2016. The conference is also being livestreamed.

Panel – Towards a MediaWiki Foundation

EMWCon_panel1

Cindy Cicalese, Anja Ebersbach, Mark Hershberger, Chris Koerner, Yaron Koren

Panel discussion to address some of the challenges around the development, maintenance, and use of MediaWiki and related software (extensions). Not a discussion of separating Wikipedia ops from MediaWiki core (which would be something similar to how WordPress is set up).

Yaron presented a case for a MediaWiki foundation that would help fund “unfunded” software. Specifically, funnel money from the users of the software to the developers of the software. Similar to other open source “foundations”, example given was the Linux Foundation. aka pay to play.

Part of the challenge is that the Wikimedia foundation is focused almost exclusively on Wikipedia and their other projects. Desire is to make sure that the MediaWiki software remains usable by other 3d party users, and actually includes development specifically to meet the needs of those 3d party users. For example, the MediaWiki Stakeholders Feature Wishlist.

An interesting mess.

Social Semantic

EMWCon_jCantRoot

Jason Bock talking about applying SMW to achieve social objectives on MediaWiki: standard profiles, rate/review articles, user point system, user badges. Shared some of the specific extensions used.

It is possible to create a template for userpages and have it pushed automatically, need to get user feedback on whether they like this or not. Some people prefer freeform userpage. Balance between user needs / desires and the enterprise’s needs.

All uses existing extensions, doesn’t require any development just “front end” SMW work. Walked through examples the “code” for each. This has potential to cause significant performance issues, will work to have some of this turned into extensions.

Questions about gamification: how do you handle cheating (don’t really), has it helped with adoption (with some people).

Improving Enterprise Findability with SMW

Laurent looking at how SMW can be used to help people find answers to their questions.

“Searching doesn’t give you answers, it gives you search results.”

Use the wiki as a wiki; keep it lean and fast. Use APIs to access other systems. Such as WordPress.

Lightning Talks

MediaWikiFarm extension

Nicolas Nallet talking about the MediaWikiFarm extension. Slides from the talk.

US Federal Government MediaWikis

Peter Meyer talking about MediaWiki installations in U.S. Federal Government. Full notes. Something to follow up on – Federal MediaWiki Demonstration and Discussion Group, a monthly gathering to share ideas, practices, and demo ideas. A call for a more consistent coherent wiki policy across the Federal Government.

“Knowledge management is not a curse word.” — Peter Woudsma

Lunch

Wiki Farms (again)

Peter Woudsma. Reasons for multiple wikis: ownership, access, control, processing. Defined: MediaWiki server install that includes core and some extensions and supports multiple otherwise independent wikis. Went through an example putting three wikis on one MediaWiki server. Reasons to use common elements: data re-use, template re-use, branding, data processing (inter-wiki data data exchange, queries, transposition)

Question: how much is common to all wikis in the farm? Many possible options.

Discussion among the group about approaches they have taken.

 

SMW Factory

Lex Sulzer spoke about SMW Factory. Offline, secure, encrypted, backup (and a bunch of other terms). Duplicity, Vagrant, Ansible (executable documentation). Details in the link. An easy way to “clone” a wiki soyou can work on it offline, then push changes back to production. (Meant for dev, not content.)

Ultimate goal – ability to have “offline” wikis that can be updated and merged back in with the main wiki. This would be huge. A project for Friday.

Single enterprise wiki

me

What’s new in Semantic Forms

Jaron giving an update.

Change 1 – Spreadsheet display. “display = spreadsheet” Each entry becomes a separate instance of  the template on the page. Less work for admins, allows for better HTML, such as <label> (important for accessibility)

Change 2 – Some input types moved from Semantic Forms Input extension to Semantic Forms, continues a process started with other input types. Few extensions, less work for admins and developers.

Semantic Forms is now its own thing, no longer requires Semantic MediaWiki.

Change 3 – removed some params, some of the long-deprecated parameters.

Thinking of changing the name, to reduce confusion with other “semantic” things.

Important takeaway – cleaning up Semantic Forms to make it better, easier for developers and admins.

Also discussed changes to Cargo (an alternative to SMW). Key feature of note (to me) – text search of articles.

Opening Session

EMWCon_pw

Peter Woudsma gave a quick overview of the history of MediaWiki in the enterprise and some of the challenges of taking a tool designed as an easy way to update content and making it useful and valuable at the enterprise level. A nice summary of Enterprise aspects of previous conferences, and the need for balance in discussing the non-technical aspects along with the technical.

He gave us quite a few questions, homework if you will, to consider as the conference progresses. Not just in how we use the tools, but how we influence the future development of the tools, and the support infrastructure.

Good discussion around the philosophy and implementation. Differences between what’s going on with MediaWiki software and WikiMedia. Need to show off who is using MediaWiki, and how they are using it. WikiReport.

Lex – SMW power user is critical to success. SMW is a digital co-worker.

Introductions

Mix of corporate, government, and non-profit interests represented. People from several places in US and Europe (France, Germany, Switzerland).