 |
| What is this encyclosphere thing anyway? We have made so much progress that a good way to learn the concept of an encyclosphere is to just look at what we’ve done. Here, then, is a guided self-tour. Begin with an encyclopedia that is not Wikipedia. Like Ballotpedia: Clarence Thomas (Supreme Court). Or the World History Encyclopedia: Great Pyramid of Giza. Or the Encyclopedia of Mathematics: Pythagorean theorem. There are a lot of great encyclopedia articles out there that aren’t from Wikipedia, and they are licensed under a similar license. But you know what? They aren’t all collected in one place. They won’t rise to the top of Google or DDG or Brave search results. These and many other fine articles are buried under a lot of garbage, frankly. It sure would be useful to have them all collected in one place. Yes, but…these sources are of course different, with different styling, a different format…so we constructed a standard encyclopedia file format. Let’s take a detour into the technical innards (skip to 5 if this bores you): Check out an example of a ZWI file: the standard encyclopedia file! The acronym ZWI is derived from “Zipped WIki”; but it’s not just wiki files. Go here to download the latest ones to be added by EncycloReader. Or just open up the ZWI file for Citizendium’s “Europe” article. Go ahead, download it. You’ll want to use a ZIP reader on it, because it is a ZIP file. Open up metadata.json (in a text editor/reader). It has info about the article—basically, a lot of the info a search engine needs, like a title and description. Look in the data/ directory. Inside data/media/images/, you’ll find a dozen images. Inside data/css/, you’ll find the basic styling you need to display the article. If you go to the top of the “Europe” archive, you’ll find article.html. If you’ve unzipped the article and you click on article.html, you’ll see that the article is display correctly, with pictures and some basic styling. “But,” you say, “what about paywalled encyclopedias like Britannica, and free-to-read but not-freely-distributed encyclopedias, like the Internet Encyclopedia of Philosophy? Look at the ZWIs from proprietary and ordinary copyrighted encyclopedias too. No, we don’t have the whole articles, so don’t get excited, now. But we do have the ZWI files. If you peek into the file for Britannica‘s “France” article, you’ll find, not the article, but the metadata.json file. It has the title, a description (“France, officially French Republic, French France or République Française, country of northwestern Europe. Historically and culturally among the most…”), and a bunch of “SignificantPhrases” that make it possible to include the article in a search engine. So we can have all the encyclopedia articles in the world in the database. Here’s a similarly stripped-down ZWI file for the IEP’s “Plato” article. Notice the “signature.json” file? Look at the signatures on these files—this is how we bring true self-ownership to a content network. The humble “signature.json” file you’ll find there might look like a lot of technical gobbledegook, but what it means is that this file can live somewhere far away from its declared origin, which is part of the ID (here it is: “did:psqr:encyclosearch.org#publish”), and yet you can guarantee that it comes from where it says it’s from. This means you will be able to use software on your blog, to digitally “sign” the ZWI file of an article you put on the blog…and anyone will be able to confirm, or disconfirm, that some other copy somewhere else actually was signed by your “DID.” Mostly it will be techies who get excited about this, but in this day and age when our presence online is functionally owned and controlled by giant corporations, this is how we rebel. But enough of the tech. Once a bunch of ZWIs have been collected—we have 700,000 from 19 sources, and counting—what can you do with them? Well, search and read them! Let’s take tours of our search engine/readers. Visit EncycloSearch. We have talked a lot about EncycloReader a lot in the past, but our other, independently-developed reader, EncycloSearch, has made great strides this year. Try searching for Aristotle. There I see results from Handwiki, Citizendium, Britannica 11th edition (public domain), World History Encyclopedia, Stanford Encyclopedia of Philosophy. Notice: no Wikipedia. Why not? It’s not shown by default. You can make it show up by clicking on the down arrow next to the search box (where you can select which encyclopedias you want in your search) and clicking on “Sources…” at the bottom, then enabling it. Click on an article title (say, the EB11 article) you’ll be given a choice: show the article in encyclosearch.org, or go to the source website. If you click on the first choice, the article comes up right there in the website. There’s a link to the “original source,” which in this case is Oldpedia.org. Have a look at the left sidebar, or if you’re on a phone or have a narrow window, the “hamburger menu” in the upper left. Fun stops: “Random article” does for the whole encyclosphere what the similar button on Wikipedia does. “Source code” links to the free (OSS) software that runs EncycloSearch, called EncycloEngine, authored by Henry Sanger. “Download ZWI file” lets you do just what it says—we’re serious about this “free knowledge” stuff. By the way, you can now click around from a Wikipedia article, and by default you’ll stay on EncycloSearch. There is a lot more to say about EncycloSearch but this is a quick tour. On to… Visit EncycloReader. Try a search like “epistemology”. We’re serious when we say we want a decentralized network. To make this a reality, we are independently developing not one but two encyclopedia search engine/readers. A lot of the development of the encyclosphere’s ZWI file format and network standards has taken place in the context of EncycloReader. It is fast, and it supports the “stay within reader” feature for more websites (like Handwiki and Citizendium), and handles misspellings fine. It also has a number of handy Preferences you can set; for example, you can choose how to display external links in articles. You can also view the log of additions EncycloReader makes to the encyclosphere here (where you can download ZWI files) and view the number of articles that EncycloReader is tracking. We are grateful to Dr. Sergei Chekanov for his great work building EncycloReader and a lot of the encyclosphere software and standards. Now, you might ask, “How could someone contribute new articles to the encyclosphere?” Well, that’s already possible, in a number of ways: Look at the options for how to contribute articles. One way to contribute articles would be to join Citizendium and start writing there. This website has installed our ZWIMaker extension for MediaWiki (the same software that runs Wikipedia), which creates a button on articles; if you push the button, a ZWI file is created, and the article is added to the encyclosphere (via EncycloReader). There is a similar plugin, ZWIdoku, for the popular alternative wiki website, DokuWiki; the STEM encyclopedia HandWiki has this installed, and you could do some work on such subjects there. Sergei wrote these as well. Another of Sergei’s projects is Enhub, which is an easy-to-use way to create encyclopedia articles and add the ZWI files to the encyclosphere. Now, what if you want to host the article yourself? That’s what I plan to do with my personal website, a WordPress site. Developer Shelly Warren has been putting finishing touches on a WordPress plugin that takes a WordPress post, makes a ZWI file, signs it, and pushes it up to your choice of either EncycloSearch or EncycloReader (and from there it can migrate to the other aggregator(s)). We can’t demo this yet, but we will as soon as it’s ready! We have other options percolating as well. So the encyclosphere can grow organically, from the bottom up! But what about support for older, public domain articles? Announcing Oldpedia.org. I recently introduced the Old Encyclopedia Digitization Project to you. Well, we’ve already made some more progress there. I have developed ZWIFormat.rb, which works with the HTML files outputted by ABBYY FineReader PDF Editor; it now converts them into signed ZWI files. That is how we now have our first three ZWI files created from the 1888 Historical Collections of Ohio: “The Geography and Geology of Ohio,” “Glacial Man in Ohio,” and “Edward Orton” (first president of OSU). Those are on Oldpedia, which (like EncycloSearch) runs EncycloEngine, and so you can download the ZWI file; if you open it up, you’ll find it’s been signed by Oldpedia.org (so, the KSF). If you look at the article.html, you’ll see the article has been marked up in ways appropriate to it, as an encyclopedia article. Oldpedia also has “canonical locations” or “the home on the web” of particular EB11 articles, like this one. The question I’m afraid people will ask is: “With EncycloReader and EncycloSearch as the only aggregators so far, isn’t this encyclosphere, you know…centralized?” That bothers me, which is why I am pushing Sergei and Henry to make aggregator software that is easily installable by others, who will be able choose and transmit their own selections of articles to share (and Henry will do this with EncycloEngine, while Sergei has said he will make ZWINetwork more widely usable for this purpose). We have several people ready to install this (soon). And get ready… The Chrome (and Brave) encyclosphere plugin for will let you seed and view articles right within your browser. “Wait, what does that mean?” you ask. It means that you will soon be able to install a browser plugin (it’s not available yet) that will let others grab articles not from any aggregator, but from the anonymous BitTorrent network. Everyone who turns on the plugin will be co-hosts of the entire encyclosphere. And the really neat thing is that you’ll be able to view articles directly in your browser, downloaded straight from BitTorrent, without going to any website at all. We have talked about other kinds of software that will do something very similar. That is the sort of dream of true decentralization that we’re working toward, where we are all participants in a leaderless, centerless knowledge network—and yet, because ZWI files are signed, orgs and individuals can credibly assert ownership over intellectual property that is distributed in this way. By the way, in December we plan to add dozens of more encyclopedias (free and proprietary both), and the number of ZWI files created is bound to go over a million. Help us finish this work! We have a lot left to do. I haven’t told you about all the essential features we still need to add, and problems we need to solve, and bugs we need to squash. I haven’t told you about the partner projects who are waiting to use different pieces of the encyclosphere body of work. This development work is important, but it’s hard to do without adequate compensation. Our budget is around 1/1000th that of Wikipedia—and we’re doing work that, if we succeed, will be much more important than Wikipedia. So, please consider donating regularly. We will not accept donations from governments, from large corporations, from reference publishers, or from any organization or individual that we feel represents a threat to the independence of this endeavor. Does this mean we’ll have more trouble raising money? Fine, I don’t care. So, yes, we do needyour support. We would love to see larger donations from wealthy freedom-loving individuals and foundations, as long as the donations are without strings; and any amount over $5,000 means that I will have to fly out to meet you face-to-face. If I’m going to take that much money from you, I need to know you. We would love to have your $5, $50, or $150. DONATE HERE. Every little bit helps. Best, Larry — Dr. Larry Sanger, President, Knowledge Standards Foundation ([email protected])  encyclosphere.org/donateThe Knowledge Standards Foundation is building a #decentralized network of encyclopedias where anyone, anywhere can access all the free encyclopedias easily. We are a 501(c)(3) non-profit organization.  |
Leave a Reply
You must be logged in to post a comment.