Konstantin Tovstiadi - DISsertation

Database driven world revisited

nospam@example.com (Konstantin Tovstiadi) — Fri, 09 Jul 2010 15:44:43 -0400

I have talked here before about a database-driven world. The idea is that the internet is largely driven by databases, and our lives are increasingly dependent on it. The invisible layer is even more powerful as it is invisible. I wrote that blog post as an attempt to understand the phenomenon. Then I wrote a paper about it.

The paper was kindly reviewed and commented on by several colleagues and friends. Then I even sent it out for publication. Long story short, I haven't heard back from the editor yet - probably never will. Maybe because they didn't like it, maybe because it was too weird (mixing translation theory, rhetorical criticism, and databases), or maybe for some other unknown reason. Whatever it may have been, a year later parts of the argument are obsolete. MySQL is still the default standard, but the big sites with billions of records are switching to new standards, the No SQL stuff.

So I have decided to release that paper to the world here - under the Creative Commons License. The main premise still holds, and it is more use here than sitting on my hard drive.

Epistemology and rhetoric of database-‐driven translation - PDF file

Reading Paul Hawken

nospam@example.com (Konstantin Tovstiadi) — Thu, 03 Jun 2010 14:32:05 -0400

Not every book is based on good science rather than anecdotes on conjecture. When it is scientific, it is rarely well written. When well written, it is not always provocative and novel. And the rarest of all gifts, few provocative academic books pass the test of who-the-fuck-cares-besides-your-tenure-committee.

Paul Hawken's Blessed unrest is unique in that it is all of the above - factual, precise, fresh, meaningful, and timely. The main argument is delivered with precision, elegance and poise. The greatest social movement that is the focus of the book is the one with no name and no unifying ideology. It is a bottom-up movement that includes a variety of people, causes, and geographies - social rights, working rights, social equality, transparency, preservation, sustainability. My favorite element of the argument is that Hawken convincingly shows that social movements of the past and the environmental movements of today are two sides of the same coin. The less privileged strata of society are usually the people who suffer the most from pollution, deforestation, erosion and other forms of environmental degradation. The same ideology of exploitation is applied to people and nature.

The answer can be summarized in one word - sustainability (a word that, according to Hawken, Exxon Mobil at some moment explicitly prohibited in its printed materials, while sponsoring "skeptics for hire" to deny climate change). Growth that is couched in terms of greater creativity not in terms of ever-increasing consumerism requiring more and more natural resources. Change that is bottom-up rather than top-down. Local solutions to global problems.

The movement is not a rigid structure with a centre of command - it is a distributed global ecosystem of hundreds of thousands of organizations. The book ends with a meticulously organized taxonomy of these organizations that takes about a hundred pages. This is just the tip of the iceberg. The iceberg itself is online - a dynamic community with its own discussions boards, groups, and other tools expected of a modern social media network site. It is the finest and the most prominent example of an ever-growing endeavor that helps us move "from a world created by privilege to a world created by community".

Digital Signage with Drupal

nospam@example.com (Konstantin Tovstiadi) — Fri, 28 May 2010 14:12:09 -0400

Project goal: build a digital signage system for the college that meets the following requirements:

- the content is modular and is easily re-arranged or re-styled
- existing content from the college CMS / web site can be displayed
- external content can be displayed (RSS, XML, etc)
- system can be controlled remotely by multiple users with different levels of access
- only open standards are used (ruling out Flash, PowerPoint), leaving the content open and accessible
- content is stored locally rather than by a third party
- system is running on open source code rather than expensive commercial software

Hardware
- Apple Xserve
- Apple mac mini (any model will do)
- mac mini mount
- an LCD panel with 1080p resolution - 40" TV in our case
- LCD mount
- an HDMI cable
- a Mini Display Port / HDMI adapter

Software
- Mac OS X Server / Apache / PHP / MySQL enabled
- Drupal 6.16 with various additional modules (details below)
- Apple Remote Desktop for remotely administering server and player machines
- Mac OS X on the player machines
- Plainview
- a custom written Apple Script

Problem 1: Web site styling must be different from signage styling

Install ThemeKey module that allows different themes to be applied to different areas of the site. Load two themes of your choice - one for the site, one for the signage. Under Blocks, disable all navigations and menus (=all interactive items that require the user input) in the signage theme. If it produces orphan elements, kill those elements in code by editing template.php file in your theme's folder. Keep a backup of the file in case you cut too much and break everything.

Problem 2: Need to have control over what content is displayed in signage

Install the CCK module. Create a new content type and give it a memorable name - like Signage slide. Make sure it has an image field for the actual slide and and a text field for the story to go with it. Set the image to resize to the size you want so that it doesn't mess up your layout. In our case, everything is sampled to 800x600 pixels.

Problem 3: Need to have scrolling effects without locking the content in a proprietary format

Install Views module. Install Views Slideshow module. Create a slideshow using your Signage Slides. Decide where and how you want to display titles, stories and thumbnails. Set the times for the transition duration and time each slide is displayed. Once you have created multiple slides, you don't want to show all of them. Set the display properties to a reasonable number of slides and thumbnails (9 in our case because that's how many fit under the 800 pixel wide image in one row) and arrange them in descending order by creation date so only the newest ones are shown.

Problem 4: Need to have external content (e.g. news items from a newspaper)

Find the newspaper RSS address. Add it using the Aggregator module that is part of the default Drupal 6 package (Core optional). Create a separate view for the feed using Views, styling and titling it the way you want (display it as a list, table, etc).

Problem 5: Need to have weather information

Install Weather of Yahoo Weather Forecast Module. Customize it to your location. You now have a weather block that you can put on any page

Problem 6: Need to have a clock with date and time

Install the Clock module. Make sure you get the Libraries API and the jquerytimers.js script and put them in appropriate places. Set the clock to display the time the way you want - which means you might have to create a new custom date and time format (Administer > Site Configuration > Date and time > Formats tab). Set the clock to update continuously using Javascript.

Problem 7: Need to assemble various pieces of content and display them on one page.

Install Panels. Create a new page in panels. Give it a memorable name (like Signage) and a memorable path (like yoursite.net/signage). Choose the layout for your page - number of columns, fluid / fixed layout, and so on. Add content to the page - signage slideshow in one column, RSS in another, clock and weather in the third one in our case. They can always be re-arranged / re-sized / re-styled later as your system evolves. Your signage page is ready - you can test in a browser from anywhere (huge advantage for troubleshooting).

Problem 8: Need to feed the content to my signage displays.

Create a barebones Mac OS X installation - no iLife, no fonts, no multiple language support, no extra software. Install a free app called Plainview that allows to run web content in kiosk mode. Clone the installation to all your player machines, as many as you choose to have. Open Plainview and set it to display your page on application startup (yoursite.net/signage, or whatever path you decide to have).

Problem 9: Need to make sure signage works even after a power failure and that it pulls new content from the server automatically as it is added.

You need your system to load Plainview on startup and to show nothing else (no dock, address bar, etc). You also need to able to reload the page periodically to retrieve new content, especially if you are using RSS / weather forecasts and other time sensitive materials. Both tasks can be achieved by a simple AppleScript:

on idle
-- the applescript only calls attention to itself when the idle period is up

tell application "Plainview"
--confirms your kiosk app is the front application
activate
end tell

tell application "System Events"
tell process "Plainview"
keystroke "r" using {command down}
end tell
end tell
--sets the idle period in seconds. i.e. plainview will refresh every 1800 seconds (= every 30 minutes)
return 1800

end idle

Cut and paste the above code into an AppleScript editor (part of the default Mac OS X install, to be found in Applications > Utilities). Hit Compile. Save the script as an Application (look for that option in the File Format menu). Give your script a memorable name (like Refresh). Add your Refresh app to your startup items (System Preferences > Accounts - click on your user account, open Login Items tab, hit the plus button below, navigate to your Refresh app).

Problem 10: Need to administer the player machines remotely.

Install Apple Remote Desktop on the machine from which you will control everything. ARD is expensive but it quickly pays for itself because it saves so much time. Connect your player machines using Ethernet cables rather than wireless. Hard code the IP addresses (DHCP with manual address option). Install ARD clients on player machines, enable remote access. Connect to player machines using ARD from your control machine.

Problem 11: Need to save energy.

On your player machines go to System Preferences > Energy Saver. Hit the Schedule button and set your boxes to run 9 to 5 on week days only - or whatever schedule you need. While you are at it, disable display sleep and screen savers so your signage display is not disrupted.

Problem 12: Need to allow several people to add various types of signage content, controlling access and the level of privileges.

You can assign access to the Signage Slide content type in Drupal administration. (Administer > User Management > Permissions). For finer gradations, install Content Access module or if that is still not enough, ACL.

Problem 13: Need to display an urgent message

Create a node with a memorable path (such as yoursite/urgent). Restrict access to it to people who are allowed to send urgent messages - i.e. main administrator and the big bosses. Keep it empty by default. Using the Panels module functionality, edit your signage page (see Problem 7) and add the urgent message node to one of your columns. When the bad things happen, the big boss logs in, edits the empty urgent message to include the actual message (CLASSES CANCELLED TODAY or whatever else it might be), saves the modules. On the next refresh (in our case within 30 minutes), the message is displayed on your signage screens.

Problem 14: I have other needs but you didn't write about them. Can't figure out how to get things to work

Post your question as a comment here.

Reading Jared Diamond

nospam@example.com (Konstantin Tovstiadi) — Tue, 16 Mar 2010 16:22:45 -0400

Last Friday I drove for 15 hours, travelling 870 miles and consuming approximately 32 gallons of gasoline. My truck (Ford Ranger with the 2.3 liter engine) is the most fuel efficient and the least polluting of all trucks currently sold in the US. It produces 279 grams of CO2 per mile travelled (which compares poorly to the cleanest car, the 2010 Toyota Prius, at 127 gm/mile, but favorably to the 476 gm/mile of the Nissan Titan or the 479 gm/mile of the the Porsche Cayenne). So my ride produced 242 kilograms of CO2. If I had the inclination to drive like that every day, I would have produced 88 metric tons of CO2 a year, four times the average of US per capita emissions, and forty times the average the global per capita emissions. In other words, even if I drove like that only once every forty days, that driving alone would produce enough CO2 to match the global per capita average.

The reality is even worse. The carbon impact of my drive includes the emissions that were needed to extract, refine, and transport the fuel as well as the emissions needed to extract, refine, and transport the raw materials for the manufacture the vehicle itself. I am also indirectly responsible for the pollution needed to sustain the infrastructure that makes driving possible - maintaining the roads, filling stations, and road police. I am responsible for the destruction of habitats needed to build the roads, noise and light pollution that they create, and the disruptions in the migration patterns of wildlife. These hidden costs are immense. They are not reflected in the MPG or CO2 emissions numbers. All these factors are connected and interdependent, sometimes linked together into positive feedback loops (where a growth in consumption in one area leads to a spiral of increased pollution and consumption in other areas).

If my contribution as an individual is so huge, what is it like to have a society of driving loving individuals? What sort of environmental impact can they produce? This is one of the questions that Jared Diamond attempts to answer in his book Collapse: How Societies Choose to Fail or Succeed. The book looks at the problem from a variety of disciplinary angles, and describes a broad range of societal settings - Norse Greenland, oil mining in New Guinea, Anasazi and Pueblo people, modern Los Angeles, and the vanished civilization of the Easter Island, to name just a few.

Easter Island © Wikipedia

It is a great read that I am not going to summarize or evaluate here. It is also a very thought provoking read, and I am going to focus on the ideas that it triggered.

1. Many of our problems can be explained by the tragedy of the commons. When a resource is shared with various parties exploiting it, they tend to each attempt to extract as much as possible. As a result, resources that could be used in a sustainable manner with quota management are overexploited, depleted, and destroyed.

2. Reducing the environmental impact often necessitates a change of values that the cultures hold dear. For example, the US would have to give up the myth of the untamed Wild West, and the exhilarating freedom of driving on a whim (among other things) to reduce its footprint.

3. World population problem is not the only problem; it is the growing proportion of the population striving for and achieving First World lifestyles, thereby driving the global per capita impact up.

4. As a corollary to both 2 and 3, one of the worst values that we have that contributes to our growing impact is defining achievement as increased consumption.

5. Recycling is wonderful, but not even remotely as wonderful as non-consumption. Recycling only solves a tiny fraction of the carbon cycle - the consumer part, not the manufacturing part, which is many times more than the consumer part.

6. The current governmental pre-occupation with global warming and green house emissions - as well as endless debates whether global warming is happening distracts us from problems that are most certainly happening - soil erosion, depletion of fisheries, deforestation, increasing consumption, toxic waste and toxic manufacture transfer to the Third World, and so on.

7. Those still doubting the gravity of these problems should be reminded of the Pascal's wager. The risk of doubting them and being wrong is the risk of destroying the Earth; this is a chance no one should want to take.

Metareality

nospam@example.com (Konstantin Tovstiadi) — Mon, 01 Mar 2010 13:14:00 -0500

There is a cluster in rhetorical criticism called apologia, or corporate apologia, that deals with image restoration in response to a crisis caused by a corporate screwup - such as an oil spill, or exposure to toxic chemicals (asbestos, lead, arsenic - take your pick).

One of the interesting conclusions of apologia research is that it takes different people to do the cleanup and to talk about doing the cleanup, while giving the audience the illusion of a unified front. It is very hard to for the same person to do both. It is unlike trying to be a photographer at your own wedding, or as Erving Goffman would put it, playing tennis with yourself. You cannot be part of the action and outside of the action at the same time. You cannot be a participant and an onlooker (unless you are in shavasana getting an extracorporeal glimpse of your own restive body, but that is beside the point). You cannot be part of the reality and of metareality (just like the map can never be the territory, but that is beside the point, too).

Yet that is increasingly the expectation if you are an active user of social networks such as Facebook and Twitter. If you want to be popular, you have to do interesting things and then be able to describe them in an interesting way. You are supposed to run both manufacturing and public relations, all by yourself. It is probably a good thing. I hope it gives us a better grasp of how these two activities relate to each other, and how to manage both successfully. It can't hurt to have such a grasp in today's workplace.

Reading Stuart Kauffman

nospam@example.com (Konstantin Tovstiadi) — Thu, 21 Jan 2010 15:26:19 -0500

There are many ways to learn about a new city or a country. You could look at its GDP data, or the exchange rate, or the atmospheric pressure and dew point, or any other point estimates and indicators. You could read a Lonely Planet guide about it. Or you could go spend a week there. If somebody asked you about it later and all you had was the GDP data, you could simply tell them what the number was. They would have the same information as you then. But you can't share your travels with them the same way. Your impressions are complex, multifaceted, irreducible to stories and pictures.

Complex phenomena are all around us - snowflakes, tornadoes, ant colonies, stock markets. Studying them requires a new approach. Converting them to point estimates - size of the flake, speed of the wind inside the tornado, number of ants, Dow Jones - doesn't give them justice. The simplifications are too crude, too much data is lost. One solution to that is to have multiple point estimates. We can map the snow flake configuration with 1 micron accuracy - and if that is not enough, even greater detail.

But it is never going to be enough. The map can never be the territory. The measurements are imprecise; initial imprecision translates into greater and greater errors as we model the reality over time using point estimates. Eventually we have to admit that some of these problems - even the seemingly simple ones - are transcomputational - we cannot know what the weather will be two weeks from now, what the stock market will do in a month, or what would be the in-thing at the Milan fashion show next year.

Stuart Kauffman tackles the issue in his new book, Reinventing the sacred. Kauffman is one of the most respected scholars in the study of complex systems. This is his fourth book, and it builds on the first three. His earliest book, Origins of order, published in 1993, is a rigorous academic account of order-for-free, bottom-up emergence of high level order from the actions of lower level agents - such as the water molecules forming a snowflake. It was followed by At home in the universe, a brilliant translation of the first book into language comprehensible to the non-academic audiences. In 2000, Kauffman published Investigations - although it would be more accurate to call it Musings or Ramblings. It reads like a notebook - powerful raw ideas in need of better argumentation and cleaner style.

Reinventing the sacred resembles all three. It has some hardcore theory that many will hard to follow (such as the discussion of quantum decoherence). It has some excellent explanations of what these concepts mean in layman terms. And it has oodles of speculation and conjecture.

But there is also a new element - one that has to do with religion, ethics, philosophy, and the emergence of a global society and culture. Emergent order is the quality of the universe. Life emerges naturally. Human consciousness emerges naturally. Culture, language, society, economies all follow similar paths - unpredictable, irreducible, forever changing, forever creating. There is no need to invent a creator God to explain it all. But there is something that can be revered - the endless creativity of the universe itself. Worshipping this creativity is the suggested foundation for a new global culture, global ethic, global religion.

It is a beautiful idea, but one that is not easy to argue. The argument annoys the hard scientists (and anyone who wants to defend reductionism), philosophers (and anyone who finds Kauffman's foray into epistemology amateurish), complexity scientists (why can't Kauffman talk about biology and leave the rest out), and last but not least, the believers in a Creator God.

The task is enormous, and it is also enormously important. Kauffman's book isn't flawless - but on a subject as daunting and broad as world ethics, no argument can be. The book suffers from occasional repetitiveness and uneven style. It oscillates between barely understandable discussions of quantum physics and bar-style down-to-earth real life examples delivered with excessive nonchalance and camaraderie. But most of the time the book stays right in the middle, and despite redundancies and style issues, it delivers a strong argument - an argument that is not only thought-provoking, but timely and important as well. I have thoroughly enjoyed it - how can you dislike a book where a scientist who understands biology, mathematics, and quantum physics devotes several pages to praising rhetoric?

But this is a reductionist narrative about the book. The book is a lot more complex than my ramblings about it. You should read it yourself, or at the very least, listen to Stuart Kauffman talk about it himself. You can also read some of his most recent thoughts on this NPR blog.

Knowing more doing less

nospam@example.com (Konstantin Tovstiadi) — Fri, 08 Jan 2010 22:20:39 -0500

In the 1990s, all books in intercultural communication started with "we have never seen so much international travel / globalization". In the 2000s, all computer mediated communication books went on about never-before-seen ways to interact online. And in the first week of the 2010s (that The Economist suggests we call "teens", "decadents" or "debtcade") I managed to screw up in an entirely new, novel, never-before-possible way.

I have been talking about the software stack; and how depending on your geekiness you either live at the very top of it or you can actually penetrate its murky depths. The stack situation, in the great scheme of things, is a consequence of the division of labor and specialization. Even the geekiest nerds don't know every element of the stack. Generally, any modern household, vehicle, appliance, is a stack where the user knows only the top layer. It used to be that a peasant could fix the tools of his trade himself. There was nothing mysterious about a sickle. A John Deere tractor is a different story.

So armed with a minimal knowledge of server technology, I spent the last week of the "oughts" setting up a Mac OS X server - all the way from network connections and DNS records to remote administration, databases, and active PHP applications. And then I managed to lock myself out of one of my accounts; and decided to re-install everything.

It was all going well in the beginning - faster than before, because now I knew what I was doing. But it all went to hell in a hand basket when it was time to set up the database server. You start the MySQL daemon with a click of a mouse; but then you have to use a third party program like Navicat to create users and databases. The first time I was doing the server setup, I looked it up online, found an answer that worked, and was creating databases in five minutes. But the second time was different. I still didn't know how to set up the database server; but I knew more about the whole process - enough to be doing the Google search for instructions with different terms, enough to not be able to find the solution that had worked before. So knowing more about the process was actually counterproductive.

Climbing the software stack

nospam@example.com (Konstantin Tovstiadi) — Wed, 23 Dec 2009 21:51:00 -0500

In 2000, I wanted my own web page. Getting one felt like magic. Then I wanted my own site. Another magical experience. Then my own virtual server. More magic. Then a chunk of a real server that somebody else managed. Kind of magical. Finally, a server all to myself to administer. Today is finally up and running, in all its 16-virtual-cores, 2-terabytes-of-online-storage, 90-gigaflops-of-computing-power glory.

So I am looking for an open-source CMS to run on it; and given Drupal's recent runaway success with kings and presidents alike, Drupal is my top choice. So I am reading e-books about Drupal; and I come across a description of Drupal's architecture and its place in the "software stack".

The Drupal code is written in PHP, a programming language designed to deliver dynamic content (rather than static HTML). The application relies on the data in the stack layer below it - the database layer, in Drupal's case MySQL. Deeper still below that is the web server layer, enabling both PHP and MySQL function over the Internet (in our case, the web server is running on Apache). At the very bottom, the web server is embedded in the operating system (we use Mac OS X Server, but it could be Windows, UNIX, or Linux). So all in all, four layers:

PHP > MySQL > Apache > Mac OS X

You could extend the stack both up and down. The content management administrator doesn't interact directly with the PHP code; he or she normally uses a graphic user interface (GUI) to access the application. The administrator sets up the experience for the end users, who interact with the system through their own GUI. On the other end, for the operating system to run, the server hardware must have some basic software (=firmware) that allows to install the operating system, and check the integrity of the hardware. So the extended stack is more like this:

End user GUI > Administrator GUI > Application > Database > Operating System > Firmware

The IT staff's goal is to make as much of the stack invisible to the end user. The goal of the advanced user is to understand how the stack functions as far down as possible. The deeper you can reach, the more power you have.

But the beauty of today's situation is that one can have power even with just top-of-the-stack access. Modern servers come ready with the whole stack set up and ready to go. There is open source software for all the layers of the stack. Besides the depth, there is breadth - there are open source applications for almost everything under the sun. Which means the end user no longer needs to dive into the stack and get lost in heady geeky things. They can stay on top of the stack and focus on the task no one else can do: creating and organizing content.

Engaging the audience: The case of institutional repositories

nospam@example.com (Konstantin Tovstiadi) — Wed, 25 Nov 2009 11:54:50 -0500

If the audiences are everything for web projects, then a successful web site is one that finds a way to engage its audience - i.e. finds ways to grow through reader contributions, comments, ratings, and usage patterns. The same applies to other projects that involve collaboration - especially to the development of online collaborative tools for academics, such as institutional repositories (IRs). The authors of the key text on IRs describe the struggles of engaging the audience very well:

Acquiring the content is slow and laborious work, and at the present time we pay for it with the sweat of our brow, rather than by dipping into our materials budget. It involves - in addition to the seminars in the library - working through lists of academics with research management responsibilities, research journal editors and senior managers, as well as 'common or garden' academics. It requires the organization and tenacity (and thick skin) to lobby powerful committees, to meet academics in their own territory - in departmental meetings, and at lunchtime seminars held by research groups. It requires the hunting down of unorganized self-archived work, and the corralling of it within the institution's managed repository. It requires the capture of author permissions and an awareness of publisher policies on copyright transfer. It involves knowledge of licensing as it applies to self-archived and open access content. ANd most of all, perhaps, it involves the practice of constant repetition of the same message over and over many thousands of times, often to the same people, because the issues are difficult and non-intuitive to academic authors. This is a liaison work, and in a large research university it can easily cost a full-time professional member of staff. (pp. 38-39).

Jones, Richard; Andrew, Theo, and MacColl, John. (2006). The institutional repository. Oxford: Chandos.

Web Mantras 2.1

nospam@example.com (Konstantin Tovstiadi) — Tue, 17 Nov 2009 10:26:27 -0500

I made my first web site in 2000. I can now claim I have been working with web technology for a decade. And since my last three jobs are directly related to delivering web content, I feel like I have a grasp on what it takes to create and run a successful web project.

I have talked about it here before, focusing on three major concepts: form, content, and audience. I have been rethinking the relationship between the three lately, and here's an updated version.

Form is really synonymous with design. These three concepts - form, content, audience - are about what you deliver (content), to whom (audience), and how you present it (form / design).

I have argued against a workflow that starts with design, then moves on to the content, and then tries to adjust to the needs of the audience. I have claimed that form is nothing, content is nothing, audience is everything. I would add one more thing today.

A good web workflow works in the reverse order. It starts with an audience analysis - what do they need? Once the needs are clear, what content can be created (by us, or better still by us with the audience's feedback or contribution)? And once we know the content, how do we design the interface to deliver the content in the most accessible and intuitive way possible?

Technology is part of every step of this process, but it should never ever be a deciding factor driving the change. You don't update the site just because it is time. Or worse still, because there is some new flash(y) technology available. You use the technology that allows you create the good interface you need to deliver the content that your audience wants.

It starts with the audience, never with technology.

Buildings and icebreakers

nospam@example.com (Konstantin Tovstiadi) — Mon, 22 Jun 2009 19:50:09 -0400

In a famous piece titled On viewing rhetoric as epistemic, Robert Scott writes:

The attractiveness of the analytic ideal, ordinarily only dimly grasped but nonetheless active in the rhetoric of those who deem truth as prior and enabling, lies in the smuggling of the sense of certainty into human affairs

This is one of my favorite papers ever written on communication or on rhetoric; but that's beside the point. The point is about smuggling certainty into human affairs.

A lot of times we drag certainty in in the metaphors we use. What do we liken our lives to? A road. A path. Our lives do indeed have a beginning and end; but that's where the likeness ends. Our lives are not linear; nor are they marked clearly so one know where to proceed next. Maybe an old 2D arcade game is a more realistic metaphor - dodging enemies all the time, not knowing what's coming next, things getting nastier over time, getting slaughtered every now and then. Except of course you can't reload the game in real life (that's where this metaphor ends, too - they all do).

Russian nuclear icebreaker Yamal on an Arctic voyage

Some scientists somewhere once asked people if they see their lives as similar to a game of cards or a game of chess. Most picked chess, quite mistakenly. When was the last time you knew the location of all your opponents pieces? Even the game of cards analogy is overly optimistic: at least in cards a jack is a jack and is always below a queen. Real life is even more random.

Which finally brings me to my point: we have bad metaphors for education, too. We think of it as a building. Getting bigger and prettier over time. Growing from a foundation. Orderly with a top to bottom hierarchy - simple facts at the bottom, differential calculus at the top.

I don't feel like my education is like that. The foundation has crumbled away. What I learned when I was ten I could not recall now - at least no more than 10% of it. I used to be fluent in Spanish. I used to know trigonometry. I used to be able to write in hiragana. No more.

And yet I am surprisingly comfortable with my current level of knowledge. I think of myself as an icebreaker - ploughing through the unknown, with enough power to crunch through any problem, one at a time. Loose ice stays loose in my path for a while, then freezes over again. Over time, I gain skill in navigation, a knowledge of the terrain, and maybe even a more powerful engine to run faster and through thicker ice. But I know I will never crush all the ice. The goal is to never be locked in it, not to crush it all.

The icebreaker metaphor lets you forget without feeling guilty. Just keep moving and don't let the water freeze around you, and the rest will take care of itself.

Mantras 2.0

nospam@example.com (Konstantin Tovstiadi) — Fri, 12 Jun 2009 17:05:47 -0400

Today I am going to revisit some of my basic How to be successful on the web mantras:

form is nothing. content is nothing. audience is everything
Meaning you could have the best graphic design layout, but with sucky content it doesn't matter. And then you can have spectacular content, but if readers can't access it nor contribute to it, it is a waste of time, too.

for every good idea, there is already an implementation
You want a photoblog? There are tons of open source PHP / MySQL (or Python / Ruby / Perl if you prefer) applications available. Want Flickr exports? There are already plugins that do that. Want to change the layout? Find a theme that would do it. Do not write new code unless you absolutely have to.

if your solution requires manual data entry, find another solution
This is really a variation on the previous rule of no custom code development. Unless you have some really unique data, the stuff that you need is already somewhere, typed up and ready to go. Find a way to acquire it.

the wider the user base, the better the product
Works this way with trails - come to a fork on a trail, not sure which one to take - always take the more trodden one (metaphorically of course the opposite is true, but that's a different story). If choosing between two platforms, pick the one with the larger user community - more people to report bugs, fix the code, write plugins, answer your questions on forums, and develop custom themes.

you gain by giving away
Putting your work online - not all of it, not in full resolution - but enough to give people a flavor of what you do and make them want to eat more - is the quickest, cheapest and the most flexible way to advertise your talents. Plus it makes you look cutting edge - and actually does make you cutting edge. Even if you do a little bit of it, you are still ahead of the game - even in 2009. An amazing number people still consider it unnecessary, too complicated, or worst of all, below them. Getty Images sneered at the microstock model when it appeared. Then they re-thought it and bought the most successful microstock agency. Now that business is pulling them through the crisis as the traditional macro stock market is dwindling and tightening.

Stealing ideas

nospam@example.com (Konstantin Tovstiadi) — Sun, 07 Jun 2009 14:26:46 -0400

I hear it all the time around me in the academic world. "I am not going to tell you what I am working on". "We won't share the custom code we have developed". "My bibliography took me ten years to compile, like hell will I put it online". Or better still, "I put my favorite photo online and so-and-so took it and used it on their site without giving me credit". "How do I put my papers on my site but prevent people from cutting and pasting my stuff?"

There are several false assumptions here. The truths are,

1. Most likely nobody gives a fuck about your ideas. Just because you saw them re-used somewhere doesn't mean they are unique or valuable. You just happened to be on the first page of Google search for a few days.

2. The best way to capitalize on your ideas is to make them public, not to guard them like a dragon with a pile of gold. You gain by giving away (more on that later).

3. Once you have put something online, consider it public property. If it is online and if it is any good, somebody will re-use it without your permission eventually. You are better off accepting it outright and smiling every time you see it happen. Put an image up as a CSS background, and somebody would look in the style sheet and copy it from there. Hide pictures in a Flash animation, and somebody will take a screen shot and steal it that way. If you don't want to share it, don't put it online.

Cartoon stolen from Hugh MacLead of gapingvoid.com

Why is it pointless to protect your data? Well, there is an obvious reason - sharing means feedback, feedback means improvement. The less obvious reason - dummies won't understand it, smart people will steal it anyway. Even if you don't explain how you did something, clever people with similar knowledge will look at the result and reverse engineer the process. Those who are not capable of doing that, are usually not capable of understanding it even if you give them the full detailed specs. If somebody gave me the full die specifications for the upcoming Intel Nehalem 8-core chip, I would gain nothing from it. It would be all Greek to me.

Your data - your photographs, your manuscripts, your ideas - serve as ads for your abilities if you put them online. Let as many people as possible see them. They are the manifestations of things no one can steal from you - your experience, your skills, and your networks.

It doesn't matter if you have access to data. Yes, you can get a million search results on Google, big deal. Do you have the experience to ask the right questions? Do you have enough experience to tell bad answers from good answers?

It doesn't matter if you have the right answers if you don't know what to do with them next. Full processor specs are useless if you don't know how to run a 45 nanometer production line. You may have a Canon EOS 1Ds Mark III with a 7 thousand dollar lens and sill take a God awful picture.

And you may have the best knowledge and skills, but nobody to hire you to do the job. Knowing everything has no value unless you know everyone. Putting ideas online, having an avant-garde, a first-strike battalion of your thoughts on the web, lets you spread the knowledge about what you can do. The benefits of that far outweigh the risks of seeing that cartoon you drew on somebody else's blog.

The known unknowns

nospam@example.com (Konstantin Tovstiadi) — Wed, 13 May 2009 11:39:28 -0400

I used to waste hours of my life playing The Batte of Wesnoth, a strategy conquest game. Some of the maps in the game are hidden: you start with a tiny part of the terrain open and visible, the rest covered in dense fog. Enemies jump out of the fog where you least expect them. As you move your troops, the explored terrain grows, more of the map becomes visible. Yet there are still too many unknowns to predict the flow and the position of new enemies. And then at some point - while a sizable part of the map is still covered in fog - you just know it all, and the enemy does not stand a chance. For practical purposes, a clear view of maybe three fourths of the terrain is as good as the complete view.

And that reminds me of the famous complexity theory thought experiment - imagine a thousand beads on the floor. They are disconnected. You take a piece of string, randomly select any two and connect them. Then you repeat the procedure again and again, until small clusters start forming - 2 beads here, 3 here, 4 over there... And then you reach a point when almost every bead is in a cluster, big or small; but there are still dozens of clusters. And finally with just a few more iterations everything coagulates into one giant cluster. You have not at all exhausted all the possible connections between the beads, far from it; yet everything is connected, albeit not necessarily directly, to everything else. That transition is not unlike the Eureka moment of playing the game and all of a sudden "knowing" the whole map.

So what kind of "knowing" are we talking about? What does it mean to say that somebody knows their stuff? Well, we only make knowledge claims about patterned phenomena. Nobody in their right mind would claim knowing a sandpile, a primarily chaotic system. Up in the Bezengi area of the Caucusus mountains is the 18 kilometer long Bezengi glacier, the longest in the Caucasus. To get to the base camps for many of the climbs, you have to walk up the glacier for about ten kilometers. It is very monotonous; everything looks the same the whole time, especially to a new comer. There is no discernible pattern. Part of the glacier may collapse overnight, shifting the optimal path a hundred yards to the left. It is hard to "know" the glacier.

But even there you can get to a point that you run up and down the glacier in dense fog without a compass and know exactly where you are going. You have learned to recognize the subtle patterns of the system, same as you would "know" a big city. You don't really "know" every street, every house, every station; but with enough knowledge about some of them (like the beads connections reaching a transition) you can predict the rest of them and find them with an almost 100% accuracy. It is almost like you have latent knowledge of every node in the system, based on the positions of some of the nodes and the overall structural principles of the system. This is what Confucius was talking about it when he said, only those who can imagine the three other corners of a quadrant when shown one corner are worthy of being a student.

For years and years and years I have been pushing myself to learn more about information technology. It always seemed like a giant map covered in fog, with me standing in some corner of it blabbering about PHP and CSS. Yet yesterday I realized how far I have come already. I had my "beads-in-a-cluster" / "this glacier has a pattern" moment.

I don't need to see the whole map. I feel (=know) the rest of it already, without having ever seen every element of it.

University of Texas at Austin lecture: Online Research Collaboration Tools

nospam@example.com (Konstantin Tovstiadi) — Fri, 17 Apr 2009 03:04:16 -0400

On April 17, 2009 I am giving the Jesse H. Jones Centennial Chair Lecture at the School of Journalism at the University of Texas at Austin. This page contains links to the resources mentioned in the lecture. You can access this page directly by following a shorter URL.

utexas.mikt.net

An attentive listener

Software links

Zotero – a Firefox plugin for personal bibliography management
EndNote – commercial package for personal bibliography management with catalog linking functionality
Connnotea – online portal for personal and shared bibliographies
Cite U Like – a portal similar to Connotea; owned by a commercial publisher
WIKINDX – open source software package for personal and shared bibliographies; needs to be installed on a server
Aigaion – another open source package similar to WIKINDX
CWIS - a digital library software package from the National Science Digital Library

Other resources

Online bibliographies 1 - an early blog post from me about the subject
Online bibliographies 2 - a more recent update
Monster searches with EBSCO host - a blog post describing how to build complex Boolean searches with multiple blocks using EBSCO databases
Creating large scale online bibliographies - a work document used by us at CRCM for harvesting citations. Tailored for the OU IT environment, but could be useful elsewhere - especially if you are an EndNote user.
A blog about bibliographic tools by Kjell Fauske - an extensive summary of local and shared solutions

gapingvoid.com - Hugh McLeod's site, the source of the cartoons used in lecture slides
How much data is that? - a set of examples of different amounts of data, from the smallest to the largest
The End of Theory - a thought provoking article about science in the petabyte age

Lecture slides

A PDF file from the lecture