Monday, April 25, 2011

Supporting Debate

Democracy, rule by the people, is a challenging goal. It's not practical above the village level to have an entire population gather to discuss a topic. Even at the village level… even at the individual level, gathering and analyzing evidence and tracing out the most likely consequences can push one's resources to the limits. Group dynamics tends to overwhelm any attempt at rational dispassionate consideration. Nowadays on the various internet discussion groups it has become all too obvious how quickly noise can swamp information.

Over the centuries, societies have worked out methods to filter out noise and bring information to the surface, to enable better decision making. Various forms of representative democracy and rules to structure debate yield decisions that surely remain imperfect but still in general vastly superior to mob or dictatorial rule. Representatives can be chosen by popular vote, as is usual for legislatures, or chosen more randomly, as with jury selection. I've read that some ancient Greek cities chose their legislators at random.

What interests me here are the new possibilities opened up by the internet. Certainly the internet already supports group decision making in many ways. Many retail sites give customers the space to offer and read ratings and opinions about products. Political discussion boards are of course filled with debates. The discussion behind Wikipedia articles is also fascinating: somehow that discussion needs to come to a conclusion, at least for the moment. I would like to sketch out here a rough idea for a system to support on-going debate on the internet.

The skeleton of the system is a set of propositions. These propositions should be statements about community affairs that might plausibly be true or false. The propositions themselves should be kept separate from the various arguments for or against them. However, a proposition might assert something about an argument or about a relationship between arguments, or about other propositions. A proposition should not, however, merely assert the truth of another proposition or the validity of some argument: such propositions would be redundant.

The flesh of the system is then the set of arguments. Each argument must support or oppose a proposition. Arguments can refer to various other proposition, some of which might be used as support, while others might be dismissed as invalid or irrelevant.

Users can then vote on propositions, supporting them or opposing them. These votes can be retracted or reversed at any time. Users can also maintain lists of the arguments for and against a proposition that they find most persuasive. A user can maintain such lists even without a current vote on the proposition.

Along with tallying votes, the system can track changes and inconsistencies in the relationship between propositions. For example, a popular argument in support of one proposition might rely on a second proposition as support. If the popularity of that second proposition declines significantly, then users could be alerted to review their support of the first proposition.

Similarly, if a widely accepted proposition asserts a logical relationship amongst the truth values of a set of other propositions, and the voting for those propositions is becoming less consistent with that logical relationship, then users could be alerted to that inconsistency.

The system would also maintain relationships between users. Users might frequently be found on the same side of the same propositions, or on opposite sides. They might find similar arguments convincing, even if they ultimately reach different conclusions on the propositions themselves. These relationships between users can then be used to cluster users along various dimensions, using methods such as principal component analysis.

Once users have been classified, then the voting on various propositions can be analyzed on the basis of that classification. More primitive analysis is also possible; for example, users who vote for this proposition tend also to vote against that proposition, etc.

The group decision making mechanisms supported by the system could also be used to maintain the system. For example, redundant propositions could be combined, complex propositions could be split into simpler components, and frivolous or abusive propositions could be deleted. Each of these operations could be suggested as a proposition which then could be argued. Judgment would generally be required to move from voting results to action: for example, users with a long record of involvement should generally be given more consideration than a flood of new users who have yet to establish trustworthiness. The goal of this system is not to automate judgment, but to support it.

The idea here is to establish a system something like Wikipedia. Wikipedia attempts to present a consensus picture of the way things are. This system, in contrast, has the goal of presenting the best arguments for and against the various alternative descriptions of how things are. The goal here is not to settle debates, but to filter out the noise and to highlight the key issues.

7 comments:

  1. I've had a similar idea floating around my head for about 20 years. I think it started back at IBM where they had some discussion forums covering really interesting topics. One I remember is a discussion about whether or not a computer could every behave like a human brain.

    There were several positions that emerged, but the arguments tended to get lost in the archives and then repeat. The same arguments would resurface, sometimes clearly, sometimes not so clear. Overall there was really not much advancement in the ideas because the best arguments were lost in history and the technology didn't support a robust debate.

    I've always wondered about how to create a site where the strongest arguments on each side of an issue could float to the surface and be preserved. My thinking on the topic usually ends up with a human moderator to identify the strongest posts and move them to the top. The idea of incorporating a good voting system could eliminate the need for the moderator. Challenges would including keeping the voting fair, free from spam etc.

    I'd love to see the idea move forward.

    I think of controversial topics like abortion, evolution, atomic power, whatever. As you've suggested, the system would not need to resolve the debate, but could capture the best arguments in favor of each position. Such a system could be an amazing resource. Newcomers to the debate could be presented with the best arguments for each position. The best supported, most clear, most easily understood arguments developed by people over time would move to the top. The key to me is to keep the different positions separate so that the strongest arguments in favor of each position can be preserved. Rebuttal is important too, but it should be possible to read through all the "pros" without having to be interrupted by the "cons." Perhaps links to opposing arguments could be integrated in there somehow.

    Rather than relying on headlines and sound bites people could quickly have access to the best arguments on all major positions when developing their own opinions. This could be an amazing resource for really advancing debate and decision-making.

    Count me in to participate - maybe even to help out.

    ReplyDelete
  2. Thanks much, Richard, you have the spirit of the idea precisely. Thanks too for bringing up rebuttals. It does seem like arguments need to refer to each other somehow, to reinforce or rebut.

    Usually when I have sketched this system out for myself, it quickly gets far too complex. Finding the right balance, enough structure but not too much, to be a container for unbounded complexity without getting tangled up in the complexity... I think it ought to be possible!

    I'll keep you in the loop... it'd be grand fun to make something happen!

    ReplyDelete
  3. I wonder, too, Richard, if we actually developed this idea together twenty years ago. Our offices were next door to each other! And we certainly took part in some intense computer-mediated debates in those days! I've been thinking about this scheme for many years too, though I don't remember the specific origin.

    ReplyDelete
  4. I'm looking a little at various content management systems that I might be able to use to build this. WordPress doesn't look powerful enough, at least this is out of their target zone. Drupal, on the other hand, looks like a good fit.

    ReplyDelete
  5. One good preliminary exercise is to study folks that have done similar things already, e.g.

    http://debategraph.org/home

    ReplyDelete
  6. Here's another:

    http://www.argunet.org/working-with-argunet/

    ReplyDelete
  7. Here is a list of tools:

    http://wiki.lesswrong.com/wiki/Debate_tools

    ReplyDelete