Showing posts with label peer review. Show all posts
Showing posts with label peer review. Show all posts

Wednesday, 24 October 2007

Academia vs Wikipedia...again

I thought I'd share this rather fabulous rant by Thomas on Techyum with you. Thomas is incensed about this article that appeared in New Scientist about Wikipedia, particuarly noting that some professors are less accurate in what they say than Wikipedia, which in my experience, is fair comment.

There has been phenomenal debate around the worth of Wikipedia, and most of the key elements of the debate can be found on Wikipedia itself. Indeed, in one of its own articles, Wikipedia itself notes that

Wikipedia acknowledges that it should not be used as a primary source for serious research.

but its value as an encyclopedia for me is endless. There is no way I could have written a thesis on the use of social media had I not had access to the definitions therein. Current information on the the terminology used in the area of social software use just wasn't available to me in books, and relevant peer reviewed articles were few and far between, in obtuse articles not held by my university.

Barry Leiba reviews research papers, and has kindly cited some examples of where Wikipedia shouldn't be used in research papers. His greatest bugbear appears to be the fact that references cited may have changed by the time the reviewer checks the reference, which is fair comment. This is mitigated however by refering to the precise version. David Gerard's comment on the same post explains how -

Click on the "History" tab and you'll see every version inthe edit history. Whereas the version at http://en.wikipedia.org/wiki/Inflation may change, the version at http://en.wikipedia.org/w/index.php?title=Inflation&oldid=162022107 (14:46
UTC, 3 October 2007) will not.


Wikipedia are attempting to get around the problem of peer review, with, suprise suprise, a peer review system...based on "trusted" sources. To earn this trusted status, users will have to show some commitment to Wikipedia, by, for instance, making 30 edits in 30 days.

This seems very much a quantity over quality strategy, but I'm sure its more robust than that (!). New software will also be used to give a "trust score" to contributors based on whether their edits are changed or not. This means that any topic in which there is healthy debate and disagreement will therefore be a no go as edits and changes will reduce trust scores.

In terms of academically accepted definitions, maybe the solution is for the Oxford English dictionary say, to produce a free online dictionary which takes contributions from us mere online mortals, to speed up the process of adding new words. As it is, you have to subscribe to even view the Oxford English dictionary, it takes an age for new words to be included, and right now, there's just no real substitue for Wikipedia.

Thursday, 26 July 2007

Wikipedia, community self-regulation and academic peer review

I've just been reading an interesting post by Seacat (Cass Nevada) demonstrating the machinations of the Wikipedia editing collective in relation to the entry on Enterprise 2.0, which has been through a long and intense process to ascertain whether the term really is "a dubious neologism".

Of Wikipedia, Seacat notes that
"The system works because it is incredibly porous. Input, output, the chorus of voices, the rag-tag team of determined editors, all keep the information and the channel incredibly vital and alive..."
Despite what I said about the term Enterprise 2.0 in a previous post, I feel for Andrew McAfee, who has experienced first hand the editing process for Wikipedia entries, which displays hugely energetic intellectual debate, and as he has recognised, isn't cuddly (see this Harvard Business School report for more on the whole episode).

George Siemans remarks on his blog that he has experienced a similar vetting process, this time by the academic community, who are making it difficult for students to reference his (in my opinion fabulous) book Knowing Knowledge.

He's been in conversation with Masters student who
"...stated that her panel felt that the theory of connectivism and the book Knowing Knowledge had not been subject to peer review."

No doubt I will suffer the same dilema as the aforementioned Masters student, as I have quoted widely from the book in my own dissertation, primarily in relation to the theory of Connectivism

I have no problem with this, as I feel I have a good case for including material not considered to have been peer reviewed, as I believe, as does George Siemans, that it is an outdated concept. I thoroughly agree the argument he makes in his post:

"Peer review plays an important role - it is intended to provide expert critical review of concepts and ideas to ensure quality and accuracy. I'm all for that. My primary concern rests with "privilege only" accepted view of peer review. The progressive advancement of educational attainment (see OECD's Education at a Glance 2006) indicates a society increasingly capable of engaging in complex dialogue. The throne of knowledge is now a seat available to many of society's members. As such, it's reasonable to assume that the opinions of even those peripherally engaged in a discipline can provide insight and value. I appreciate experts, excellence, and established processes. But I despair when the processes of validation inhibit, rather than advance, thinking and idea sharing in a discipline."

Referencing Knowing Knowledge and the theory of Connectivism might lose me marks but it's worth it to make the point. After all, I'm not studying to progress my academic career, but to improve practice in my chosen field.

I've also been warned against using definitions from Wikipedia, which is, as far as I'm concerned, a valid source for the current use of terminology around web 2.0 and associated concepts. Thanks again to George Siemans, who pointed out the rather wonderful Wikipedia page entitled Errors in the Encyclopædia Britannica that have been corrected in Wikipedia which says it all really...

As for the concept of peer review, maybe it's time the academic establishment recognised that communities of online users really do self regulate effectively and started to reconsider their "priviledged" stance...and if they could do it quickly, before my dissertation hand in date, that would be really great (I don't really wanted to be a martyr :-)