Exploring the potential of Webpa

Well the first use of webpa has just about come to an end here.

Webpa was is open source, summative group peer assessment tool developed by loughborough uni amongst others. We are using a legacy version and have not connected it to the VLE. The tool is really well design and well constructed for the job that it performs. It uses a likert scale criteria and a single free text box for student justification of marks.

This was basically a road test making sure the system could get data in and out, which it has done very successfully. There are a couple of issues reported which were users and admin errors. 

Exploring the possibilities 

Interestingly, many staff who are interested in the tool want to use it as formative feedback to groups as well as final marks. They intend to run webpa a number of times during the group project. Trying to extend the webpa tool has so far involved using Excel sheets to create a mail merge of data back to the students. This involves organising all the feedback aimed at any particular student, creating an average fore the group on each criteria and linking this to individual average scores. This could be greatly improved by working at the SQL level within webpa, but at the me spent I simply don't have the time.

An obvious extension to this would allow groups to construct their own criteria, and time tabling. This would need to prohibit students to seeing each others results. I believe that this might improve engagement in formative feedback, allow greater group autonomy in the decision making process and cut staff workload. 

Cohort and group level health checks. Staff are interested in receiving overview reports of each group using the formative process. This would allow them to flag up issues when meeting the groups.Identifying dysfunctional groups might involved identifying the split of marks, looking for above normal distributions of marks, i.e. one or two students receiving very low marks compared to others. Also sub group marking patterns, i.e. when a section of the group are giving each other high marks and low marks for the rest. This would indicate factions within the group. 

I'm sure there are other ideas out there webpa has a lively community

Survey

I need to write a survey to gather information about staff use of group summative assignments. This would help me understand the level of interest and need for this tool. 

Q1. Have you used summative group work within your marking schemes.

Q2. How did the marking mark in terms of process. Eg give same mark to all students in the group.

Q3. What were the reasons for using summative group work within the programme

Q4. Did any issues occur for staff and students during this process

A more localised survey for students would be good to see what they thought of the formative feedback.

Q1. Did you read your feedback from webpa

Q2. Did the feedback change or influence your behaviour when relating to the group.

Q3. How would you improve the process of feedback within a group

So, a promising start, just need more an extra day in the working week to complete this!

What would a MOOC be like if the BBC made one?

This post is completely of the cuff. I haven't had a search around on this one, its just something that's been buzzing around in my head.

I've been taking a few MOOCs recently, lets face it who hasn't dipped there toes in to get a feel of the experience. Out this comes many questions, but one is about old media an new media, old broadcasting and new social engagement with media. It's not escaped anyone's attention the phenomena of tweeting while watching TV. It's Newsnight on BBC being a typical example. This is a new mass social experience of media and interaction. Lets not focus on the detail of the quality of that experience, because it is a live programme, the reaction to it on social media from the viewers has to be instant, which can make it difficult to sustain concentration and analyse at any great level. Old media and broadcasting is not in conflict with this audiences desires to interact, but it's far from perfect.

What would be better, is debate and discussion happened over a longer period, and the various streams of information were less distributed over different platforms and discussion spaces.

Now there are many design faults within the MOOC platforms yet to be solved, but the experience of listening and reacting to content is far slower and more contained that the example above. What would a MOOC be live if the BBC made one

1. Content would be far more creative. Like it or not, unis are not great at producing sophisticated content, the visuals are a little stayed, the joke awful, the budgets just a little too small. 

2. It would super inflate the interest in the general public of engaging in this kind of educational experience as MOOCs my provide. It's still a niche market, by the BBC have always had a remit to educate.

3. The chosen topic would be overwhelmingly populist, and disregard the academic approach to knowledge and fields of study

4. There would be a few errors in there, the BBC are getting better, but the academic rigour would just not be there

5. It wouldn't happen because of there fears of control over the discussion spaces, public broadcasting has distanced itself over the years from social media, particularly ones that encourage debate. A fall out this would mean, more quiz elements than discussion and losing the whole meaning of the process.

6. Certification. There is no university of the BBC, so would anyone else step in to offer a badge of the area being studied. Open Uni would be ideal, and I'm sure somewhere the discussions have gone on.
7. Licencing, open content is still very difficult for companies particularly the BBC to adapt to

Never the less it's an interesting idea. The gap between old and new media is growing, leaving most large media corporations desperately exploring ways to reconnect with some audiences.

But if we turn this on its head, if all a MOOC is, is a space for collective learning and a badge to follow, are there not other spaces where this is already happening. Is the MOOC just offering a poorer experience to the wider social Internet?

If it did happen, what would be the first subject, I'm thinking cake baking, how about you?

Ten years is a long time in learning technology?

 

This post is part of FDOL course, I read 2 of the suggested papers . . .

Online learning: it is all about dialogue, involvement, support and control- according to the research by Marion Coomey* and John Stephenson available here

Openness, Dynamic Specialization, and the Disaggregated Future of Higher Education, by David Wiley, John Hilton, available here

Reflections:
Cultures of Digital Research
Firstly the 2 documents are written in two very different styles, using almost opposite methodologies and styles. 'Online Learning' is seeking to support practice and identify benefits. It uses systematic methods to categories and collect benefits and 'tips' for practice from 100 other papers. It believes the gap between adoption can be filled with with these tips.

'Openness' by contrast feels bombastic, ideas are gathered from many places to give some kind of vision.
e.g.
"People are more connected to people, content is more connected to content, and systems are more connected to other systems than ever before."
You could say exactly the same statement at any time in human history, surely.
The future in this paper is something that has to be addressed, and change is massive and wholesale. Adoption of practice is replaced by a need for urgency to remain 'relevant'.

Both papers suffer from the weaknesses of their approach, and perhaps point to a larger weakness in the limited ways we have to describe and demonstrate learning technology and learning research. 

Relevance is mentioned 7 times in 'Openness' but there seems little discussion of what it is to be relevant. This theme is still with Higher Education, and the active experimentation that this paper calls for is one way of seeking the answer, but there are others. There needs to be a far great, deeper and louder discussion on the importance of the values that underlie the relevance we seek.

Wikipedia staff session

This post is about a session I am running with staff on Wikipedia. I have called it  . . .

Wikipedia. friend or foe

Title: Wikipedia – wisdom of the crowds or confederacy of dunces

This session will provide you with an overview of the wild and wiki world of the Wikipedians. It will dispel a few myths, discuss the recent research and explore the recent moves by academics to create Wikipedia assignments. There will be a little hands on editing of Wikipedia too.

If you can’t make the session – I recommend this paper

Learning Chronobiology by Improving Wikipedia

JOURNAL OF BIOLOGICAL RHYTHMS; AUG, 2012, 27 4, p333-p336, 4p.

This paper shows a possible way to incorporate Wikipedia within a summative assignment. It provides evidence of the motivational and learning effectiveness of the process and all the assignment details to allow others to use the same process. http://www.nslc.wustl.edu/courses/Bio4030/wikipedia_project.html

http://search.ebscohost.com/login.aspx?direct=true&db=edselc&AN=edselc.2-52.0-84864628701&site=eds-live

The aims also include the demonstrating the ways in which social media and many other forms of social interaction allow opportunities for teaching practice to engaage, either formally e.g. let's make an assessment out of this or informal, lets discussion, analyse and explore this together. In the session I intended to:

1. Start with introducing a wikipedian, describe how wikipedia works from their perspective, eg. workgroups, looking after pages
2. Then show a particular group of pages that are from their subject area. I'm in the arts faculty. One of the pages was created by the wikipedian just discussed http://en.wikipedia.org/wiki/2010s_in_fashion. I'll talk about the 'talk' area of pages and 'history' and their relation to discussion and debate. Also the ways in which wikipedians display areas of concern in the page, e.g citation needed, and other warning signs. 
3. I'll finish of with an example of assessment using wikipedia to hopefully lead into a discussion about innovation within teaching practice.

This is a very short session, so I might not get time to mention all the wiki research. What follows are some interesting links that have helped me prepare.

Facts and figures
From Townsend (2013) (apologies to wikipedians!!)

• 17M registered users.
• most are male, internet experts, childless
• 5000 of these are active users
• 1459 administrators within the system who are super users and step in to arbitrate over content
• Administrators can block users or limit edit rights to some pages. 0.05 articles are blocked
• 10% of authors – 90% of the acrtiles Ortega (2009, p. 106) in Shen (2013)

Research / Analysis

Rosenzweig's (2006) is the most notable study looking at the validity of the entries. 25 US historical figures where research in 3 different resources, eg two encyclopedias and wikipedia. Wikipedia was better than one and close behind the other. He critises the writing style of its choppiness.
But also see Nature 2005 article what compared accuracy and moved the debate from rejection to reluctant acceptance. Britannica had 3 errors per entry wiki 4

Shen (2013) conclusion is wikipedia is so huge power, control and hierarchy have to grow. Therefore it is not a true display of 'the wisdom of the crowds'

Systemic Bias - really interesting veiwpoint from a set of slides here

And the geographical location of page edits - http://www.zerogeography.net/2009/11/mapping-geographies-of-wikipedia.html 

References

Shen, XL,What leads students to adopt information from Wikipedia? An empirical investigation into the role of trust and information usefulness. BRITISH JOURNAL OF EDUCATIONAL TECHNOLOGY; MAY, 2013, 44 3, p502-p517, 16p. 

Roy Rosenzweig, "Can History be Open Source? Wikipedia and the Future of the Past", June 2006
Giles, Jim. “Internet Encyclopaedias Go Head to Head.” Nature 438, no. 15 (2005): 900–901.
Townsend, S. Wicked Wikipedia? Communities of Practice, the Production of Knowledge and Australian Sport History. International Journal of the History of Sport Apr2013, Vol. 30 Issue 5, p545 15p.

half and half - summative and formative quiz in one

Research in Learning Technology 2013; 21: 19153 - http://dx.doi.org/10.3402/rlt.v21i0.19153

Combining the formative with the summative: the development of a twostage online test to encourage engagement and provide personal feedback in large classes 
Susanne Voelkel





This paper explains a cycle of 3 years of different uses of online test technology.
Cycle 1 students have the choice of taking formative tests each we. Take up is small but learning gains appear to make it a useful activity.

Cycle 2 not surprisingly the author tries it as a summative  exam worth 20% of final mark. Take up is higher but learning gains decrease. Student feedback is also poor.

Cycle 3 a combination, students have to achieve 80% in a formative test in order to take the online test which are still worth 20%. Learning gains are the same as cycle 1, but number of attempts of formative have increased and student feedback is far more favourable.

Well worth a read

Wiki, Blogs and Discussion boards

I'm putting information together to a session on these tools. I had some old nots that took me to Warwick University blog site. You might remember, but when blogs first became popular Warwick were very quick to make blog spaces available to all students. The site is still going, but as you can see from the stats, it tells a story of the rise in interest in this technology and the decline caused mostly by Facebook dominating university campus communications.

Beautiful designed study into academics changing identities

So jealous, I've been looking for a good process for my doctorate for years now. And in my final write up I come across this. So simple but very effective, qualitative and quanta give. Oh poo.

Note to self do more background reading!!

Negotiating professional identities in higher education: dilemmas and priorities of academic staff.
Authors: Calvert, Mike
Lewis, Thérèse
Spindler, John
Source: Research in Education; Nov2011, Issue 86, p25-38, 14p, 1 Diagram
Document Type: Article
Subject Terms: *CASE studies
*UNIVERSITIES & colleges -- Employees
PROFESSIONALISM
EMPLOYEE empowerment
Geographic Terms: ENGLAND
Abstract: This article reports a case study of how staff in a new university in England are making choices about their use of time. Weekly time logs, small discussion groups and individual interviews reveal that the principle of 'service' is embedded in the professional identities of staff. The paper explores how this perspective is reinforced by notions of 'professionalism' that are encouraged within the community of practice. It indicates how keeping the narrative of service going can result in self-denial, with implications for reflection and scholarly activity. The paper concludes by posing questions about how staff can take care of their professional selves, as their work intensifies and resources fail to keep up with increasing government expectations of Higher Education. [ABSTRACT FROM AUTHOR]
Copyright of Research in Education is the property of Manchester University Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
ISSN: 00345237
Accession Number: 70101060

Persistent link to this record (Permalink): http://search.ebscohost.com/login.aspx?direct=true&db=ehh&AN=70101060&site=eds-live