Per Ola Kristensson

The Facebook study and the precise purpose and scope of ethical review

July 9th, 2014

The recent Facebook study controversy highlights the discrepancy in ethical review requirements imposed by private companies compared to universities and other government-funded organisations. Companies only need to adhere to legal requirements while universities and other government-funded organisations must also adhere to ethical frameworks defined by for instance funding bodies. In some countries, university research ethics is even governed by legislation, which can lead to unintended consequences.

I do not think anyone is really surprised that companies perform research on human beings all the time in search for better products, more effective advertisements and so on, and that any ethical review of such research is left for the companies themselves to decide upon.

The point when the discrepancy is becoming problematic is when companies carry out traditional university research on human beings in their laboratories or via their commercial platforms and publish in the scientific literature but can choose to ignore ethical review requirements imposed by universities carrying out similar or even identical research.

One of the broader questions brought by the Facebook study controversy is the precise purpose and scope of ethical review. Let me sketch three potential purposes of ethical review.

First, is ethical review an essential process to protect human beings from harm? If so, it is nonsensical to limit it to university research and it would be reasonable to introduce a legal requirement that an ethical review board must approve all research on human beings regardless of where it is carried out.

Alternatively, is ethical review a process that serves to ensure the public remains reassured scientists are undertaking research in a responsible and thoughtful manner? In this case, it is reasonable to expect that scientific publishers, as representatives of the broader scientific community, demand evidence that research on human beings has obtained prior ethical approval before any results are published.

Finally, is ethical review of most research on human beings, perhaps in particular in human-computer interaction and social computing, too much of a bureaucratic burden given the minimal risks on participants? After all, an ethical review requirement has an opportunity cost: if the hurdle is too great, useful research may not happen in the first place, or it might be carried out in secrecy but never published.

In my mind, the big question raised by the Facebook study controversy is the precise purpose and scope of ethical review and why it is limited to research carried out at universities and other government-funded organisations. I don’t have a clear answer but my gut feeling is that university researchers are exposed to too much ethical review bureaucracy while industry researchers could benefit from a little more.

Posted in computer science, ethics, facebook | No Comments »

My second paper reaches exactly 100 citations

August 29th, 2012

On August 3, 2010 I wrote that my first paper reached exactly 100 citations on Google Scholar. Now my second publication, a UIST 2004 paper, has exactly 100 citations as well. Coincidently, we will host UIST 2013 in St Andrews next year.

Posted in acm, bibliometrics, computer science, hci, research, science | No Comments »

List of questionable open access publishers

July 11th, 2012

I have previously written about how some open access publishers spam researchers for submissions and demand high fees to publish papers and book chapters (here, here, here and here).

There is a website that attempts to list such open-access publishers:

Beall’s List of Predatory, Open-Access Publishers (2012 Edition).

As I suspected, the website lists both InTech and Bentham.

Posted in open access, research, science | 1 Comment »

IEEEtran.bst and the mysterious missing item

March 22nd, 2012

If you were naive like me and downloaded the IEEE conference template from IEEE’s website you may have experienced a puzzling error:

“Something’s wrong–perhaps a missing \item”

There is no point in trying to fix this as the reason this is happening is because the IEEE website gives you a template with an old IEEEtran.bst file. Go download the new version of IEEEtran.bst here from CTAN. Then delete all intermediate files (.aux, .bbl, etc.) and recompile. This worked for me.

Lesson learned: Never download IEEE templates from IEEE. Always use CTAN.

Posted in fail, ieee, latex | No Comments »

The Chronicle writes about open access spam journals

March 7th, 2012

I have blogged before about how low-quality open access “publishers” indiscriminately spam researchers for manuscripts for the most ridiculous journals and “edited books” (here, here and here). The peer-review and quality-control appears to be minimal and the publication fees are outrageously high.

Now The Chronicle of Higher Education has an article about how ‘Predatory’ Online Journals Lure Scholars Who Are Eager to Publish. It is excellent and very important this is brought to attention. There is of course nothing wrong with a desire to communicate research in the form of open access articles. However, having tax-funded researchers paying ridiculously high fees for someone essentially hosting PDF files without any real peer-review whatsoever is a huge waste of tax money. Even worse, having researchers passing these “publications” off as properly peer-reviewed scholarly articles is essentially a form of academic misconduct in my view.

Posted in open access, peer review, research, science, spam, spamming | 1 Comment »

The beginning and the end of the world

October 6th, 2011

Having moved to St Andrews about five months ago I felt I should try to read up on the town’s rich history. However, I felt that the first book I picked up to read was just too dry to be interesting (it started with the ice age…). What I missed was a story about the people of St Andrews. Surely a 600-year old university town must have some interesting stories to tell from its past?

Therefore I was delighted when I read a review in the Guardian of a recently published book called The Beginning and the End of the World: St Andrews, Scandal, and the Birth of Photography by Robert Crawford, a professor in the School of English at the University of St Andrews.

This fantastic book paints a vivid picture of life within the University of St Andrews in the 19^th century. It tells the tale of Sir David Brewster, an extremely distinguished scientist at the time and Principal of the United College in the University of St Andrews. This man, who among his many achievements invented the kaleidoscope, helped start the Literary and Philosophical Society of St Andrews. Despite St Andrews awkward location (it took about a day to travel from Edinburgh to St Andrews at the time), the Lit & Phil managed to attract individuals who went on to have a profound impact on both the British society and the world at large. Among other things, their ability to keep up with the research frontier on photography resulted in St Andrews becoming the first town in the world to be comprehensively photographed.

I very much recommend reading this fascinating book. It was a great pleasure to read this well-written and thoroughly researched account of 19^th century St Andrews.

Posted in book review, research, science, st andrews | No Comments »

More open access spam

October 6th, 2011

Relating to a previous post on academic spam it seems the spammers from inTech now have competition in the open access “book chapter” business.

I just received the following email:

Dear Per Ola Kristensson,

On behalf of iConcept Press, I would like to invite you to submit a paper
to our new book project under the working title “Handbook of Pattern
Recognition: Methods and Application”. The editor of this book is Khalid
Hosny. The book description can be found at:
http://www.iconceptpress.com/01/site/publication.viewPapers.php?publicationID=BK020A

We notice that you have some publications related to this book project. For
instance, we are interested in your paper entitled “Parakeet: a
demonstration of speech recognition on a mobile touch-screen device
(2009)”. (Please note that we are not asking you to republish this paper).
You are welcome to submit an extended/expanded version of this paper, or
any work that is related to this book. We expect each chapter has a minimum
of 16 pages.

All iConcept Press books are published as hard copy with ISBN and as open
access. For more information, please visit our web site:
http://www.iconceptpress.com.

Since we are open access, we would like to ask our authors to contribute
part of the publication expense. This publication fee might be
waived/reduced if: Most authors are from countries that are not classified
as high-income economies
(http://www.iconceptpress.com/01/site/info.lowIncomeCountries.php). Please
contact us for more information. The standard publication fee for each
accepted manuscript is USD$38/page for the first 16 pages and USD$18/page
thereafter. The corresponding author of each paid chapter will receive one
hard copy for free.

I sincerely hope that you would accept the invitation and that you support
the open access idea.

Next step:
1. Please inform us about your decision via email by: 18 Oct 2011
2. Submit a tentative abstract and title via our online submission system
(http://www.iconceptpress.com/01/site/process.register.php) by: 10 Nov
2011
3. Upload a full chapter by: 15 Jan 2012

Please free feel to contact us if you have any question.

Regards,

iConcept Press
Web: www.iConceptPress.com

Again, I wonder why I would pay for an open access book chapter? These chapters seem to undergo minimal (if any) peer review (I have never been asked to review any and I have never heard of anyone else being approached either) so all an open access book chapter amounts to is a PDF document hosted on a website. I might as well make the text available on the web myself. For free.

Posted in computer science, open access, peer review, publishers, spam, spamming | 9 Comments »

SLPAT 2011 workshop report

July 31st, 2011

Yesterday I attended the Second Workshop on Speech and Language Processing for Assistive Technologies (SLPAT) in the Informatics Forum at the University of Edinburgh. It was held in conjunction with ACL’s annual conference on Empirical Methods in Natural Language Processing. Last Thursday I had presented a paper on our work on AAC language modelling work at EMNLP. Therefore, even though I did not have anything new to present at the workshop, the SLPAT workshop was of high interest to me.

I have attended quite a few workshops before and my impressions so far have been mixed. Sometimes workshops fail to attract high-quality papers. Other times the organisation is in chaos. Very rarely do the workshop organisers manage to get everything right. Therefore I was very pleased to note that SLPAT 2011 managed to both attract high-quality workshop papers and deliver on an ambitious workshop schedule that packed the day with talks interwoven with user panels, posters and demonstrations.

Besides the talks, two other features stood out. First, the workshop had two ambitious poster & demo sessions. Demonstrations included a Lightwriter, a popular text-to-speech AAC device. I had never used one before and it was instructional to learn how it was designed. I also think it was a brilliant move to invite the AAC vendors Toby Churchill and Tobii to the workshop. I think researchers in AAC have a duty to try to interface with industry and government so that our research results can translate into actual benefits for end-users.

Related to this point, another feature that stood out was the AAC user panel. The organisers had invited AAC users to deliver prepared speeches about their reflections and ideas of AAC based on their own first-hand experiences with AAC devices. Thereafter the audience had an opportunity to ask questions. It was a very instructional session and also a reminder to me about just how much AAC devices can still be improved. I was also amazed by the eloquence of some of the talks delivered.

In summary, SLPAT 2011 was a highly ambitious and well-organised workshop. It was run just as well as a full conference, albeit for a smaller and perhaps more dedicated audience. I had a great time, I met lots of interesting people and I learned a lot. I will definitely try to attend the next workshop.

Posted in aac, acl, assistive techology, computer science, nlp, research, science, slpat, workshops | No Comments »

AIED 2011 conference report

July 12th, 2011

Queen Street, Auckland

I recently attended the 2011 Artificial Intelligence in Education (AIED 2011) conference in Auckland, New Zealand.

The main focus of the AIED conference series is various approaches to designing and evaluating intelligent tutors. An intelligent tutor is basically a computer program that uses AI techniques (such as planning algorithms) to teach students new concepts or techniques based on a model of what the students already know, or are supposed to know. The conference is quite old, this was the 15^th biannual conference. The conference is organized by the International AI in Education Society. It also seems that the AIED conference is liaised with the Intelligent Tutoring Systems conference (ITS) so that ITS runs every even year and AIED runs every odd year. Collectively ITS and AIED publish the vast majority of new research in the intelligent tutoring field.

I was at AIED to present a poster on the first steps towards designing intelligent tutors for text entry methods. The idea is to increase user adoption of novel text entry methods by using highly effective and engaging intelligent tutoring. I wouldn’t classify myself as an intelligent tutoring researcher so attending this conference was a good opportunity to get an insight into the field’s frontier.

AIED conference proceedings

The conference was held at the Faculty of Engineering at the University of Auckland’s city campus. The conference had a poster session, an interactive event, and three parallel paper tracks. In my opinion the overall organization of the conference and the program worked out very well.

The presented papers fell into three categories: systems, user studies, or data mining. I learned that the latter is apparently so popular that people have felt the need to start a conference on just that, the Educational Data Mining conference (EDM). I also found that machine learning is becoming increasingly popular in intelligent tutoring systems. Some of the machine learning techniques people used were POMDPs and Bayesian models. An interesting concept I didn’t know about before was Open Learner Models (OLMs). OLMs exploit the fact that while a student is using an intelligent tutor the system develops a model of the student. The idea behind OLMs is to figure out how to present the entire model (or aspects of it) to the student. Presumably this can aid the student’s learning.

Some of the papers presented intrigued me. In particular a paper on motivational processes seemed to point in good directions and another paper on confusing students (!) had led to some rather surprising results. I also found out that another active research area is authoring tools for ITSs, some which use techniques from the end-user programming field.

Overall I very much enjoyed going to this conference. It was refreshing to learn about an area of research outside of my comfort zone. Time will tell if I will submit a full paper to AIED 2013!

Posted in aied, computer science, computer science education, conferences, design, hci, research, science | No Comments »

Per Ola Kristensson | Blog