A map of Design Science Research

The images presented here are the focus of this article. I will summarize what I got from the images, and why I think they’re useful.

The ‘finished’ image roughly positions various Design Science Research techniques on two dimensions of continuous space.  The X-dimension approximates the levels of participation in the design process between experts (to the left) and users (to the right). The Y-dimension approximates whether the approach is led by research (to the bottom) or by solving problems (to the top).

A research question can be approached with techniques from anywhere on the map – indeed if we had infinite budgets, every research question could be approached with all of the techniques. With finite resources, how effectively a technique gets answers to a research question is important. I’d like to know if there is any research done on the heuristic effectiveness of techniques for solving various problems – for example, is it better to tackle a new problem with expert or participatory mindset. I imagine working closely with users will give an excellent feel for a new problem space, but that only experts will be capable of innovating in well-established problem spaces.

It’s interesting that the commentary on the article picks fault with the accuracy of the map. Researchers are a community, and at the moment the design research community seems pretty fragmented. There are clearly terms in circulation that are different, yet mean the same thing [design based research, design science research seem to be equivalent]. This map lays out one clear description of the field. It gives a concrete perception to discuss – rather than speculating, experts can ‘point’ say, “this looks wrong”. Perhaps it will eventually lead to a better standardization of terms, and a map that community experts all agree is correct.

The original article raises some interesting uses for the map, for me the most interesting one being to position your own research on the map, and then see if you can move your future research to a different area. Another use I see would be to repeat prior research with a technique from elsewhere on the map as a form of triangulation.

Based on article published in interactions – Designing games: why and how, Volume 15 Issue 6, November + December 2008 Pages 13-17

Design Science Research

This article will give a brief introduction to “design science research” and how it relates to other design methods in Educational Technology.

The hardest thing with discussing design is choice of words. For the purpose of this article, I will adopt the following strict definitions:

  • Science is the creation of knowledge
  • Design is the creation of artifacts

Design Science Research [DSR] prescribes creative design and development of solutions to real problems as a way to build knowledge. In DSR, the results must be reported, and the report must contain what the design aims to do, a documentation of the design process and an evaluation of the processes and artifact. All this can be done iteratively and combined into one report.

If a design instance does not build knowledge, it is not DSR. Likewise, if research does not create a useful artifact, it is not DSR. It requires both components.

Whether a design is DSR or not depends on what qualifies as “building knowledge”.  An educator may make a discovery while developing an online course that benefits all their future online courses. Their personal knowledge has certainly grown, but if they do not share that knowledge, can it be said they have contributed to overall knowledge?

In the same way, whether some research is DSR is difficult because “what is a useful artifact” must be answered. A research may develop an artifact, but if it is never used outside lab conditions, can it truly be considered useful?


Pasteur’s quadrant is a category defined by Donald Stokes (1997) where a research technique generated both useful artifacts and pure knowledge, as opposed to “Bohr’s quadrant” of pure basic research and “Edison’s quadrant” of pure applied research. All DSR techniques fall into Pasteur’s category of “Use-inspired research” as they both build knowledge and design a practical artifact.

[Corrected - originally stated that Pasteur's quadrant was the way of determining researchiness/usableness]


DSR is a recognized methodology that can be adopted while doing EdTech research.  It recognizes the value of creative development of an actual artifact.

DSR does not contradict design methods – the choice of design method is still down to the researcher.

Instructional Design in an Image

In a Google search for “Instructional Design Methods”, I think this image by Giulia Forsythe is very useful.

cc licensed ( BY NC SA ) flickr photo shared by giulia.forsythe

Unlike the “powerpoint boxes” that abstract away all the process, this image is messy, just like the reality of educational technology. It makes clear that a specific starting point not clear, though the weighting and positioning of the texts suggests a user centered start is best – who is learning, then what you want them to learn. It also provides a practical resource for finding new ideas, as encourage by Moonen (2002).

The image seems to focus on the left on traditional systematic delivery methods, and on the right more on constructivist techniques. This accurately reflects the hybrid of the two which is used, as Jonassen, McAleese, & Duffy (1993) describe, going from introductory material delivered traditionally to expertise through constructivism.

We can also clearly pick out elements of the ADDIE model – the analysis of users and tasks, the design of material and evaluation techniques, and the feedback. We see rough designs for the development and implementation, but no actual mention of them – perhaps as this image was from an article focused on designing web courses.

This image clearly focuses on a user centered design – WHO the users are is almost the first word a reader notices. Likewise, we see the emphasis on recursive and reflective experiences, with lots of complex feedback and loops between parts of the design.

It’s also interesting for me that this is teaching ‘teachers’ on important points to consider when using educational technology, and in it’s development, used these same processes. On Giulia’s blog you can see she asked for feedback, and this is actually a version 2 of the image with extra focus on feedback.

Sources

  1. Moonen, J. (2002). Design methodology. In A.A. Adelsberger, B. Collins, J.M. Pawlowski (Eds.), Handbook on Information Technologies for Education and Training. Springer-Verlag, 153-180
  2. Jonassen, D. H., McAleese, T. M. R. & Duffy, T. M. (1993). A Manifesto for a constructivist approach to technology in higher education. In Duffy, T. M., Lowyck, J. & Jonassen, D. H. (Eds.) The design of constructivistic learning environments: Implications for instructional design and the use of technology, Heidelburg, FRG: Springer-Verlag
  3. Forsythe, G (2012). Planning an Online Course. Available online at http://gforsythe.ca/planning-an-online-course/ (retrieved 31.03.2014)

 

The Golden Bullet in Educational Technology

The elusive golden bullet- an educational technology that fulfills every goal. This theoretical bullet will solve every problem. The analogy would be a bullet that, when fired, would destroy every problem in the world without any negative side effects.

Rather than developing golden bullets, focus on your goals like the weapon designer would focus on a single target type. The first part of educational design is to pick out a suitable, singular target. Perhaps that’s a group of self motivated business people that want learn basic Chinese, or a government wishing to raise the average level of basic computer literacy in their teenage population.

Only when the smallest, roundest goal is defined should a solution be developed. Analogously, if the target is an old can, perhaps the solution is a rifle, if it’s a military bunker, perhaps it’s a cruise missile. Both serve different but valuable purposes, and both have radically different designs.  A cruise missile for a can is expensive, and a rifle for a bunker is ineffective.

If you’re an educational technology developer, start by focusing on the goals you can reach your current capabilities – whether they are axes and swords or missiles and tanks. As your funding and skills improve, you may work steadily upwards, or you may improve at making axes quickly and effectively. Know your abilities and don’t aim for a goal that’s five miles away with a boomerang, or you’re only going to hurt yourself. (unless you’re good at catching)

Cafeteria Nipponica Recipes – A progression tree

Cateria, my second restaurant

I’ve been playing Cafeteria Nipponica on my phone recently, and found myself regularly checking Happy Enna’s Recipe Reference to work out how to do the things I “remember” from last playthrough in the best order. The information there is invaluable but spreadsheets hurt my eyes. So, I created these flow charts!

All dishes come from a “root” dish, you can find out how to get them here.

Colors indicate the max * ranking, keys on each image. I wonder why rice and soba are so lame?

Rice

Rice recipes
So much recipes. But whole team is babies! Also how does rice become french fries, or pizza, or salad? Mysterious.

Udon

Udon recipes
Udon! This all seems pretty logical.

Ice cream

Ice cream recipes
Ice creaaaaaaaaam.

Sauteed Chicken

Sauteed chicken recipes
Sauteed chicken obviously develops into steak!

Grilled Fish

Grilled Fish Recipes
Grilled fish sucks at first, but makes great end products!

Soba

Soba Recipes
Soba seems pretty lame :<

Introduction to AI for game programming students

I was recently invited to give a talk on “AI in games” to Jyväskylä’s AMK game programming students. The lecture could have gone better if I had prepared more material for it, but the catch was that I didn’t know what to prepare. These notes act as a guide for “next time”. I won’t go into any depth on the subjects here.

  1. Briefly introduce scientific AI, point out useful libraries [A*, speech recognition, opencv] then waft it away
  2. Focus on architecture of players, representation of game state and move logic
  3. Establish the simplest AI player for tic-tac-toe, then develop it to a perfect player ["smart as possible AI"]
  4. Make the tic tac toe player more fun to play against
  5. Consider expanding it to more complex games (chess, civilization) and optimizations
  6. Human behavior simulation – know the limits of senses, program them in. FPS/Stealth AI is a good example. ["dumb as a human AI"]
  7. Planning and decision making architecture
  8. Techniques for more realistic behavior

Additionally…

  • Examples of AI in video games: planner in fear, neural networks aren’t used much in industry (notable exception, Supreme Commander 2) and only for decisions, Quake 3 doesn’t have neural net bots
  • Data structures and algorithms are ever useful
  • Observe nature and mimic behaviour

Matt Bond's stuff

Follow

Get every new post delivered to your Inbox.