Design Science Research

This article will give a brief introduction to “design science research” and how it relates to other design methods in Educational Technology.

The hardest thing with discussing design is choice of words. For the purpose of this article, I will adopt the following strict definitions:

Science is the creation of knowledge
Design is the creation of artifacts

Design Science Research [DSR] prescribes creative design and development of solutions to real problems as a way to build knowledge. In DSR, the results must be reported, and the report must contain what the design aims to do, a documentation of the design process and an evaluation of the processes and artifact. All this can be done iteratively and combined into one report.

If a design instance does not build knowledge, it is not DSR. Likewise, if research does not create a useful artifact, it is not DSR. It requires both components.

Whether a design is DSR or not depends on what qualifies as “building knowledge”. An educator may make a discovery while developing an online course that benefits all their future online courses. Their personal knowledge has certainly grown, but if they do not share that knowledge, can it be said they have contributed to overall knowledge?

In the same way, whether some research is DSR is difficult because “what is a useful artifact” must be answered. A research may develop an artifact, but if it is never used outside lab conditions, can it truly be considered useful?

Pasteur’s quadrant is a category defined by Donald Stokes (1997) where a research technique generated both useful artifacts and pure knowledge, as opposed to “Bohr’s quadrant” of pure basic research and “Edison’s quadrant” of pure applied research. All DSR techniques fall into Pasteur’s category of “Use-inspired research” as they both build knowledge and design a practical artifact.

[Corrected – originally stated that Pasteur’s quadrant was the way of determining researchiness/usableness]

DSR is a recognized methodology that can be adopted while doing EdTech research. It recognizes the value of creative development of an actual artifact.

DSR does not contradict design methods – the choice of design method is still down to the researcher.

Instructional Design in an Image

In a Google search for “Instructional Design Methods”, I think this image by Giulia Forsythe is very useful.

cc licensed ( BY NC SA ) flickr photo shared by giulia.forsythe

Unlike the “powerpoint boxes” that abstract away all the process, this image is messy, just like the reality of educational technology. It makes clear that a specific starting point not clear, though the weighting and positioning of the texts suggests a user centered start is best – who is learning, then what you want them to learn. It also provides a practical resource for finding new ideas, as encourage by Moonen (2002).

The image seems to focus on the left on traditional systematic delivery methods, and on the right more on constructivist techniques. This accurately reflects the hybrid of the two which is used, as Jonassen, McAleese, & Duffy (1993) describe, going from introductory material delivered traditionally to expertise through constructivism.

We can also clearly pick out elements of the ADDIE model – the analysis of users and tasks, the design of material and evaluation techniques, and the feedback. We see rough designs for the development and implementation, but no actual mention of them – perhaps as this image was from an article focused on designing web courses.

This image clearly focuses on a user centered design – WHO the users are is almost the first word a reader notices. Likewise, we see the emphasis on recursive and reflective experiences, with lots of complex feedback and loops between parts of the design.

It’s also interesting for me that this is teaching ‘teachers’ on important points to consider when using educational technology, and in it’s development, used these same processes. On Giulia’s blog you can see she asked for feedback, and this is actually a version 2 of the image with extra focus on feedback.

Sources

Moonen, J. (2002). Design methodology. In A.A. Adelsberger, B. Collins, J.M. Pawlowski (Eds.), Handbook on Information Technologies for Education and Training. Springer-Verlag, 153-180
Jonassen, D. H., McAleese, T. M. R. & Duffy, T. M. (1993). A Manifesto for a constructivist approach to technology in higher education. In Duffy, T. M., Lowyck, J. & Jonassen, D. H. (Eds.) The design of constructivistic learning environments: Implications for instructional design and the use of technology, Heidelburg, FRG: Springer-Verlag
Forsythe, G (2012). Planning an Online Course. Available online at http://gforsythe.ca/planning-an-online-course/ (retrieved 31.03.2014)

The Golden Bullet in Educational Technology

The elusive golden bullet- an educational technology that fulfills every goal. This theoretical bullet will solve every problem. The analogy would be a bullet that, when fired, would destroy every problem in the world without any negative side effects.

Rather than developing golden bullets, focus on your goals like the weapon designer would focus on a single target type. The first part of educational design is to pick out a suitable, singular target. Perhaps that’s a group of self motivated business people that want learn basic Chinese, or a government wishing to raise the average level of basic computer literacy in their teenage population.

Only when the smallest, roundest goal is defined should a solution be developed. Analogously, if the target is an old can, perhaps the solution is a rifle, if it’s a military bunker, perhaps it’s a cruise missile. Both serve different but valuable purposes, and both have radically different designs. A cruise missile for a can is expensive, and a rifle for a bunker is ineffective.

If you’re an educational technology developer, start by focusing on the goals you can reach your current capabilities – whether they are axes and swords or missiles and tanks. As your funding and skills improve, you may work steadily upwards, or you may improve at making axes quickly and effectively. Know your abilities and don’t aim for a goal that’s five miles away with a boomerang, or you’re only going to hurt yourself. (unless you’re good at catching)

Cafeteria Nipponica Recipes – A progression tree

I’ve been playing Cafeteria Nipponica on my phone recently, and found myself regularly checking Happy Enna’s Recipe Reference to work out how to do the things I “remember” from last playthrough in the best order. The information there is invaluable but spreadsheets hurt my eyes. So, I created these flow charts!

All dishes come from a “root” dish, you can find out how to get them here.

Colors indicate the max * ranking, keys on each image. I wonder why rice and soba are so lame?

Rice

Udon

Ice cream

Sauteed Chicken

Grilled Fish

Soba

Introduction to AI for game programming students

I was recently invited to give a talk on “AI in games” to Jyväskylä’s AMK game programming students. The lecture could have gone better if I had prepared more material for it, but the catch was that I didn’t know what to prepare. These notes act as a guide for “next time”. I won’t go into any depth on the subjects here.

Briefly introduce scientific AI, point out useful libraries [A*, speech recognition, opencv] then waft it away
Focus on architecture of players, representation of game state and move logic
Establish the simplest AI player for tic-tac-toe, then develop it to a perfect player [“smart as possible AI”]
Make the tic tac toe player more fun to play against
Consider expanding it to more complex games (chess, civilization) and optimizations
Human behavior simulation – know the limits of senses, program them in. FPS/Stealth AI is a good example. [“dumb as a human AI”]
Planning and decision making architecture
Techniques for more realistic behavior

Additionally…

Examples of AI in video games: planner in fear, neural networks aren’t used much in industry (notable exception, Supreme Commander 2) and only for decisions, Quake 3 doesn’t have neural net bots
Data structures and algorithms are ever useful
Observe nature and mimic behaviour

WordPress domain – Redirect loop problem

I recently changed my wordpress host to be, well, wordpress. I already owned this URL, bomadeno.com, and wanted to remap it. I followed the instructions here, but got a redirect error in chrome.

A little googling and this problem seems to be person specific, indeed there was no redirect loop in incognito mode. I went to chrome’s content and cookies settings and deleted all cookies for bomadeno.com and wordpress.bomadeno.com. Sure enough, the problem was fixed.

Troubleshooting the Database

A customer sent me a short email about formatting in their new system. Simple enough, I created a sample in the system and sent them a specification for formatting, and told them “just delete the sample when you’re done”.

Alas. Ten minutes later, an email arrives “I deleted the sample… everything is gone and I can’t log in?!”

Shock.

Shock fades into denial – it must be the platform. Right? A database failure, or a virus. Glancing at the database, it appears to have rolled back 3 months. I sent off a support request to Heroku and they pulled a friendly face of “that’s very odd, our databases don’t do that kind of thing on their own”.

So, I took a closer look, and sure enough there are suggestions it isn’t just a rolled-back database – a few missing things were created before a few present things, so it isn’t just a plain rollback, more like a systematic deletion. And the automated database backup hasn’t run since last night, so the most recent work by the customer is lost.

Despite it apparently being a bug in the delete method, no matter how hard I tried, I wasn’t able to reproduce it in the CMS locally or on production. I created almost exact replicas of the structure prior to the error, and couldn’t get a repeat of the bug – maybe there was something malicious at play?

I played around for hours, loading up content, deleting it, renaming it, trying to be malicious, and nothing came up. I pored over the logs from the client working on the CMS, and as I got further back in time noticed that they were testing content in the “client” side application as they added it.

I connected the client side application and worked through some exercises, deleted a few from the CMS. Nothing. Edited a few more things, completed a few more exercises, hit delete.

Long pause… “Signed out of CMS”. Ok… How odd. I couldn’t sign back in, either – user not found. I’m a system administrator, how can I be not found! This sounds exactly like the reported problem. There’s a lot to be said for examining the logs carefully – not just the immediate problem, but the surrounding context. Get a feel for what the user was doing when they encountered the problem!

Another database dump, and sure enough, my user account is deleted, along with quite a few (but not all) of the slides. Mixed feelings at this point – on one hand, I’ve reproduced the bug and am making some headway. On the other hand, the bug and subsequent data loss must be my fault – depressing.

At this point I wrote down the facts as I knew them

The user needs to use the application as well as the CMS for the bug to occur
A few User(s) are deleted
Many Exercises are deleted

Not much links users to exercises in the system. Just one little table called “UserProgress” which has a many-1 relationship with users and exercises, which entries are created in as the user progresses through exercises. A quick check of the database, and indeed all UserProgress rows are deleted as well. A lot of fingers now point at this relation.

Opening up the rails models, the UserProgress model has dependent => destroy set on it’s relations to User and Slide. On the other end, that’s also set – so if one end is destroyed, so is the other, regardless of which end is destroyed. This is not good. If a slide is destroyed here, it cascades – all the Exercise’s Progress relations are deleted, then all the Progress’s Users, then all the User’s Progresses, then all the Progress’s Exercises. We have come full circle – and it doesn’t have to end here… This explains why almost all users, and absolutely all slides vanished from the production database.

I created a simple test case and deleted a slide the user has marked progress on. The Bug is reproduced! This is a vital step in any troubleshooting. Another two unrelated deletion-of-progress bugs raise their heads – unrelated in the sense they don’t help fix the initial bug, but they take an hour to fix (they get in the way of running test cases for the show-stopper bug).

A quick removal of the offending dependent => destroy association from the UserProgress model (http://guides.rubyonrails.org/association_basics.html – deleting a customer deletes orders, which is sensible. Not the other way round!) and I am ready to re-run my test case.

Fixed! Hurray – off to the autobuild it goes.

Lessons learned:

Bugs usually clump together around bad code
Bad code is usually the less tested code
“System” level bugs that don’t occur until everything comes together require more complex tests
Be humble, expect the mistake to be in your code, not others
Don’t just throw answers together until they work. (i.e. putting the association property on both sides “just to be sure”)

Now on wordpress.com

Just shuffled this blog from a dedicated host to here on wordpress.com, so if you notice something awry just leave a comment and I’ll fix it up!

Delicious omelette

I just made a delicious omelette. I’m not sure which bit is the delicious factor, but I suspect it’s the occasional hit of sun-dried tomato amidst the fairly salty omelette.

To construct the artefact you need

Eggs (obviously).
A handful of black olives,
A handful of diced salami,
A few thinly sliced sun-dried tomatoes.

I seasoned with a sprinkle of garam masala, paprika, tiny bit of balsamic vinegar and a little (very little!) salt. Salt might be completely unnecessary given the salami.

Then did roughly this

Whisk the eggs till they’re a consistent color (not much effort needed), sling in the other bits, and cook on a medium heat (not hot!) till it’s pretty firm.
Turn it using a plate inversion technique and cook for a minute or so more (don’t use a spatula or you have to clean up the kitchen like I now have to).
Eat it. I would upload a picture but I already ate it. (the omelette, not the picture) Maybe next time.

Using the Redmine Kanban

We’re working on a small project and wanted to see how well the Redmine kanban works – it fell to me to set it up.

I started by followed the instructions from the Kanban plugin website, however, it requires a few extra steps to get it working at all:

Install the gem block_helpers (gem install block_helpers)
Read the README.rdoc that comes with the project – to save you some time here it is in short – you need to go to the administration page in Redmine, then configure the kanban plugin. There, you need to configure the pane settings – any panes you want to use must link to a status within your own redmine – there’s a list of recommendations in the readme but it’s just common sense.

Now, how does it fare in use? It unfortunately has a showstopper bug – moving something twice on the kanban logs you out. In addition to that, some users were unable to see the kanban. This means that the kanban plugin as it stands, is unusable for anything but an ‘overview’ of the projects (and an non-interactive overview at that).

Seeing as development seems to have ended a while ago, this plugin isn’t worth installing, and we have removed it from our Redmine setup. If it resumes active development, I will do a more full review of the plugin.