# Designing an experiment design with the support.CEs package: comparing designs (2)

Today I will follow up on designing choice experiments using R by exploring further the support.CEs package develoepd by Aizaki (2012).

Aizaki (2012) has functions to generate both unlabeled and labeled designs. Unlabeled rather than labeled designs are of interest to me, but the syntax is very similar, so I will only generate unlabeled designs. The functions developed by Aizaki (2012) are based on orthogonal main-effect arrays, which is no longer the most common way to generate experimental designs. Nonetheless, the functions are so rich that, if the researcher is contemplating a choice experiment with very few attributes and levels, it is worth learning.

Let me do exactly this: produce a choice experiment that is unrealistically simple. I will use a similar experimental design to that from the previous blog post, which tried to replicate Duran et al. (2015)’s. Let’s say we have two attributes: number of employed fishermen and the cost of a programme to keep them in employment at the fishing sector. The attributes and levels are described in the table below:

Similarly to our first design in the previous blog post, I want a choice card with two alternatives and a status-quo option, wherein the individuals would not have to pay. So I set the number of alternatives equal to two.

The code to generate such a design is (using the rotation method):

``````design_1 <- rotation.design(attribute.names = list(FISHERS_N = c("300", "600", "900"),

COST = c("5", "10", "20" )),
nalternatives = 2 ,
nblocks = 1 ,
randomize = FALSE )
design_1
``````

And the ouput is as follows:

``````> design_1

Choice sets:
alternative 1 in each choice set
BLOCK QES ALT FISHERS_N COST
1     1   1   1       900   10
2     1   2   1       300   10
3     1   3   1       900    5
4     1   4   1       300   20
5     1   5   1       600   10
6     1   6   1       600    5
7     1   7   1       900   20
8     1   8   1       300    5
9     1   9   1       600   20

alternative 2 in each choice set
BLOCK QES ALT FISHERS_N COST
1     1   1   2       300   20
2     1   2   2       600   20
3     1   3   2       300   10
4     1   4   2       600    5
5     1   5   2       900   20
6     1   6   2       900   10
7     1   7   2       300    5
8     1   8   2       600   10
9     1   9   2       900    5

Candidate design:
A B
1 2 2
2 1 2
3 3 2
4 2 3
5 3 3
6 1 1
7 3 1
8 1 3
9 2 1
class=design, type= full factorial

number of blocks = 1
number of questions per block = 9
number of alternatives per choice set = 2
number of attributes per alternative = 2 ``````

This experimental design consists of 9 choice cards, with two alternatives in each.

After running this design, we can use the function questionnaire to visualize how the design would look like, as such:

``````> questionnaire(choice.experiment.design = design_1)

Block 1

Question 1
alt.1 alt.2
FISHERS_N "300" "600"
COST      "10"  "20"

Question 2
alt.1 alt.2
FISHERS_N "600" "300"
COST      "5"   "20"

Question 3
alt.1 alt.2
FISHERS_N "600" "600"
COST      "10"  "5"

Question 4
alt.1 alt.2
FISHERS_N "300" "300"
COST      "20"  "5"

Question 5
alt.1 alt.2
FISHERS_N "600" "300"
COST      "20"  "10"

Question 6
alt.1 alt.2
FISHERS_N "300" "600"
COST      "5"   "10"

Question 7
alt.1 alt.2
FISHERS_N "900" "900"
COST      "20"  "5"

Question 8
alt.1 alt.2
FISHERS_N "900" "900"
COST      "5"   "10"

Question 9
alt.1 alt.2
FISHERS_N "900" "900"
COST      "10"  "20" ``````

So, in question 9, the respondent has to make a choice between a program costing 10, 20 or zero euros, and employing 900, or the status quo number of fishermen. I can already see a problem with this: a respondent should never choose to pay more to obtain the same level of employment for the fishing sector. This design works, but it may produce redundant choice cards.

Now, I show the problem of having a more “standard” choice experiment. In this design, I have six attributes (including the cost attribute) and several levels within each:

Using the rotation method, the resulting experimental design is:

``````> design_1 <- rotation.design(attribute.names = list(FISHERS_N = c("300", "600", "900"),
+                                                    FLAG_PROJ = c("0", "25", "50"),
+                                                    VESSELS_N = c("207", "400", "600", "800"),
+                                                    FESTIVALS = c("0","1","3"),
+                                                    KNOWLEDGE = c("low", "medium", "high"),
+                                                    COST = c("5", "10", "20", "35" , "60", "95")),
+                             nalternatives = 2 ,
+                             nblocks = 1 ,
+                             randomize = TRUE )
The columns of the array have been used in order of appearance.
For designs with relatively few columns,
the properties can sometimes be substantially improved
using option columns with min3 or even min34.

>
> questionnaire(choice.experiment.design = design_1)

Block 1

Question 1
alt.1    alt.2
FISHERS_N "300"    "300"
FLAG_PROJ "0"      "0"
VESSELS_N "800"    "800"
FESTIVALS "1"      "3"
KNOWLEDGE "medium" "high"
COST      "95"     "20"

Question 2
alt.1  alt.2
FISHERS_N "600"  "600"
FLAG_PROJ "50"   "25"
VESSELS_N "600"  "400"
FESTIVALS "1"    "0"
KNOWLEDGE "high" "low"
COST      "60"   "60"

(...)

Question 71
alt.1    alt.2
FISHERS_N "600"    "600"
FLAG_PROJ "0"      "25"
VESSELS_N "400"    "400"
FESTIVALS "0"      "3"
KNOWLEDGE "medium" "high"
COST      "5"      "95"

Question 72
alt.1  alt.2
FISHERS_N "600"  "600"
FLAG_PROJ "25"   "0"
VESSELS_N "600"  "600"
FESTIVALS "3"    "0"
KNOWLEDGE "high" "medium"
COST      "5"    "95"  ``````

The resulting design includes 72 choice cards, wherein not all choice cards make a lot of sense. For example, in question 72, the respondent is asked to choose between two alternatives (and status quo) wherein the second alternative is more expensive and worse at the same time (lower number of festivals and lower number of projects). Again, these types of designs may yield redundant choice sets.

Overall, I think this package is great if the researcher has very simple choice experiments. After getting an experimental design, one can by-hand inspect and delete choice occasions that are not plausible, thus decreasing the cognitive burden for the respondent. For more complex designs, it is worth exploring other options, for example using NGENE software or more recent R packages such as the idefix package. I hope to illustrate how to use the latter in a future blog post.

References:

Aizaki, H. (2012). Basic functions for supporting an implementation of choice experiments in R. Journal of statistical software50, 1-24.

Durán, R., Farizo, B. A., & Vázquez, M. X. (2015). Conservation of maritime cultural heritage: A discrete choice experiment in a European Atlantic Region. Marine Policy51, 356-365.