21w.789 field evaluation lecture notes


quick feedback — get mvp out within weeks of idea, can see things not see in lab — fit into daily life, creative uses of app

mvp should address specific question or test experience

social groups — if it’s social app, make sure alpha users are friends
public betas — can have feedback button
but this mvp proto — usu friends sending emails
people do not accurately report how often they do things — instrumentation is important –VCs look for number engagements, number of mins of engagements per day. and for our own understanding.

can have voicemail daily or weekly depending on length of study.

talk to them as quickly after use as possible

for story finding, call researchers right after find story, and parents call after get comms with children

if creating content, use content analysis

processing data — affinity analysis, statistical analysis if lots of data

good example: civic content — about cities and governments: what was style of story telling,  how long ago story occurred.

instrumentation and app analytics – features used or not (if not exposed to user) — see problem and then use qualitative method to understand why problem exists

power of combining = what and why people use system

facebook analytics team blog is very interesting. google is famous for this a/b testing.

this is a whole career. sifting patterns out of data.

einmann (sp?) elctures are very interesting.

contextual inquiry — h&r may be most relevant to this, where they watched people do taxes — improved online tax system to be more helpful. think aloud = can learn what people are thinking at the moment.

Experience Sampling Methods
how does all of britain spend their time? ?__? and how it changes over time (watch tv, read books). used to call people so have data for 50s, 60s (time on domestic acctivities has completely plummeted thanks to dishwashers etc). also studied location privacy / location fidelity –> learned likely no algorithm will solve this (e.g. buying a surprise gift etc.)

would people lie about locat? no usu. just say busy.

regardless should prolly always do semi-structured interviews
to understand where data comes from. vs structured have set list of questions
use specifics, not generalities

multicast for new updates — sent repeatedly
wifi access points –> drilling stadium walls to run wires
enterprise class phone — okay to drop on floor etc.
researchesr were the ones producing the updates
controlled access area — so can get phones back
initially thinking rental where people can rent phones during game
other researchers watch people using it
after goal, during midtime, after game = spikes in use
final cut pro -> cannot handle four goals in last ten minutes (fans expect <30sec latency)
^– focus on that. want different angles, commentary

created app now 5 secs to edit, <30 secs to user.
2005-06 data plans not common
made sure people recruited had unlimited texting
reading contact dbase hard back then
big battery vs. writing algorithm to optimize energy use
to users mental model -> wrong, locations versus large locations
get broad group as possible (altho may be difficult for this class) but for hci paper publishing
corporate research lab -> if paying them, they may be very positive to you, and talk about it to friends is much more telling
hours-long prototype
send text messages when people play song – init and artist/song
recruited close friends with diverse music tastes
(they primarily play music on desktop at time) yep primary devices

depend on country whether require payment as a corporation — can just be pizza
tell them –> getting paid no matter what, use it like any other app you downloada

the plan: daily email manual log, vs instrumentation of android (tech accuracy)
location sharing experience — prefrosh at CPW (ask them to email us / survey those who come to build party)