my brain is totally fried

right, saturday was workshops were i was upskilling and connecting with people on linkedin and sort of my coworkers

sunday — when i really started to dive into informatics. sunday was infinitely more exhausting, just way more people. interesting thoughts but no one to talk to about them, not possible to go to all the sessions. but i did learn yesterday (saturday at WINE, the women .. informatics… networking thing) you can swap between sessions during paper presentations, so i marked mine out carefully and have been going in and out of sessions.

sunday morning

morning i went to sessions on social media research. as usual i was late, insomnia and turning off my alarm and i find it really hard to stop ironing once i start but i’m also bad at it. but it was freeing that no one knowed or cared if i showed up on time, since my lab group went to alcatraz without me … first half of workshop was a bunch of paper presentations. afterward was group discussion. and at the end there was coffee break, so you could stick around and talk to people.

i learned through brief chat after: best practice is to contact a forum moderator and explain what research you want to do. if they don’t respond, tell university irb. university irb will likely say exempt since it’s social media research. prior big discussion on whether this is reasonable, observation that junior faculty are encouraged to use social media if irb approvals taking too long. note that irbs are overworked, so either have inter-institutional irb focused on this, or perhaps have subject matter experts to call in on these topics. dataset release: also occurs, and just run an off-the-shelf name stripper. (i stressed that names will slip through, but i guess… it’s… fine?)

i went over my idea of how to flip the legal power dynamic: companies agree to users’ privacy policy when we agree to theirs. incentive: perhaps just that

keynote

the keynote in the early afternoon was a really good crash course on theoretical ethics. no longer felt so obscure and hard to understand. i should start watching more talks.

–> differential privacy — the simplest example is randomized response. for a survey about behavior people might lie about, e,g, their social distancing habits, we can give people plausible deniability. tell them to flip a coin, if it’s heads tell the truth, if it’s tails, flip the coin again: if it’s heads, tell the truth, if it’s tails, tell the predetermined response. that default response is yes to illegal behavior, so you have plausible deniability. on the individual level, who knows; at the aggregate level, you can work out mathematically an estimate of the truth.

i’d heard of this technique long ago, but never realized it counted as a differential privacy algorithm.

throw money at it, aka bug bounties, and concatenate models for more accuracy without less fairness

bounty a: find subgroup with different outcomes. bounty b: if not only subgroup where outcomes different for current model, but also find model with optimal performance on that subgroup;

then you can just concatenate models! and don’t have to trade fairness with accuracy — strict (theoretically proven) increase in accuracy. if it’s in subgroup z then use model B, otherwise use model A. downside: do not know full model complexity ahead of time so risk overfitting.

bug bounties in traditional security: avoid adversarial interaction (propublica, facebook, compas) and shift to more collaborative interaction.

(will insert rest of summary of talk tomorrow)

sunday afternoon

i flitted between sessions.

demos: people’s screens were illegibly tiny, and i couldn’t get my camera to zoom enough. next time i’m bringing binoculars or a phone camera binocular attachment. one presentation was mostly a webapp llm, but the enthusiasm was catchy. (the topic was connecting pregnant women to sources of info). little practical tidbits, like they tried conversation history but the llm tended to stop referring to primary srcs and start reflecting user inaccuracies. makes sense since the user input will be the most recent input. the other was a recommendation for LLM engineers handbook. also, models can run on laptops (macbook m series i’m guessing?).

-> lost-in-the-middle: issue with RAG where references at the beginning (ranked relevant) and end (due to how llms are trained to predict next token) are highly rated but ones in the middle are ignored.

epic: packed. the ten minutes i heard were a lot of “physicians hate changes usually but they actually sometimes liked this!” a lot of focus on a. we have so many users! and b. they’re happy with us! and not a lot of mention of hallucinations and integrity checking. i will have to find someone who went to the epic session to debrief me. otherwise, it matched my fears of industry: focused on putting gen AI in everything regardless of if reasonable or not and not really caring about unknown bugs.

~ epic: my own thoughts on ethics: i find there are multiple “choice” frames to approach this from. one: if it makes care 1% better for 1000 patients and 50% worse for one patient, is it worth it? two: if taking an extra month to implement some quality assurance avoids that one bad outcome, is the extra month worth it? three: what if it means we can provide care for one person we couldn’t see at all before due to lack of physician time? etc.

JAMA: i learned that physicians are exhausted and frankly rather scared of how AI is taking over. the FDA has approved over a thousand devices with AI components the past year. there is work on a podcast to explain recent research to physicians.

JAMA – methods move fast: if used same methods as five years ago in a paper submitted today, paper would be returned with tons of methods critiques

child maltreatment public (state-level) policy analysis: how different state laws correlated with different pediatric outcomes.

used LLMs to summarize a bunch of public policies. does maltreatment definition include physical abuse? is reporting centralized or not (hotline)? then factor analysis to find factors to make cohorts (find underlying patterns in policies): mandated reporter? reporter training? then cluster around to find specific factor combinations (few/moderate/high levels of training, or penalties, etc.). then look at outcome differences. (will insert screenshots later)

policy -> difference-in-difference: check if results still hold between 2019 and 2021 when policies were in-place and not-in-place.

dinner / expo

they have free headshots! i desperately need one, but [tmi] i have a giant pimple right now :'( maybe that’s what AI is for lol. the expo … somehow i was completely uninterested. i think i felt rejected by my coworkers. i called a friend for moral support lol

personal thoughts

overall the conference is more diverse than i expected (which isn’t saying that much though tbh). i wonder how the stats compare to IROS/RSS/ICRA.

i need to remind myself i’m utterly new to this field and i’m not here presenting a paper or poster. of course, i will have to work harder to connect technically with people, and surface a whole different set of my interests than i usually nerd out about. people here are not excited about welding robots or maker faire or engineering education. nor are they nerding out about the latest theoretical advances and architectures and benchmark leaderboards. people here talk about weird interesting healthcare things.

i’ve only officially been to robotics conferences as a presenter

so there’s multiple factors at play here for my social sore thumb feeling.

mostly, i was really surprised when a fellow attendee who also went to the boston VA for many years declined to connect on linkedin. but i guess we exchanged conference app qr codes? in any case – the main thing i learned talking to her is that there a huge disparity in resources in regional VAs. i thought it was a national organization but apparently not. we in boston have the resources to figure out how to get at data sets but researchers in other VA regional sites do not. even though the data sets are VA datasets! at least, this was her experience a decade ago. also, generally feeling a bit surprised that my coworkers are not meeting up between sessions and discussing topics in real time constantly. i guess we are not really in the same lab. i gave up on organizing meals with my labmates and just messaged randomly people (around my career level) to find someone to go to dinner with. a person from WINE invited me to a dinner with a bunch of student volunteers for the conference. note: if you join the working group, there’s a chance you can be invited to volunteer for the conference and the registration fee is waived. (but travel, hotel, food not covered).

people expect postdocs to have a research idea and focus, so i need a different self-intro hook.

that makes sense. i’m not really working with a professor directly on a specific idea. as a result of my lack of topic, i need a different self blurb than “i do data science at the VA.” i think the only thing i’ve done of interest to folks is my thesis research topic.

alternatively, i could just not talk to anyone and do my own work. but oh, i felt a bit sad to not have people to compare notes with after the sessions. i really wanted those interesting and thought-provoking technical conversations i would have late at night in undergrad. (i never really got that in grad school except for classes, not research). i really did yesterday. but y’know, this is like one day into the field for me depending on how you count lol

grad school – i wish that after leave of absence they’d invested in my success instead of throwing me in and asking me to perform better than other people with a handicap already. i learned so much in so little time from this conference ! !

i guess for many of the older/senior folks, AMIA is like putzgiving, where people fly in every year and it’s a reunion.