Uncategorized

Diary #89 – lifespan clock

yea ok it’s good to split these into diary / not diary since only half my blog posts are about technical things anymore

i’m here to chronicle some issues i never thought about. they suddenly coalesced over the last few months

i’ve always been consciously against the idea of a biological clock as causing semi-sexist complications (in terms of employer discrimination, career ambitions, etc.), kind of allergic to the concept actually

i mean wow way to give me FOMO anxiety BBC, why didn’t I hear about this when I was 20

https://www.bbc.com/worklife/article/20220603-why-women-have-to-sprint-into-leadership-positions

There is immense pressure for women to reach a certain level of career and financial success before becoming parents, says Karin Kimbrough, Chief Economist at LinkedIn, who conducted the research into the 10-year window to leadership.

Kimbrough calls this process a “sprint” to leadership, meaning that women who don’t scale the leadership ladder very quickly are less likely to make it to the top at all. This might mean they end up overworking or making enormous personal sacrifices in order to ascend to C-suite level during this crucial decade. Much of this urgency to sprint – and the exhausting overwork it involves – stems from women needing to make sure their careers don’t sink once they begin families.

They are racing the clock against the so-called motherhood penalty. In this phenomenon, women find their careers stalling in areas such as promotion and pay once their children are born (while, conversely, men’s careers accelerate after becoming fathers). This effect, as well as the enormous burden of caregiving responsibilities that women take on, is well documented (and similarly affects other types of caregivers, like looking after ageing parents, says Kimbrough).

Katie Bishop, for BBC

but through conversations (maybe 2 or 3 a year) i’ve started to realize things

  • parents don’t stay spry forever, and it’s nice if they’re lively enough to help care for kids (b/c they’re a lot of work, and that reduces the career impact)

this is the result of talking to friend who consciously wanted her kid to know their grandparents are full independent adults, and be able to get to know their grandparents as adults also

+ reflecting that yea, my relationship with my parents is vastly different at 30 than at 20. my grandparents all died relatively young, and I wish I’d gotten to know them a bit more when they were healthy.

+ also staying with an older person, who I think even 5 years ago was really spry and independent, but is now almost housebound

  • my other recent realization, it can take a while to get pregnant even. apparently on average it can take 6 months

I mean like this is so terrible from a project management / timeline perspective lol.

(Of course there is fostering, adopting, surrogacy, etc. Though I know relatively little about these. Other interesting rabbit holes — single dads by choice — see appendix.)

  • then i looked up more details… you’re supposed to wait (according to science) between 1.5 to 2 years after giving birth before starting to try to have another kid

all this starts to add up!

  • also, if i want to wait to run for senate until after my kids are off to college, i don’t want to be another geriatric geezer in congress 0:

anyhow, basically it was the combination of spacing (interpregnancy interval) + delay (time to actually get pregnant) which was like … that’s an extra 2-3 years on top of my timeline estimate 0: 0: 0:

anyhow yea not very technical but just stuff i never thought about — there’s a lifespan clock separate from fertility clock (which! btw! exists for guys too!)

i wonder if i’ll look back in 30 years and laugh at the “geriatric geezer” comment haha

appendix – single fathers by choice

Recently i learned there are also single fathers by choice, but only very few studies on them. Many studies are on gay dads having surrogate children (often in Israel — apparently the government / religion / culture really emphasizes having kids o__o — but it’s complicated because (according to the quotes in the papers) the conservative government also doesn’t really approve of LGBT. So the dads almost viewed it as an empowering act to have surrogate families). But far fewer on *single* dads by choice.

  • https://www.semanticscholar.org/paper/The-Social-Experiences-of-Single-Gay-Fathers-in-An-Tsfati-Segal%E2%80%90Engelchin/8405f0b875742b805de49aa91292638385c4293f
  • https://www.semanticscholar.org/paper/Children-of-Single-Fathers-Created-by-Surrogacy%3A-Pereira/ee815c5b824b37e96bc2f42088a66aa709e6dff9
  • https://www.theguardian.com/society/2020/jan/29/i-always-wanted-to-be-a-dad-the-rise-of-single-fathers-by-choice

Uncategorized

single-file scrapy spider (without all the default folder/project overhead)

just documentation for myself while still fresh in my head.

Here’s the simplest version. More details to follow (coloring terminal output, adding timestamps to a log file, explaining some of the delay and log settings, etc.)

Overview

  1. Create a CrawlSpider class
  2. Inside have a start_requests() class, where you take a list of urls (or a url) and yield a scrapy.Request() that calls a function you define (e.g. self.parse_page())
    • Note use of a base_url, this is because often links on a page are relative links
  3. Create the parsing function called earlier, e.g. most commonly use response.css() (and same as normal css selectors – period to select a class, :: to select the plaintext that’s inside a set of tags, or ::attr() to select a specific attribute inside the tag itself
  4. Create a dictionary of the desired values from the page, and yield it
  5. Create a CrawlerProcess that outputs to CSS
    • c = CrawlerProcess(settings={
      "FEEDS":{ "output_posts_crawler.csv" : {"format" : "csv"}}
      })
  6. Call the CrawlerProcess on the CrawlSpider defined earlier, then start the spider
    • c.crawl(PostSpider)
      c.start()

Here is some

example code

$ vi my_spider.py

import scrapy
from scrapy.spiders import CrawlSpider
from scrapy.crawler import CrawlerProcess

import numpy as np
import pandas as pd

class PostSpider(CrawlSpider):
    name = 'extract_posts'

    def start_requests(self):
        self.base_url = 'https://somesite.com'

        threads_list = pd.read_csv('some_list_of_urls.csv')
        # csv has headers: | link |
        urls_list = threads_list.dropna().link

        for url in urls_list:
            url = self.base_url + url
            self.logger.error(f'now working with url {url}')
            yield scrapy.Request(url=url, callback=self.parse_page)

    def parse_page(self, response):
        self.logger.info(f'Parsing url {response.url}')
        page_name_and_pagination = response.css('title::text').get() 

        # - get post metadata
        posts = response.css('article.post')
            # <article class='post' data-content='id-123'>
            # <header class='msg-author'> <div class='msg-author'>
            # <a class='blah' href='index.php?threads/somethreadtitle/post-123'>
            #   No. 2</a>
            # </div> </header> </article>

        for post in posts: 
            post_id = post.css('::attr(data-content)').get()
            post_ordinal = post.css('.msg-author a::text').get() 

            self.logger.info(
                f'Now scraped: {page_name_and_pagination}')

            page_data = {
                'post_id': post_id,
                'post_ordinal': post_ordinal}
            yield page_data

c = CrawlerProcess(
    settings={
        "FEEDS":{
            "_tmp_posts.csv" : {"format" : "csv",
                                "overwrite":True,
                                "encoding": "utf8",
                            }},
        "CONCURRENT_REQUESTS":1, # default 16
        "CONCURRENT_REQUESTS_PER_DOMAIN":1, # default 8 
        "CONCURRENT_ITEMS":1, # default 100
        "DOWNLOAD_DELAY": 1, # default 0
        "DEPTH_LIMIT":0, # how many pages down to go in pagination
        "JOBDIR":'crawls/post_crawler',
        "DUPEFILTER_DEBUG":True, # don't rescrape a page
    }
)
c.crawl(PostSpider)
c.start()

$ python my_spider.py

end

that’s all folks

Uncategorized

yea, you’re pretty sleepy, and we don’t know why

aka idiopathic hypersomnia

Idiopathic hypersomnia (IH) is a neurological sleep disorder that can affect many aspects of a person’s life. Symptoms often begin between adolescence and young adulthood and develop over weeks to months. People with IH have a hard time staying awake and alert during the day (chronic excessive daytime sleepiness). They may fall asleep unintentionally or at inappropriate times, interfering with daily functioning. They may also have difficulty waking up from nighttime sleep or daytime naps. Sleeping longer at night does not appear to improve daytime sleepiness. The cause of IH is not known. Some people with IH have other family members with a sleep disorder such as IH or narcolepsy.

sometimes i think it’s made up, but then other times i remember how, even if it’s made up or not that severe, my sleepiness does impact many parts of my life. and life is short, why not happier if i can?

  • going to cafe by myself: good for waking up, but it’s awkward if I fall asleep there, so can’t go
  • going to student center (or library): good to get out of house (not just sleep for half a day or more), but at e.g. the Harvard student center, they’ll actually have security come around and wake you up if you fall asleep. needless to day it’s very unpleasant to come out of a dead sleep to a security person waking you up
    • i have gotten into a surprising amount of situations due to napping in public places. generally not an issue in some of the out-of-the-way parks in cambridge/somerville. generally is an issue in downtown boston. also depends on if you’re sitting up with a laptop in your nap, vs. fully laid out head on a bookbag jacket pulled up over your head
    • one time i decided that napping on the stairs along mass ave was needed, woke up to concerned strangers
  • a labmate once commented that they think it’s really rude if someone goes to a lecture and falls asleep, and you should not go in that case. i didn’t say that, if i needed to guarantee i wouldn’t fall asleep, in my case i’d never go to any lectures. but it’s important to go still to stay in the loop…
  • one time i fell asleep while sitting across from a professor waiting a few minutes for my turn to speak…
  • i’ve definitely gotten sleepy or overloaded to the point of all-consuming desire to find the nearest spot to fall asleep on, e.g. bench on the side of the street. like when you’re really hungry and it’s hard to think of anything else
  • i’ve really avoided 9-5 jobs, i just remember staring uselessly at my screen scrolling mindlessly for an hour because i couldn’t fully sleep and couldn’t fully not sleep
  • which is why i tend to want to work close to home

with my other medical issues stabilized i understood that my sleepiness definitely wasn’t part of e.g. depression, but just inherent to me. so eventually i went to get a sleep study, and got a diagnosis of idiopathic hypersomnia.

the sleep dr. treated this as a 5-minute appointment situation, and didn’t particularly care how i was treated / left it up to me, so i left feeling discouraged and that maybe i was making it all up anyway. so it went for several years until finally, i made an international trip that was supposed to be a “workcation”. instead i just slept 13 hrs a day long after i should have been un-jet lagged, despite being in a foreign country by myself which should have been exciting (admitted i had a severely sprained ankle and had been too scared of medical fees / didn’t get traveler’s insurance to go to a clinic / had never sprained anything so didn’t know to get a brace, i just got compression socks and a cane… and hopped around). combined with the falling asleep sitting up at a table, i really decided to handle this once and for all

anyway, medication has helped, but then (do to recent supply chain issues) i briefly came off of medication, and wow. how did i used to live like this? lol

it’s not so much the 1-3 hr long naps (despite sleeping 8-9hrs), but the randomness of them. and constantly devoting brainpower / feeling like i’m desperately running in circles — can i get more exercise? go out to a cafe? eat less carbs? scheduling meetings gets really stressful — what if i don’t wake up? should i try afternoon meetings? first thing in the morning?

the main thing of meds is I can reliably be awake from 9-12, and can thus have a semi-regular schedule

also, having a really supportive doctor who is willing to listen to me and patiently explain potential side effects, what to watch out for, side effects, how to test dosage, experience with other patients, and take my concerns seriously. so that i feel like there is something legitimate to my issues. vs. the original person just writing a script and saying “schedule a follow-up in four months, feel free to text.” and at the same time, i read things online and know that my issues are not nearly as severe as others.

anyway, some links:

https://www.dynamed.com/condition/classification-of-sleep-disorders#IDIOPATHIC_HYPERSOMNIA

https://rarediseases.info.nih.gov/diseases/8737/idiopathic-hypersomnia

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5558858/

https://emedicine.medscape.com/article/291699-overview

https://www.sleepfoundation.org/excessive-sleepiness

it’s complicated diagnostically

All these disorders have in common a subjective complaint of excessive sleepiness. ICSD-3 defines this as “daily episodes of an irrepressible need to sleep or daytime lapses into sleep.” For those disorders, such as narcolepsy and idiopathic hypersomnia (IH), which require demonstration of objective sleepiness by the multiple sleep latency test (MSLT), a mean sleep latency of < 8 min on the MSLT is required. This criterion is unchanged from the ICSD-2 and represents the best compromise between sensitivity and specificity.
However, physicians must recognize that there is substantial overlap between pathologically sleepy individuals and “normal” (often sleep-deprived) persons. Therefore, in establishing a diagnosis of a central disorder of hypersomnolence, physicians must be keenly aware that sleep deprivation, especially in those with longer sleep requirements, may account for abnormal MSLT results. The use of sleep logs and actigraphy for at least 1 week prior to MSLT is strongly encouraged to rule out insufficient sleep, sleep-wake schedule disturbances, or both as potential explanations for abnormal MSLT findings. Limited data suggest that one-off subjective reports and sleep logs alone may significantly overestimate total sleep time in the days prior to MSLT. Conversely, some patients with legitimate central hypersomnolence conditions may not consistently demonstrate mean MSLT latencies of < 8 min. Clinical judgment is required in such cases. Repeat MSLT at a later date may confirm objective sleepiness.

even MSLT is suspect though

First, neither short MSL nor SOREMPs are specific. Up to 30% of the normal population may have a MSL ≤ 8 min, the current cutoff for the hypersomnolence disorders.

Second, the MSLT may not be adequately sensitive, especially for IH. The 8-min cutoff was determined for patients with narcolepsy and extended to IH for “simplicity,” without independent determination

Third, while MSLT test-retest reliability is high in patients with narcolepsy with cataplexy restudied within 3 weeks,53 in clinical practice, test-retest reliability of the MSLT in narcolepsy without cataplexy and IH is poor. More than one-half of subjects with these disorders are given a changed diagnosis on repeat testing

some more interesting thoughts: ( i will need to cite all these later )

clinical presentation includes irresistible attacks of daytime sleepiness, unwanted, unrefreshing daytime naps ≥ 1 hour long, difficulty waking up from naps, and sleep inertia (sleep drunkenness)

Long sleepers feel fully refreshed and do not experience
daytime sleepiness if they are allowed to sleep as long as they need, in contrast with
patients with IH who continue to feel sleepy regardless of prior sleep duration.
In contrast to narcolepsy, patients with idiopathic hypersomnia generally have high sleep efficiency, sleep drunkenness, and long, unrefreshing naps.

Sleepiness is typically experienced as the inability to stay awake when desired, yet the MSLT measures “sleepability,” or the ability to fall asleep on command. These two constructs, while related, are clearly not identical.

my conclusions / specifics

Total recording time was 528 minutes, from 23:53-8:32. Total sleep time was 385 minutes, with a sleep efficiency of 74%. Sleep and REM latencies were 17 and 116 minutes respectively.
There was 19% stage 1, 54% stage 2, 8% slow wave sleep and 18% REM, with 2 REM cycles.

There was moderate to severe hypersomnolence, with a mean sleep latency of 6 minutes, and no sleep onset REM in any of 4 naps. The sleep latencies were 8, 12.5, 1.5, 3 minutes respectively. The nap times were 10:05, 12:28, 2:23, 4:43

moderate to severe hypersomnolence with no objective sleep onset REM. Delayed sleep phase. The sleep latencies do not just reflect tendency to delayed sleep phase. Consistent with diagnosis of idiopathic hypersomnolence.

complicated in diagnosis, but not in treatment, so in some sense it doesn’t matter in the end to me as a patient…

projects blog (nouyang)