With my Part D work, I am concerned and trying to protect our ability to do a strong causal inference study when I am worried about the power of the data structure that we have to do such a study.
We are trying to estimate the effect that pharmaceutical access expansions have on long-term care use.
We assume that for the treatment population ages 65-69, the population ages 60-64 offers a good counterfactual for change in long-term care use. The population 65-69 has a shock, exogenous to long-term care use trends, that causes a portion of its uninsured population to switch to an insured population, and this uninsured-to-insured population is a good representation of [##ask-control or treated##] population of interest.
The policy question that Aparna is looking to address is
The empirical intention is to “estimate the impact of prescription drug insurance on elderly individuals’ utilization of formal and informal long-term care” – but impact for whom? the uninsured is assumed
The second empirical intention is to examine how changes in long-term care use affects informal caregivers, but is written as “how changes in LTC use affected labor market and mental health outcomes of informal caregivers.” I’ll need clarification here.
Furthermore, we will do “heterogeneity tests”
The treated group is all Medicare-eligibles. Because they had a shift in drug access caused by Part D. But grouping the three major treated categories (uninsured->uninsured, uninsured->insured, insured->insured) together will dilute the ability to best test our hypothesis about the effect of increased access upon LTC.
The whole endeavor seems ripe for SEM.
But I am also coming around to using
Future topics to cover:
– heterogeneity
– power analyses
– ATET or ATC estimand – or can we develop a weighted
When you selected the IV method in your 2018 proposal, did you choose not to do propensity score matching/weighting or synthetic control methods for any particular reason? Do you view the IV method as equivalent to using propensity score weights, or as fundamentally different? (I’m thinking that using uninsured-hat changes the estimand from average treatment effect on the treated to ATE on the treated & uninsured.) Are we generally flexible about the estimand? i.e., do we want to estimate average treatment effect of Part D on the treated (which is an overlap group – it includes pre-treatment uninsured and insured), and/or on the treated & pre-treatment uninsured?
If we’re flexible, perhaps we could try to use our study to extend the estimated effect over to today’s elderly population, weighting our study’s estimated ATE based on the demographics & insurance characteristics in today’s post-Part D Medicare-eligibles.
I continue to study ways we can strengthen our causal inference, which we would need to settle prior to specifying a power analysis. I am concerned about the applicability of an IV method using demographics because I think, in theory, that demographics were relevant to changes in LTC use during the studied period. I am worried about the pooling of effects of the Part D treatment across multiple groups. I.e. they would not satisfy exogeneity to the DV except for the ways they relate to Rx insurance. There may be a set of demographic variables that we could carefully select as instruments that in theory relate only to Rx insurance, and we could test their exogeneity in the data. I also am studying if we could generate an estimand of ATE for
but first, I want a fresh understanding of the alternatives to Diff-Diff designs.
Synthetic Control Method
I read about this on Sunday and totally forgot how it would be different from PSM
Synthetic control method (SCM) matches according to the Y variable in pre-intervention periods, as a time series. Untreated comparison cases are identified according to similarity to the treated case during the period (can be multiple but typically one or few case[s]).
– Parallel trends assumption is dubious
– Assume unobservable confounders influence the Y variable and desire to get most accurate (how) estimates of treatment effect \alpha = \Y_treated_t=1,i=1 \minus \Y_untreated_t=1,i=1
– Economists with stronger design backgrounds tend to pool multiple treated cases – notably, they have also had multiple treatments, multiple cases. The inventors of SCM are usually
Kreif, Noémi, Richard Grieve, Dominik Hangartner, Alex James Turner, Silviya Nikolova, and Matt Sutton. “Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units.” Health Economics 25, no. 12 (2016): 1514–28. https://doi.org/10.1002/hec.3258.
“This paper extends the limited extant literature on the synthetic control method for multiple treated units. A working paper by Acemoglu et al. (2013) uses the synthetic control method to construct the treatment‐free potential outcome for each multiple treated unit and is similar to the approach we take in the sensitivity analysis, but weights the estimated unit‐level treatment effects according to the closeness of the synthetic control. Their inferential procedure is similar to the one developed here, in that they re‐sample placebo‐treated units from the control pool. Dube and Zipperer (2013) pool multiple estimates of treatment effects to generalise inference for a setting with multiple treated units and policies. Xu (2015) propose a generalisation for the synthetic control approach, for multiple treated units with a factor model that predicts counterfactual outcomes. Our approach is most closely related to the suggestion initially made by Abadie et al. (2010), to aggregate multiple treated units into a single treated unit. In preliminary simulation studies, we find the method reports relatively low levels of bias in similar settings to the AQ study.”
Propensity Score Matching
is ideal in cases where
– Assignment to the treatment group correlates with variables relevant to the outcome variable (treatment assignment bias)
– Few cases eligible for comparison group are comparable to treatment group case (on covariates deemed relevant)
– Many relevant dimensions on which to match
Tactic: Generate a “propensity score” via logit regression of participation on confounders, giving the predicted probablity of participation in the treatment group.
Then: Each treatment participating case gets one or more matched comparison cases based on their confounding variables, which give their propensity to have been participants. To do this, we need measures and thresholds of nearness.
I’m unclear about: But is it nearness on P-hat or is it nearness on the confounders? If the latter, does that still involve it like there is a variable P for participation and P~X and P~Y, so model P~X, pick comparisons that look like group for whom P=1, and then assume relationship to Y operates similarly in treatment & comparison groups?
Neoclassical economics runs in my ideological veins. In my office, I am often the sounding board for economic policy ideas: couldn’t we impose a $30/hour wage for in-home caregivers? “No – inevitably,” I begin, “… inevitably, the demand for formal in-home care will plummet, and in fact families will turn to a black market for low-wage work.” This inevitability is despite the enormous value caregivers provide to society, and this is also because of the low socioeconomic status of the caregiving workforce — they’re immigrant women, and if they held any higher status, they would not be in the industry: better options would be available at McDonald’s or at Walmart. The issue is too far gone.
This example embodies the failure of liberalism as Patrick J. Deneen sees it in Why Liberalism Failed. We are not free, and the more we learn about how our state and our markets work, the less we consider our world to be one of opportunity. This example is a failure of our state (a failed immigration system) and our markets (a failure to meet dire need, reward valuable work, and establish true freedom for either consumer or worker).
Deneen claims that trends in the markets and the state are pushing liberalism away from its core values for humankind — “to secure liberty and human dignity through the constraint of tyranny, arbitrary rule, and oppression” — and toward a “remaking of the world in the image of a false anthropology.”
This is Deneen’s key thesis: we began with a structure that held honorable goals that protected us from oppression but required us to collectively define the liberty, dignity, and culture we desired; at some point, our definitions rotted; since that point, the entire structure is decaying from its core, spread only more quickly by a polar politics pulling that ideology out to every last piece of the structure.
For the conscious-but-partisan among us, we see the rot only when it is spread by the other team’s favored institution: the market or the state. But in fact, we adopted the rotted definitions ourselves, and we are responsible for its spread. At best, we are left to be angry cynics mad at the state of affairs but acquiescent, unwilling to improve it except marginally. At worst, we are cruel cynics: both unwilling to act and uncaring about the harms arising. In either case, we are shepherded toward cynicism while chanting that the arch of the system bends toward liberty if it will bend our way. As Deneen wrote, “our liberation renders us incapable of resisting these defining forces — the promise of freedom results in thralldom to inevitabilities to which we have no choice but to submit.”
Take the Ideology of Competition, for Example
Now I want to return to that “false anthropology,” and to use the ideology of competition as an example of what Deneen may have meant by “false anthropology.” I interpret “anthropology” to mean a mental model of what humans value. Leaning on my caregiver example, the most obvious example of a false anthropology that’s common in liberalism (and neoclassical economics) is the equivalence of human incentives with competition: that direct competition is a key component of human life, and therefore that competition can suitably motivate all structures that govern human activity in our liberal society. The market is competitive, therefore its results are good. (Ah, Milton Friedman.) And the effect of this on human life and the state are clear. We made humanity subservient to competition.
Specifically, under this competition ideology — just one ideology of many that has rotted the core of liberalism by improperly defining “liberty and human dignity” — human life has been shifted into a near-constant state of anxiety. We are not secure in our access to necessities. We are never secure. We live in a society where the elderly — even with the Social Security benefits due from a lifetime of hard (but low-paid or “competitively paid”) work — face 8-month waiting lists to access Meals on Wheels in many cities. The cruel cynic supposes they can simply make do in those 8 months; the angry cynic calls for more government action to support the individual’s need for state support under a “glitch” in competition’s benefits. But it’s not a glitch. The system is not a leaky boat (which we tell ourselves it is), but rather a life vest (which we’ve held onto for far too long). Its inadequacies are uncountable, but I’ll continue: your access to health care could disappear if your employer went out of business (especially pre-Obamacare). And, because of the subservience of humanity to the “false anthropology” of competition, we feel no responsibility to help our nearest neighbors in the event that they are without home, food, or health care. The competition example is just one of many to be made. But I will explore it further still.
(As I explore the competition example, I’d like you to consider what “false anthropology” means as you read it, and what exemplary definitions of liberty, dignity, and culture draw from a “false anthropology” as you see it. I’m reachable on Signal and iMessage at 831-402-2736.)
For a jarring break of pace about competition, I will recount a story from my childhood. In a made-for-radio contest called “hold your wee for a Wii,” a local mother competed for a Wii to give her children by drinking gallons and gallons of water. She died. The pressure of competition placed blinders over her vision, disappearing her humanity, long-term thinking, and health.
A true anthropology would exalt humankind’s ability to plan for the long term, to take care of future generations and of health and well-being, and would consider connectivity and collaboration as a wellspring of problem-solving. It need not be utopia, but it would be hard to summarize in a word and diminish to a process that induces anxiety to motivate human action. In conclusion, and applying it to a contemporary crisis: a true anthropology would not struggle to reconcile “motivat[ing] liberty and human dignity through the constraint of tyranny, arbitrary rule, and oppression” with the changes in human activity required to confront climate change.
This concludes my example about competition and draws me into some very-current events. Today, it is 2020; today, the political moment is coronavirus pandemic tumult. Today, debate rages about whether to reopen society from shelter-in-place orders after a month of social distancing. Merkel is on her way out — steady leader of the liberal order that she was — and Macron was embattled but is newly seen as steady-handed. Trump, Bolsonaro, Obrador, Erdogan, Duerte are in, but their styles are currently questioned by the fearful masses. For all the fuss over US’ top-level populism, it is Europe’s nationalists that set the pace for ascendant illiberalism in the West. The last five years, from Brexit to the EU elections, have demonstrated visceral disgust by the individuals liberalism liberated and the markets and states that perpetuate their individual liberties. But hold on, what liberties? The definitions we take to answer that question answer Why Liberalism Failed, and I have sampled above how those definitions shortchange humanity. Want to move to a new system and stop shortchanging ourselves? Pick a new, truer, richer anthropology to guide the future. Define and defend culture.
Macron’s Anthropological Shift
This is precisely what French president Emmanuel Macron charted out yesterday, in an interview with FT. (The article, the video.) He may well have read Why Liberalism Failed himself during his sheltering in place. (I imagine he is having less travel time.)
Emmanuel Macron, in his April 16, 2020 interview with the FT, shared a vision of liberalism’s shortcomings that echo the thesis of Patrick J. Deneen’s Why Liberalism Failed down to a shared use of the word “anthropology.” If he is sincere, careful, and humble, the project of restoring France after Covid can kickstart this project of replacing liberalism’s rotten core — which he and Deneen both suggest is a misunderstanding of what humans value.
France is notorious among my peers in economics for the “friction” they introduce in their markets: “inevitably,” we jeer, “making harder to fire people makes a company slower to hire people.” After his bold actions to reduce these frictions to ensure economic competitiveness, the people seemed liable to fire Macron and hire a nationalist for president. Years-long protests undermined his political credibility. His pro-EU position faced the obvious skepticism of a people anxious about their security to access the necessities and anxious about their jobs and companies in the flows of trade, migration, and finance. But it is unlikely that the people would be satisfied with a tyranny of the majority and the results of a nationalist pivot — even if they were to vote one in. Nationalism does not solve the problems of liberalism; it extends them, because it has at its core the same rotten ideologies. Macron doubtless would make that argument on nationalism.
What Macron instead proposes is to replace the rotten core of liberalism and hope that the structure is renewed by a stronger, truer anthropology at the very center of it all. (Even if rotted, perhaps the structure is salvageable with a new set of definitions of liberty, dignity, and culture.) Macron names — multiple times — “anthropology” as the crux of what he wants to restore. This suggests that Macron is now thinking beyond just how policies can bandage the individuals’ wounds from competition or assuage a key constituency’s anxieties. No: Macron is thinking about nothing less than redefining what the state and market are geared toward, e.g. subbing out competition with collaboration.
Rather than hiding how the structure is shortchanging our dignity and culture, he lists off the anxieties liberalism wrought — anxieties he exacerbated — with a serious tone. He soberly states that liberalism is not just a post-Covid failure: it was a failure before Covid, and the writing was on the wall. The writing on the wall was addressed to him, liberalism’s poster boy.
He expresses and exudes optimism, and he promises to replace the core ideology of our state and market. If he — and our generation generally — succeed to replace the ideology, we could move sustainably past this fail-state tug-of-war between liberalism and nationalism, between right and left. Human dignity and connectivity should come first, and economics should be subservient to those intuitions. We should not feel insecure in a society that’s so technically advanced. Speaking of Covid’s role in this shift, Macron hits the nail on the head: “I think it’s a profound anthropological shock.” he says. “We have stopped half the planet to save lives, there are no precedents for that in our history.” Covid is a temporary shock to the liberal order, not from a nationalist front but from an exogenous mortal reckoning. If used correctly, we can use it to pivot away from a painful decline. A decline of liberalism and resultant, unsatisfactory surge in nationalism. Instead, we can pivot toward enhancing our collective ability to plan and act on shared priorities, and to cherish life and a dignified existence.
We will know society’s rot has been excised from its liberal core when we neither ignore the suffering of a neighbor due to the supposed inevitability of that suffering nor feel so routinely insecure that we cannot act collectively or individually to protect our lives and dignity. And, you and I must pray that we excise the rot without excising the rest of the protective structure we cherish about this society. (In which case we will face both arbitrary oppression from above and anxiety from the dreadful sights just below.)
It would be easy to see how Macron would be moved upon reading Deneen: he is a former investment banker, recently confronted with the incredible pains his rural constituents are feeling. He was the type of guy who began counterarguments with “inevitably, under competition…,” but suddenly he was faced with the projections of havoc from a climate crisis, and then he was faced with the real pain of the French layperson, and now he is faced with most of his country fearing their lives or those of their parents. After facing the enormity of the challenges, the scope of his concern likely changed. “No,” he realized, “trade adjustment assistance and free community college won’t heal these anxieties.” (My apologies to Bernie, Warren, Biden, Buttigieg, 2019 Macron, Merkel, Obama, and Cameron.) The takeaway from Covid-19 is that our economic and state structures do not value what we value — even if they have bandages.
You can’t repeat “suffering is inevitable” for long before you realize you’re shortchanging anthropology. In fact, now that we look around at our desolate place in the big wide ocean, and our mortality becomes salient, we realize we’ve been holding onto a rotten life jacket all along.
We need something more solid: we need our friends and family to be valued, and ourselves to be secure. That was supposed to be the promise of society. What ever happened?
We need something more solid now. We need Macron, liberalism’s heir apparent at the end of this first quarter-century of a new millennium, to (1) keep thinking about that “truer anthropology,” (2) install credible changes in French society that better define its core values, including needed actions that humbly reverse his past policy positions and that genuinely empower the French people including his current opponents, (3) report honestly how this project succeeds and why it met resistance, and (4) remain committed to a true anthropology above his commitment to saving the vestiges of liberalism.
After all, he will have only several more years to take this Covid awakening forward into practice and into structural changes. There will be an unprecedented demand for those changes, but unprecedented external turbulence as well. The projects of fixing the Covid fallout must be seen as the substrate of piloting a new social order, not seen as a distraction from doing so.
In my work for Dr. Aparna Soni, we are interested in estimating the effect of expanded access to prescription drugs on long-term care (LTC) utilization. Below I sketch out determinants of LTC utilization.
LTC use reflects need for personal assistance, which is caused by some decline in function (disability) and mediated by access to care.
LTC need is determined partially by
Disability
Social determinants
Social isolation
Home and neighborhood environment
Nutrition
Income, wealth, and health care access history
Employment and employment type
History of stressors including discrimination
Medical determinants
Chronic illness
onset
management
exacerbation
Progressive illness, esp. dementia
onset
progression
Behavioral health
substance use disorder
mental illnesses
Pain, its management
Vision
Accidents
Home hazards
Polypharmacy
Vision
Nutrition
Medical care history
History of fitness, nutrition, stressors
Use of supportive services that substitute for LTC to allow aging in place with disability
Social services
Zoning/Housing walkability
Accessibility of housing stock
Transitional care management and other medical-provider programs to reduce the burden of complex care needs
LTC access involves
affordability
financial resources
insurance status
private LTC insurance (rare – 5% of older adults have plans)
Medicare for home health
Medicare/Medicaid for the PACE program
Medicaid for home, community-based, and facility LTC
availability of LTC options
nearby formal providers: facilities, including LTACs and IRFs, home care agencies
market structure especially Medicaid program design
rurality
informal help: friend or family member helps
objective social isolation/connectedness
persons for whom the economic tradeoff of help is worthwhile
Many nursing facility residents are assessed with the Minimum Data Set (MDS) 3.0 item set. This contains a Brief Interview for Mental Status (BIMS) or a longer staff assessment for mental status.
The BIMS does not contain questions required for the long-popular Cognitive Performance Scale, but it also cannot be completed in many cases involving varying cognitive impairment severity.
So, the Cognitive Function Scale for the MDS 3.0 can categorize residents as either cognitively “intact” or as “mildly,” “moderately,” or “severely impaired.” Each category on the Cognitive Function Scale maps to a resident’s scores on the BIMS or CPS.
Cognitive function of nursing facility residents should be assessed at admission, and perhaps in all assessments using the Minimum Data Set assessment tool. These assessments are reported by nursing facilities’ staff to the Centers for Medicare and Medicaid Services (CMS) and therefore generate a panel of data for each resident about their condition and planned care in the nursing home.
The MDS is not a research tool, but researchers can use it carefully to better understand nursing facilities and their residents. In particular, researchers can access MDS assessments that are generated during a Medicare-covered short stay (e.g. post-hospital care) and those for Medicaid-covered long stays. These may be from the CMS Integrated Data Repository, from the CMS Chronic Conditions Warehouse, or from the ResDAC’s HRS merged datasets. This past week, my task with Dr. Joanne Lynn, MD, has been to define cohorts of frail Medicare beneficiaries using the Chronic Conditions Warehouse MDS 3.0 records, among other data sources (OASIS assessments for home health, and perhaps Medicare claims for Parts A & B).
Some fields in the MDS are to be filled out for care planning and assessment, but filling them out is not required for payment. Staff fill these fields out less frequently than those tied to payment, but are less incentivized to report the fields to the nursing home. My task this past week has revealed this trade-off: MDS information about cognitive functioning of nursing facility residents is frequently missing; MDS information on physical functioning (ADL deficits) is not often missing, but it may be biased for financial reasons precisely because the pay received depends on assessors’ answers there.
Cognitive Functioning Disrupted by MDS 3.0 Switch
When the MDS changed to a new version, from v2.0 to v3.0, in 2010, researchers faced a problem: a measure used for cognition was no longer consistently available. In the v2.0 era, researchers and CMS alike had drawn on several data fields — most crucially about short-term memory and daily decision making — to compute a Cognitive Performance Scale (CPS) based on MDS 2.0 records. The v3.0 MDS stopped consistently capturing those crucial memory and decision fields, to accommodate faster mental screening. The default MDS workflow instead uses the Brief Instrument for Mental Status (BIMS), which is brief and gives an intuitive ordinal score from 0 (all at issue) to 15 (no issue). The BIMS is not administered to all residents: if the “resident is rarely/never understood,” or if the resident does not complete the BIMS, then the assessors instead fill out a longer section containing the CPS’ requisite memory and decision fields. In that minority of cases, the CPS can be measured. However, not all of these cases necessitate the resident is usually or even currently truly unable to complete the BIMS. In the MDS 3.0’s early years, the BIMS was uncompleted for 17% of cases where a long-stay resident would be eligible — so the lack of a BIMS is a noisy signal of severe cognitive dysfunction. We’d be remiss to use the BIMS alone and to deem all non-BIMS-completers as severely impaired. How, then, do we crosswalk from the BIMS to the CPS?
If the BIMS were completed by all but the residents with most severe cognitive impairment, uncompleted BIMS could be treated as severely impaired, and otherwise the BIMS score would indicate cognitive function. However, some 17% of long-stay nursing facility residents eligible for the BIMS did not complete it. These cases would then skip the BIMS scoring process of the MDS 3.0 and have a staff assessment, even if they were not “rarely/never understood.” So, researchers created the Cognitive Function Scale (CFS) to crosswalk the BIMS score with the Cognitive Performance Scale (CPS), a previously used scale that requires other fields, which are skipped if the BIMS is completed.
Resources for Analyses of Frailty and Cognition in the MDS 3.0 and OASIS
[Forthcoming]
The CPS, the Cognitive Performance Scale from MDS 2.0
The Cognitive Performance Score has seven categories, notably “intact” and “borderline intact” and then five escalating categories of “mild impairment” to “very severe impairment.”
The “very severe impairment” class has a coma or has both severely impaired decision making and total dependence for eating. The “mild impairment” class has no severe impairments but has some impairment.
The Cognitive Performance Scale algorithm, updated for MDS 3.0, in which version it is usable if a non-BIMS assessment was conducted for mental status.
The BIMS, the Brief Interview for Mental Status
The BIMS assesses for three things:
ability to repeat three words, for
ability to recall those words from the interview’s start again at its end, and for
accuracy in recounting the current year, month, and weekday.
The BIMS generates a score from 00 to 15, and this field’s name in the MDS records starts with C0500. It appears this will have the value 99 if BIMS was not conducted. Probable, correct skip patterns are discussed later.
Many nursing facility residents are assessed with the Minimum Data Set (MDS) 3.0 item set. This contains a Brief Interview for Mental Status (BIMS) or a longer staff assessment for mental status. Because the BIMS does not contain questions required for the long-popular Cognitive Performance Scale, but it also cannot be completed in many cases involving varying cognitive impairment severity, researchers Kali Thomas, David Dosa, Andrea Wysocki, and Vincent Mor constructed a Cognitive Function Scale for the MDS 3.0 that categorizes residents as either cognitively “intact” or “mildly,” “moderately,” or “severely impaired.” Each category on the Cognitive Function Scale maps to a resident’s scores on the BIMS or CPS.
The CFS, the Cognitive Function Scale using either the BIMS or CPS
The Cognitive Function Scale simply provides a crosswalk between the BIMS and CPS patterns in MDS 3.0: it does not try to replicate the CPS’ eight-level structure.
Validation still occurred: its creators analyzed concordance between the scale and BIMS and CPS, and the CFS was validated against MDS 3.0 behavior items that should concord with cognitive status. It was also validated against CPS from MDS 2.0, using 2 records <100 days apart that were 3.0 and 2.0.
At first, I thought that the CFS’ most arguable assumption is that completing the CPS indicates the resident’s cognition cannot be intact. However dubious I felt about the assumption a priori, of all long-stay residents
CFS Score
CPS Score
BIMS Score
Intact
N/A
13–15
Mild Impairment
0–2
8–12
Moderate Impairment
3–4
0–7
Severe Impairment
5–6
N/A
The Cognitive Function Scale for MDS 3.0 crosswalk. Source: doi.org/10.1097/MLR.0000000000000334
Calculating the Cognitive Function Score
For constructing the BIMS and CPS without attending to data quality issues, one will need from their Chronic Conditions Warehouse MDS 3.0 extract:
C0500_BIMS_SCRE_NUM, for BIMS score or indication it was uncompleted.
If C0500 has the value 99, you will need all of the following.
C0700 Staff Assessment of Mental Status – Short Term Memory Code, C0700_SHRT_TERM_MEMRY_CD, 1 or 2.
C1000 Cognitive Skills for Decision Making Code, C1000_DCSN_MKNG_CD, integers 0-3.
B0700 Makes Self Understood Code, B0700_SELF_UNDRSTOD_CD, integers 0-3.
B0100 Comatose Code, B0100_CMTS_CD, integers 0 or 1.
Brief Interview or Staff Assessment? An addendum about data quality.
In the CCW, it seems three patterns for the BIMS or longer staff assessment (here, called SAMS) should exist:
No BIMS, yes SAMS: In the BIMS section, whether-to-conduct C0100 == 0, then C0200 through C0400C have carrots indicating blank because of skip pattern, and finally C0500 == 99. In the SAMS section, whether-to-conduct C0600 == 1, and then SAMS items C0700 through C1600 have 0s, 1s, or 2s.
Yes BIMS, no SAMS: In the SAMS section, whether-to-conduct is 0, and SAMS items are blank. In the BIMS section, whether-to-conduct is 0, then BIMS items are filled with integer among 0 through 3, and BIMS score is an integer among 0 through 15.
Incomplete BIMS, yes SAMS: In the BIMS section, whether-to-conduct is 1, but some BIMS items are blank or incomplete; then, BIMS score is 99. In the SAMS section, whether-to-conduct is 1, and SAMS items C0700 through C1600 have 0s, 1s, or 2s.
Social phenomena unfold at a human level. But these phenomena are most often observable only in the aggregate. MIT researchers do not have access to your paystubs to study your job history, but they can examine aggregate incomes among you and your neighbors according to census tract, or they can look at people like you in nationally representative surveys. Resultantly, your mayor or county supervisor must lean on nationally- or internationally-centered evidence to ascertain the best programs. Because the evidence is often poorly synthesized and not designed by researchers to be yet generalizable, officials more likely will turn to innovating by anecdote. (If not by ideology alone.) This seems like a failure by academia and funders in coordinating the work of social scientists so that their efforts hold use for society: as the medical research enterprise aligns its work (in private and public efforts alike) to benefit professionals who serve society, so should social science.
I share others’ excitement toward social observatories. We need more granular information, more diverse cohorts from whom it is collected, and more rich data sources that can be combined. We also need new ways of reporting science: there are topological relationships between topics we study, between the entities and phenomena we estimate. It is not enough to have a landscape of research populated by dense PDFs of written words that imprecisely communicate why we did the research, how we understood the research’s potential contributions in ongoing discourse, what concerns we hold about the research conducted, and how we interpret the research.
We waste immense effort because so much information is siloed, both at the source and at the finish line. Funders pay research teams to nosedive into data dictionaries trying to merge datasets that should, as a default, be compatible, colocated, and even co-produced. In the US, even researchers with authorized access to microdata (person-level data) typically cannot connect those data with relevant data at a similar level, either because the data does not exist at that level or because there is not information in each dataset allowing one to be merged to another.
You shouldn’t — and wouldn’t — allow me to make this argument without recognizing the work and great concern from convening institutions like the National Academies of Sciences Engineering and Medicine, funders like National Institutes for Health and the Robert Wood Johnson Foundation, or the editorial boards of academic journals. The Information Mapping section of Wikipedia teems with articles, which will point you toward the large literature on knowledge representation. They care deeply about this. Yet the problem of underdetermination in science persists — and likely always will. So what’s new?
Nothing, under this sun.
Log the logic. We just need a community-moderated website where researchers log the logical relationships they study. Consider they understand from existing science, propose new ones, and add basic information about what their studies imply about relationships. “Relationships” here connect multiple entities or events: for instance: “immigrant labor ~ job creation.” (More formal descriptions of how to log relationships can be found at the Genome Biology article linked below.) The platform would allow standardized descriptions of research results to be infinite depth of relationships to be reported, but would seek to standardize the information
Create Data Observatories at a Local Scale. Consider the vision of the Social Observatory Coordinating Network. From https://socialobservatories.org/vision:
… a representative sample of the places where people live and the people who live there. Each observatory would be an entity, whether physical or virtual, that is charged with collecting, curating, and disseminating data from people, places, and institutions in the United States. These observatories must provide a basis for inference from what happens in local places to a national context and ensure a robust theoretical foundation for social analysis
A national framework for studying local contexts.
A national framework for interdisciplinary collaboration.
These could be regionally organized community-level data collection efforts. One of the Network’s white papers calls for drawing on public and private databases to create a coherent data environment documenting local activity and life experiences. It also suggests conducting web-scraping and automated media analysis of local online experiences and content. These would all be wonderful ideas, accounting for rigorous commitment to individual privacy. For that reason, the priority setting would occur locally or regionally but the observatory areas would have a substantive core activity that is common nationwide. These ideas are developed here by John Schulz: https://socialobservatories.org/papers/paper_20130205.173152
For population health, there may be no more impressive goal than to stand up this envisioned observatory. UMD’s Christine Bachrach writes on how the social determinants of health would be a timely and useful topic to train the observatories’ lenses upon. Consider the value of merging biomedical testing data, hospital data, insurer claims data, e-health records, environmental, situational, demographic, and public-program data to give meaningful, geographically relevant and in-all-other-ways rich information on health. This 2013 paper sounds familiar especially now as the NIH undertakes its All of Us research project around the country, especially for underrepresented populations to gather lifecourse health data from voluntary participants.
Like All of Us, Bachrach’s vision would hand useful learning back to the communities engaged in the studies, and allow local leaders monitoring. This highlights a great opportunity generally for the observatory model: we need model communities that know their populations’ needs and can draw on good info too meet them in the long term and bringing the nation’s best minds to bear on the problems of the community. Participating “observatory communities” would be periscopes, and beneficiaries; crucially, they should also help steer the observatory’s principles for studied topics and privacy. Bachrach’s idea is detailed here: https://socialobservatories.org/papers/paper_20130703.0001 [Reader]
One final thought: in addition to creating a new familiarity with local social-behavioral data collection and use, we should welcome the observatory model as a substrate for new ways of talking about social research. New ways of organizing social science should emerge from the project, both in how scientists can transcend disciplinary silos and in how study topics and interpretation can benefit from participatory community-based research.
Use new ontologies. Here, there are inspiring strides taken in studying ontologies themselves by the Open Biological and Biomedical Ontologies Foundry, which is well described in an open-access article in Genome Biology here: https://genomebiology.biomedcentral.com/articles/10.1186/gb-2005-6-5-r46. The Foundry studies ontologies and has been consistently proposing improvements to how we can most efficiently represent knowledge. Efficiency does not mean lack of nuance; instead, the nuance is retained with new dimensions added to allow better meta-analysis. Consider for instance topological research of the loops, shapes, and networks formed among relations. Other dimensions, represented wisely, could be inputs to those topological evaluations.
A trail up Mount Toro, peering out between Coast Live Oak and a fenced cow pasture to the left. Foothills like these surround the valley, populated with chaparral interspersed among dry grasses.
An afternoon blaze takes out a sizable tract of maritime chaparral in 2014.
Lizard blends into the sand of a dune at Fort Ord National Monument.
Salinas Valley from Mount Toro in drier times.
A Spreckels foothill faces a setting sun, eclipsed by the next hill over.
Overlooking a seldom-filled “vernal pool” just after sundown.
The foothills of Fort Ord from Spreckels.
High-school-days shenanigans from a path overlooking the Salinas Valley’s farmlands.
Overlooking the Salinas Valley from Mount Toro.
The sun peers through smoky fog. The smoke comes from the Soberanes Fire, having drifted north from Big Sur. The Soberanes Fire was then the most expensive fire in world history.
Coastal live oak and native grasses rejoice after an unusual rainy season. My mother and her dog rejoice in the grasses and oak.
A dry creekbed. 36.6249582,-121.6919874
Fungi creep onto wood in a dry creekbed.
A path splits in Fort Ord, foregrounded by coyote bush.
Also worth discussing the commons and grace in terms of food production in Salinas.
And picking apart the moral obligations we have around our food and its production in a country of 340 million people.
Below, seasonal migrant farmworkers pick cabbage from a field along the Salinas River.
“The larger problem here is software observability, or more accurately, the pronounced lack of it. We have built mind-bogglingly complicated systems that we cannot see, allowing glaring performance problems to hide in broad daylight in our systems. How did we get here? And what can be done about it?”
If you join me for a hike or a run, you’d see I toggle between two modes: cruising and perusing. When cruising, I move fast and use my senses purposefully, ignoring the amazing detail around me. When perusing, though, I’m willing to spend travel time in an observant trance, captivated by the squirrels at my feet or the late-summer fruits ripening in the trees.
Both have some value, but I believe I more deeply enjoy the forest when I don’t rush, when I don’t pay attention to my time and speed. A healthy and fortuitous benefit from the observant approach is often a better sense of my own activity: Where, why and how am I in the system around me?
Detail without Reduction
Working in health policy, I can catch myself in that dichotomy: either I’m traveling intently on some transect of a large system or I visit a subtopic for days at a time. From this dual experience, I advocate for and prefer time in the weeds. That time in the weeds offers the only route, an intensive one, to really appreciate the where, why, and how of the system under consideration.
When we consider where knowledge emanates, it comes from closer and closer proximity to some basic unit and moving outward: physics underlies chemistry, chemistry underlies biology, biology (may) underlie psychology, psychology may underlie economics. This assertion should not make us all envy the physicist, but it should suggest we appreciate how depth can guide us to new distances, informing our direction and increasing our enjoyment.
Did you know that DC’s public-health chiefs stayed at their post about half as long as the national average between 1980 and 2017? A study of state health officer tenures in the Journal of Public Health Management and Practice that indicated that DC had the most frequent state health officer turnover, with directors of the DC Department of Health averaging a 2.1-year tenure compared to a 4.1-year national average [1]. The study’s authors emphasized the particular importance of merit-based selection criteria for health officials, urged creation of regional exchanges, and recommended investing in leadership development as early as possible in a leader’s tenure.
Resources available from the Center for State and Local Government Excellence (CSLGE) [2] and the Association of State and Territorial Health Officials (ASTHO) [3] suggest and build out the need for succession planning to resolve both the likelihood and impact of turnover in upper echelons of health departments and behavioral health agencies. Backdropping the urgency of health-official turnover are demanding changes to policy and technological landscapes that pile onto the enormous existing task of managing health service provision in an environment constrained by politics, budget, and workforce [1, 2, 3]. The effects of officials’ departures in such agencies are especially harsh because the agencies often contract out service delivery, and departures of department administrators place strain on contractor relations, according to a 2003 article published in the State and Local Government Review [4].
Succession planning should be ongoing, formalized, and undertaken prior to a leader’s announced departure [2, 3]. Succession planning scopes out the possible effects of agency departures, identifies core competencies, relationships, and tasks required in the present position, and develops a clear pipeline of potential replacements into the positions [2].
Successful Succession Planning for Agency Leaders
Succession planning requires asking,
Which roles require or benefit from succession planning?
What are the probability and effect of departures from these roles?
What is desired from a successor to fill this role well?
How well are these needs met by the bench of possible successors?
How can we groom possible successors to familiarize them with these needs?
And, for top roles, asking,
If this individual departed today, how immediately could the role functions and relations be continuously carried on, and by whom? [2]
Succession Planning is Strategic Planning
When done correctly, lessons gleaned from succession plans integrate with strategic planning for the organization. For instance, the ASTHO Succession Planning Guide highlights a convocation of public health officers and providers around Arizona to discuss leadership needs, common workforce concerns, and organizational staff development plans [3]. The subsequent Arizona Department of Health Services strategic plan included objectives and plans developed at the gathering.
There are plenty of ways a farsighted government might implement succession planning, but Darriell and others’ 2013 report to the CSLGE reported that fewer than one in thirteen local health departments have formal succession planning for “key leadership,” representing just 16 (7%) of 225 departments [2]. This indicates prevalent processes don’t match prevalent needs: more than half of surveyed health departments (56%) had sought a new key leader in the past 18 months. Planning around so common an organizational occurrence, formally and in advance, would mitigate the predictable disruption of an unplanned departure. In fact, planning at all would be ideal: about one-third of health departments (29%) performed no succession planning at all according to the 2013 survey.
Let’s hope that DC, with its particular rocky history, can implement these best-process, does not fall into that bucket. A cursory check [5] doesn’t look promising.
Notes
[1] Halverson, P. K., Lumpkin, J. R., Yeager, V. A., Castrucci, B. C., Moffatt, S., & Tilson, H. (2017). High Turnover Among State Health Officials/Public Health Directors: Implications for the Publicʼs Health. Journal of Public Health Management and Practice, 23(5), 537–542. https://doi.org/10.1097/PHH.0000000000000639; Public manuscript available at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5548508/
[4] Clingermayer, J. C., Feiock, R. C., & Stream, C. (2003). Governmental Uncertainty and Leadership Turnover: Influences on Contracting and Sector Choice for Local Services. State and Local Government Review, 35(3), 150–160. https://doi.org/10.1177/0160323X0303500301
