Can a routine prescription become a clue to possible Alzheimer's protection?

How a transformer reads the hierarchy inside medical codes

Medical codes are not flat strings. A diagnosis code for a pancreatic disorder sits inside broader endocrine categories, and a prescription code points to a drug class before it names a molecule. A generic transformer can read the sequence of codes, but it often treats each code as one undifferentiated item, so the internal ladder of meaning gets flattened. The hierarchy-aware pretraining design gives every code five sub-tokens, one for each level in that hierarchy. Each sub-token receives its own vector embedding, and a small transformer encoder lets those five vectors exchange information before they are summed into a single representation for the whole code.

For diagnoses, sex is embedded like a medical token, while age is turned into a vector by a linear layer, and both are added to the diagnosis representation. Procedures and prescriptions are joined into treatment vectors, then processed separately from diagnosis sequences. A bidirectional cross-attention block compares diagnoses with treatments in both directions. That step lets the model learn which therapies tend to follow which conditions, rather than assuming every code is just another symbol in a list.

The training task also respects hierarchy. Fifteen percent of tokens are chosen as targets, and each target is split into its five sub-tokens. In most cases, eighty percent, the first higher-level sub-token stays visible while the lower four are hidden. Ten percent receive full masking, and ten percent remain unchanged. The model is trained to predict only the third sub-token, a middle level of the code. This makes the task less brittle than guessing the most specific code, because many fine-grained codes share the same broader category.

On pretraining tests for predicting that middle-level diagnosis token, the full design reached an accuracy of 0.5867 and a Matthews correlation coefficient of 0.5797. A version with hierarchical sub-token aggregation but without partial masking fell to 0.3913 accuracy, while a self-attention ablation using both methods reached 0.5804. The gap shows that the hierarchy-aware split helps, and the diagnosis-treatment cross-reference adds a small but consistent gain.

What the learned representations predict about dementia and hospital admission

A year of Japanese insurance claims was turned into a compact picture of each person’s health, and that picture was then asked to anticipate two events. One event was the first appearance of dementia, defined by diagnosis codes for Alzheimer’s disease and related dementias. The other was hospital admission during the follow-up period. The model did not read raw notes or images; it worked from coded diagnoses, procedures, prescriptions, sex, and age recorded in claims.

For each month, the system built a diagnosis vector and a treatment vector. Those two monthly summaries were joined into one clinical snapshot. The twelve snapshots formed a short sequence that captured how care had changed over the year. A final layer converted that annual summary into probabilities for dementia onset and hospital admission.

The prediction task was checked on separate people who had not been used to build the representation. Ten percent of patients were held out as a test group, while the rest were divided into nine folds so every patient could be tested once. Performance figures are averages across those folds. The people in these sets were mostly older adults, with a mean age near 73 or 74 years and about nine months of claims during the year.

For dementia onset, the strongest model reached a ROC-AUC of 0.8164 and an MCC of 0.2577. Its PR-AUC was 0.2204. A simpler version of the same architecture, without the two coding-aware training tricks, had a PR-AUC of 0.1949. That gap is about a 13 percent relative improvement in precision-recall area. Dementia was uncommon in the final sample, with an incidence of 6.2 percent. In such a setting, ROC-AUC can look encouraging while still leaving many cases missed. The precision-recall area gives a stricter view because it focuses on how well the model ranks true dementia cases above noncases.

Hospital admission was much more common in the evaluation sample, with an incidence of 31.6 percent. The same annual representation produced an admission probability for each patient. Because admissions were frequent, a useful predictor had to distinguish people who would be admitted from those who would not without simply flagging most patients.

From routine prescriptions to prioritized clues for Alzheimer’s protection

Routine medical records are full of small signals that never reach a clinician’s attention. A prescription for blood pressure, a lab visit, and a diagnosis code entered on an ordinary day can become part of a pattern when millions of such entries are compared. A screening approach uses Japanese claims and insurance data from local government insurers to turn those routine entries into ranked clues about possible Alzheimer’s protection. It does not claim that any everyday medicine prevents Alzheimer’s disease. Instead, it asks whether some prescription patterns appear less often among people who later develop the condition.

The repositioning screen began with a random sample of 100,000 people drawn from a larger pretraining group. After applying eligibility rules, 80,308 remained. None had dementia or Alzheimer’s disease during January 2017 through December 2018, and all had continuous insurance enrollment for the required period. In the final sampled sets, Alzheimer’s disease onset occurred in about 2.8 percent of people, a rate low enough that a weak association can easily be mistaken for noise. That is why the workflow uses statistical control for false discoveries before treating any drug-disease link as worth further attention.

The practical trick is to keep the question honest while ranking candidates. If the model simply sees that a person took a certain medicine, it may learn who was treated rather than what treatment might be associated with lower risk. To reduce that shortcut, prescription tokens for the target drug and related drugs are removed when diagnosis vectors are used for prioritization. The result is a shortlist of protective-association hypotheses that passed screening. They are clues to test, not established facts about Alzheimer’s protection.

Why observational screening still needs separate validation

A screen built from routine medical records is useful because it can turn millions of ordinary encounters into a short list of ideas that deserve closer attention. The claims data used here came from Japanese insurers and covered several million people, so the patterns were not drawn from a small clinic or a single hospital. Still, the output of such a screen is a signal, not a verdict. It says that a medicine appears more often among people who did not develop dementia in the observed window; it does not say that the medicine caused the lower risk.

This distinction matters because routine records mix treatment with illness. A patient who receives many prescriptions may be sicker, older, or under closer monitoring than someone who receives few. The same person may also have better access to care, different insurance rules, or a physician habit that shapes what is written in the file. If those factors are not fully measured, an association can look protective even when the drug has no effect at all. That is why the workflow treats its results as hypotheses for prioritization rather than evidence of causal benefit.

The statistical controls help narrow the list but do not remove this problem. After false discovery rate control, some candidate links showing protective associations passed the screen, yet no specific drug or disease is named in that part of the finding. Propensity score matching can balance measured differences between treated and untreated people; for two severity indicators, most standardized mean differences were small after matching, often below 0.1. That is a good sign, but it only covers what was recorded. Unmeasured habits, family history, diet, social support, or coding choices remain hidden in the numbers.

A separate validation step is therefore needed before any clinical meaning can be attached to a flagged link. An independent dataset can test whether the same pattern appears outside the original sample. A randomized trial can place patients into treatment and no treatment groups, which blocks many of the confounders that observational data cannot fully control. Until then, the screen has done its real job: it has sorted routine noise into plausible clues, not proven protection.

How billing patterns can point toward Alzheimer’s protection

A model trained on Japanese insurance claims can turn billing codes into a map of how diagnoses, procedures, and prescriptions move together. By learning the middle level of medical coding, it captures disease patterns without drowning in every fine-grained code. Its strongest signal appears in dementia, where a year of monthly claims gives a stronger predictive signal than established transformer models or simple tabular methods. Hospital admission is also predicted from the same annual representation, giving a second check that the learned patterns track real clinical events. The same representations then become a screening lens for drug repositioning. Four candidate drugs, vildagliptin, candesartan, loxoprofen, and pitavastatin, showed protective associations strong enough to survive false discovery control. These are not proven treatments; they are ranked hypotheses that deserve closer clinical scrutiny. The broader promise is a discovery engine built from data already collected for payment, where rare treatment patterns can be compared across millions of patients.

The obstacles are equally clear. Because claims are written for billing, a diagnosis may appear only after tests or prescriptions have already shaped care. The population is also older and Japanese, so results need testing in other age groups, genetic backgrounds, and health systems. Sampled cohorts may leave rare diseases and infrequent drugs underrepresented, even though the full database contains more than five million people. Scaling to complete claims, validating candidates in independent cohorts, and moving from association to trial-ready questions will decide whether this workflow becomes a routine part of drug discovery. If billing records can reveal protective signals hidden in ordinary care, how many other diseases are waiting for the right representation?