Large Language Model Architectures in Health Care: Scoping Review of Research Perspectives

Background:
Large language models (LLMs) can support health care professionals in their daily work, for example, when
writing and filing reports or communicating diagnoses. With the rise of LLMs, current research investigates how LLMs could
be applied in medical practice and their benefits for physicians in clinical workflows. However, most studies neglect the importance
of selecting suitable LLM architectures. Objective: In this literature review, we aim to provide insights on the different LLM model architecture families (ie, Bidirectional Encoder Representations from Transformers [BERT]–based or generative pretrained transformer [GPT]–based models) used in
previous research. We report on the suitability and benefits of different LLM model architecture families for various research
foci.

Methods:
To this end, we conduct a scoping review to identify which LLMs are used in health care. Our search included
manuscripts from PubMed, arXiv, and medRxiv. We used open and selective coding to assess the 114 identified manuscripts
regarding 11 dimensions related to usage and technical facets and the research focus of the manuscripts.
... mehr
Results: We identified 4 research foci that emerged previously in manuscripts, with LLM performance being the main focus.
We found that GPT-based models are used for communicative purposes such as examination preparation or patient interaction.
In contrast, BERT-based models are used for medical tasks such as knowledge discovery and model improvements.

Conclusions:
Our study suggests that GPT-based models are better suited for communicative purposes such as report generation
or patient interaction. BERT-based models seem to be better suited for innovative applications such as classification or knowledge
discovery. This could be due to the architectural differences where GPT processes language unidirectionally and BERT
bidirectionally, allowing more in-depth understanding of the text. In addition, BERT-based models seem to allow more
straightforward extensions of their models for domain-specific tasks that generally lead to better results. In summary, health care
professionals should consider the benefits and differences of the LLM architecture families when selecting a suitable model for
their intended purpose.