JCSIS
019928311823 info@jcsis.org
JCSIS

Transformer-Based Natural Language Processing Model for Clinical Text Mining and Drug Adverse Event Detection

Weiguo Gee
Volume 5

Abstract

 

The rapid growth of unstructured clinical text within Electronic Health Records (EHRs), discharge summaries, medication notes, progress reports, and spontaneous drug safety reports has created a valuable but underutilized source of information for pharmacovigilance and clinical decision support. Drug Adverse Events (ADEs) represent a major challenge in healthcare systems, as they may lead to prolonged hospitalization, increased medical costs, treatment discontinuation, and serious patient safety risks. Traditional adverse event detection methods rely heavily on manual review, rule-based systems, and voluntary reporting mechanisms, which are often time-consuming, incomplete, and limited in their ability to capture complex linguistic patterns in clinical narratives. To address these limitations, this paper proposes a transformer-based Natural Language Processing (NLP) model for automated clinical text mining and drug adverse event detection.

The proposed framework leverages domain-specific transformer architectures, including BioBERT, ClinicalBERT, PubMedBERT, and RoBERTa-based clinical language models, to extract contextual semantic representations from heterogeneous clinical documents. The model is designed to identify drug entities, adverse event mentions, dosage-related expressions, temporal cues, negation patterns, and drug–event relationships within free-text clinical narratives. A comprehensive preprocessing pipeline is applied, including text de-identification, tokenization, abbreviation normalization, spelling correction, section segmentation, clinical entity recognition, and handling of negated or speculative statements. To enhance the model’s ability to distinguish true adverse events from unrelated symptoms or historical conditions, the proposed architecture integrates named entity recognition, relation extraction, and sequence classification within a unified multi-task learning framework.

The framework can be trained and evaluated using publicly available biomedical and clinical NLP resources, such as ADE corpora, MIMIC-based clinical notes, and FDA Adverse Event Reporting System data, along with institution-specific annotated clinical records where available. Fine-tuning is performed using supervised learning with class-weighted loss functions and data augmentation strategies to address the imbalance between positive adverse event cases and non-event clinical mentions. In addition, attention visualization and explainability techniques, including SHAP and token-level attribution analysis, are incorporated to highlight clinically meaningful words and phrases contributing to model predictions. These explanations allow clinicians and pharmacovigilance experts to verify whether the model focuses on relevant evidence, such as medication names, temporal associations, symptom descriptions, laboratory abnormalities, and documented treatment reactions.

Experimental evaluation is conducted using standard NLP and pharmacovigilance metrics, including precision, recall, F1-score, area under the receiver operating characteristic curve, and relation extraction accuracy. The proposed transformer-based model is expected to outperform traditional machine learning approaches, rule-based extraction systems, and non-contextual word embedding models by capturing long-range dependencies, domain-specific terminology, and subtle contextual variations in clinical language. Furthermore, the integration of explainability improves transparency and supports safer adoption of automated ADE detection in real-world clinical workflows.

The proposed system provides a scalable, accurate, and interpretable framework for mining clinical text and detecting drug adverse events from large-scale healthcare data. By transforming unstructured clinical narratives into actionable pharmacovigilance insights, the framework can support early safety signal detection, reduce manual review burden, improve medication monitoring, and enhance patient safety. Its transformer-based and explainable design makes it suitable for integration into hospital EHR systems, pharmacovigilance platforms, and clinical decision support applications.

Keywords

References

  • [1] El-Kenawy, E. S., & Eid, M. (2020). Hybrid gray wolf and particle swarm optimization for feature selection. Int. J. Innov. Compu t. Inf.
  • [2] El-Kenawy, E. S. M., Mirjalili, S., Ibrahim, A., Alrahmawy, M., El -Said, M., Zaki, R. M., & Eid, M. M. (2021). Advanced meta -
  • [3] Khodadadi, N., Abualigah, L., El -Kenawy, E. S. M., Snasel, V., & Mirjalili, S. (2022). An archive -based multi -objective arithmetic
  • [4] Ibrahim, A., Mirjalili, S., El -Said, M., Ghoneim, S. S., Al -Harthi, M. M., Ibrahim, T. F., & El -Kenawy, E. S. M. (2021). Wind speed
  • [5] Abdelhamid, A. A., El -Kenawy, E. S. M., Alotaibi, B., Amer, G. M., Abdelkader, M. Y., Ibrahim, A., & Eid, M. M. (2022). Robust
  • [6] El-Kenawy, E. S. M., Mirjalili, S., Alassery, F., Zhang, Y. D., Eid, M. M., El -Mashad, S. Y., ... & Abdelhamid, A. A. (2022). Novel
  • [7] Hassib, E. M., El -Desouky, A. I., Labib, L. M., & El -Kenawy, E. S. M. (2020). WOA+ BRNN: An imbalanced big data classification
  • [8] Eid, M. M., El -kenawy, E. S. M., & Ibrahim, A. (2021, March). A binary sine cosine -modified whale optimization algorithm for feature
  • [9] El-Sayed Towfek, M. (2018). El -kenawy. Trust Model for Dependable File Exchange in Cloud Computing. International Journal of
  • [10] Abdelhamid, A. A., Towfek, S. K., Khodadadi, N., Alhussan, A. A., Khafaga, D. S., Eid, M. M., & Ibrahim, A. (2023). Waterwhee l plant
  • [11] El-Kenawy, E. S. M., Abdelhamid, A. A., Ibrahim, A., Mirjalili, S., Khodadad, N., Alduailij, M. A., ... & Khafaga, D. S. (2023). Al-
  • [12] Alhussan, A. A., Abdelhamid, A. A., El -Kenawy, E. S. M., Ibrahim, A., Eid, M. M., Khafaga, D. S., & Ahmed, A. E. (2023). A binary
  • [13] Alhussan, A. A., Khafaga, D. S., Abotaleb, M., Mishra, P., & El -Kenawy, E. S. M. (2024). Global Potato Production Forecasting Based
  • [14] Towfek, S. K., & Alhussan, A. A. (2024). Potato Production Forecasting Based on Balance Dynamic Biruni Earth Radius Algorithm for
  • [15] Mishra, P., Alhussan, A. A., Khafaga, D. S., Lal, P., Ray, S., Abotaleb, M., ... & El -Kenawy, E. S. M. (2024). Forecasting production of
  • [16] El-Kenawy, E. S. M., Mirjalili, S., Abdelhamid, A. A., Ibrahim, A., Khodadadi, N., & Eid, M. M. (2022). Meta -heuristic optimization
  • [17] Abdelhamid, A. A., El -Kenawy, E. S. M., Khodadadi, N., Mirjalili, S., Khafaga, D. S., Alharbi, A. H., ... & Saber, M. (2022).
  • [18] Alharbi, A. H., Towfek, S. K., Abdelhamid, A. A., Ibrahim, A., Eid, M. M., & Khafaga, D. S. & Saber, M.(2023). Diagnosis of
  • [19] Alhussan, A. A., & Towfek, S. K. (2024). 5G Resource Allocation Using Feature Selection and Greylag Goose Optimization Algori thm.
  • [20] Towfek, S. K., & Alhussan, A. A. (2024). Potato Production Forecasting Based on Balance Dynamic Biruni Earth Radius Algorithm for
  • [21] Abdelhamid, A. A., Alhussan, A. A., Qenawy, A. S. T., Osman, A. M., Elshewey, A. M., & Eed, M. (2024). Potato harvesting pred iction
  • [22] Eed, M., Alhussan, A. A., Qenawy, A. S. T., Osman, A. M., Elshewey, A. M., & Arnous, R. (2024). Potato Consumption Forecastin g
  • [23] Mahmood, S., Sun, H., Iqbal, A., Alhussan, A. A., & El -kenawy, E. S. M. (2024). Green finance, sustainable infrastructure, and green
  • [24] El-Kenawy, E. S. M., Alhussan, A. A., Khodadadi, N., Mirjalili, S., & Eid, M. M. (2024). Predicting potato crop yield with machi ne
  • [25] Radwan, M., Alhussan, A. A., Ibrahim, A., & Tawfeek, S. M. (2025). Potato leaf disease classification using optimized machine learning
  • [26] Ghasemi, M., Khodadadi, N., Trojovský, P., Li, L., Mansor, Z., Abualigah, L., ... & El -Kenawy, E. S. M. (2025). Kirchhoff’s law
  • [27] Yassen, M. A., El -Kenawy, E. S. M., Abdel -Fattah, M. G., Ismail, I., & Mostafa, H. E. D. S. (2025). Explainable artificial intelligence
  • [28] Radwan, M., Alhussan, A. A., Ibrahim, A., & Tawfeek, S. M. (2025). Potato leaf disease classification using optimized machine learning
  • [29] Mozhdehi, A. T., Khodadadi, N., Aboutalebi, M., El -Kenawy, E. S. M., Hussien, A. G., Zhao, W., ... & Mirjalili, S. (2025). Divine
  • [30] El-kenawy, E. S. M., Alhussan, A. A., Mattar, E. A., & Radwan, M. (2026). Feature selection and hyperparameter tuning in transfo rmer -
  • [31] Chen, L., Xu, C., Lim, W. H., Sharma, A., Tiang, S. S., Chong, K. S., ... & Khafaga, D. S. (2025). Transparent and reliable c onstruction
  • [32] Khodadadi, N., Towfek, S. K., Zaki, A. M., Alharbi, A. H., Khodadadi, E., Khafaga, D. S., ... & Eid, M. M. (2025). Predicting
  • [33] Alhussan, A. A., El -Kenawy, E. S. M., Khafaga, D. S., Alharbi, A. H., & Eid, M. M. (2025). Groundwater resource prediction and
  • [34] Mozhdehi, A. T., Khodadadi, N., Aboutalebi, M., El -Kenawy, E. S. M., Hussien, A. G., Zhao, W., ... & Mirjalili, S. (2025). Divine
  • [35] Lopes, J. M., Pinho, C. S., Martins, R. C., & Bandeira, T. (2026). Sustainability Gets Smarter: Competitive Pressure and AI L eading the
  • [36] Radwan, M., Ibrahim, A., Abdelsalam, M. M., Alhussan, A. A., Mattar, E. A., & El -Kenawy, E. S. M. (2026). Optimizing solar and wind
  • [37] Alhussan, A. A., El -Kenawy, E. S. M., Eid, M. M., & Khodadadi, N. (2026). Hybrid Al -Biruni and Puma Optimization (BERPO) for
  • [38] El-Kenawy, E. S. M., Ibrahim, A., Alhussan, A. A., Khafaga, D. S., Ahmed, A. E., & Eid, M. M. (2026). Smart city electricity loa d
  • [39] El-Kenawy, E. S. M., Khodadadi, N., Mirjalili, S., Zaki, A. M., Ibrahim, A., Alhussan, A. A., ... & Eid, M. M. (2026). Glider sn ake
  • [40] Mekaret, F., Rabehi, A., Zebentout, B., Tizi, S., Douara, A., Bellucci, S., ... & Alhussan, A. A. (2024). A comparative study of Schottky
  • [41] KOUADRI, Ali, RABEHI, Abdelhalim, BENZIANE, Ali, et al. A Robust Multi -Transform Watermarking Scheme for Medical Images
  • [42] Tibermacine, I. E., Russo, S., Scarano, G., Tedesco, G., Rabehi, A., Alhussan, A. A., ... & Napoli, C. (2025). Conditional VA E for
  • [43] Kouadri, A., Benziane, A., Rabehi, A., Rabehi, A., Alhussan, A. A., Khafaga, D. S., & El -Kenawy, E. S. M. (2025). A novel hybrid
  • [44] Russo, S., Tibermacine, I. E., Randieri, C., Rabehi, A., Alharbi, A. H., El -Kenawy, E. S. M., & Napoli, C. (2025). Exploiting facial
  • [45] Bentegri, H., Rabehi, M., Kherfane, S., Nahool, T. A., Rabehi, A., Guermoui, M., ... & El -Kenawy, E. S. M. (2025). Assessment of
  • [46] Tibermacine, I. E., Russo, S., Citeroni, F., Mancini, G., Rabehi, A., Alharbi, A. H., ... & Napoli, C. (2025). Adversarial de noising of EEG
  • [47] Ouahabi, M. S., Benyounes, A., Barkat, S., Ihammouchen, S., Rekioua, T., Rabehi, A., ... & Alharbi, A. H. (2025). Real -time sensor fault
  • [48] Belaid, A., Guermoui, M., Khelifi, R., Arrif, T., Chekifi, T., Rabehi, A., ... & Alhussan, A. A. (2024). Assessing suitable a reas for PV
  • [49] Mehallou, A., M’hamdi, B., Amari, A., Teguar, M., Rabehi, A., Guermoui, M., ... & Khafaga, D. S. (2025). Optimal multiobjecti ve
  • [50] Rabehi, A., El -Hadi, M., Benmahmoud, S., Rabehi, A., Alharbi, A. H., & El -Kenawy, E. S. M. (2025). SOCA -CFAR Processor in A
  • [51] Bakria, D., Beladel, A., Korich, B., Teta, A., Mohammedi, R. D., Laouid, A. A., ... & El -kenawy, E. S. (2025). A novel enhanced Grey
Download