The Skill Explained
What Are NLP and Text Mining?
NLP (Natural Language Processing) is the field of AI that lets computers work with human language: reading, classifying, extracting information from and generating text. Text mining is finding useful patterns and facts in large amounts of text.
Why AI Employers Ask for Them
Organisations hold huge amounts of unstructured text: reports, emails, notes, publications. NLP turns it into structured, usable information. Large Language Models have made NLP far more powerful, and employers want people who can apply it accurately to their own documents.
How It Works
Classic NLP tasks include text classification (sorting documents into categories), named entity recognition (finding names of drugs, diseases, people or places), relation extraction (finding how entities connect), sentiment analysis and summarisation.
Methods have moved from rules and word counts, to machine learning on features, to Transformer models such as BERT fine-tuned on labelled examples, and now to Large Language Models that can do many tasks from instructions alone. Choosing between a small fine-tuned model and a large prompted model is a common design decision.
Domain language matters: medical and legal texts use specialised vocabulary, abbreviations and negation ("no sign of infection"), so models are tested on real domain data and terms are often linked to standard vocabularies.
What You Do With It at Work
- Classify documents or messages automatically.
- Extract entities and facts from free text into tables.
- Build search and summaries over large document sets.
- Label data and evaluate models on domain text.
- Link extracted terms to standard vocabularies.
Tools and Terms You Will Meet
- Named entity recognition
- Finding and labelling names of things in text.
- Text classification
- Assigning categories to documents.
- Information extraction
- Pulling structured facts out of text.
- Tokenization
- Splitting text into units a model processes.
- Negation detection
- Recognising when a statement says something is absent.
Industry Example: Pharma
A pharmacovigilance team uses NLP to extract drug names, reactions and dates from case reports and medical literature, then maps the reactions to MedDRA terms so safety signals can be analysed.
A Common Mix-Up
NLP did not start with chatbots. Many valuable NLP systems are quiet classifiers and extractors running in the background, often with smaller models.
How Deep Do You Need to Go?
- Know itExplain common NLP tasks such as classification and entity extraction.
- Use itBuild and evaluate NLP models or LLM prompts on domain text.
- Master itDesign NLP systems for specialised language at scale.