Text Annotation Services

Giveyourlanguagedataaclearerpointofview.

Language is labelled against the page or thread that made it true: entities with honest offsets, fields that still know their header, relations with a direction, and document classes that do not flatten a table into a string.

Text annotation labels language data with entities, intents, sentiment, document fields and preference rankings so NLP and LLM systems learn from consistent examples. VIPRO measures inter-annotator agreement as a quality target and resolves disagreements through adjudication under versioned guidelines.

Start a Pilot All Services

Document with entities and fields linked to a structured panel — TEXT / CLAIM_118.PDFREVIEW PASS · 000%

QUEUE / LIVE

Compatible workflows

Tools we can work with

Roboflow
SuperAnnotate
Darkhorse
Roboflow
SuperAnnotate
Darkhorse

Start smaller than the backlog

Resolve language ambiguity with examples.

A focused document set helps align field boundaries, nested entities, taxonomy choices, and cases where layout changes meaning.

Pilot first

What a text delivery contains

Spans point back to the source. Fields name their region. Classification labels name the taxonomy version. You should be able to open an item and see why it was labelled that way without calling us.

Entity spans with offsets
Document and passage classes
Key-value / table structures
Relation graphs
Reading-order sequences

Interactive example

Move from source document to structured fields

Switch between the original document and extracted labels to see how entities remain linked to their source context.

DOCUMENT / AGREEMENT_031STRUCTURED

Paper document workspace used to demonstrate text and document annotation

ENTITYVipro Data ServicesDATE18 September 2026DOCUMENT TYPEService agreement

review period and delivery scope linked to source text

Document and language guide

Text work that refuses to flatten the page

Entities, fields, reading order, and document type are labelled against the source. If a number’s meaning depends on a header, a stamp, or a column, that context stays in the annotation.

Every part of a document matters

Headers, footers, columns, lines, and tables are marked so each part of the page matches the field a model needs to find. That makes the issuer, amount, tax, and due date easier to recognize.

The same labels teach the difference between a phone number, a date, a quantity, and a financial figure instead of treating the page as one block of text.

When a stamp, a strike-through, or a handwritten note changes a printed value, that state is labelled rather than silently overwritten. A tidy field that hid the correction is a future training bug.

Newspapers and magazines

Older editions mix columns of text with advertising and pictures, so a model can easily read the page in the wrong order.

Annotation marks each spread and column in the sequence it should be read, so headlines, ads, and body text stay separate.

Reading order is a first-class label here. Without it, extraction models concatenate a caption into a paragraph and learn a sentence that never appeared in the edition.

Digitizing archives

Libraries and organizations still hold records that exist only on paper, from forms to handwritten entries that have never been transcribed.

Labeled pages let a model move through those archives, read the fields, and return structured results without a person retyping every sheet.

We keep the page image beside the text whenever ink quality, a stamp, or a folded corner would make a plain-string export dishonest.

Paper read where it actually appears

Document reading is no longer limited to a clean scan. A robot cashier or assistant has to read paper in the place where it appears — on a counter, in a hand, under shop lighting.

Human labels supply the context to tell a glossy magazine, a clinical form, and a till receipt apart before the system acts on what it sees.

The ontology for this work includes document type as well as fields, because a date on a receipt and a date on a prescription are not the same training signal.

Quality on this modality

Text quality is an offset you can highlight back onto the page.

If we cannot paint the span onto the source and have a second reader agree, the label is not finished. Taxonomy arguments are settled with examples, not with a longer synonym list.

Span fidelity

Start and end offsets are checked against the original characters, including punctuation rules you chose. Off-by-one errors are treated as defects.

Layout honesty

When a number's meaning depends on a header or a stamp, that context remains in the annotation, not only in someone's memory.

Class separation

Neighbouring intents and entity types have counter-examples. If two classes keep collapsing, we merge or rewrite them instead of hoping annotators will 'know what we mean'.

Relation direction

Links between entities have a type and a direction that were tested on the calibration set. Orphan relations are not left in the export.

Text Annotation

Where text and documents need structure

Four places this work earns its keep. Read them as scenes, not as a pile of chips.

01Search and ranking relevance
02Support and operations language
03Moderation and policy
04Records and archives

Search and ranking relevance
Query–document pairs graded against a written scale. The work is teaching a model the difference between exact, useful, and merely related — a distinction that collapses if the grades are undefined.
Support and operations language
Tickets, chats, and emails labelled for intent, urgency, and entities the routing model must see. Tone is labelled only when the product actually uses tone.
Moderation and policy
Spans and document classes tied to a policy version. Borderline cases stay flagged. A silent forced class here becomes a silent production incident later.
Records and archives
Forms, statements, clippings, and handwriting turned into fields a system can query. We keep the page image beside the text whenever ink, stamp, or strike-through changes the value.

Questions that belong only here

Ask these before this work scales.

Yes, after calibration. We do not pretend a general entity set covers clinical, legal, or industrial terms. Your examples become the gold, and uncertain mentions are documented instead of guessed.

When layout changes meaning, yes. Pages, regions, columns, and reading order stay in the workflow. Plain-text-only projects are possible when you truly have no layout to lose.

Yes. Relation types, direction, and allowed entity pairs are part of the ontology. We test them on nested and crossing mentions before production.

Access, redaction expectations, and retention are agreed before transfer. Annotators see only what the brief allows. We will not move a claims file through an informal channel to save a day.

With a written rule: prefer the correction, keep both with a state, or escalate. The worst outcome is an annotator silently choosing the tidier number.

A packet that includes a clean document, a damaged scan, a table, and a thread where intent is mixed. That mix exposes whether the taxonomy is ready.

Entities

Domain-specific named entity recognition

Generic entity sets like person, organisation and location rarely match what a domain model needs. The work starts with a taxonomy built for your text and your downstream decisions.

Taxonomy design

We draft the entity types with you, write a definition and positive and negative examples for each, and test the draft on a calibration sample before production. Types that annotators keep confusing are merged, split or redefined at this stage, while changes are still cheap.

Nested and overlapping entities

Real text nests. 'Bank of England base rate' holds an organisation inside a financial term. The guideline states whether the schema is flat, where only the outermost or innermost span is kept, or nested, where both are kept. Overlapping spans that aren't strictly nested are allowed only when the schema supports them, and are marked explicitly.

Boundary rules

Rules cover articles, titles, possessives, punctuation, abbreviations in brackets and line breaks. For example: is it 'the Supreme Court' or 'Supreme Court'? Is 'Dr.' part of the person span? Character offsets are taken from the original text, never from a cleaned copy, so spans map straight back to your source.

Domain glossaries

Legal text needs parties, clauses, obligations, dates and jurisdictions. Finance needs instruments, amounts, currencies, tickers and reporting periods. Medical text needs conditions, medications, dosages, procedures and negation. E-commerce needs brands, product attributes, sizes and materials. Each project gets a glossary of domain terms and their canonical labels, maintained alongside the guideline and reviewed by someone familiar with the domain.

New to the process? Start with how data annotation works.

Meaning

Intent and sentiment labelling

Intent and sentiment labels look simple, but they're judgement calls. Consistent labels come from written definitions and agreed edge-case rules, not from annotator intuition.

Label definitions

Each intent gets a one-sentence definition, a list of typical phrasings, and the neighbouring intents it is most often confused with, with a rule for telling them apart. Sentiment labels are defined by the writer's attitude toward a stated target, not by whether the topic sounds positive or negative.

Single-label versus multi-label

Single-label schemes force one answer and suit routing decisions. Multi-label schemes allow a message to be both a complaint and a cancellation request. The guideline says which scheme applies. In single-label work it also sets a priority order for messages that really do contain more than one intent.

Sarcasm and mixed sentiment

Sarcasm is labelled by intended meaning, and a sarcasm flag records why the surface words disagree with the label. Mixed sentiment is handled either with aspect-level labels, such as positive about delivery and negative about price, or with a dedicated mixed class. Annotators never average two opinions into neutral.

LLM training

RLHF-style preference ranking and response comparison

Preference data teaches a model which of two or more responses is better. Its value depends entirely on whether different raters mean the same thing by 'better'.

Rubrics

Raters judge responses against named criteria, such as instruction following, correctness, helpfulness, safety, tone and formatting, each with a written description of what good, acceptable and poor look like. Where criteria conflict, for example a helpful answer that breaks a safety rule, the rubric sets which one wins.

Tie rules

Ties are allowed only when the responses really are equivalent on every criterion, and raters pick a reason for the tie. Without that rule, ties become an easy way out of hard comparisons. When a ranking uses a strength scale, such as 'slightly better' or 'much better', each step is defined with examples.

Annotator calibration

Before production, raters work through a shared calibration set with reference judgements and written explanations. Disagreements are discussed and turned into rubric clarifications. Calibration items are reused throughout the project, so drift in a rater's judgement shows up early.

Evaluation

LLM hallucination and factuality evaluation

Scoring a whole response as 'accurate' hides where it went wrong. Factuality evaluation works at the level of individual claims.

Claim-level checking

Evaluators break a response into atomic factual claims, then mark each one as supported, contradicted or unverifiable. The response score is built up from its claims, so a long answer with one serious error isn't rated the same as a short, clean one.

Source grounding

For retrieval or document-grounded tasks, each claim is checked against the provided sources only. The evaluator records the supporting passage, or notes that none exists. A claim that is true in the world but missing from the sources is labelled as ungrounded, because that is the failure a grounded system needs to see.

Severity tiers

Errors are tiered, for example minor (imprecise wording), major (a wrong fact that changes the meaning) and critical (harmful or seriously misleading content). The tiers are defined for your domain, with examples, and reported separately so a critical medical error is never averaged away by many minor ones.

Consistency

Inter-annotator agreement and adjudication

Agreement tells you whether a label set is learnable. We measure it on overlapping samples and report it per label, as a measured quality target agreed in the pilot, not a guarantee.

Measurement

A share of items is labelled independently by more than one annotator. Depending on the task we use chance-corrected statistics such as Cohen's or Fleiss' kappa or Krippendorff's alpha for categories, and span-level F1 for entity boundaries. Low agreement on a label is read as a guideline problem before it is treated as an annotator problem.

Adjudication

Disagreements go to an adjudicator, who picks the final label and records the reason. Recurring disagreement patterns are fed back into the guideline as new rules or examples.

Why consistency matters to model performance: why training data quality matters.

Change control

Versioned guidelines

Guidelines change as edge cases turn up. Every change is versioned, so you always know which rules produced which labels.

How versions are managed

Each guideline version has a change log with the rule changed, the reason and an example. Every delivered item records the guideline version it was labelled under. When a change affects earlier work, we agree with you whether to relabel affected items or keep them flagged.

Quality control

QA stages for text annotation

Every project starts with a calibration pilot, runs on written rules and goes through two-stage review.

Two-stage review

Annotators self-check spans, labels and rubric scores before submitting. An independent reviewer then audits a sample of every batch. They check boundaries, label choice, rubric reasoning and source citations, and log errors by type. Batches that miss the agreed threshold are returned for rework, and recurring issues update the guideline.

Delivery

Delivery formats

Text labels are delivered in the shape your training or evaluation pipeline expects.

Common formats

JSON or JSONL with character-offset spans and attributes; CoNLL-style token files with BIO or BILOU tags; classification tables in CSV; and preference or evaluation records in JSONL with per-criterion scores, rationales and guideline version. Custom schemas are agreed in the pilot and tested against your loader.

Inspect Sample Deliverable & QA Rubric

Illustrative sample

{
  "info": "Illustrative sample — not client data",
  "doc_id": "contract_0042",
  "guideline_version": "v1.3",
  "text": "The Supplier shall deliver 500 units to Acme Retail Ltd by 30 June.",
  "spans": [
    { "start": 4, "end": 12, "label": "PARTY_ROLE" },
    { "start": 27, "end": 36, "label": "QUANTITY" },
    { "start": 40, "end": 55, "label": "ORGANIZATION" },
    { "start": 59, "end": 66, "label": "DATE" }
  ]
}

Stage 1 — Annotator self-check

Spans match boundary rules and original offsets
Labels follow the definitions and glossary
Rubric scores carry a written rationale
Each claim has a verdict and source reference

Stage 2 — Independent reviewer audit

Independent reviewer audits a sample of each batch
Agreement measured per label against the target
Disagreements adjudicated and reasons recorded
Failed batches returned for rework before delivery

Text annotation technical FAQ

How do you handle nested and overlapping entities?

The guideline first states whether the schema is flat or nested. In flat schemas a written rule picks the outer or inner span. In nested schemas both spans are kept with their own labels. Overlaps that aren't strictly nested are only allowed when your schema supports them, and are always marked explicitly.

How is inter-annotator agreement measured and reported?

A share of items is labelled independently by multiple annotators. We use chance-corrected statistics such as kappa or Krippendorff's alpha for categories, and span-level F1 for entity boundaries, reported per label. The agreement level is a measured quality target agreed in the pilot, not a guarantee.

When are ties allowed in preference ranking?

Only when two responses really are equivalent on every rubric criterion, and the rater records why. Otherwise raters must choose, using a defined strength scale. That stops ties becoming an easy way out of hard comparisons, which would weaken the preference signal.

How do you evaluate hallucinations in LLM outputs?

Responses are split into atomic claims. Each claim is marked supported, contradicted or unverifiable against the provided sources, with the supporting passage recorded. Errors are tiered by severity, such as minor, major and critical, and reported separately so serious errors are never averaged away.

How are sarcasm and mixed sentiment labelled?

Sarcasm is labelled by intended meaning, with a flag recording the mismatch with the surface words. Mixed sentiment is captured with aspect-level labels, or a dedicated mixed class if your schema uses one. It is never collapsed into neutral.

Test us on your hardest text first.

Start with a Free Calibration Pilot

Start a conversation

Send the page that changes meaning in the margin.

A table, a strike-through, a thread with two intents. If the taxonomy survives those, it can survive production.

Request a Quote