Data Type Detector

What the data type detector does

Upload a CSV or spreadsheet export and the tool estimates the most likely type for each column: integer, decimal, date, boolean, string, or empty. That gives you a fast schema check before importing the file into a warehouse, API pipeline, BI tool, or modelling workflow.

Why type detection matters

Many data issues are really schema issues. Numbers stored as text sort incorrectly. Dates stored as strings break timeline analysis. Boolean flags written as yes, no, true, false, and 1, 0 in the same field often cause silent failures. Detecting the likely type early helps you spot columns that need normalization before they reach a downstream system.

How to interpret the results

• Integer: whole numbers with no decimal places.
• Float: decimal numeric values.
• Date: values that consistently match common date patterns.
• Boolean: flags such as true/false, yes/no, or 1/0.
• String: free text or mixed values that do not fit a cleaner type safely.

Common schema problems this catches

CSV files often look fine until they are imported into another system. A column called policy_number may look numeric, but it should stay as text if it contains leading zeros. A column called invoice_date may include a few text values like "TBC" or "pending", which prevents reliable date parsing. A currency column may contain commas, symbols, or blanks that need cleaning before totals can be trusted.

The detector is especially useful before building dashboards, loading warehouse tables, validating bordereaux files, or sending a dataset to someone else. It gives you a quick map of columns that are ready, columns that need standardisation, and columns that need a human decision because the values are mixed or ambiguous.

How to decide whether to override the detected type

Automatic detection should not replace business meaning. Product codes, account numbers, policy references, zip codes, and SKU values are often made of digits but should usually remain strings. Dates should only be treated as dates if the format is consistent enough for the target system. Boolean fields should use one agreed convention, such as true/false or yes/no, rather than a mixture of labels. If the detected type conflicts with how the field is used, document the intended type and clean the source.

Limits of automatic detection

Type detection is a strong first-pass heuristic, not a contract. Locale-specific dates, mixed identifiers, padded codes, and Excel serial dates may need manual review. If a field contains a combination of numbers and text, the safest fallback is string. Use the detector to identify suspicious columns quickly, then clean them with more intent.

Checklist before importing a CSV

• Confirm identifiers with leading zeros are stored as text, not numbers.
• Standardise date formats before loading the file into a database or BI tool.
• Remove currency symbols and thousands separators from numeric fields when the destination requires plain numbers.
• Decide how blanks, nulls, unknowns, and not-applicable values should be represented.
• Review any column detected as string when you expected a number, date, or boolean.

Examples from everyday data work

A sales export might contain order IDs, dates, customer names, quantities, prices, and refund flags. The order ID should usually be text even if it contains only digits. Quantities should be integers, prices should be decimals, dates should parse consistently, and refund flags should use one boolean convention. If the detector finds that a price column is mostly strings, it is a warning that symbols, commas, spaces, or text labels may break totals later.

In insurance or operational datasets, the same issue appears in policy references, loss dates, claim counts, locations, and status fields. The detector helps you find those problems before a validation tool, bordereaux upload, or dashboard calculation fails downstream.

What to do after detection

After reviewing the detected types, create a short target schema for the file. Write down the intended type for each important column, then clean the values that do not match. That might mean converting dates to one format, replacing text placeholders with blanks, removing currency symbols, or keeping numeric-looking identifiers as text. A target schema turns a one-off cleanup into a repeatable rule that another analyst or system can follow later.

Columns that deserve manual review

Pay extra attention to columns where the detected type is different from the column name. A field called amount that is detected as string, a field calleddate that is detected as mixed text, or an identifier detected as a number can all create downstream problems. These columns are usually the best place to start because they often explain broken imports, failed joins, and unreliable dashboard totals.

Related tools and guides

• Missing Values Checker to review completeness alongside schema quality.
• CSV Column Extractor to isolate the columns you want to normalize first.
• Data Governance Tool to document the schema after review.
• Data Governance Guide for broader schema and ownership practices.