What Is a Historian?
A historian, also known as a process data historian or operational historian, is a specialized software system designed to collect, store, and retrieve time-series data from industrial processes. Unlike general-purpose databases, historians are optimized for the unique demands of industrial data: high-frequency ingestion rates, efficient compression of time-stamped values, and fast retrieval of both real-time and historical trends.
Historians are a cornerstone of modern SCADA, MES, and industrial automation architectures, providing the long-term data storage needed for process optimization, regulatory compliance, and operational intelligence.
Why Historians Matter in Industrial Automation
Process data historians serve several critical functions that make them indispensable in industrial operations:
Regulatory Compliance
Many industries are required by law to maintain detailed records of process conditions over extended periods:
- Pharmaceutical (FDA 21 CFR Part 11): Batch records, environmental conditions, and equipment parameters must be stored with full audit trails
- Food and beverage (FSMA): Temperature, pressure, and sanitation data must be traceable
- Water and wastewater: Flow rates, chemical dosing, and quality parameters must be archived for regulatory reporting
- Oil and gas: Custody transfer measurements and safety system events require long-term storage
Process Optimization
Historical data enables engineers and operators to:
- Identify trends and patterns in process behavior over time
- Compare current performance against historical baselines
- Analyze root causes of quality deviations or equipment failures
- Optimize setpoints and operating parameters based on historical performance
Operational Intelligence
When combined with analytics tools, historian data powers:
- Key performance indicator (KPI) dashboards
- Overall Equipment Effectiveness (OEE) calculations
- Energy consumption analysis and optimization
- Predictive maintenance models based on historical equipment behavior
Data Compression Techniques
One of the defining features of a historian is its ability to compress time-series data efficiently. Industrial systems can generate thousands of data points per second, and storing every raw value would quickly exhaust storage capacity.
Common compression methods include:
- Deadband compression: Only stores a new value when the change exceeds a configured threshold (deadband). This eliminates redundant readings when a value is stable
- Swinging door compression: Uses a geometric algorithm that fits data within a deviation corridor, storing only the points needed to reconstruct the trend within a specified accuracy
- Lossy vs lossless: Deadband and swinging door are lossy methods (some precision is sacrificed for storage efficiency). Some historians also offer lossless compression similar to standard data compression algorithms
Effective compression can reduce storage requirements by 90% or more while preserving the meaningful shape of process trends.
Store-and-Forward
In distributed industrial architectures, network connectivity between remote sites and the central historian may be intermittent or unreliable. Store-and-forward is a critical capability that addresses this:
- Data is collected and buffered locally at the remote site (on an edge device or local historian)
- When connectivity is restored, the buffered data is automatically forwarded to the central historian in chronological order
- No data is lost during network outages, ensuring complete and accurate historical records
This capability is essential for geographically distributed operations such as water utilities, oil fields, and renewable energy installations.
Historian vs Standard Databases
While it might seem logical to use a standard relational database (such as MySQL or PostgreSQL) for time-series data, historians offer significant advantages for industrial applications:
- Write throughput: Historians are optimized for millions of writes per second, while standard RDBMS are limited by transaction overhead
- Data compression: Built-in industrial compression in historians, absent natively in RDBMS
- Query patterns: Optimized for time-range and trend queries in historians, versus relational queries in RDBMS
- Storage efficiency: 90%+ compression ratios for historians, no compression by default in RDBMS
- Integration: Native OPC, PLC, and SCADA connectivity in historians, requiring custom integration in RDBMS
That said, modern approaches (like Ignition's) increasingly use SQL databases as the storage backend for historian data, gaining the benefits of standard database tools while adding an optimized historian layer on top.
Ignition's Tag Historian Module
Inductive Automation's Ignition platform includes a powerful built-in historian through its Tag Historian module:
- SQL-based storage: Historian data is stored in standard SQL databases (MySQL, Microsoft SQL Server, PostgreSQL, MariaDB), making it accessible with standard SQL tools and BI platforms
- Tag-level configuration: Each tag can have individual history settings including sample rate, deadband, and storage provider
- Partitioned storage: Data is automatically partitioned by time period for efficient querying and archival
- Store-and-forward: Built-in buffering ensures no data loss during database or network outages
- Historical tag groups: Tags can be organized into groups with shared historian settings
- Easy trending: The built-in Easy Chart and Power Chart components provide powerful tools for visualizing historical data
- Ad-hoc queries: The system.tag.queryTagHistory scripting function enables programmatic access to historical data
Advantages of Ignition's Approach
Ignition's SQL-based historian offers several unique advantages:
- No proprietary data format: Data is stored in open, accessible SQL tables
- Unlimited tags and connections: No per-tag licensing for historian storage
- Cross-platform: Works with any supported SQL database
- IT-friendly: Standard database administration tools, backup procedures, and replication mechanisms apply
- Seamless integration: Historical data is immediately available for reports, dashboards, and analytics within the Ignition platform
Best Practices for Historian Deployment
- Define retention policies: Establish how long data should be kept at full resolution versus downsampled or archived
- Configure appropriate deadbands: Set deadband values based on the precision requirements of each measurement
- Plan storage capacity: Calculate expected data volumes based on tag count, sample rates, and compression ratios
- Implement redundancy: Use database replication or clustering to prevent data loss
- Monitor performance: Track database size, query response times, and ingestion throughput
- Secure access: Implement role-based access to historical data, especially for regulated industries