October 2017

Trends and Resources

Business Trends: Smart refineries—Leveraging data for operational intelligence

In the highly competitive, capital- and asset-intensive oil and gas industry, demand is growing for the implementation of information technology (IT) practices covering the following functional requirements.

Agrawal, M. D., Morawala, A., Tata Consultancy Services

In the highly competitive, capital- and asset-intensive oil and gas industry, demand is growing for the implementation of information technology (IT) practices covering the following functional requirements:

  • Informed decision-making capabilities—collaborative decision-making and intelligent decision automation
  • Analytic capabilities—descriptive diagnostics, predictive and prescriptive techniques
  • Information management capabilities—describe, organize, integrate, share, govern and implement.

Executives do not want to look at past performance, but would rather use present and historical performance, as well as operational models, to navigate the road ahead, including how to satisfy near-term changes in demand. They must know which actions to take now to yield the most profitable outcomes tomorrow—or even later today.

A convergence of IT and real-time operational technology (OT) is raising the bar for managing downstream operations. The volume and subsequent values of the information generated in production operations are increasing, and the convergence of IT and OT is increasing the volumes and complexity of data and information.

The need for an operation intelligence framework, a methodology and the technology needed for implementation are discussed here, along with business outcomes.

About the domain

Downstream refining is a potentially risky business. Global oil and gas economics, local recessions and plant productivity are all factors in defining business profitability. A trend toward “smart manufacturing” is seen, through  which the risks posed by debilitating factors can be mitigated, to a large extent.

Smart refineries are the result of informed, near-real-time decision-making that is based on historical and present process and business data, providing the ability to predict near-future productivity. When built into operations, such intelligence results in operational intelligence (OI)—the ability to analyze information to predict future performance.

Operational intelligence: An industry perspective

Embedding intelligence in daily refinery operations is one of the key steps toward achieving OI, according to global best practices. OI monitors, identifies and detects situations related to inefficiencies, opportunities and threats, and provides operational solutions that help refineries address all aspects of plant performance in real time.

Informed real-time decisions are achieved in key areas,

  • Improved feedstock flexibility
  • Growth in high-value product yield
  • Guaranteed availability
  • Increased logistics capabilities
  • Reduced risk.

OI delivers visibility and insight into data, real-time events and business operations, and enables businesses to make decisions and act on these analytic insights through manual or automated actions. Data consolidation and the integration of operation and business systems offer a firm foundation for business transformation. Making sense of the surfeit of data is a major step toward OI, which:

  • Enables a common view of refinery key performance indicators (KPIs)
  • Provides a uniform enterprise monitoring system that encompasses all operations
  • Maximizes data visibility and asset availability
  • Delivers multidimensional analytics on critical business parameters
  • Improves response to change in operational parameters and takes proactive action through predictive analytics
  • Develops business and operations forecasting
  • Optimizes resource allocation and production capacity
  • Increases profitability.

These performance indicators rely on a strong real-time data management strategy. Data is continuously generated through multiple interactions between refinery processes and functional areas. The goal is to aggregate such data, contextualize it, generate useful information through advanced analytical tools, and then disseminate this actionable information to its target audience.

Role of data management in OI

In any production enterprise, data is generated by numerous systems and processes. These can be broadly classified into four sets: production data, process data, master data and business data (FIG. 1).

FIG. 1. Sources of data in a downstream refinery.
FIG. 1. Sources of data in a downstream refinery.

When data is generated from any base system, it contains no links to other processes; the raw data generated relates to only those parameters that create it. Multiple systems may make use of similar data, but only from their individual databases. Such data has limited “information value” because:

  • These data clusters are islands of information
  • They vary with different systems at different locations
  • They are individualistic and lack comprehensive information of the entire system
  • The same data may be available at multiple locations (i.e., duplicity of data)
  • Data is generated from disparate systems and applications across the organization.

Once all data is consolidated in one unified, structured database, it can be accessed by all systems and processes to achieve close coordination through feedback loops.

Downstream refining—data dependencies

Downstream refining comprises numerous processes and functions that are mutually related through information exchange. TABLE 1 provides insight into the interdependencies of processes and functional areas in a downstream refinery.

For example, the “optimization” function occurs across the entire refinery process chain, from crude intake to final product selling. Depending on the process, optimization can happen in any form: technical (asset strategy, blending); business-oriented (crude cost, product cost); customer relation management (customer segment servicing, product pricing); and retail (risk management, improved payment schedule).

Similarly, a process can derive information from multiple functional areas. Each of the functional parameters, optimization through “process control,” has information to offer to the “product management” process.

Managing data

FIG. 2. Indicative architecture of a data management system, integrating disparate data silos into a single source of enterprise information. Source: Gartner, 2009.
FIG. 2. Indicative architecture of a data management system, integrating disparate data silos into a single source of enterprise information. Source: Gartner, 2009.

Effectively controlling such complex operations requires real-time insight into the various process cycles. Data management includes:

  • Gathering, aggregating and contextualizing data. Linking data to other devices or systems can increase its effectiveness. For example, data from a temperature sensor by itself is not as useful as data that is linked to various, temperature-dependent processes.
  • Analyzing data to identify actionable insights that create better outcomes.
  • Predicting refinery performance based on the analysis of past and present data.
  • Generating actionable information, which differs with respect to the perspective. For business users, this information can be in the form of charts, dashboards, reports, etc., to decipher the state of the refinery in real time.

From an IT perspective, FIG. 2 depicts the basic architecture of a data management system.


Shown in FIG. 3, a rule-based approach—an essential abstract tool to generate actionable intelligence from data—enables decision-making. The major containments within this approach are briefly described here.


Oil and gas operations involve distributed data sources in process, business and asset management systems that must be consolidated and aggregated. The realities of such data sources include:

  • Dispersed geography—covering multiple assets and plants, distribution stations, refineries, etc.
  • Data source variety—manual and digital sensors on pumps and distribution lines, refinery process lines, laboratories
  • Data mode variety—continuous data from DCS/ historians, discrete data from field operator manuals, mobile device data collection, laboratory sample data.


FIG. 3. An approach to building intelligence from refinery data.
FIG. 3. An approach to building intelligence from refinery data.

Create and maintain functional/operational relationships between data elements from disparate sources. In FIG. 3, contextualization is evident within each horizontal block. Production system information can be functionally related to overall performance metrics, so business systems information can be related to performance as demanded by operations.


Transfer relevant operational performance information to the appropriate business-level systems. For example, inventory management and control information are available across the refinery in real time, based on the produced items. This allows procurement departments to monitor low inventory stocks and initiate procurement without being informed by others. This may be considered as closing a feedback loop. Alternatively, refinery operations may redefine schedules or other indices based on up-to-date information.


Data analytics transform data into real-time performance intelligence through the application of business rules. This part of the framework is responsible for advanced features of smart refineries, such as predictive maintenance, which utilizes historical data from multiple sources to build accurate, testable predictive models, allowing the generation of predictions and risk scores. This provides a more effective and efficient way to maintain and monitor critical assets, with high availability and reliability.


Graphical representations of KPIs support the context- or role-based navigation of information based on persistent interrelationships.

FIG. 4. An indicative dashboard of overall utility functions.
FIG. 4. An indicative dashboard of overall utility functions.

In some instances, this means enabling drill-down from multi-plan representations to individual facilities and systems. In the process unit overview and utility overview dashboard in FIG. 4, critical parameters are graphically adapted to suit operator intuition. Such visuals assist rapid decision-making. This framework is built using multiple IT tools and digital technologies.

Platforms and management approaches

Platforms and management approaches that provide a foundation to build robust OI solutions include:

  • Data modeling: The ISA 95 standard describes a framework for connecting plant floor operations
    to enterprise applications. Data models specific to refineries based on this standard provide a good starting point. However, proper configuration of the models is required to accurately reflect operations.
  • Data lifecycle management: Short-term storage of actionable data and longer-term archival and retrieval of selected OI data. Providing full resolution of archived and short-term data is necessary, but the difference is in the access rate. Short-term operational data must be immediately available for analysis. Retrieval of long-term archived data will be used for strategic decision-making.

0   Real-time KPI and extensive historian data access are part of data lifecycle management.

  • Business process management, data mining and discovery: These business process-centric approaches are helpful for developing a desired layer of data aggregation, analysis and virtualization.

0   A business process is a collection of related, structured activities that produce a service or product that meets the needs of the business.

0   Data mining techniques help in the aggregation of data from heterogeneous sources.

0   Business process management helps provide modeling tools, establish links of common objects and enable process modeling, an important requirement of OI and real-time visualization.


Some examples of how data consolidation helps the refinery are listed here. Major parameters depicted are energy consumption, production parameters and KPI variance. Other information related to production includes asset health (maintenance), HSE parameters and human resources availability.

Production monitoring

Production managers looking at the consolidated dashboard can see the whole picture at a glance, including:

  • Amount of crude processed and the yield
  • Available inventory
  • Basic quality parameters of the products
  • Energy consumed and other utility (water, gas, steam, etc.) information
  • Unit KPI variance (actual KPI vs. target KPI)
  • Asset information and human resources availability.

Production managers are informed of core production parameters, asset availability, work permits issued and human resources availability. These outer circle parameters ensure awareness of production efficiencies and provide foresights on threats to normal production. Real-time information about related metrics, KPIs and performance highlights enable timely action and escalations. Asset maintenance and work order(s) issued for each major unit and its ancillary assets can be visualized, and it can be determined if multiple work orders have been issued, which will hamper production. Engineering departments generally track scheduled and breakdown maintenance to monitor the mean time between failure (MTBF) of assets. This historical information can be used to effectively schedule preventive maintenance.

Such historical information, combined with manufacturer design and operational parameters and rule-based algorithms, can be directed toward predictive maintenance. While preventive or scheduled maintenance must be carried out irrespective of asset functionality, predictive maintenance saves resources by generating alerts only when the asset’s functionality tends to exceed its usual parameters.

A dashboard, like that shown in FIG. 4, is used by process engineers and provides a total graphic representation of the various resources consumed and the cost of production at the unit.

Tracking the energy and resources consumed is necessary, as they represent a high cost in the production cycle. Unit efficiency is also measured by how resources are spent in the production cycle. In the dashboard in FIG. 4, the consumption of various resources, such as steam, water, fuel and nitrogen, is tracked to monitor unit performance.

Key operating parameters, such as boiler efficiency and compressor running hours, indicate the efficiency of the unit’s production. The overall cost of resources during production is also shown. This cost is compared with budgeted estimates and is a measure of the effectiveness of the production process.

The architecture of OI

OI is not a batch of applications that churns out the resulting intelligence, but is rather an architecture or design involving multiple platforms and systems spanning across timeframes and roles.

The architecture includes: historians that provide a robust foundation for an OI database, with the ability to aggregate, contextualize and analyze large volumes of data from various sources such as DCS, control systems and other OT platforms; MES systems with built-in intelligence capabilities that operate on data within the context of the application’s data; and business analytics that offer historical, present and predictive views of business operations.

To date, no single vendor delivers a complete OI solution. Customers generally take a blended approach, and IT service providers interface and integrate multiple applications to suit a customer’s requirements.

Business outcomes

OI enables an improved decision-making capability based on enhanced information quality and availability extracted from a mass of data points. Such information produces alerts based on KPIs and key risk indicators.

Often, the greatest opportunities for key improvements and increased efficiencies occur when silos of previously independent data are integrated or related in a new and meaningful way. This requires effectively spanning numerous disparate sources of information from tag-based systems (control systems) and relational data sources (e.g., a production database) to enterprise applications (e.g., an ERP system). Information from these diverse systems must be capable of being combined in expressions or calculations and related through structures, filtering and navigation.

Diverse operation and business data converge as a single, virtual database that contains all information that is otherwise distributed in various systems, such as SAP, CMMS, LIMS, planning, production, inventory, historian, machine monitoring, etc.

This integration allows multi-dimensional analytics that can be analyzed from various perspectives (production, supply chain, planning, maintenance, etc.).

A proprietary scalable and reliable platform delivers manufacturing integration and intelligence. This flexible solutiona ensures full integration between shop floor systems, MES and SAP ERP to deliver benefits that include:

  • A comprehensive platform that enables composite applications
  • The creation of an environment that integrates disconnected environments
  • Role-based visibility
  • A technology agnostic platform that interoperates across multiple platforms.

In addition to functionality, platform scalability and delivery expertise, reusable templates, data models and a configuration kit reduce the overall development and deployment time.

Key takeaways

The level of OI achieved is almost entirely dependent on how well the plant, operational and business data are managed. This involves complex data management frameworks, coupled with process changes, to absorb the intelligence that is spelled out by the transformed data, as well as adherence to a disciplined, structured manufacturing system.

While many tools can assist in OI, such tools must be selected carefully based on individual needs. Each enterprise—even within the same sector—is an individual entity with its own quirks and needs, and the tools must be tweaked to give optimal results.

OI is a prequel to operational and business excellence, and an important milestone to achieve. HP


  a TCS Global Operations Solution

The Authors

Related Articles

From the Archive



{{ error }}
{{ comment.comment.Name }} • {{ comment.timeAgo }}
{{ comment.comment.Text }}