July 2019

Special Focus: The Digital Refinery

Advanced crude management by NIR spectroscopy combined with topology modeling

Crude characterization is critical for refinery optimization.

Lambert, D., Saint-Martin, C., Benkhelil, K., Ribero, B., Topnir Systems Co.; Valleur, M., Val Innovation

Crude characterization is critical for refinery optimization. Real-time knowledge of crude properties is the only way to ensure that full value is extracted from the crude, based on simulated plant operations and products quality. A good understanding of crude oil has an impact on planning and affects all refinery operations.

Crude quality is changing rapidly. In the past, oil wells were characterized by a crude assay that would change slowly over time, allowing fairly stable feedstock qualities. Now, the availability of shale oil and other opportunity crudes is a cause for wider feedstock variability.

Unknown fluctuations of feed properties introduce instability into the crude distillation unit (CDU), necessitating larger offsets from constraints. On the contrary, a closer monitoring of crude feed quality enables better optimization of the CDU by pushing against the appropriate constraints.

It is necessary to be able to characterize crude feeds more frequently and not to rely only on crude assays. The appropriate answer lies in a technology able to analyze, both rapidly and accurately, any crude feed received at the refinery. This technology should also assist in allocating the correct crude tanks, depending on quality, and allow the setting of CDU targets based on crude quality. Moreover, promptly knowing the crude quality will help planning and scheduling activities in setting target yields and qualities for process units.

Inline near-infrared (NIR) spectroscopy has been increasingly used to replace hazardous manual sampling and low-frequency laboratory analysis.1 It is made possible by an appropriate modeling technology, allowing the prediction of the full property vector of any crude oil. Topology-based NIR models are able to identify and characterize any crude mixture. In less than 1 min, neat crudes ratios are predicted from any crude mixture, as well as properties such as full distillation curve (TBP), API gravity, total acid number, sulfur, SARA (saturate, aromatic, resin and asphaltene) and more. In addition, the full crude assays, integrating distillation cut yields and properties, can be delivered.

NIR spectroscopy

NIR range has been optimized for crude NIR analysis, as shown in Fig. 1. The best spectral domain to analyze crude oil was identified in the combination band, lying between 4,000 and 5,000 cm–1, which provides the maximum of information without residual band absorption coming from the visible spectral region. Those residual absorptions would make the analysis, and therefore the model prediction, more unstable and inaccurate.

FIG. 1. NIR range for crude oil analysis.
FIG. 1. NIR range for crude oil analysis.

The spectrum of any product is directly linked to the physical and chemical properties of this product. It is like a fingerprint of the product. Topology modeling is based on spectra matching, working through pattern recognition and database densification. It means any spectrum is used as a fingerprint of the sample. This spectrum will be positioned in the database and characterized with the closest neighbor’s spectra. Then, the full set of properties of the new sample will be predicted from the standard average of properties of closest spectra (Fig. 2). Indeed, the closest spectra means samples having the closest physical and chemical properties.

FIG. 2. Spectra matching for prediction of properties.
FIG. 2. Spectra matching for prediction of properties.


It does not use any linear model, it uses only a criteria of distance of neighbors to calculate the sample properties from the neighbors. Then it delivers, in less than 1 min, all the properties required for a given application. Moreover, it offers the possibility of extrapolation from the initial calibration range.

Only one model is used for all the properties, with the property prediction being performed from a reference database. Having one model for all the properties gives a better robustness and reliability to the prediction than does having many simple, linear models. This approach keeps the inter-correlations between the properties and makes sense in terms of sample chemistry.

This spectra-matching approach allows, for instance, the prediction of the full distillation curve as per ASTM standards by keeping the continuity of the distillation process.

The topology model is not limited by number of properties, as it deals with a single reference database. Therefore, the number of properties can be easily extended, if required, without impacting the modeling effort.

CDU, NIR spectroscopy and topology model

The early applications of NIR for crude quality have covered:

  • Oil production sites with variable quality at well clusters, with the objective of having a crude commercial grade at gathering stations2
  • CDUs in refineries with diversified crude slates, with the primary objective of avoiding transient throughput reduction during crude swings3

The first online NIR application was installed at BP’s Lavera refinery in 1992, using the topology modeling technique. Typically, about 40 quality determinations at high frequency on crude feed and four side streams were used by the CDU advanced control application. A sample of these determinations are shown in Table 1.


A screenshot from the distributed control system (DCS) for TBP online prediction is shown in Fig. 3. The efficiency of crude feed quality monitoring during a transient between two crude oil tanks can be seen. Even tank stratification can be detected and quantified with topology-based NIR models. This analysis of crude TBP has no equivalent in terms of an online system, giving the full TBP analysis every minute.

FIG. 3. Crude TBP points monitoring during crude transient.
FIG. 3. Crude TBP points monitoring during crude transient.


The precision of the TBP curve is coherent with ASTM Standard D2892 from initial boiling point (IBP) to 370°C, and with ASTM Standard D5236 from 370°C to final boiling point (FBP). Another example of the capability of topology is shown in Fig. 4, with a very efficient spectral discrimination between the origins of crude oils. This achievement is possible because topology software uses the entire NIR spectra, as well as specific proprietary axes to plot the samples.

FIG. 4. Crude oil identification and discrimination.
FIG. 4. Crude oil identification and discrimination.

Lessons learned from this methodology include:

  • The combination band spectral domain (5,000 cm–1–4,000 cm–1) is the most appropriate for crude quality determinations (Fig. 1)
  • Topology-based models have demonstrated a unique efficiency (Fig. 4) to identify and discriminate crudes, as well as to deliver a precision compatible with ASTM in less than 1 min
  • The sampling system is the most critical item of the NIR system from design and maintenance viewpoints
  • Besides increased throughput, the topology-based NIR crude application also enables a very tight control of side streams and, therefore, offers improved stability of process units downstream of the CDU.

A typical validation stage display is shown in the table within Fig. 5, where all property predictions are compared with conventional results. It is clear that all predicted properties perfectly match the standard measurements from conventional analyzers. Therefore, critical properties such as TAN, simulated distillation, assay yields, sulfur, API gravity and density are accurately predicted by the topology-based NIR models every minute.

FIG. 5. Crude oil analysis during validation stage.
FIG. 5. Crude oil analysis during validation stage.


All the described advantages allow reliable advanced process control (APC) with effective closed-loop control. The use of topology-based NIR models to enhance the CDU APC offers a rewarding application:

  • Better fractionation control, effectively using side stream qualities in real time
  • Better cutpoint control by APC
  • Better response to crude mix quality swings, reducing transient products downgrading such as gasoil and residue
  • Yield increase of selected streams without impacting quality, in particular cloudpoint and freezepoint
  • Throughput increase by better swing procedure,
    using real-time TBP of crude mix
  • Better tuning of LP and scheduling models
    • Better representation of refining scheme flexibility, using potential yields from real TBP
    • Real-time quality determinations on intermediate streams will improve continuity between short-term planning, scheduling and optimization systems (e.g., allocation of naphtha to reformer or steam cracker).

These advantages can easily capture benefits of several million USD/yr.

Other applications of topology-based NIR models

Several other NIR-based applications have emerged in a context of increasing crude slate and logistics complexity. In particular, crude traders have diversified their supply sources with opportunity crudes (including very heavy, high sulfur), distressed cargoes, condensates, shale oil and other unconventional crudes.

The receipts and storage facilities have generally not been designed to cope with such a diversified crude slate, and the crude tank farm can face the severe problems of crude segregation and mitigation of compatibility issues.

Crude receipt and storage

Topology NIR systems have the ability to provide a fast check of crude receipts from tank samples or from automatic samples to detect quality excursions from crude assays.

Given the constraints of crude segregation, it is not unusual to find mixtures of several neat crudes in a single tank. An NIR system with topology is used to provide a reliable estimate of the tank composition, which is essential for crude blending and to mitigate compatibility risks. One example of achievement is shown in Fig. 6.

FIG. 6. Crude identification by NIR topology model from a mixture of five crude oils.
FIG. 6. Crude identification by NIR topology model from a mixture of five crude oils.


It can be seen that the topology-based NIR model identifies the origin of crude oils in the blend and precisely quantifies the ratio of each pure crude in the blend. This is particularly useful to accurately track the crude tank composition in tank farms.

Crude blending

Neat crudes are often blended at the gathering stations of oil fields. These blends are mixed in the refinery receipt tanks that are themselves blended to provide the feed to the CDU under many constraints, such as thermal, condenser capacity, TBP profile, compatibility and TAN.

Some refineries use crude blenders to saturate these constraints. In that respect, spectral blending is an efficient tool to compute optimal blend recipes. A screenshot of proprietary blending software using spectral blending and topology is shown in Fig. 7.

FIG. 7. Blending software using full crude spectra and topology.
FIG. 7. Blending software using full crude spectra and topology.


The software uses spectra to perform the calculation based on topology to estimate properties from a spectrum. A spectral plane displays the spectral boxes used to identify the various origins of crude oils, as well the spectral box used to set up the crude blend target.

Short-term crude scheduling

Uncertainty on crude receipts timing, approximate knowledge of crude quality and a large number of constraints make short-term crude scheduling a challenging task, particularly when the refinery is producing bitumen, lubes or specialty products (e.g. arctic diesel) that add constraints on molecular species. The requirement for fast crude assay prediction is a challenge for each site.

Using NIR with topology enables more up-to-date crude oil assay information and a better prediction of cuts on nonlinear properties for complex crude mixtures—one major requirement to ensure reliable scheduling.4

The topology-based NIR model is able to predict the full crude assay, integrating distillation cut yields and properties from only crude oil spectra. This analysis data sheet is delivered in less than 1 min, which is an order of magnitude compared to the days required to obtain such results with conventional methods. An example of such an analysis is displayed in Fig. 8.

FIG. 8. Prediction of full crude assay.
FIG. 8. Prediction of full crude assay.


NIR spectroscopy, linked to topology modeling, is a powerful technology for identifying and characterizing crude oils. It has two major advantages:

  • It is fast, providing a comprehensive crude assay in a matter of minutes, rather than days as seen with conventional methods.
  • It is robust and can be supported (from both hardware and model viewpoints) by refinery laboratory chemists, which is not the case for all spectroscopic methods. Model support and updating requires only the addition of sample in the database. This is not the case with traditional regression models, where chemometricians are required to redevelop the model equations.

Several decades of constantly applied research and continuous development based on field-proven system experiences have enabled the capture of significant benefits in various levels of operation in the refinery:

  • For planning and scheduling:
    • Better tuning of linear model for planning and optimization of crude slate
    • Faster crude quality information to scheduling system for better adherence to optimal plan
  • For the crude tank farm:
    • Better crude segregation and crudes compatibility management
    • Detection of crude tanks stratification
  • For the CDU:
    • Faster crude swings without loss of throughput
    • Higher ADU throughput by simultaneous saturation of upper and lower fractionation sections
    • More stable downstream operations
    • Lower risk of corrosion of CDU process lines and optimal additives injection.

Typical benefits collected from installed bases are within the range of several million USD/yr. A promising future exists for inline, topology-based NIR applications, as molecular characterization will become essential for future complex crude slates and crude-to-chemicals plants. HP


  1. Valleur, M., “Inline laboratory and real-time quality management,” Hydrocarbon Processing, March 2009.
  2. Lambert, D., “Crude control,” Hydrocarbon Engineering, October 2007.
  3. Lambert, D., “Online optimization with Topnir Systems,” ARTC 9th Annual Meeting, Kuala Lumpur, Malaysia, March 2006.
  4. Kelly, J. D. and J. L. Mann, “Crude oil blending scheduling optimization, an application with multi-million dollar benefits,” Hydrocarbon Processing, June 2003 and July 2003.

The Authors

Related Articles

From the Archive



{{ error }}
{{ comment.comment.Name }} • {{ comment.timeAgo }}
{{ comment.comment.Text }}