Data Volume, Efficiency Changing Oil, Gas High Performance Computing Needs


The need to quickly analyze and storage growing volumes of data in oil and gas operations are driving the need for greater capacity in high performance computing (HPC), or the use of parallel processing to run advanced application programs efficiently, reliably and quickly.

While HPC has been used by the oil and gas industry for some time, the need to quickly make high-stake business investment decisions and stay competitive in the current market is creating a shift in the industry’s HPC needs.

Technologies that have allowed oil and gas companies to unlock previously inaccessible resources, and the need of geoscientists to better understand subsurface prospects, have created massive growth in dataset sizes and computational needs, according to a Hewlett-Packard (HP) webinar on how HPC is accelerating exploration and production. Oil and gas companies also are tracking a wider variety of data from reservoir, sensor and machinery parameters means that the amount of volume generated by oil and gas operations.

Rising infrastructure costs and lower budgets, the need to quickly turn data into insight that is available to the right people, and the “digital oilfield”, or managing, measuring and tracking global oilfield data through web-based visualization, are other trends that have created demand for a new style of IT, according to HP webinar.

The growing volumes and complexity of data generated in global oil and gas operations has created the need for high performance computing capability in the public cloud, according to officials with Dallas-based Nimbix. Earlier this year, the company introduced its Jarvice cloud-based, high performance data analysis (HPDA) solution platform, and the expansion of Jarvice’s graphics processing unit (GPU) acceleration in the solution. This solution delivers advanced computer aided engineering, Big Data analytics and seismic processing capabilities for the oil and gas industry. Jarvice works in tandem with the company’s public cloud platform to provide high performance computing capacity to oil and gas companies.

Nimbix Chief Executive Officer Steve Hebert told Rigzone that the company has seen an increase in the number of oil and gas companies seeking new ways to access HPDA. According to IDC, worldwide HPDA server revenue will rise at a 13.3 percent compound annual growth rate for HPDA-focused servers until at least 2017. HPDA will bring HPC into the mainstream enterprise, powered in part by GPU-accelerated Big Data solutions for both simulation and analytics-based data analysis.

“GPU acceleration will power the next generation of these complex analytics, and Nimbix will offer both Software-as-a-Service and Platform-as-a-Service models to meet these needs,” said Hebert in an Oct.28 press release.

Nimbix supports companies in three domains, traditional seismic processing, or reservoir simulation, engineering simulations, and Big Data. All three require the same underlying computing technology, Hebert said.

The company’s public cloud solution differs from the high performance computing solutions offered for e-commerce, which require low power in terms of actual calculations but require the scale to handle many users, Nimbix CTO Leo Reiter told Rigzone. Instead, Nimbix caters to the niche demand in industries such as oil and gas, which need high power and performance with the self-service and availability of mass market cloud solutions. Jarvice is powered by NVIDIA GRID GPUs and Tesla GPU accelerators.

“The large data sets associated with seismic processing and growing use of sensors creates lots of computational complexity in oil and gas means that very complex algorithms must be used to get accurate information,” said Hebert.

Growth in horsepower demand is occurring both on the exploration and production side. In terms of manufacturing components for pipelines and rigs – which operate under pressure – the complexity of the simulation needed to get higher fidelity responses is increasing. To run the necessary simulations, companies are needing to use more cells, and the volume of cells needed and complexity of the simulations means that more computing horsepower is needed, Hebert explained.

On the geophysics side, higher amounts of data are being gathered as the number of streamers for collecting seismic data increases. 

“The types of algorithms needed to process data efficiently keep getting smarter and smarter, which in turn demand more and more computing capacity,” Hebert said.

Many cloud solutions cater to developers who have to program, but Nimbix caters to the platform and service side of the market, and offers the ability for some developers to build these algorithms, and then package them as applications to be run by geologists on demand. The standardization of algorithms making their way into the analytics suite means that they can be provided on demand versus handing out raw capacity and leaving companies to handle it on their own, said Hebert.

Although some use cases exist for Nimbix’s technology downstream, the company has primarily been working with upstream oil and gas companies. Earlier this year, the company announced plans to expand its data center capacity in Houston, meaning customers can run computations closer to data. Even if they don’t want to put data in the cloud, the company will now be within fiber optic range of data, Hebert said.


In the past, oil and gas companies met their HPC needs by investing in their own supercomputers, or timeshared on a mainframe or shared supercomputing resources at a bureau to do this type of work. The trends of growing horsepower and complex computing demands has prompted oil and gas companies to examine whether to expand their high performance computing capacity.  

However, building a traditional data center may not be an option, particularly in inhospitable environments. In many cases, companies also would overbuy capacity or resources to account for spikes in demand to allow work to continue without crashing the system, said Sam Coyl, president and CEO of Netrepid, which offers hybrid cloud and data center services to organizations. However, that excess capacity was only need 3 to 5 percent of the time each year, meaning that most of the capacity went unused.

Now, a shift is occurring towards cloud hosting that allows companies to dynamically expand their capacity as needed, but only pay for what they are using. Similar trends are occurring in healthcare, banking and other industries. Now, these companies can leverage HPC technology they previously couldn’t afford, couldn’t implement or didn’t have the scalability they needed.

“Most of these companies are using hybrids,” Coyl told Rigzone. “There is still some stuff on site to handle day to day operations, but the majority of those technological needs can be put in a cloud environment so they rapidly and dynamically expand as capacity requirements are needed.”

An accounting firm during tax season is a good example of the need for dynamic HPC capacity. During tax season, a firm may increase the number of accountants from 100 to 150. Previously, the accounting firm had to pay for 150 users, even though 150 were only needed three months out of the year. By only paying for the capacity the need for their actual workload, companies can better manage costs.

As a geophysicist working in the oil and gas industry in the 1970s, 80s and early 90s, SAS’s Keith Holdaway worked with very large mainframe parallel computers. These were big old IBM mainframes that would do seismic processing and would take up a whole room – the size of a football field. The arrival of PCs in the oil and gas workplace makes it possible to take the football field of computing capacity and put it on a desktop.

“In HPC, there’s been an uptrend in this area, especially in exploration, because that’s where the really big data is and the algorithms are very complex,” said Holdaway.

This trend is occurring as industry pursues numerical simulations and dynamic simulations and 3D earth models. Many companies are going this route – even service companies are partnering with universities in the United States and in the Middle East to investigate new algorithms for processing.

 “When you think about capacity, it’s not just about the number of servers or modes you have, it’s about how powerful and dense the processing power of each machine is,” said Reiter.

Eventually, a company will run out of floor space for machines because of having to add machines. To meet this need, companies such as Intel are making hardware for systems that have more dense computations cores.

Jarvice allows users to scale capacity up and down, rather than trying to guess how much to spend in terms of data and hardware. Instead of making huge bets in terms of time and resources,

“Nimbix provides a way to democratize access to these capacities for decision-making,” said Hebert.

Nimbix’s technology also allows for calculations to be performed in real-time or in batches of data from multiple days or hours. Customers also can use public storage, but use Nimbix’s cloud for crunching data.

Nimbix’s cloud-based high performance computing infrastructure can be used for large and small data sets. While Nimbix’s solution is very much about Big Data, it’s about a specific kind of Big Data that requires big computing capacity.

“It’s not the amount and size of data that is interesting, but the value you drive from it,” said Reiter, noting the difference between Big Data and large volumes of data.

Big Data is data that can’t be processed using traditional methods.

“Big Data is more about taking information from all sorts of different domains and deriving analytical value from data processing.”

An example of large amounts of data that’s not considered Big Data is counting people of a certain age from a database with information on every single person in the world is not Big Data.

“It’s a simple query that can be done with databases. It’s huge data but simple processing,” Reiter noted.

“What would make it a Big Data problem is if we wanted to take names from databases and try and determine, based on these people’s social media habits, when they will buy their next pair of pants,” said Reiter. “Big Data involves running complex analytics, algorithm and machine learning.”

Regardless of whether data qualified as Big Data, more horsepower is needed computing, and companies are taking advantage of high performance analytics to compute data spread across HPC clusters.

It’s not all about Big Data necessarily – the volume, variety and frequency of data — but more around trying to solve much harder problems, said Holdaway.

“High performance computing is essentially an environment that enables high performance analytics – such as grids, the cloud, memory and database analysis – in which the analytics are moved to the data because the data is so big.”


HPC gives the oil and gas industry a competitive advantage in modelling pockets and finding the best spots to drill.

“High performance computing means getting to the answer faster and applying more computing power to the problem,” said Dr. Tim Lynch of Quincy, Mass.-based Psychosoft.

Traditionally, this has been done by using clusters, which are a bunch of computers acting as one in a distributed fashion attacking the problem. The latest trend in this arena is the use of GPU computing, especially in the NVidia Tesla CUDA platform, said Lynch. This puts the power of a small cluster in the form of a single machine, which can sit on a person’s desk, allowing complete sole access to the supercomputer instead of shared time on a cluster.

“This is revolutionizing the use of high performance computing in oil and gas discoveries as well as other areas,” Lynch said. “CUDA applications can also be distributed over multiple machines provided that those computers have an NVidia Graphics card so it is possible to utilize every machine on a network for certain problems.”

Because of security issues, the use of the cloud in high performance computing hasn’t caught on in the oil and gas industry like Hadoop-type situation has, said Holdaway, but anticipates that HPC will move into the Hadoop and cloud due to the need to quickly access and store data.

“As they push boundaries in new algorithms, it takes a lot more sophisticated algorithms to run in parallel on lots of data.”


According to the Council on Competitiveness report, Solve, a publication of the High Performance Computing Initiative, HPC is viewed industry leaders as a cost-effective tool for speeding up the research and development process, and two-thirds of all U.S.-based companies that use HPC say that “increasing performance of computational models is a matter of competitive survival.”

While U.S. industry representatives surveyed struggled to imagine specific discoveries and innovations from HPC, they were confident that their organizations could consume up to 1,000 increases in capability and capacity in a relatively short amount of time. However, software scalability is the most significant limiting factor in achieving the next 10x improvements in performance, and it remains one of the most significant factors in reaching 1000x. The links between U.S. government and industry also need to be strengthened to allow U.S. industry to see a benefit from government leadership in the investment in supercomputing technologies.

In mid-November, the U.S. Department of Energy announced it two new HPC awards to help put the United States on the fast-track to next generation exascale computing, which DOE said would help advance U.S. leadership in scientific research and promote America’s economic and national security.

Secretary of Energy Ernest Moniz announced $325 million to build two state-of-the-art supercomputers at DOE’s Oak Ridge and Lawrence Livermore National Laboratories. In early 2014, the joint Collaboration of Oak Ridge, Argonne and Lawrence Livermore (CORAL) was established to leverage supercomputing investments, streamline procurement processes and reduce costs to develop supercomputers that will be five to seven times more power when fully deployed than the fast systems available today in the United States.

DOE also announced approximately $100 million to further develop extreme scale supercomputing technologies as part of FastForward2, a research and development program. Computing industry leaders such as NVIDIA, AMD, Cray, IBM and Intel will lead the joint project between DOE Office of Science and the National Nuclear Security Administration.

“DOE and its National Labs have always been at the forefront of HPC and we expect that critical supercomputing investments like CORAL and FastFoward 2 will again lead to transformational advancements in basic science, national defense, environmental and energy research that rely on simulations of complex physical systems and analysis of massive amounts of data,” Moniz said in a Nov. 14 press release.



Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.