“You’ve Got to Have Courage!” BP on Going “All-In” on the Cloud

“We’ve rationalised 30% of our applications…”

An “all in” BP cloud migration will see the energy supermajor shut down its two mammoth Canary Wharf data centres in favour of shifting 900+ mission critical applications to Amazon Web Services (AWS)’s infrastructure.

The project is no lift-and-shift of a few customer-facing apps.

The software being moved to the cloud from January 2020 will include real-time oil and gas trading applications handling billions of pounds of transactions and running on custom hardware (“thankfully, no mainframes!” says BP’s Stewart Fry – the man in charge – wryly).

It will include SAP software handling HR and finance functions for a 17,500-strong global workforce; databases capturing metrics on everything from European spark spreads to upstream oil production in Angola.

What drove BP to make this move, after having taken a “dual-cloud” approach using both AWS and Azure in the US? And how did the project team build a business case?

Computer Business Review joined Stewart Fry, BP’s global VP for enterprise IT and security at Re:Invent in Las Vegas, for a refreshingly frank conversation about the motivations for and challenges of the shift.

Anticipating what it will cost to run nearly 1,000 applications in the cloud instead of on-premises is no mean feat, irrespective of the oil and gas industry’s notoriously robust ability to to crunch the numbers of high CAPEX, long life-cycle projects.

Stewart admits that early on some guesswork was involved – and keeping executives informed with rolling updates as the project evolved was crucial: “We did our first data centre exit in 2016, we’re just finishing that (our Houston data centre).

“When did the original economics, it really was based on like-for-like IaaS comparison”

“When did the original economics, it really was based on like-for-like IaaS comparison between what it would cost you to run your on-premise data centre – you do the OpEx, the people, the power – mapping that to what you think that would cost you in the cloud. And of course, it isn’t easy: it was tight! (Laughs).

“But we knew if we just picked up the existing on-premise environment and put it on the cloud it would be more expensive and there was a bunch of things that we knew we couldn’t hope to guess. So with leadership internally we developed a rolling plan: ‘this is how we’ll start with…’, and kept checking in to report what we were learning. And out of our US data centre, we’ve rationalised 30 percent of our applications.

“We guessed that into our business case and [made it happen] by forcing it through. We’d tried rationalisation programmes before and this was the most effective…”

Computer Business Review: Can you give a few examples of applications you got rid of?

“Just small things that hung around, doing internal processing work… they may not of cost anything but just consumed a virtual slice somewhere, and held data that was all locked down for a small transactional bit of one [BP] unit.

“When we’ve moved to the cloud, we’ve looked at groupings of those [semi-dormant or highly siloed applications]together and standardised on single applications, or brought datasets together.

“What’s really interesting is that 70 percent of our workloads on the cloud now aren’t what we migrated. They’re new things. The enablement of the move has opened people’s minds to the art of the possible. Because how could you have hoped, in 2016, to know Kinesis [Ed: An AWS data streaming service] and all these other new cloud feastures was coming along and what you could do with that?

“Ultimately though, if you’ve got a base case that looks pretty good, all the rest is extra value. The rolling plan we’ve had with the executive has really helped us.”

How much of an opportunity was this for to rethink the applications you are running – and how much of a database shift was there, as a result?

“A massive one. On-premises, we used to buy services from vendors with a lot of application maintenance: we were one of the most outsourced [companies] you could be.

“Our partners didn’t do anything wrong: we’d asked them to do it, but databases were sort of managed with the applications. As we move to the cloud, we’ve moved to database as a service and so we’ve sought to standardise our offerings around things like RDS [Amazon Relational Database Service].

“What this has allowed us to do is to pull workloads  together, provision the right fit-for-purpose capabilities for the demand workloads. And the other interesting thing is that all of our partners were managing the databases slightly differently.

Which databases have you gone for?

“Wherever, wherever we can, we want to go to open source. Clearly, like for many others, we have to use Microsoft, we use Oracle, we use Hana for SAP…

But we’ve been able to get our own management of the databases far more in control.”

You say you’re starting your “all-in” move to AWS in Europe in January 2020. What does the current environment look like?

Some of our most complex applications are Europe, and that’s going to be critical to our migration path. Just due to the types of technology that they have been running on; proprietary hardware that is not transferable in its current state to an AWS landing zone…

We have two major European data centres in Canary Wharf, and a secondary centre in Watford. That covers Europe and actually pulls in some non-European workloads; we do some work for Angola and services the a whole bunch of the Middle East and Africa.

“We’ve got a whole floor of a [co-located] global switch environment: but we owned all of the equipment; all the infrastructure… it’s a heavy, capital-intensive world and when we decided not to be hybrid it was a relief.

In the US we’re close to shutting off our first data centre now and the ‘big rock’s are always at the end, as we call them, the most complex big things that we’re trying to innovate on. But AWS love a problem: they’ve got really smart engineers, we’ve got really smart engineers and they’re working together to solve some of the problems that no one else has and you can navigate through that.

In the US, we’re one of the top three traders in the world of oil and gas and we’ve successfully moved all of our trading applications out of Houston onto AWS. We’ve moved over 2000 applications already.

How much have partners been part of this process?

And I meet a lot of vendors who tell me they’ve got a silver bullet for fixing my migration problems. But we’ve got great partners who work with us. We know we don’t have all the answers. But with the right partners and our own teams will we’ll find a way through the solutions.

It sounds challenging…

“You have to have courage. The energy industry has changed many times over the past 100 years. We fix big, complex problems. We don’t shy away from it. That’s been that’s our culture as an engineering company or as a technology company.

“We’re moving away from what was a traditional application maintenance agreement, which has run its course, to partners joining our squads”

“When we declared our cloud-first strategy in 2016, CloudReach really helped us, supplement our engineering teams and build out our regions, with our technical design patterns; and billing.

“(Our businesses tell me all the time they don’t want an allocation of a share of a box. They want to pay for what they consume: they’re happy to pay more if they consume more, but they want to pay less if they consume less. Getting billing right has been crucial for everyone…)

“The other two vendors we’ve worked with a lot are Wipro and Veeam; Accenture have also helped on the SAP environments.

“We’ve been building and growing talent internally though a lot.

“AWS have really worked with us on training, so we have programmes from the very basic to ‘cloud ninja’. We’ve also had to really retrain our management teams so that they think differently about managing the products they’re responsible for.

“We operate everything in BP as a service. We’ve put everything into a service team, and there are squads within that. These squads are nearly always a blend of BP and partners. We’re moving away from what was a traditional application maintenance agreement, which has somewhat run its course, to partners joining our squads.

“In that way, we may buy partners in as an FTE, as a sprint, as an outcome we’ve defined. But partners have to be part of our squads.

“What does the final model of the BP Cloud Migration look like?

When we first built our cloud environments [in the US], we basically stretched our VPN out around them. We had a big central hub… we set strict rules.

“We were learning and didn’t want anarchy to reign, so we said ‘these are the security rules, these are the builds you can have; if you want new features, we’ll put it into our central teams and they’ll release them’. Of course, the team could never keep up with AWS nor our own application teams’ demands of what they wanted.

“We didn’t wanted anarchy to reign…”

What we’re moving to now is a hub and spoke model where the security and the access sit out on the spokes.

We have a few core rules that we know matter: security, types of build, billing approach, some break-class capabilities for patching. (Applications [team] needs to decide but if they haven’t done it in a certain time we break the glass and do it).

But now the ‘spokes’ are able to innovate at pace, as they are not constrained by a central hub; they can accelerate away. I wouldn’t move away from the journey we’ve been on as we’ve learned it was constraining, and we needed to change it, although some teams still want and get an internal managed service.

A lot of the application migration examples given in your original announcement seemed to be downstream ones. Why not the upstream? 

A lot of our upstream activity is on Azure.

We’ve just found it a natural better fit in terms of the partnerships we’ve formed. Then again, Shell have put their’s [upstream applications] on AWS…

“Having a dual-cloud strategy works for us. If you’re doing a lot of engineering upwards, we find AWS great; for a lot of the ready-made platforms we find Azure good for that. And it boils down to how partnerships are forming.

How much of that had to do with your energy supply agreement? (Under which BP will provide over 170 megawatts of green energy annually to AWS)

In Europe we’ve come to a fantastic solution.

We had to test ourselves on why we didn’t go with a dual-cloud strategy, but we pulled together something unique.

Partnerships are changing: everybody has got to lean in to the fact that the world is changing and test boundaries… the days of just being a buyer and a seller are changing and AWS were open to a different kind of partnership.

“We were looking for a partnership that was that was bigger than just an IaaS Migration, which is why we announced the power agreement. Data centres are massive power hungry things, and we’ve got a renewables business… It made sense.”