The last eighteen months has seen drastic change across the data world. With momentum building around data tooling, organisations would be forgiven for thinking they finally had what they needed. However, March 2022 signals the end of support life for data platforms that have dominated the market for years. Whether organisations are mid-migration or yet to start their journey, complex, risky and expensive big data cloud migrations are staring them in the face.

So what can organisations do to simplify and de-risk these big data cloud migrations? Nimbus Ninety, in partnership with Unravel Data, Appsbroker and Google Cloud, brought together members from across industry to lend support, share challenges and, perhaps most importantly, enjoy a cheese twist or two. The evening began with presentations from Firat Tekiner, EMEA Data Analytics Practice Lead at Google Cloud and Andy MacInnes, CTO at Appsbroker before breaking out into a highly enjoyable and productive virtual roundtable discussion.

BUILDING A UNIFIED ANALYTICS DATA PLATFORM

Firat began by emphasising the need for cloud migrations that attend to the nuances of specific organisations. Organisations have to deal with varying degrees of data governance and different organisations may be more suited to hybrid or multi-cloud solutions. Therefore, to bring whole organisations - from business users to data engineers - into one unified data platform, flexibility is key. Firat put forward four key characteristics for creating unified data platforms:

• A flexible tool stack: Through providing a ‘lego brick’ style tool stack, cloud platforms are able to offer a broad range of BI, analytics, visualisation and dashboard tools for organisations to pick from.

• A solution to the data warehouse or lake dilemma: Cloud platforms decouple storage and compute capabilities allowing organisations to easily and economically scale storage and compute as required by their big data processing needs.

• Offer emerging trends and technologies: Emerging trends and technologies can solve challenges and empower analytics. For instance, new options such as dataplex, a system by which to centrally manage, govern and monitor data, enables federated access to data at scale.

• Legacy technologies: Migrations involving legacy technologies are labelled as highly complex and risky migrations. The ability to support both low risk, conservative approaches, such as ‘lift and rehost’, and more complex, high risk modernisation migrations is crucial.

DATA TRANSFORMATION IN ACTION 

Andy presented a case study of data transformation when working with John Lewis Partnership. Upon joining, Waitrose and John Lewis operated with completely different data platforms, preventing any sharing of trends that could be mutually beneficial. So, Andy and his team set about a journey to create a shared data platform, oriented around reusing and remodelling. 

The first key takeaway from the data migration journey was data management. They soon found that the quality of the data would hamper any efforts to migrate. For instance, across both organisations there was data for 1,000+ black lipsticks, different definitions of profit and different data management offices. Solving the data management problem was key to a successful platform. Additionally, Andy emphasised that data migrations are an evolving journey. For instance, once they implemented the platform, propensity modelling capabilities grew from 1 model every three weeks to 1000 models an hour. Then, AI tracking to identify fraudulent activities was added. All of this was possible through following a clear plan to reuse and remodel.

DRIVING AND SIMPLIFYING MIGRATIONS

Members then broke out into roundtable discussions. The key takeaways from the discussion were:

1. Choosing technology

The discussion was heavily dominated by the factors for choosing cloud technologies. A clear consensus emerged here around performance, customer support, security, speed, regulation factors, sustainability, business as usual and previous case-studies as important factors when choosing between technologies. Although, more in-depth discussion emerged around price as a double-edged sword. All organisations are looking for the most cost-effective solution but larger investment initially could not only boost operational efficiency but also prevent further investment if it was not done right the first time. Furthermore, members felt that business outcomes were crucial for choosing technology. For example, members noted that the technology to achieve agility within the market might be very different to that needed for building a customer-first cloud platform. Tooling was also a key factor in technology decision-making for members. Some members noted that in large organisations with different silos and usages, the ability to support a variety of tools is not only where value is achieved but also a way of remaining independent from the cloud itself.

2. Managing stakeholders

A theme that gained a lot of support from members was about managing stakeholders. One member suggested that a cloud migration is a balancing act between the defensive point of view, of a CDO or CIO looking to ensure that data is safe and meeting regulation, and the offensive view, of a CMO looking to gain value from data insights and analysis. Ultimately, a compromise here is only possible when these stakeholders buy into the process and engage with the risks and benefits of cloud migration.

3. Knowing your data

Many members emphasised the importance of contextual understanding around data. This is not just around the quality of data; although the phrase “rubbish in, rubbish out” still holds true here. It is about identifying the value and purpose of data, understanding what it means to you, to customers, to all employees and, ultimately, what you want to do with it.