Once they occupied the top of the food chain. For millennia. And then the unthinkable happened: the clouds closed the sky, and they ceased to exist. At the other end of the world events that changed the climate: increased cloud cover. Dinosaurs became too large and slow: their attempts to survive were doomed to failure. Higher predators ruled the Earth for 100 million years, growing bigger and stronger. They evolved into what seemed like an ideal being at the top of the food chain, but the Universe instantly changed the face of our planet.
Ironically, it was the clouds that obliterated the dinosaurs 66 million years ago. In the same way, clouds today are destroying classic data storage systems that are “at the top of the food chain.” In both cases, the problem was not in the clouds themselves, but in the ability to adapt to a changing world. In the case of dinosaurs, everything happened quickly: the destructive effect of the clouds occurred within days or weeks from the moment the meteorite fell (or a volcanic eruption - the choice of theory remains with you). In the case of classic data warehouses, the process takes years, but it, of course, is irreversible.
Triassic: the age of the great iron and the emergence of migratory applications
So what happened? In the existing ecosystem, there were entry-level and mid-range storage systems, enterprise-level systems and direct-attached storage systems (DAS). These categories were determined by analysts, had their own market volumes, indicators of cost, reliability, performance, scalability. And then something strange happened.
The advent of virtual machines meant that many applications, probably several owners, could work on one server at once - such changes immediately called into question the future of direct-attached storage. Then the owners of the largest hyper-scalable infrastructures (hyperscalers): Facebook, Google, eBay, etc., tired of paying huge sums of money for storage, developed their own applications that made data available on regular servers instead of large "iron" storage systems. Then Amazon introduced something strange to the market called Simple Storage Service or S3. Not a block, not a file, but something fundamentally new: it became impossible to buy a system, it became possible to buy only a service. Wait, what kind of bright light is visible in the sky? Another asteroid?
Jurassic period: the era of "good enough"
We entered the phase of storage development with the ideology of "good enough." Customers using storage, noticing what the hyperscalers did, began to question the fairness of the ten- or even a hundred-fold added value in excess of the iron that they paid for their corporate storage. Mid-range arrays began to win market share from top-level systems. Products like HPE 3PAR have shown rapid growth. EMC Symmetrix, once the dominant array (from the word "massive") of an enterprise class, still held some territory, but it was rapidly decreasing. Many users began to transfer their data to AWS.
On the other hand, storage innovators began to borrow ideas from hyperscalers using technologies of distributed horizontally scalable systems - an ideology opposite to vertical scaling. It is expected that the new storage software will be able to work on regular servers, just like hyperscalers. No more 10-100 multiple prices in excess of the cost of the equipment itself. In theory, you can use any server - the choice depends on your preference. The era of software-defined storage systems (SDS) has begun: clouds have covered the sky, temperatures have dropped, and the population of higher predators has begun to decline.
Cretaceous period: the beginning of the evolution of software-defined storage systems
The early days of software-defined storage were stormy. Very much was promised, but few were delivered. At the same time, an important technological shift occurred: flash memory has become a modern alternative to “rotating rust” (HDD). This was the period of the emergence of many SHD startups and easily distributed venture money. Everything would be great, if not for one problem: data storage requires a serious attitude. It turned out that customers like their data. If they lose access to them, or a couple of incorrect bits are found in terabytes of data, they are very worried and worried. Most startups did not survive. Customers got cool functionality, but not everything was good with basic tools. Bad recipe.
Cenozoic period: storage arrays dominate
Few people talk about what happened after, because it’s not very interesting - customers continue to buy the same classic arrays of storage. Of course, those who moved their applications to the clouds moved the data there too. But for the vast majority of customers who do not want to switch to the cloud completely, or do not want to switch at all, the same Hewlett Packard Enterprise continued to offer classic arrays.
We live in 2019, so why is there still a multibillion-dollar storage business based on technologies from the time of Y2K? Because they work! Simply put, the requirements of critical applications were not implemented by products created on the wave of hype. Products such as HPE 3PAR have remained the best options for corporate customers, and the new round of evolution of the HPE 3PAR architecture - HPE Primera - this only confirms.
In turn, the capabilities of software-defined storage were excellent: horizontal scalability, the use of standard servers ... But the payback for this was: unstable availability, unpredictable performance and specific scalability rules.
The complexity of customer requirements is that they never become easier. No one will say that loss of data integrity or increased downtime is acceptable. That is why architecture is so important for storage systems that simultaneously meet the requirements of modern rapidly evolving data centers and at the same time, in search of a compromise, is not devoid of key characteristics of enterprise-class storage systems.
Tertiary period: the emergence of new forms of life
Let's try to figure out how one of the newcomers to the storage market - Datera - managed to cope with such a difficult mix of historically established and new storage requirements. First of all, due to the implementation of architecture oriented towards solving the dilemma described above. It is impossible to modify the old architecture to solve the problems facing a modern data center, just as it is impossible to modify the architecture of an average software-defined storage system to meet the requirements for enterprise-class systems: dinosaurs did not become mammals because the temperature dropped.
Building a solution that meets the requirements of enterprise-class storage and at the same time takes into account the full value of the dynamism of a modern data center is not an easy task, but that was exactly what Datera intended to do. Datera specialists have been working on this for five years and have found a recipe for “preparing” an enterprise-class software-defined storage system.
The main difficulty that Datera encountered was that it was necessary to use the logical AND operator instead of the noticeably simpler OR. Stable availability, “AND” predictable performance, “AND” architectural scalability, “AND” orchestration-like-code, “AND” standardized equipment, “AND” implementation of management policies, “AND” flexibility, “AND” analytic management, “AND” security, “AND” integration with open ecosystems. The logical operator "AND" is one character longer than "OR" - this is the main difference.
Quaternary period: modern data centers and drastic climate change determine the development of software-defined storage systems
So how did Datera create an architecture that meets the requirements of traditional enterprise-class storage systems and satisfies the demands of a modern data center at the same time? It all comes down again to this annoying AND operator.
There was no point in solving one task at a time to meet individual requirements. The sum of such elements will not become a single whole. As in any complex system, the careful study of the whole complex of balanced compromises was important here. During development, Datera specialists focused on three main principles:
- application-specific management;
- a single mechanism for ensuring data flexibility;
- high productivity due to reduced overhead costs.
A common property of these principles is simplicity. Simple system management, simple data management with a single elegant mechanism and providing predictable (and high) performance by reducing costs. Why is simplicity so important? Experienced masters from the world of storage know that it is not possible to meet storage requirements for a modern, dynamic data center using only granular control, many data management tools and hyperoptimization to increase productivity. A set of such techniques is already familiar to us as the storage system dinosaur.
Familiarity with these principles has served the Datera well. The architecture they developed has on the one hand the availability, performance, and scalability of a modern enterprise-class storage system, and on the other hand, the flexibility and speed necessary for a modern software-defined data center.
Availability of Datera in Russia
Datera is a global technology partner for Hewlett Packard Enterprise. Datera products are tested for compatibility and performance with various HPE ProLiant server models.
You can learn more about Datera architecture at the HPE webinar on October 31.