Voxxed Days Microservices 2018
from Monday 29 October to Wednesday 31 October 2018.
Hi there! I am a engineer and an open-source enthusiast working for Confluent, the company founded by the creators of Apache Kafka. In the past couple of years, I have worked closely with clients of all shapes and sizes - from small startups to Tier 1 bank - on topics such as designing event sourcing system and integrating microservices together.
For me, event sourcing and microservices are a solution to the growing complexity of software development and I expect it becomes a standard in the future. Furthermore, playing lego with containers is really funny!
When building a microservices-based systems, we don’t generally think too much about data. If we need data from another service, we ask for it. This pattern works well for whole swathes of use cases, particularly ones where datasets are small and requirements are simple. But real business services have to join and operate on datasets from many different sources and this can be slow and cumbersome in practice.
These problems stem from an underlying dichotomy. Data systems are built to make data as accessible as possible - a mindset that focuses on getting the job done. Services, instead, focus on encapsulation - a mindset that allows independence and autonomy as we evolve and grow. But these two forces inevitably compete in most serious service-based architectures.
Understanding and accepting this dichotomy is an important part of designing service-based systems at any significant scale. In this talk, we will explain how to make use of a shared, immutable sequence of records to balance data that sits inside their services with data that is shared, an approach that allows the likes of Uber, Netflix, and LinkedIn to scale to millions of events per second.