Leveraging AWS For Rapid Infrastructure Evolution
The Deutsche Börse is the German Stock exchange providing a marketplace for the trading of shares and other securities. It is also a transaction services provider and gives companies and investors access to global capital markets.
In 2017, the Deutsche Börse needed an APA developed for their RRH business to support MIFID2 regulations, to be fully operation by January 3, 2018. The service provides real-time MIFID2 trade reporting services to around 3000 different financial services clients. After an open RFP process, The Deutsche Börse approached Risk Focus to build this system, resulting in an on-time, problem-free launch. The Deutsche Börse then approached Risk Focus again in May 2018 to expand their system to allow the Börse to process twenty times the volume of messages, with no increase in latency, and deliver the system to their on-premises hardware within 4 months.
Though the implementation was technical, Risk Focus was ultimately recruited by the business unit at the Deutsche Börse to provide an implementation of their service, requiring us to determine and implement both technical and business requirements. The stakeholder group also included the internal client IT team and the Bafin (German Financial Regulator), as the choice of technology, infrastructure and cloud provider was decided in tandem with all three groups.
Risk Focus’s deep domain knowledge in Regulatory Reporting and Financial Services was crucial to understanding and proposing a viable solution to the client’s need that satisfied all stakeholders. That domain expertise in combination with Risk Focus’s technology acumen then allowed for delivery of the service under very tight constraints.
The client hardware procurement timelines and costs precluded the option to develop and test on-premises. Instead, Risk Focus developed, tested and certified the needed infrastructure in AWS and applied the resulting topology and tuning recommendations for the onsite infrastructure.
- Proposed a radical infrastructure overhaul of the client systems that included the replacement of their existing Qpid bus with Confluent Kafka, involving architecture changes and configuration tuning.
- Implemented an automated CI/CD system that built both environment and application to find the optimal configuration, allowing developers and testers to create production-scale infrastructure on-demand cost- and time-effectively.
Finding optimal configuration required executing hundreds of performance tests with 100s of millions of messages flowing through a complex mission-critical infrastructure, and it would have been impossible in the few weeks available without the elasticity and repeatability provided by AWS.
- Docker Swarm: client chosen Docker orchestration framework
- Redis: in-memory cache
- Confluent Kafka: scalable replay log
- TICK: monitoring framework
- Greylog: log aggregation
- Jenkins: CI/CD pipeline
The system was delivered to the client data center on-time within a startlingly short timeframe. Risk Focus worked with the client to allow their internal IT departments to take over the delivered solution, allowing Risk Focus to disengage from the process. All test environments and automation were made available to the client, allowing them to further tune and evolve the system.
- Make precise hardware purchasing decisions as volume demands change
- Maintain an environment for further development to adapt to new regulatory requirements.
Risk Focus’s work provides a pathway to public cloud migration once that path is greenlighted by regulators.