Flagship Application Migration to AWS Leveraging Cloud-Native Technology and DevOps Best-Practices

Flagship Application Migration to AWS Leveraging Cloud-Native Technology and DevOps Best-Practices

The Client

A market-leading fund administrator with global presences and over $1 Trillion in AUM.

Risk Focus delivered excellent technical staff with deep knowledge of and insight into the operations and applications of Amazon Web Services, and strong governance and oversight of projects with effective use of AGILE methodologies. The Risk Focus team was able to quickly understand the complexity of our on-prem environment and apply their experience with other customers to guide our solution. Most importantly, the Risk Focus team created a strong partnership with my team to deliver the project.

Managing Director, Head of Development and Application Support

The Challenge

The Client’s flagship application had grown unwieldly due to a deeply layered architecture, the result of multiple generations of development. The Client was also looking to exit their data centers and wanted to use this as a catalyst to re-architect the applications and implement DevOps best practices. After initially failing with two other PS firms, this leading Financial Services provider engaged Risk Focus to take-over and complete the migration of their flagship product to AWS.

The Solution

Risk Focus helped the client by:

  • Introducing good development and release processes, increasing velocity and reliability
  • Introducing build and release automation, increasing reliability and repeatability
  • Introducing on-demand environments, decreasing cost
  • Rearchitecting the Flagship app, increasing flexibility

Release frequently requires automation, as there is no time for long, error-prone manual processes. Risk Focus specializes in identifying and opening bottlenecks with a combination of technology introduction and process coaching. Prior to their migration, many of the processes for the client were time-intensive, manual, and error-prone.  This led to severe contention in areas such as test environments for QA processes, where new application releases were bottlenecked by the availability of these environments. Additionally, though the implementation was technical, the base application of the firm supports back office processes for hedge funds.  Strong domain knowledge derived from building similar trading and risk systems at some of the largest banks and exchanges in the world guided Risk Focus in designs of architecture and recovery processes.

For this client, Risk Focus:

  • Introduced CI/CD pipelines for both applications and the infrastructure components such as networking and VPC on which applications depended.
  • Worked with the client to design and implement customer onboarding automation using DynamoDB and Lambda.
  • Used CloudFormation templates and CodePipeline to create On-Demand environments for QA processes, automating the process of creating and destroying test environments as needed.

The solution relied on CodePipeline, CodeBuild, and CodeDeploy, with code stored in CodeCommit to create the pipelines.

Migrating to the cloud is most effective when combined with application refactor.  One of the key victories of the client’s migration to AWS was an update to the architecture of their application.

Risk Focus assisted the client with:

  • Migrating their core application from a monolithic Java application in WebSphere backed by an Oracle store to a micro-services framework.
  • Designing a complete multi-region DR solution, effectively lowering their exposure to unforeseen regional events.

The services are run in Docker containers on ECS in AWS, with a managed-services backend running on a set of Postgres RDS databases.

The Benefits

Risk Focus’s work has substantially increased the client’s efficiency by introducing automation in numerous areas.

  • The newly introduced pipelines allowed the client to migrate from an arduous build process on a developer’s laptop to continuous integration complete with an automated test framework built into the deployment pipeline.
  • Automating onboarding has replaced a manual, error-prone system that took weeks with an automated system where the heavy lifting was complete in minutes.
  • Automating QA has not only allowed the client to remove the bottleneck in creating test environments, it has allowed for much more efficient resource management for a significant cost savings. When combined with automated tests, the weeks-long testing cycle was reduced to days. Risk Focus continues to work with the client to shrink that cycle from days to hours, as the legacy manual validation are increasingly captured in automation.

In the end, the client has achieved their key goals. Their application is onboarding customers, their team is moving forward at a greater velocity, and their lead time to release new features is starting to shrink dramatically. The client continues to move forward aggressively in their adoption of AWS. Risk Focus remains engaged with this client, continuing to provide guidance and focused technical deliverables.

Application Refactoring and DevOps Implementation for Industry Leading Regulatory Reporting Software Solution 

Application Refactoring and DevOps Implementation for Industry Leading Regulatory Reporting Software Solution 

The Client

RegTek.Solutions, now part of Bloomberg, is one of the premier software providers in the Regulatory Reporting space with 15 of the 20 largest global banks as customers. RegTek.Solutions, provides modular software solutions built around actionable regulatory intelligence.

Due to business developments, we needed to quickly evacuate our data center. Risk Focus moved us quickly into the cloud and re-engineered our development and deployment processes, allowing us to become more agile and increase delivery velocity.  Moreover, they helped us deliver our first SaaS offering and onboard our first clients quickly and seamlessly.  The key differentiator that Risk Focus brought to the table is the unique combination of deep domain knowledge coupled with technical expertise and delivery excellence

Brian Lynch

CEO, RegTek.Solutions, A Bloomberg company

The Challenge

RegTek’s software was developed and tested inside its own datacenter, with VMs running on a small set of Hyper-V racks. The software was delivered exclusively to run inside its clients’ datacenters as binary artifacts, but RegTek was also asked by clients to offer some of its products as a SaaS offering.

RegTek approached Risk Focus to help it:

  • Create a fully-automated CI/CD pipeline outside of its datacenter that would allow it to provision environments on demand, build the binary artifacts, and run large-scale testing on its suite of products
  • Create the ability for RegTek to produce different deployable binary artifacts, both traditional WARs as well as Docker Containers
  • Create a secure automated ability for it to offer some of its products as a SaaS offering to prospective clients
  • Onboard its first batch of clients onto the newly-built SaaS offering

The Solution

The project consisted of three parts:

  • Constructing CI/CD Pipelines
  • SaaS Architecture Design
  • Client Onboarding

We proposed that RegTek move to AWS, and because its software needs to retain the ability to run in any datacenter, we ensured that the software is cloud portable and not tightly bound to the AWS cloud native offerings. Additionally, because of the sensitivity of the data being reported, all SaaS clients had requested complete isolation from one another.

AWS was used to:

  • Create single-tenant VPCs with Oracle RDS instances provisioned for each client, which were created by using CloudFormation templates.
  • Deploy some of the RegTek products as SaaS offerings into separate AWS accounts under an Organization leveraging Consolidated Billing.
  • Send logs to CloudWatch. All access to the deployed resources is monitored by CloudTrail.
  • Achieve resiliency by relying on ELB, Multi-AZ deployments and Auto-scaling groups.

The domain knowledge of Risk Focus was especially useful in Client Onboarding, which RegTek had subcontracted fully to Risk Focus.

This involved:

  • Placing Business Analysts at the client sites to identify the necessary feeds and design their delivery process.
  • Perform required data mappings and enrichment to ensure that the raw trading feeds that were delivered by the client could be submitted to the SDR (Swaps Data Repository) of the DTCC.

Additional 3rd party technology used for this solution included:

  • Jenkins
  • Ansible
  • OneLogin for Authentication and Authorization of client users
  • DataDog for monitoring

The Benefits

Risk Focus’s work has helped RegTek.Solutions operate a highly successful business that features both client data center installations and a SaaS reporting solution.

In particular:

  • The elasticity provided by AWS allows RegTek develop and test much faster by provisioning and tearing environments down in an automated way.
  • It also allows their clients to keep growing their Financial Services business, while staying compliant and avoiding the hefty fines levied on businesses that do not report in an accurate and timely manner.

This allows RegTek.Solutions to provide a higher value service to their clients.

Deutsche Börse – Leveraging AWS For Rapid Infrastructure Evolution

Deutsche Börse – Leveraging AWS For Rapid Infrastructure Evolution

The Client

The Deutsche Börse is the German Stock exchange providing a marketplace for the trading of shares and other securities. It is also a transaction services provider and gives companies and investors access to global capital markets.

Given the extreme time pressure that we were under to deliver a mission-critical platform, together with Risk Focus’ we decided to use AWS for Development, QA, and UAT which proved to be the right move, allowing us to hit the ground running. The Risk Focus team created a strong partnership with my team to deliver the project. Maja Schwob

CIO, Data Services, Deutsche Börse

The Challenge

In 2017, the Deutsche Börse needed an APA developed for their RRH business to support MIFID2 regulations, to be fully operation by January 3, 2018. The service provides real-time MIFID2 trade reporting services to around 3000 different financial services clients. After an open RFP process, The Deutsche Börse approached Risk Focus to build this system, resulting in an on-time, problem-free launch. The Deutsche Börse then approached Risk Focus again in May 2018 to expand their system to allow the Börse to process twenty times the volume of messages, with no increase in latency, and deliver the system to their on-premises hardware within 4 months.

The Solution

Though the implementation was technical, Risk Focus was ultimately recruited by the business unit at the Deutsche Börse to provide an implementation of their service, requiring us to determine and implement both technical and business requirements. The stakeholder group also included the internal client IT team and the Bafin (German Financial Regulator), as the choice of technology, infrastructure and cloud provider was decided in tandem with all three groups. Risk Focus’s deep domain knowledge in Regulatory Reporting and Financial Services was crucial to understanding and proposing a viable solution to the client’s need that satisfied all stakeholders. That domain expertise in combination with Risk Focus’s technology acumen then allowed for delivery of the service under very tight constraints.

The client hardware procurement timelines and costs precluded the option to develop and test on-premises. Instead, Risk Focus developed, tested and certified the needed infrastructure in AWS and applied the resulting topology and tuning recommendations for the onsite infrastructure. Risk Focus:

  • Proposed a radical infrastructure overhaul of the client systems that included the replacement of their existing Qpid bus with Confluent Kafka, involving architecture changes and configuration tuning.
  • Implemented an automated CI/CD system that built both environment and application to find the optimal configuration, allowing developers and testers to create production-scale infrastructure on-demand cost- and time-effectively.

Finding optimal configuration required executing hundreds of performance tests with 100s of millions of messages flowing through a complex mission-critical infrastructure, and it would have been impossible in the few weeks available without the elasticity and repeatability provided by AWS.

Additional 3rd party technology used for this solution included:

  • Docker Swarm: client chosen Docker orchestration framework
  • Redis: in-memory cache
  • Confluent Kafka: scalable replay log
  • TICK: monitoring framework
  • Greylog: log aggregation
  • Jenkins: CI/CD pipeline

The Benefits

The system was delivered to the client data center on-time within a startlingly short timeframe. Risk Focus worked with the client to allow their internal IT departments to take over the delivered solution, allowing Risk Focus to disengage from the process. All test environments and automation were made available to the client, allowing them to further tune and evolve the system.

The ability for the client to continue developing and experimenting in AWS enables them to:

  • Make precise hardware purchasing decisions as volume demands change
  • Maintain an environment for further development to adapt to new regulatory requirements.

Risk Focus’s work provides a pathway to public cloud migration once that path is greenlighted by regulators.

CFRA-AWS Equity Publishing Platform Case Study

CFRA-AWS Equity Publishing Platform Case Study

The Client

CFRA Research is one of the best-known Financial Research companies and offers global coverage of individual stocks, ETFs and Mutual Funds to its retail and institutional clients.  The research that CFRA provides is based on quantitative and qualitative processing of large amounts of financial information on companies and their financial instruments.

Risk Focus understood our business and had the right technical expertise to design and execute a modern serverless architecture. This allowed us to increase agility, improve resiliency and cut costs at the same time.  They were able to engineer and rebuild the core of our platform in timeframes that were so compressed, that we did not believe were possible. Risk Focus replaced a low-cost outsourcing firm but proved to be much better value for money due to their expertise, productivity and ownership of delivery. They were committed to our success and were obsessed with automation, security and the total cost of ownership of the deliverable.

More impressively, they worked as true partners to our organization and helped us build out our teams, processes and IT infrastructure.

Eram Schlegel

CTO, CFRA Research

The Challenge

In 2016, CFRA acquired S&P Global’s Equity and Fund Research business including the core systems that were supporting it.  The systems were moved to AWS in a classic Lift & Shift approach.  In May of 2019, CFRA engaged Risk Focus to help them rationalize their whole architecture, replace certain raw data sources, integrate a new publishing platform from Eidos Media and create a new API layer to facilitate modern data access. Everything had to go live by October with no disruption to existing clients. 

The Solution

Our development team and business analysts worked with CFRA to map out the use cases to a logical architecture. This was then converted into a physical architecture and we created mocks for the various service implementations.  The API and Service design, the ingestion pipelines as well as the design of the database schemas required an intimate understanding of the business domain. Given the very tight timeframes, Risk Focus approached the challenge with a solution that would allow us to work in parallel. Risk Focus:

  • Partitioned the system into microservices
  • Agreed on the interfaces between them
  • Aligned the team structure to the microservices
  • Rolled-out a simple CI/CD solution based on Jenkins to allow the teams and developers to quickly roll out their pieces without breaking anyone else’s

The diagram below reflects the high-level architecture.

For each of the components, Risk Focus looked at the most applicable AWS offerings. The systems demand of the CFRA’s business are very spikey: multi-hour flat lines with big surges at various times.

This is a great use-case for many of AWS’s serverless services. Ultimately, Risk Focus used:

  • AWS Glue for the ETL service and AWS Lambda for most processing tasks
  • AWS API Gateway and Cognito for both the internal and client-facing APIs
  • Aurora for the main RDBMS, as it was both very performant and cost effective.

Additional AWS technology used for this solution included:

  • Glue
  • IAM
  • SQS
  • SSO
  • Aurora BeanStalk
  • CloudWatch

Additional 3rd party technology used for this solution included:

  • Jenkins
  • Terraform
  • Ansible

The Benefits

CFRA was able to go live within a very compressed timeframe with a new architecture and software stack that

  • Allows them to run with much lower data and software licensing fees
  • Resulted in over 90% cost reduction on average
  • Increased the resilience of the solution by leveraging the various AWS Serverless offerings such as AWS Glue, Lambda and Beanstalk

The delivery encompassed not just the end software, but also the necessary tooling and processes that enable CFRA to have a fast, automated, predictable Software Delivery Lifecycle.  In particular:

  • The CI/CD pipeline encompasses both the application and the underlying infrastructure
  • New environments can be provisioned on-demand with a partial or full stack of the software in minutes
  • Production releases can be made daily instead of quarterly
  • Fully automated pipelines ensure a consistent and repeatable security posture that has tightly controlled role access and secrets which are centrally managed and rotated

Risk Focus’s work has allowed CFRA to grow their analysis and distribution platform with increased agility and improved predictability.

Consolidated Audit Trail (CAT) resurrects the age-old question ‘Build vs. Buy?’

Consolidated Audit Trail (CAT) resurrects the age-old question ‘Build vs. Buy?’

Well, like any other complicated problem there isn’t a ‘one size fits all’ answer. Multiple variables will be at the heart of your decision making, including your firm’s scope, long-term strategy, maintenance, etc. Hopefully below will give your firm some guidance on things to consider making the right strategic decision.

Case for working with a dependable vendor:

Price: Your budget will likely be a #1 driver. Cost that will be associated with building an in-house solution will likely far exceed a pre-determined solution. Depending on your size, it may be economical to give up convenience of proprietary built vs. out of the box solution.

Time: Conforming with expected regulatory timelines is critical, both from reputational standpoint and having to avoid a potential regulatory fine/action. Possibility of slippage is a reason enough for you to consider a vendor solution.

Industry knowledge: SME knowledge is not replaceable, but where proficiency is lacking utilizing a vendor may be optimal way to go to market. A reputable vendor will ensure the solution aligns to the actual rule and regulator’s expectations.

Scale: Overtime the vendor will receive continuous feedback as it relates to the solution, and economies of scale will dictate that their solution will continue to improve and serve your broader needs.

Reasons to consider looking to internal solution:

Accountability: Don’t confuse ‘service offering’ that meets your needs with your obligations. Just because you are utilizing a given solution that seems to ‘work’, it doesn’t alleviate your overall responsibility for the accuracy of the reporting. ‘Safety in numbers’ will not work when external audit is conducted.

Ongoing maintenance: Going with a vendor is never truly ‘plug n’ play’. You will have to ensure you implement and deploy the solution. Involvement and scope will depend on size/needs of your firm.

Limitations: You will find yourself locked into the solution and proposal that aims to address broader needs but may not be customizable to your specific business and overtime may generate unwanted limitations which will be difficult to overcome.

Cost: up-front price may be attractive but consider the longevity and ongoing dependability. Price point shouldn’t be limited to ‘go-to-market’ mentally.

In Conclusion

Overall, in-house built is not replaceable, but it may be more practical to consider outside solutions. The decision is never one dimensional and never in the moment, as it will transcend scope and time. Your decision should balance practical short-term considerations and long-term strategy/vision.