Modern Microservices Architecture and DevOps Solution Increases Stability and Efficiency of Systems
Case Study: Application Development
Situation and Business Challenge
An American department store chain with hundreds of brick-and-mortar locations was losing market share to Amazon in online purchases. The organization had made significant investments in its digital platforms and business optimization efforts to compete with a giant e-commerce competitor, but its merchandising organization was bogged down by siloed development processes that required labor-intensive nightly batch updates as well as trigger updates of spreadsheets and other tabular data stored in databases.
The organization wanted to switch from their existing .NET technologies to a modern microservices architecture. The new solution would be based on an event-driven Java-based microservices architecture, Amazon Web Services (AWS), and Oracle RMS, enabling outputs at greater efficiency, frequency, and stability with reduced maintenance and support costs.
The project, however, faced numerous challenges. As a .NET-heavy division, the group in command of the Oracle RMS integration had almost no experience with the desired Java ecosystem, database architectural components, or AWS. Additionally, the IT organization, composed of 95% vendors and only 5% FTEs, lacked the necessary organizational structure. Only one team, which was dedicated to an entirely separate effort, was well-versed in agile processes, and no one had the know-how to implement and run DevOps effectively, which would be essential to maintaining and improving the solution in future iterations.
Realizing the need for external expertise to lead the project, the organization reached out to AIM Consulting for its robust reputation in Application Development and DevOps. As the organization’s culture was also resistant to change, they needed a consulting partner that could not only provide the expertise needed to implement the new solution, but also the change management to rapidly foster cooperation, achieve stakeholder buy-in, demonstrate value add, and provide ongoing mentorship, training, and support.
Solution
A team of AIM consultants, including a Product Owner, a Lead Developer/Architect, three Developers, and two SDETs implemented the microservices architecture solution in just five months using an agile approach (Scrum methodology) with two-week sprints and a customized DevOps model. The team worked autonomously (after the initial ramp-up period), interfacing with client stakeholders as necessary and mentoring client resources in the technologies and processes by cycling them through the team in 2-4 sprint cycles (4-8 weeks).
Services implemented were built on top of an AWS cloud architecture utilizing the following components:
- AWS: DynamoDB, Lambda, SQS, SNS, EC2, CloudWatch, etc.
- Java Microservices (REST), Spring Boot, and Docker
The implemented solution included collaboration with the internal Platform team on fleshing out and being on-boarded on a brand-new CI/CD platform and implementing a fully revamped end-to-end communication channel between the source of the data (an enterprise performance management system) and multiple versions of their Oracle RMS system. The modernized, scalable solution connects the systems via event-driven workflows, is built with high fault tolerance, and can automatically scale instances and services as needed. The solution was implemented with no incidents.
For the solution’s quality component, the AIM solution team imparted a behavior-driven development (BDD) approach with automated testing, which differs from the test-driven development (TDD) process in that it allows non-technical resources to create test cases.
The DevOps model utilized error alerts pushed live via Slack and integrated with telemetry as well as detailed, standardized logs in Kibana to enable data flow tracking across applications. The model allows the team to be alerted to production issues in real time, quickly investigate their causes, and if necessary, develop, build, test, and deploy a hotfix in a matter of hours. With the older architecture, these tasks took days and sometimes weeks to perform.
The project initially called for the building of an internal team to manage the solution upon its release, but because the IT organization could not have the required skillsets for DevOps available by project’s end, AIM Consulting was contracted to maintain it. AIM also developed thorough documentation for every component of the solution, including Cucumber for automated tests (BDD), API specifications and other tools via markdown files and other collaboration applications.
Results
AIM successfully helped the company transition to a modern Java-based microservices architecture in less than six months. The organization shared that it would have taken more than 1.5 years for the company to produce the same solution on its own.
The organization now has an event-driven microservices architecture in place with an automated system that listens for events and moves retail data from its source to the destination without human interaction or lengthy batch processes.
With agile methodologies and DevOps at the heart of their software development lifecycle (SDLC), the organization has noticed stark differences in the time required to build and deploy applications, run tests, and resolve post-deployment and runtime issues. Deployments that used to take days/weeks now take hours. Troubleshooting and root cause analysis that used to take hours now takes minutes. Deploying hotfixes that used to take hours/days now takes under two hours.
As AIM Consulting’s approach demonstrated rapid value, the IT organization’s initial resistance to change eroded. A series of executive demos conducted by AIM Consulting garnered high visibility across the organization so modernization has now been embraced.
