How moving to AWS and modernising their technology stack paid sustainability dividends for Future Super

Future Super was using a Kubernetes Cluster on the Google Cloud Platform to manage their workloads. However, the platform encountered challenges in meeting the evolving demands of the business, resulting in reliability and performance issues. They worked with their trusted AWS Partner, Fitzroy IT, to re-platform and modernise their services, using Amazon Elastic Container Service at the core of their solution. After a successful cutover, Future Super had a reliable platform, which performed far better and was trusted by the business. As an added benefit, Fitzroy IT upskilled Future Super’s staff. It implemented security and automation improvements whilst following best practices to create a solution that will enable Future Super’s forecast growth into the future.

Future Super faced several technical challenges as it expanded its investor base and funds under management, prompting a reassessment of its hosting infrastructure, and leading to a decision to move everything to AWS.

With most of its workloads running on a Kubernetes Cluster with GCP, Future Super realised that Kubernetes was not the optimal solution for their business needs. The complexity and steep learning curve associated with Kubernetes deployment and maintenance proved challenging for the team. Managing Kubernetes, especially without prior experience, required significant effort and resources. Moreover, the Google Kubernetes Engine (GKE)-specific complexities further compounded the learning curve, making it even more daunting for the team to effectively manage the platform.

Additionally, the operational overhead of managing a Kubernetes cluster posed a significant burden. The regular updates, patches, and maintenance tasks required for cluster management consumed valuable resources and detracted from the team’s ability to focus on strategic initiatives. The desire for a more streamlined and hands-off approach to infrastructure management became increasingly apparent as the team sought to optimise efficiency and scalability.

The challenges with Kubernetes configuration and maintenance also impacted cluster stability, leading to disruptions and challenges in maintaining service reliability. To mitigate these issues, the team resorted to scheduling problematic jobs during off-peak hours, albeit at the cost of added complexity and inconvenience.

Furthermore, Future Super encountered frustrations with other infrastructure inefficiencies, such as the manual management of databases, which resulted in convoluted processes for updates and maintenance.

The limitations of the GCP setup, including the lack of dedicated development environments beyond Staging and Production, further hindered development and QA efforts. This resulted in some staff resorting to running local development environments with mock data, disrupting deployment processes and impacting overall productivity.

In response to these challenges, Future Super wanted to rearchitect and replatform its infrastructure. By taking proactive action to address these issues, Future Super aimed to streamline operations, enhance scalability, and ensure seamless customer experiences as the company continued to grow, while remaining carbon neutral.

Fitzroy IT prides itself on its breadth and depth of expertise and its ability to be nimble and adapt to our customers’ changing needs. As a Managed Service Provider and an AWS Advanced Consulting and Well-Architected Partner specialising in Microsoft Windows Migrations, Fitzroy IT assisted Future Super with migrating the Windows workloads of an acquired superannuation fund. This first-hand experience showed Future Super that Fitzroy IT was competent and efficient, with a team wholly committed to achieving quality outcomes.

Fitzroy IT’s experience with the Well-Architected framework, and their ability to undertake regular right-sizing reviews also provides Future Super with the confidence to know that their compute layer is sustainable and maintains a minimal environmental footprint.

What Future Super really valued was Fitzroy IT’s experience with technologies beyond just core AWS infrastructure. With our team of developers and systems administrators, when Future Super needed assistance with modernising their Application Layer, they simply had to turn to their AWS Partner to provide additional assistance.

Future Super’s decision to adopt AWS company-wide was influenced by the need to rearchitect the platform to accommodate multiple funds. They had exposure to AWS when it acquired another superannuation fund and migrated its infrastructure to AWS. This showed them several things that were key to their decision to adopt AWS company-wide:

1. AWS provides tools like AWS Organisations and AWS Resource Groups for centralised management of resources, simplifying administrative tasks, monitoring, and governance.
2. AWS's robust security features enable isolation between different tenants or projects, ensuring security through Virtual Private Clouds (VPCs), Security Groups, and Network Access Control Lists (NACLs).
3. AWS had an extensive set of services that made system architecture and implementation quick and easy.
4. All services had simple, granular control interfaces, including simplified management via the AWS console.
5. There was a lot of appeal in being able to “start afresh”, building the infrastructure from the ground up, using best practice and Infrastructure-as-Code tooling. This approach also enabled significant modernisation to occur.
6. The AWS Account team were able to support both Future Super and Fitzroy IT with SA support and financial incentives.
7. AWS are targeting to be powered by 100% renewable energy by 2030 (and are on-track for 2025).
8. The AWS Customer Carbon footprint tool enables Future Super to accurately measure the emissions associated with their use of AWS.
9. AWS' Carbon Emission Factor for its ap-southeast-2 data centre was lower than GCP’s australia-southeast1.

Source: https://www.climatiq.io/blog/cloud-computing-amazon-google-microsoft-helping-companies-go-green

Fitzroy IT was initially engaged to work solely on infrastructure design and implementation. Planning began, and it soon became apparent that the advanced orchestration features of Kubernetes were excessive for Future Super and that the AWS Elastic Container Service (ECS) was the ideal solution. ECS supported the Docker images being used by Future Super, so there would be a degree of familiarity. The rollout began with a standard, containerised environment featuring AWS’s core tools for secure networking and a managed ECS Cluster: VPC, ECS, ECR, ELB, S3, EFS and RDS. AWS SSM is used for secure remote access, and CloudWatch provides logging. This multi-account setup was deployed and managed using Infrastructure-as-Code tools, centrally managed by AWS Control Tower, with compliance being enforced with AWS Config and inter-account networking being handled by AWS Transit Gateway.

During discussions with Future Super’s application development team, it quickly became apparent that there was an opportunity to perform substantial modernisation during the re-platforming; modernisation that would radically improve the security, performance and usability of the software as a whole. Fitzroy IT’s engagement was expanded to include modernisation and testing of the application layer. This work included updating the operating systems and software used on all Docker images and establishing new processes for building and deploying the Docker images, utilising AWS native tools, such as AWS Inspector.

Security improvements were also factored into Future Super’s different software services. Previously, in GCP, services had communicated securely with each other using encrypted network traffic that traversed the public Internet. In AWS, it was possible to refactor the services so that all cross-service communication remained within the private AWS network.

A range of tools and services were configured to improve and manage Future Super’s cloud security posture, ensuring it follows best practices and provides automated remediation.

Authentication was a core component of Future Super’s solution and was built around least privilege, zero trust access. Role-based access controls (RBAC) were implemented, allowing easier management of access into the different environments.

The modernisation process also led to the adoption of several other AWS services to provide additional functionality and improved performance. AWS API Gateway and Lambda were utilised for secure, public-facing APIs, while AWS Transfer Family provided SFTP services. Elastic File System (EFS) was utilised extensively, and Route53 resolvers allowed EFS to be shared across accounts, provided centralised storage, and optimised costs. Amazon MemoryDB for Redis was utilised to provide a performance boost.

Finally, Future Super and Fitzroy IT worked together to implement an extensive Disaster Recovery plan with AWS Backup at its core.

The benefits of Future Super’s new AWS-based platform are numerous. The benefits with the biggest positive impact on the business have been:

1. A more resilient, sustainable and scalable infrastructure that Future Super can trust
2. An upskilled team
3. Improved security and access for the entire team, making deployments and application setup straightforward.

The new Future Super infrastructure is far more resilient and functional than the GCP setup they moved away from. All application layers now scale individually for compute, so load-related problems have been eliminated, and performance has increased substantially. The batch processing time for scheduled jobs has been reduced from four (4) hours to just one (1) hour. Load testing led to the compute services being right-sized during the re-platforming. The infrastructure is in a great place, and the business can easily add new funds and track compliance.

Ongoing right-sizing efforts have led to a continuous reduction in carbon emissions, even as additional services are added.
Source: AWS Customer Carbon Footprint Tool

The Future Super team was upskilled and brought along for the modernisation journey. Migrating the workloads was important, but not as important as ensuring that the entire team had a deep understanding of their infrastructure so that they could manage it confidently in the future. Indeed, the Future Super team have already tackled an Auth0 implementation using the new platform, something they would not have dreamed of doing with GCP.

Future Super is now running four environments - development, staging, QA and Production. These are all managed with Infrastructure-as-Code, and CI/CD pipelines exist for everything, making deployments a breeze. Multiple layers of testing are now possible, and the management of access controls is centralised, consistent and straightforward, meaning developers are developing and not fighting for access.

These are all great outcomes, and they are complemented by the comprehensive and detailed documentation that Fitzroy IT has put together, ensuring that the knowledge of the infrastructure stays within the institution and is not lost with staff turnover.

The relationship between Future Super and Fitzroy IT has been mutually beneficial, with an excellent rapport developed between the two teams. Work is continuing on the new platform, with the next engagement looking at cost-savings techniques, additional right-sizing efforts and expanding the monitoring, alerting and logging tools being used.

As AWS pursues 100% renewable energy by 2025, Future Super and Fitzroy IT will continue to pursue their own sustainability goals, firmly practising what they preach.

Future Super and AWS – on the path to 100% renewable energy.
Source: AWS Customer Carbon footprint tool