As financial services enterprises integrate AI into their operations, the demands on their compute infrastructures grow. For many organizations, this growth leads to higher cloud costs. Organizations now realize that cloud solutions might not meet the performance consistency or budget predictability necessary for their crucial workloads. This report, commissioned by Dell Technologies, examines whether on-premises servers—specifically Dell™ PowerEdge™ servers equipped with 4th Generation AMD EPYC™ processors and NVIDIA® GPUs—can deliver a more secure, better-performing, and more cost-efficient alternative to cloud-hosted environments for real-world AI workloads.
Prowess Consulting assessed the total execution time, cost per job, and anticipated total cost of ownership (TCO) of using a hybrid machine learning (ML) and AI pipeline. The pipeline used in this assessment ranges from classic sentiment analysis to innovative applications of transformer-based large language models (LLMs), and it is similar to pipelines used by banking firms for customer analytics and digital marketing. The pipeline mirrors enterprise applications such as customer experience management tools, social media solutions, and semantic search technologies. To conduct this assessment, we compared a Dell PowerEdge R7615 server and a comparable single-GPU instance in Microsoft Azure®. Our analysis measured raw performance and price-performance, while also evaluating operational efficiency, system versatility, and the ability for enterprises to optimize workloads and manage data governance.
Our findings indicate that Dell PowerEdge servers have the potential to surpass cloud-based equivalents in AI workload performance, even with less-powerful GPU resources. Furthermore, our study underscores the architectural strengths of on-premises servers. These strengths include lower latency, consistent resource availability, and tailored hardware configurations that can help avoid the expenses of cloud overprovisioning.