Engineering blog

Our team discusses how spark and kubernetes are used to ingest, normalize and process millions of scientific papers for NLP.

At BenevolentAI we work with a Kubernetes based infrastructure that is completely containerized. Using Kubernetes we run many sophisticated workflows for data processing, model training, model serving and metrics and evaluation.

MetalLB is a software load-balancer for bare metal Kubernetes clusters. In this post we will go through the process by which we deployed MetalLB to our production cluster at BenevolentAI.

BenevolentAI’s System Software Engineer, Rory, explores the benefits of pair working on research and decision making. In software development, this collaborative way of working can help build team support and cohesion, and ultimately help teams to excel.