Polyglot benchmarks

The site uses an Elixir front-end powered by Phoenix, LiveView and Ash connected to a PostgreSQL database with an entirely custom built CMS.

Anywhere that Speakers or Sponsors are displayed on the site, one of our Polyglot APIs are used to serve the request, displayed in the upper right corner of your screen. The Polyglot APIs all connect to a read only database connection via pgbouncer, accessing the Speaker and Sponsor data via versioned database views to simplify management across the fleet. If we need to change how data is fed across the APIs, we can update the VIEW rather than modifying each of the APIs.

Telemetry for each API is tracked in ETS (in memory storage for the BEAM VM used by Erlang/Elixir) by tracking the API round trip. New deployments clear the stats. The site is hosted on Fly.io, which will suspend each VM when not active for a few minutes to save costs. The startup time is relatively quick on a new request, but it can affect the benchmarks. In order to ensure rapid response, the next language up in the list is sent an async request following the current page load to keep it warm.

A couple of tweaks have been made to protect the user's browsing experience. First, Elixir will enforce a timeout behind the scenes if the response time for a language is to slow for some reason. This will lead to moving to another language or simply serving the request with SQL as a fallback.

Additionally, every request is made through a Dynamic GenServer with Cachex based on the data being requested and the language used with a 1 second TTL. If you make the exact same request to the exact same language within 1 second, you will be shown a cached result or you will share the result from the request in progress. The language and time stats for the original request will be displayed in the upper right, but the telemetry data for the request will not change.

The initial APIs that were developed were completed with Ruby (Sinatra) and Go (net/http). Following the initial development, a common AGENTS.md file was created and further language were created with a simple /goal Using our previous criteria, create a polyglot API for the site using <LANGUAGE> which included language specific tests, linters, static security checks, dependency security checks, gitleaks checks and compiler checks if needed as part of the precommit hooks.

Later changes were applied with a local script using herdr to start a new session across each language with a specific prompt. Some additional tuning was done on specific languages if a problem was apparent. For example, the JVM cold start problem was observed in this environment so we attempted to let the AI setup CRaC.

You may click on any language to review it's source on Github and we're happy to review PRs if you have suggestions! We aren't ready to open up for new language submissions just yet but we do have an API starter project for you if you are considering one.

Enjoy exploring and please feel free to share your thoughts on any of our socials!