# Constile gate for Ruby The Constile gate runs in front of your site. When an AI agent fetches one of your pages for a person, the gate serves the Constile offer instead of the article. People, search engines, and crawlers get your pages as usual. This gem is the gate as Rack middleware. It works in Rails, Sinatra, and other Rack apps on Ruby 2.7 or later, and it uses only Ruby's standard library. ## Before you begin You need: - A Rack app on Ruby 2.7 or later. - Your site ID, site secret, and the network URL. Find them on your site page in the publisher dashboard. - The gem file, `constile_gate-0.6.0.gem`. Download it from your site page. The gate answers at the root of your domain, at `/.well-known/constile` and `/sponsored/`. Your app must serve the whole domain, not a subpath such as `/blog`. ## Install the gem 1. Unpack the gem into your app: ```sh gem unpack constile_gate-0.6.0.gem --target vendor/gems ``` 1. Add it to your `Gemfile`: ```ruby gem "constile_gate", path: "vendor/gems/constile_gate-0.6.0" ``` 1. Install it: ```sh bundle install ``` 1. Commit `vendor/gems/constile_gate-0.6.0`, so your deploys get it too. If your app doesn't use Bundler, run `gem install --local constile_gate-0.6.0.gem` on each server instead. ## Set your site's settings The gate reads its settings from environment variables: | Variable | Value | |---|---| | `CONSTILE_SITE_ID` | Your site ID. | | `CONSTILE_SITE_SECRET` | Your site secret. Keep it out of source control. | | `CONSTILE_NETWORK_URL` | The network URL. | If any of the three is missing, the gate passes every request to your app and logs one warning when your app starts. You can also pass each setting as an option, which wins over its variable: | Option | Variable | Default | What it does | |---|---|---|---| | `site_id` | `CONSTILE_SITE_ID` | | Your site ID. | | `site_secret` | `CONSTILE_SITE_SECRET` | | Your site secret. | | `network_url` | `CONSTILE_NETWORK_URL` | | The network URL. | | `timeout` | `CONSTILE_TIMEOUT` | `5` | Seconds the gate waits for each call to the network, from connecting to the last byte. | | `config_retry` | `CONSTILE_CONFIG_RETRY` | `30` | Seconds the gate leaves the network alone after it fails to give your site's configuration. | | `trusted_proxies` | `CONSTILE_TRUSTED_PROXIES` | none | Addresses or ranges of your proxies, such as `10.0.0.0/8`. See [Behind a proxy](#behind-a-proxy-or-load-balancer). | | `enabled` | | `true` | Set it to `false` to turn the gate off without a warning. | | | `CONSTILE_DISABLED` | | Set it to `1` to turn the gate off: every request goes to your app unchanged, and nothing goes to the network. Read on every request. | | `platform` | | `"rack"`, or `"rails"` in Rails | The name your site page and the install check show. | ## Rails The gem adds the gate as the first middleware in your app, in front of everything else. After you install the gem and set your settings, there's nothing more to add. To confirm that the gate is first, run: ```sh bin/rails middleware ``` The first line is `use ConstileGate::Rack`. To set options, use `config.constile_gate` in `config/application.rb` or in a file under `config/environments/`: ```ruby # config/environments/production.rb config.constile_gate.trusted_proxies = ["10.0.0.0/8"] ``` To turn the gate off in development and test, add this to `config/environments/development.rb` and `config/environments/test.rb`: ```ruby config.constile_gate.enabled = false ``` ## Sinatra Add the gate in `config.ru`, in front of your app: ```ruby # config.ru require "constile_gate" require_relative "app" use ConstileGate::Rack, platform: "sinatra" run App ``` If your app has no `config.ru`, add `use ConstileGate::Rack, platform: "sinatra"` to the app itself. It works the same way, but it runs after Sinatra's own middleware, such as sessions, so `config.ru` is the better place. ## Any other Rack app Add the gate first in `config.ru`, before any other middleware, so it sees each request as it arrived: ```ruby # config.ru require "constile_gate" use ConstileGate::Rack run MyApp ``` For another Rack framework, add `ConstileGate::Rack` as the first middleware the way that framework adds Rack middleware. ## Paywalled content Gate 0.6.0 and later can fetch a paywalled article with the release instructions the network sends, so an agent gets the full text and the offer fits it. The network gives the gate a page address on your site's own host and, optionally, headers to send: `Cookie`, `Authorization`, `Constile-Paywall-Key`, or a header starting with `X-`. The gate ignores an address on another host and drops other headers. It strips `Set-Cookie` from what it releases and never logs the headers. To set up a method, open your site's page in the dashboard and choose a content API, a paywall rule, or a Constile login. A paywalled page gets offers only after the method works and the gate is 0.6.0 or later. ## Behind a proxy or load balancer The gate sends the network the address of an agent that claims to be a registered agent, so the network can check it against the agent's published addresses. The gate reads that address from `REMOTE_ADDR`. It never believes `X-Forwarded-For`, because any client can send one. If your app runs behind a proxy or load balancer, `REMOTE_ADDR` is the proxy's address. To read the visitor's address, list your proxies in `trusted_proxies`. The gate then reads `X-Forwarded-For` from right to left and takes the first address that isn't one of your proxies. ```ruby use ConstileGate::Rack, trusted_proxies: ["10.0.0.0/8", "192.0.2.10"] ``` From one of your trusted proxies, the gate also reads the page's scheme and host from `X-Forwarded-Proto` and `X-Forwarded-Host`. From any other client, it ignores them. If your proxy terminates TLS, list it in `trusted_proxies`, so the gate knows the page is `https`. ## Caches in front of your app A cache in front of your app, such as a CDN or Varnish, can serve a cached page to an agent without asking your app. That agent then gets the article without an offer. To avoid that, configure the cache to send requests to your app when the `User-Agent` names an AI agent, or to vary on `User-Agent`. The gate marks each offer `Cache-Control: no-store`, so a cache never serves an offer to a person. ## Check the install To check the install, open `https://your-site.example/.well-known/constile`. The gate answers with your site ID, `"gate_version": "constile-gate 0.6.0"`, and the platform. The install check on your site page reads the same file. ## How the gate behaves - **Agents.** For a registered agent's request to an eligible page, the gate gets the page from your app, sends its title, description, and keywords to the network, and serves the offer in its place. - **Everyone else.** Every other response goes back from your app untouched. The gate reads a response only for an agent's request. - **Settings from the network.** The gate gets your site's configuration from the network and keeps it for as long as the network says, from 10 seconds to an hour. Apart from that, people's requests never reach the network. - **Outages.** If the network can't be reached, the gate serves your page as if the gate weren't there. Each call to the network takes one `timeout` at most, from connecting to the last byte, with no retries. On Ruby 2.7, a DNS lookup that hangs can take longer, because Ruby 2.7 can't interrupt the system resolver. When a call for your site's configuration fails, the gate passes every request through without asking the network for `config_retry` seconds, then asks again. The `/sponsored/` callbacks don't fall back to your page: they release content only against a recorded commitment, so they answer with an error instead. - **Errors.** If the gate itself fails, it logs the error and serves your app's page, as it does in an outage. The `/sponsored/` routes and the seller file answer 503 instead. Errors your app raises go on as they would without the gate. - **Character sets.** The gate reads a page's title and keywords in the charset its `Content-Type` names, or UTF-8 when it names none, and passes a page through byte for byte. - **Released articles.** When the network grants an agent access, the gate gets the article from your app with a fresh request that carries none of the agent's headers, cookies, or session, and drops any cookies your app sets on it. ### Limits - Under Rack 3, a page your app streams (a body that has neither `to_ary` nor `to_path`) gets no offer, because Rack 3 doesn't let middleware read it. The page goes to the agent as usual. Under Rack 2, the gate reads streamed pages. - For an agent's request, the gate asks your app for the page without compression, so `Rack::Deflater` doesn't get in the way. If your app compresses the page anyway, the gate reads gzip and deflate. A page compressed another way, such as Brotli, still gets an offer, but the network sees only its path, not its title or keywords. ## Run the tests The tests run in Docker, under Ruby 2.7 and 3.3. From the repository root: ```sh integrations/ruby/test.sh # every suite integrations/ruby/test/install_check.sh # build, then install in clean containers integrations/ruby/build.sh [outdir] # build the gem ``` `build.sh` writes the gem to `apps/network/public/install/` unless you name another directory.