How-to use Templated Prepared Queries with for Multi-Service Discovery

Avatar
Amit Chahal
  • Updated

Introduction

Consul 0.6.4 and later support prepared query templates. These are created similar to static queries, except with some additional fields and features. In environments where enormous number of services run and require Consul for Service Discovery, the templated Prepared queries can help avoid creating multiple prepared queries for individual services, and a single query can potential match most or all services in environment.

 

Use-Case Served:

  • One reusable query definition.

  • Dynamic matching for any service.

  • Compatibility with DNS lookups used by downstream applications.

 

Pre-Requisites:

  • Consul Version >=0.6.4
  • Tools installed: bind-utils , systemd-resolved
  • Consul cluster installed with Services running and registered in Consul.
  • DNS Forwarding configured as per this Document

 

How to Set Up:

Note: Some example service/query names used as below:

  • backend.service.consul
  • frontend.service.consul
  • backend.query.consul
  • frontend.query.consul

  • Templated Prepared Query to match the Full service name:

In Below example, it shows how to create a prepared query in order to match any service using full service name with the help of variable ${name.full} for a Service to be matched.

# full-svcname-query-template.json

{
    "Template": {
      "Type": "name_prefix_match",
      "RemoveEmptyTags": true
    },
    "Service": {
      "Service": "${name.full}",
      "SamenessGroup": "",
      "Failover": {
        "NearestN": 3,
        "Datacenters": [],
        "Targets": []
      },
      "OnlyPassing": true,
      "IgnoreCheckIDs": [],
      "Near": "",
      "NodeMeta": {},
      "ServiceMeta": {},
      "Connect": false,
      "Peer": ""
    },
    "DNS": {
      "TTL": ""
    }
  }

 

Next, Lets Register the prepared query, and we get a 32 character long UUID identifier for prepared query:

$ curl --request POST -d @full-svcname-query-template.json \
http://127.0.0.1:8500/v1/query

{5806cfe6-7952-69a7-0905-b42231916511}

 

Assuming there's a service already running, called backend and registered in consul, We can use a static lookup using API call to test the above template against backend to see if it fetches the service :

curl -sq http://127.0.0.1:8500/v1/query/backend/execute|jq

<snip>--------------
{
  "Service": "backend",
  "Nodes": [
    {
      "Node": {
        "ID": "4ac8a9a3-1409-8b03-8a3b-f020c15c529f",
        "Node": "lima-dc1-cli-01",
        "Address": "192.168.105.11",
        "Datacenter": "dc1",
        "Partition": "default",
        "TaggedAddresses": {
          "lan": "192.168.105.11",
          "lan_ipv4": "192.168.105.11",
          "wan": "192.168.105.11",
          "wan_ipv4": "192.168.105.11"
        },
--------------<snip>
}

 

What we saw above, is the fact that our generic templated prepared query has resolved an existing service named backend in cluster.

Note that, there is no prepared query created specific to backend or any other existing services. The lookup for backend was matched by templated query and it returned the results.

Now, Lets verify and see that backend.query.consul is actually running and returning a response in dig output:

 

$ dig backend.query.consul

<snip>-------------
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64868
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;backend.query.consul.         IN      A

;; ANSWER SECTION:
backend.query.consul.  0       IN      A       192.168.105.11
-------------<snip>

 

$ dig frontend.query.consul

<snip>-------------
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60929
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;frontend.query.consul.          IN      A

;; ANSWER SECTION:
frontend.query.consul.   0       IN      A       192.168.105.19
-------------<snip>

 

For more practical use, the prepared query based full service name, example: backend.query.consul, can be used in downstream application(frontend) to discover and resolve upstream services on the fly.

Now, Lets run the frontend application by using BACKEND_URL environment variable for backend service, which gets intercepted by templated prepared query. Also run backend service as well:

 

  • Running Frontend application:

BACKEND_URL=http://backend.query.consul:7000 ./frontend

2025/10/28 01:56:45 Starting server listen_addr="0.0.0.0:6060"
[GIN] 2025/10/28 - 01:57:00 | 200 |    5.390416ms |   192.168.105.1 | GET      "/"
[GIN] 2025/10/28 - 01:57:00 | 200 |     469.875µs |   192.168.105.1 | GET      "/static/binos.png"

 

  • Running Backend Application:
$ ./backend
2025/10/28 01:57:00 Starting server listen_addr="0.0.0.0:7000"
2025/10/28 01:57:00 Connection=close
2025/10/28 01:57:00 User-Agent=Go-http-client/1.1
2025/10/28 01:57:00 Accept-Encoding=gzip
[GIN] 2025/10/28 - 01:57:00 | 200 |     951.832µs |  192.168.105.19 | GET      "/bird?delay=0&error-rate=0"
2025/10/28 01:57:00 User-Agent=Go-http-client/1.1
2025/10/28 01:57:00 Accept-Encoding=gzip
2025/10/28 01:57:00 Connection=close
[GIN] 2025/10/28 - 01:57:00 | 200 |     164.417µs |  192.168.105.19 | GET      "/bird?delay=0&error-rate=0"

 

In above example, the BACKEND_URL=http://backend.query.consul:7000 will discover and resolve healthy instance(s) of backend service, which we saw running at 192.168.105.11:7000 in previous dig output. And this address will be used by frontend to make contact with backend. 

 

  • Access Frontend application from CLI, which shows HTTP status 200 OK :

$ curl -vv http://192.168.105.19:6060/
10:50:54.614395 [0-0] * [SETUP] added
10:50:54.614516 [0-0] *   Trying 192.168.105.19:6060...
10:50:54.614687 [0-0] * Connected to 192.168.105.19 (192.168.105.19) port 6060
10:50:54.614710 [0-0] * using HTTP/1.x
10:50:54.614861 [0-0] > GET / HTTP/1.1
10:50:54.614861 [0-0] > Host: 192.168.105.19:6060
10:50:54.614861 [0-0] > User-Agent: curl/8.11.1
10:50:54.614861 [0-0] > Accept: */*
10:50:54.614861 [0-0] >
10:50:54.614907 [0-0] * Request completely sent off
10:50:54.617680 [0-0] < HTTP/1.1 200 OK
10:50:54.617722 [0-0] < Content-Type: text/html; charset=utf-8
10:50:54.617745 [0-0] < Date: Wed, 29 Oct 2025 10:50:54 GMT
10:50:54.617769 [0-0] < Transfer-Encoding: chunked

Conclusion:

With the help of templated prepared query, which made use of interpolation variable for service as ${name.full}, multiple services can be dynamically discovered without having to write separate prepared query for each service, especially where number of services are high in numbers.

 

References:

Articles in this section

Related articles