Private Search Engines

Search Expander supports private search engines by allowing you to proxy all your requests to our API as well as proxy all image and audio requests. We also allow you to remove any calls to various third-party services that cannot be proxied, such as Google Maps. This will ensure that no (unproxied) third-party calls are made and no third-party cookies are set.

Protecting requests with an API key

Requests to Search Expander are typically authenticated using a unique search engine ID together with the referrer domain. If you want stronger authentication, you can proxy requests to the Search Expander API and add a secret API key (saved under your Search Expander account) to the requests.

Once your proxy is set up, you will need to set an appropriate value for apiUrlTemplate in the settings object passed to sxpr(), so that the JavaScript library knows that it should call your proxy server rather than Search Expander.

See the page on API keys for an example of how this is set up. (The examples on the rest of the current page assume that an API key is not used for requests.)

More details

Search Expander will make HTTP requests to its own server and to various third-party services in order to obtain API responses, CSS files, images, audio and other data. You may need to limit its functionality and/or proxy its requests in order to adhere to your site's privacy policy. This can be taken care of by setting appropriate properties on the settings object passed to the Search Expander JavaScript library's sxpr() function.

The following features may result in client-side calls to third-party services. Each can be disabled by setting the appropriate property to false:

boolean

boolean

boolean

boolean

Search Expander also provides the ability to set proxy URLs for its requests to third parties using template strings:

string

string

string

If you wish to proxy calls to Search Expander itself, you can do so in a similar way:

string

string

If these URL templates are set with appropriate tags, Search Expander will make requests to the specified proxy server instead of the original URL.

(Note: you can see a complete list of sxpr() settings here.)

Below is some example JavaScript code for a privacy-friendly use of Search Expander:

sxpr({
    se: 'your-search-engine-id',
    apiUrlTemplate: 'https://my-proxy.example.com/sx-api/?path={path}', // Proxy Search Expander API calls
    assetsUrlTemplate: 'https://my-proxy.example.com/sx-assets/?path={path}', // Proxy Search Expander CSS/JS/JSON/images
    imageUrlTemplate: 'https://my-proxy.example.com/?url={url}', // Proxy third-party images
    audioUrlTemplate: 'https://my-proxy.example.com/?url={url}', // Proxy third-party audio
    wikipediaApiUrlTemplate: 'https://myproxy.example.com/?url={url}', // Proxy Wikipedia API calls
    googleMapEmbed: false, // Disable Google Maps embeds
    // ... other settings
});

Proxying Search Expander API Requests

You may wish to proxy your requests to the Search Expander API. For example, your website may have a policy of no client-side calls outside of your own domain.

There are two approaches to this:

1. Set up a CNAME record or a simple reverse proxy (using Nginx, Apache or similar), in order to forward client-side API requests to Search Expander's domain (api.searchexpander.com).
2. Set up a custom proxy on a domain you control, filtering and modifying API request data before it is sent on to Search Expander

With either approach, you need to specify a proxy URL template using the sxpr() setting apiUrlTemplate, so that sxpr() knows where to send its API requests.

For example, if your CNAME/proxy domain is my-proxy.example.com, you will need to set apiUrlTemplate to https://my-proxy.example.com{path}. The {path} tag will be replaced with the appropriate API path, e.g. /v1/query.

Note that all Search Expander API requests are JSON POST requests.

API proxy URL templates

Example 1

If you are set up to directly forward all requests from domain my-proxy.example.com to the Search Expander API, you will need to set your apiUrlTemplate to https://my-proxy.example.com{path} in the settings object passed to sxpr():

sxpr({
    se: 'your-search-engine-id',
    apiUrlTemplate: 'https://my-proxy.example.com{path}',
    // ... other settings
});

In this case, when sxpr() needs to call the Search Expander API endpoint /v1/query (for example), it will call https://my-proxy.example.com/v1/query instead of using Search Expander's own domain.

Example 2

Let's say you have a custom proxy script written in the PHP file api-proxy.php which is located at https://example.com/api-proxy.php. This PHP script reads a query string variable called sx-api-path and takes care of processing request data before sending and receiving an API response. Finally, the PHP script sends the API response back to the client.

In this case, your URL template (apiUrlTemplate) would be:

/api-proxy.php?sx-api-path={path}

(Note that sx-api-path can be replaced with any query string key that your proxy script understands.)

The {path} tag will be automatically replaced by the appropriate Search Expander API path before your proxy is called. For example, if the required Search Expander API path is /v1/query, it will make a POST request to /api-proxy.php?sx-api-path=%2Fv1%2Fquery, instead of calling the Search Expander API.

Add this URL template as the apiUrlTemplate property on the object passed to sxpr():

sxpr({
    se: 'your-search-engine-id',
    apiUrlTemplate: '/api-proxy.php?sx-api-path={path}',
    // ... other settings
});

There is example proxy server code to illustrate this further below.

Custom proxy server

You may wish to use a custom proxy server to filter out or modify client data before it is sent on to Search Expander.

Your proxy endpoint will receive:

• The path of the Search Expander API to which your proxy will forward requests, as part of the request URL
• JSON-encoded POST data to forward to the Search Expander API

Your proxy code will need to add a referer header containing your search engine's domain before forwarding on the request.

Important: Requests to the Search Expander API must be POST requests.

Here is some PHP example code to illustrate this, which assumes you have set apiUrlTemplate in the way described above:

// api-proxy.php

/**
 * WARNING: This is simplified code for instructional purposes. It has no security precautions or error handling!
 */

// In this example, the API path will be passed to the proxy in a query string variable named 'sx-api-path':
$api_path = $_GET['sx-api-path'] ?? null;
if (!$api_path) {
    die('No sx-api-path set in query string');
}

// Create SX API request URL:
$api_url = 'https://api.searchexpander.com' . $api_path;

// Get the JSON-encoded POST body sent by the sxpr() JS function:
$request_data = json_decode(file_get_contents('php://input'), true);

// Prepare cURL request to the SX API:
$ch = curl_init($api_url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($request_data));
curl_setopt($ch, CURLOPT_REFERER, 'example.com'); // Your search engine domain name
curl_setopt($ch, CURLOPT_HTTPHEADER, ['Content-Type: application/json']);

// Get the JSON response from the SX API:
$response = curl_exec($ch);

// Check for an error:
$error = curl_error($ch);

curl_close($ch);

if ($error) {
    die('There was an error: '.$error);
}

// Output the JSON response from Search Expander:
echo $response;

If you wish to authenticate requests to the Search Expander API using an API key, see the API keys page for a modified example of proxy server code.

(Note on PHP: Search Expander uses JSON across the board for its requests and responses, which means you cannot normally read its values directly from the $_POST superglobal.)

» API keys